Project Case Study: Institutional AI Market Intelligence System

A high-precision market intelligence engine that scans 5+ raw NSE/Media data sources, filters 90%+ noise using a 22-rule deterministic AI scoring engine, and dispatches real-time institutional-grade signals.

AsyncioDeterministic AISQLite WALOCR PipelineMarket IntelligenceSarvam 30B

Live Demo

Overview

Official Website: Bulkbeat TV

Bulkbeat TV is a production-grade Market Intelligence System designed for high-frequency signal discovery in the Indian equity markets. In an environment where 1,000+ corporate filings and news items are generated daily, the "Engineering Moat" is not just scraping—it is Deterministic Noise Reduction.

The system identifies high-impact signals (bulk deals, order wins, USFDA approvals) using a tiered verification pipeline: Async Ingestion → Content Hashing → PDF OCR → 22-Rule AI Scoring → Institutional Alerting.

Technical Valuation: Replacement Cost: ₹52.5 Lakhs | Adjusted Complexity: 156 FP. Built for resilience on a 1GB RAM VPS with zero-loss data integrity.

Engineering Moat: The "Ghost" Identity

To ensure uninterrupted data ingestion from the NSE portal without IP pressure or 403 blocks, I engineered a robust Identity Rotation & Session Warmup system:

@retry_with_backoff(retries=3, backoff_in_seconds=2)
async def _init_session(self):
    """Warms up the session with fresh cookies and identity rotation."""
    ua = random.choice(USER_AGENTS)
    headers = HEADERS.copy()
    headers["User-Agent"] = ua
    self.session._default_headers.update(headers)
    
    # 1. Visit Home Page (Baseline cookies)
    async with self.session.get(NSE_BASE_URL, timeout=30) as resp:
        await resp.text()
    await asyncio.sleep(random.uniform(1, 3))
    
    # 2. Visit Corporate Filings Page (Contextual Legitimacy)
    url = f"{NSE_BASE_URL}/companies-listing/corporate-filings-announcements"
    async with self.session.get(url, timeout=30) as resp:
        await resp.text()

This "Ghost" mode mimics a legitimate browser traversal, rotating User-Agents and refreshing cookie jars before every high-frequency crawl.

The Intelligence Engine: 22-Rule Determinism

Unlike generic LLM wrappers, Bulkbeat TV uses an Institutional Intelligence Prompt (v1.0) with 22 strict rules to evaluate market impact. This prevents "AI Halucinations" and FOMO-driven alerts.

22-Rule Scoring (Excerpt)

Rule 1: No-FOMO: Discard event if it has already occurred (past tense).
Rule 2: Forward-Looking Filter: Prioritize "Wins order", "Bags deal", "LOI received".
Rule 4: Crore Value Multiplier: Boost score based on deal size relative to company cap (SmallCap vs LargeCap).
Rule 14: Strict Exclusion: Automatic 0 score for routine AGM/Compliance filings.

This ensures that only signals with an Impact Score ≥ 8 reach premium subscribers, reducing notification fatigue by 90%.

Architecture & Scalability

1. SQLite Concurrency (WAL Mode)

To run a high-frequency ingestion engine alongside live Telegram bot inquiries and API webhooks on a single CPU core, I configured SQLite in Write-Ahead Logging (WAL) Mode with a 30-second busy-timeout. This allows simultaneous reads and writes, resolving database locks when crawler workers insert news items while the subscription manager is updating user statuses.

2. Multi-Job Scheduler Architecture

The application runs an APScheduler engine locked to the Asia/Kolkata (IST) timezone, running 7 distinct background cycles:

08:30 IST: Triggers the pre-market alert sweep and dispatch.
Every 3 minutes: Scrapes the corporate filings and exchange APIs.
Every 5 minutes: Verifies payment webhook logs with Razorpay.
16:00 IST: Triggers the daily EOD subscription decrements.
Sun 03:00 IST: Automatically syncs NSE market holidays for the upcoming year to keep calendar math accurate.

3. The "Hisab" Billing System

Unlike standard calendar billing, Bulkbeat TV utilizes a custom Market Day Billing Engine. Active users are debited 1 credit only on actual NSE trading sessions. On weekends, official holidays, or market halts, the billing job automatically pauses credits, ensuring users only pay for active market intelligence.

Quantitative Audit (FP Analysis)

Outcome & Impact

Accuracy: 99.2% signal reliability achieved via deterministic scoring.
Resilience: 99.9% ingestion uptime on a $5/mo VPS.
Latency: Sub-3 minute discovery-to-dispatch time for high-impact signals.

Visit Bulkbeat TV - Live NSE Alerts | Request Access via Telegram Bot