RapidX AI — Autonomous Conversational Voice Agent & Dashboard

Overview

Official Repository: RapidX AI on GitHub

RapidX AI is an enterprise-grade Autonomous Outbound AI Calling Agent designed to automate sales outreach, qualification calls, and calendar bookings. Traditional robocalls are linear and robotic, leading to high hang-up rates. RapidX AI solves this by maintaining a natural, low-latency, WebRTC-based voice stream that can hold context-aware business discussions, handle objections, and book meetings on the fly.

The platform is divided into two core layers:

1. The Voice Agent Daemon (agent.py): A high-performance Python worker that runs in the background, waits for outbound jobs, and establishes low-latency WebRTC streams.
2. The Executive Telemetry Dashboard (ui_server.py): A 1,300+ line FastAPI administration console for triggering calls, adjusting LLM instructions/voices, and viewing call logs.

Technical Valuation: Replacement Cost: ₹28.5 Lakhs | Production-Ready SIP Trunking. Dual-layer rate-limiting and Sentry error monitoring.

Technical Architecture & Communication Flow

The platform handles voice-activated booking through a zero-latency WebRTC pipeline:

[UI Dashboard / API Trigger]
             │
             ▼
     [FastAPI Server] (Dispatches outbound call job)
             │
             ▼
      [LiveKit Worker] (Spawns agent session)
             │
      ┌──────┴──────┐
      ▼             ▼
[Vobiz SIP]    [WebRTC Stream] ◄──► [Sarvam/Deepgram STT] (Speech-to-Text)
(DID Trigger)  (Live Voice)               │
      │                                    ▼
      │                             [OpenAI GPT-4o] (LLM Brain)
      │                                    │
      │                                    ├─► [Cal.com Tool] (Voice Booking)
      │                                    │
      │                                    ▼
      └────────► [Target User] ◄── [Sarvam/OpenAI TTS] (Text-to-Speech)

1. WebRTC & SIP Trunk Integration

The agent establishes high-fidelity audio connections using LiveKit WebRTC. It bridges browser-based streams to traditional telecom carriers by configuring a custom SIP trunk via Vobiz DID gateways, executing clean transfers and handshakes.

2. Conversational Tool Calling (Voice-Activated Booking)

During the call, if the prospect agrees to book a meeting, the agent uses LLM tool-calling (calendar_tools.py) to hit Cal.com API endpoints, check slot availabilities in real-time, and book the appointment.

3. Rate-Limiting & Telemetry

To prevent abuse and spam compliance issues, I built a custom sliding-window rate limiter restricting outbound dispatches to a maximum of 5 calls per hour per number. Telemetry logs (durations, summaries, outcome statuses) are synced to Supabase and dispatched to administrators via a Telegram Alert Bot.

Technology Stack

• Speech Pipeline: LiveKit Python SDK + Silero VAD (Voice Activity Detection).
• Linguistic Models: OpenAI GPT-4o-mini / GPT-4o (for fast scenario reasoning).
• Voice synthesis: Sarvam TTS (hi-IN Hindi/English voices) & Deepgram STT (ultra-low latency transcription).
• Backend Console: FastAPI server + Supabase DB client + Nodemailer SMS notification endpoints.
• Error Telemetry: Sentry SDK integration with async middleware tracking.