Auto Call Sales & Service Agent System
Hardware + Software · ESP32 HFP→SIP Upgrades Any Phone to a SIP Terminal for ~20 CNY
Turn "sales/service calls" into an autonomous voice agent: on the front end, an ESP32 add-on takes over any phone's call audio via Bluetooth using the HFP protocol with zero friction; on the back end, it pushes via SIP/WebRTC to the Rust-based active-call voice engine, where an LLM drives real-time dialogue, sales conversion, and auto-answer. Hardware BOM under 20 CNY, with offline ASR/TTS and cloud dual engines — privacy-compliant and plug-and-play.
Core Modules
Sales/Service Call Agent · Real-Time Dialogue UI
Hardware · Software · Orchestration: Three-Layer Synergy
ESP32 HFP Bluetooth gateway + active-call Rust voice engine + Playbook Markdown orchestration, upgrading any phone into an AI call terminal.
ESP32 HFP → SIP Bluetooth Gateway
The ESP32 module masquerades as a Bluetooth headset, capturing bidirectional call audio via HFP (Hands-Free Profile), then pushes to a SIP server over Wi-Fi. No root/jailbreak, no app install; covers Huawei / Xiaomi / OPPO / vivo / iPhone. Hardware BOM ~15-25 CNY, about the size of a Bluetooth dongle, clippable to a phone case or pocket.
active-call Voice Engine (Rust)
A Rust-based voice-agent engine (miuda-ai/active-call, MIT). Three audio entry points (SIP UDP/TCP/TLS/SIPS, WebRTC SRTP, WebSocket), dual dialogue engines: classic VAD→ASR→LLM→TTS pipeline + OpenAI/Azure Realtime full-duplex streaming. Built-in nnnoiseless denoise, WebRTC AGC2, barge-in, ambient mixing.
Playbook Orchestration (Markdown)
Define persona / scene / flow in Markdown files, with Jinja2 templates, SIP-header extraction, DTMF collection, HTTP tool calls, variables, and human handoff. E.g. "after-sales scene" calls the CRM API, "order scene" uses <collect> for digits, "busy scene" uses <refer to="sip:agent@..."> to hand off to a human. Execution engine decoupled from the dialogue brain — strategy swappable wholesale.
Offline AI · Privacy First
Optionally enable SenseVoice (zh/en/ja/ko/yue ASR) + Supertonic (en/ko/es/pt/fr TTS) offline models — no cloud API calls at all. Docker one-click model download (hf-mirror.com in mainland). Suited to finance / healthcare / government strong-compliance scenarios, and as a cloud-failure fallback.
Industry Pain Points
The harness industry has no shortage of CAD tools — what it lacks is a system that truly reads drawings, extracts BOMs, validates connections, and generates board diagrams.
OEMs Block Call Recording
Most domestic Android brands (Huawei, Xiaomi, OPPO, vivo) restrict third-party call recording at the system level for privacy compliance; users need root or flashing to bypass.
iOS Fully Closed
Apple iOS does not expose call-recording APIs to third-party apps; iPhone users can hardly record calls, losing key business info.
Complex Permissions
Even where supported, granting call-log, mic, notification permissions is a high bar for ordinary users; low adoption.
Severe Info Loss
Sales reps forget ~30% of key call info daily (budget, promised delivery, etc.); memory or handwritten notes error-prone, costly to backfill.
Inefficient Manual Organizing
Manual post-call organizing averages 5-10 min; reps with 10+ calls/day waste large amounts of time, with uneven quality.
Core Capabilities
Hardware Bluetooth gateway · Rust voice engine · Markdown orchestration · dual-engine dialogue
Multi-Protocol Audio Gateway
SIP (UDP/TCP/TLS/SIPS/WebSocket) + WebRTC (SRTP) + raw PCM WebSocket, three entry points unified; register to FreeSWITCH / Asterisk / RustPBX, or go PSTN via Twilio / Telnyx elastic relay.
Dual-Engine Dialogue
Classic pipeline (VAD→ASR→LLM→TTS) supports OpenAI/Alibaba/Azure/Tencent/Deepgram; OpenAI/Azure Realtime API gives full-duplex streaming, sub-1s first response for latency-sensitive service.
Playbook Orchestration
A Markdown file defines a stateful voice bot: scene switching, <http> tool calls, <collect> digit capture, <set_var> staging, <refer> handoff, <hangup_headers> SIP BYE writeback. Zero-code business integration.
High-Perf Media Core
TinySilero VAD: ~60ms for 60s audio (RTF 0.0010), 2.5× faster than ONNX Silero; codecs PCM16 / G.711 PCMU/PCMA / G.722 / Opus; built-in AGC2 + denoise + ambient mix + smart barge-in.
Full-Link Observability
tracing-based structured logs + /list, /events/{id} (SSE), /api/records call-event streams; TIMELINE multi-layer Gantt (USER/ASR/LLM/BOT) shows each call's event timing and latency live, aiding QA and optimization.
Offline/Cloud Dual Deploy
The default offline feature (SenseVoice + Supertonic) runs the full chain without cloud APIs; keep cloud-vendor switching for compliance, cost, latency flexibility. Docker one-click deploy + graceful shutdown.
Full Lifecycle of One Call
From phone dialing to cloud AI dialogue · 10-step closed loop
Key Results
Crystallize technical capability into quantifiable, reusable outcomes.
Key Breakthroughs
The difficulty is not a single-point breakthrough, but engineering the breakthroughs into something stable and reusable.
Zero-Friction Hardware
A 20-CNY ESP32 module turns any phone (incl. iPhone) into a SIP terminal, bypassing OEM and OS permission walls — no comparable same-price solution exists.
Execution Decoupled from Brain
PlaybookRunner only dispatches events and executes commands; DialogueHandler is the strategy implementation, swappable wholesale for rules/NLP/custom LLM without touching core call logic.
Playbook Markdown Orchestration
<http> tool calls / <collect> DTMF / <set_var> variables / <refer> handoff / <hangup_headers> business writeback — zero-code integration with CRM, ticketing, knowledge base.
Offline ASR/TTS Fully Local
SenseVoice + Supertonic fully local inference, no cloud API; with Docker one-click deploy, meets finance / healthcare / government strong-compliance needs.
Business Value
Convert technical capability into quantifiable, reusable business value.
Content Under Active Update
Product content has been published based on internal materials. The following areas are planned for further development:
Other Products & Solutions
Explore Xianma AI solutions in other domains