How Retell AI Scales Voice Operations to 100M Calls Using Advanced LLM Benchmarks
Voice AI platform Retell AI handles over 100 million monthly calls by relying on rigorous model benchmarks and specialized LLMs for enterprise automation.
Voice AI platform Retell AI has expanded its monthly traffic from 30 million calls in early 2026 to more than 100 million by September, serving over 4,000 businesses. To maintain consistent response quality across complex corporate workflows, the company uses OpenAI models to power responses for 80% of its total call volume. This scale has pushed Retell past 1 trillion total tokens consumed across OpenAI endpoints, while helping client companies maintain resolution rates above 80%.
Evaluating Models on Real-World Performance
To ensure agents perform reliably during live interactions, Retell created its own vCX benchmark. This test suite evaluates models using multi-turn conversation datasets collected from real operational environments, scoring candidates on instruction following, tool execution, grounding, and language naturalness.
In Retell's October 13, 2026 benchmark, GPT-6 Astra took the top spot for instruction following with an 89.7% score. Earlier evaluations conducted prior to GPT-6 Astra's release showed GPT-5.5 leading rival architectures, including Gemini-3.1-Pro and Claude-4.6, in both non-hallucination rate (88.0%) and tool-call correctness (88.3%) when reasoning was enabled. Latency tests from the October benchmark revealed that GPT-5.6 Luna achieved the fastest median first-sentence latency at 497 milliseconds, compared to 883 milliseconds for both GPT-6 Astra and GPT-6 Sol.
"We choose models the way our customers choose vendors: on real calls, not demos. OpenAI keeps earning its spot," said Bing Wu, Co-founder and CEO of Retell AI.
Real-World Enterprise Deployments
Retell's voice harness manages live call state, processing incoming audio and orchestrating when an agent speaks, listens, or handles interruptions. During live interactions, models such as GPT-6.1 Sol, GPT-6 Astra, and GPT-5.5 analyze context and trigger internal tools or write back to corporate systems. If human intervention is needed, the system transfers the caller after giving the representative full context and confirming they are prepared.
After calls conclude, GPT-5.6 Terra processes transcripts to record outcomes and extract structured data. This architecture has driven high success metrics in client deployments:
- Anker: Uses agents for Tier-1 support across the US and UK to track orders, check warranties, and troubleshoot products, achieving an 80.4% resolution rate and a Net Promoter Score of 63.
- Pine Park Health: Employs agents for proactive scheduling and appointment reminders, saving the equivalent of 2.1 full-time medical assistant roles and reaching a 55.7% booking rate on contacted patients.
To handle traffic surges—which can reach 50 times overnight levels during peak daytime hours—Retell relies on dedicated server infrastructure and regional API endpoints. This setup minimizes latency while meeting data residency obligations in regulated sectors like healthcare and finance.
What it means for developers
Retell's operational choices offer concrete lessons for developers building real-time AI agents. Internal productivity plays a major role in managing scale: Retell's engineering team uses Codex to explore unfamiliar code, build features, and debug log files against live database records. They also use GPT-6 Astra's computer use capabilities to automatically test web interface pull requests before deployment. This automated workflow allows Retell to generate $1.6 million in annual revenue run rate per employee without expanding engineering headcounts at the same pace as business growth.
As Bing Wu noted, "By using Codex, the engineering team has a virtual teammate that can pair with them on any kind of difficult task."
For teams designing their own voice or text workflows, selecting the right model requires testing against domain-specific multi-turn interactions rather than relying on standard static evaluations. Developers looking to benchmark and implement multi-provider setups can try top AI models cheaply through one API at https://apixoai.online. Retell's multi-layered setup—combining reliable telephony backups with targeted model benchmarking—demonstrates that high resolution rates depend as much on practical evaluation frameworks as they do on raw model performance.
Source: Retell uses OpenAI models to help power 100M monthly calls — OpenAI News. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
