Building a Hybrid AI Decision Router in C#
Discover how developers can build a local C# routing architecture that combines cheap string similarity scoring with high-end models for uncertainty cases.
Introduction to Model Routing in .NET
Recent explorations in agent evaluations and entity resolution highlight a common dilemma: relying entirely on expensive models for every task is inefficient, yet cheaper models often feature unreliable confidence scores. A practical approach is to score all decisions using a fast, low-cost method, trust the model at extreme thresholds, and escalate ambiguous cases to a more capable system or human reviewer. A recent project implemented this hybrid routing pattern completely in .NET without requiring paid cloud APIs or Python sidecars.
The system utilizes a lightweight C# architecture featuring string-similarity features, logistic regression, and Platt scaling for probability calibration. When decisions fall into a defined uncertainty band, requests are passed to a local model running via Ollama, such as qwen2.5:14b, using structured outputs with an explicit unsure option.
Implementation Architecture
The implementation relies solely on the .NET base class library, keeping dependencies minimal. Similarity features like Jaccard distance, Jaro-Winkler, number overlap, and length ratios are extracted from product matching data (using the Abt-Buy benchmark). A simple logistic regression model handles initial scoring, while Platt scaling adjusts raw probabilities to address calibration issues.
To manage uncertain queries effectively, the Ollama integration enforces a structured JSON schema forcing the model to choose between match, no_match, or unsure. Introducing an explicit unsure category prevents the larger model from guessing on difficult items where errors are most costly. Unresolved or timed-out queries are automatically sent to a local human review queue.
What it means for developers
Hybrid routing architectures offer a compelling way to optimize resource usage by offloading routine tasks to fast, local algorithms and reserving powerful models only for complex, ambiguous cases. For developers seeking flexibility without maintaining complex local infrastructure, Apixo makes it simple to try top AI models cheaply through one API at https://apixoai.online. Implementing similar caching and evaluation harnesses locally also lets engineering teams test thresholds efficiently before deploying production workflows.
Source: The Hybrid Beats Both: Routing Only the Uncertain Decisions to a Bigger Model, in .NET — Towards AI. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
