Five Structural Rules for Engineering Reliable LLM Agent Loops
Dragos Bilaniuc outlines five architectural practices learned from building Kept, an open-source e-commerce support agent, to eliminate runtime failures in production agent loops.
When autonomous agents move from experimentation to handling critical operational tasks like processing refunds and returns, runtime failures carry immediate financial consequences. Dragos Bilaniuc addressed this challenge while developing Kept, an open-source post-order support agent designed for e-commerce platforms. Rather than relying on simple iterative prompts, Bilaniuc outlined five architectural ground rules to embed reliability directly into the core agent loop.
Architecting the loop with state machines and hosts
At its simplest, an agent runs in a continuous loop: it sends messages to a large language model, executes requested tool calls, returns the outputs to the model, and repeats until reaching a final response. Bilaniuc initiated the design of Kept by defining a formal finite state machine before writing application code. By identifying every possible event—such as transport errors, settlement of tool calls, and distinct model responses—against execution states like awaiting_model and executing_tools, the system maps all potential edge cases into explicit terminal outcomes like replied, conversation_full, or specific failure categories.
A fundamental structural choice was decoupling the loop runner from its execution environment, known as the host. In Kept, the host sits alongside the HTTP handler to guarantee prerequisites that the loop takes for granted. It enforces conversation locks to prevent concurrent interactions, validates configuration parameters at boot, tracks identity, commits turn histories, and wraps responses into network envelopes. This separation keeps the loop runner focused entirely on processing state transitions. It also streamlines evaluation pipelines, where an evaluation runner functions as an alternate host with deterministic dependencies, mock clocks, and in-memory trace collectors.
Treating errors as outcomes and keeping history atomic
Standard software architectures often allow unhandled exceptions to crash execution frames, but doing so inside an agent loop can truncate traces and leave conversation histories corrupted. Bilaniuc advocates converting operational errors into structured outcomes. For instance, context window overflow exceptions—often returned as HTTP 400 errors by providers like Anthropic—are converted into a conversation_full terminal state. This allows the host to close the dialogue cleanly without triggering cascading failures on subsequent messages.
To preserve visibility, error taxonomies must remain strictly defined rather than dumping anomalies into generic error buckets. During Kept's implementation, an ambiguous unknown status was split into unknown_stop_reason and empty_reply to provide clear feedback during automated evaluations. Tracing layers also serve as active guards: unclosed spans are configured to throw errors during development and are flagged with an undetermined status in production.
Maintaining data integrity also requires atomic history updates. Network drops or tokens cut short mid-generation can easily poison conversation context if half-formed interactions are retained. Kept enforces an all-or-nothing model: failed turns leave the persisted conversation history completely untouched, preserving aborted rounds exclusively within diagnostic traces. To prevent irreversible side effects, round-limit checks are enforced prior to running tool calls rather than after they execute.
Rigorous three-way assertions and provider contracts
Validating an agent loop requires looking beyond the textual response delivered to an end user. Kept implements a three-part testing strategy for each state transition, verifying the final response, the persisted state of the conversation history, and the exact structure of the telemetry trace.
Because external model APIs frequently evolve, Bilaniuc created a provider-agnostic interface backed by a unified contract suite. This suite exercises all potential completion outcomes across providers—including parallel tool execution, refusals, timeouts, and network drops. During Kept’s development, this contract suite detected two provider drift issues before they could affect live conversations.
What it means for developers
As development teams increasingly rely on coding assistants to generate agent logic, engineering responsibilities shift toward defining system boundaries and managing failure modes. Bilaniuc’s implementation demonstrates that production-grade reliability requires treating agent loops as rigorous state machines rather than open-ended prompt chains.
When standardizing multi-provider agent infrastructure, running shared contract suites is an effective way to protect against silent provider drift and unexpected SDK changes. Developers testing and benchmarking different provider implementations can try top AI models cheaply through one API at https://apixoai.online. By implementing atomic history commits, structured terminal outcomes, and deep tracing assertions, engineering teams can build resilient agent workflows capable of safely managing sensitive production actions.
Source: Reliability at the Core: Five Ground Rules for Building an Agent Loop — Towards AI. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
