OpenAI and Anthropic Outline Safety Architectures for Autonomous Financial Agents

Anthropic echoed this paradigm while discussing its recently launched Claude for Financial Advisors. Belinda Neal, Managing Director of Financial Services at Anthropic, noted that when a model crosses capability thresholds, fundamental working patterns shift. To manage that shift, Keon Shahab of Anthropic's applied AI team detailed the creation of a three-step "trust layer" for agents:

1. Establishing identity and permission layers to define who the agent acts as.
2. Implementing sandboxing and an approval layer to ensure human sign-off before actions execute.
3. Verifying that agent workflows remain stable when operational variables change over time.

Shahab illustrated this separation between proposing and acting with a trade break investigation agent. Rather than allowing the agent to execute modifications independently, the system is designed strictly to formulate and suggest order amendments for human review.

Deterministic Controls and Mitigating Approval Fatigue

While foundation model providers build foundational trust layers, institutional firms are engineering explicit architectural controls to govern agent behavior. John Feminella, SVP Architecture at hedge fund Two Sigma, warned against the dangers of "approval fatigue." Requiring humans to greenlight every single agent action creates bottlenecks and introduces risk when reviewers lose focus due to volume.

To counter this, Two Sigma uses composable controls—modular guardrails that each perform a single safety function. Among these are auto-mode classifiers that handle routine operations automatically while routing high-impact or ambiguous cases to human specialists. For straightforward scenarios, deterministic rules govern execution. As Feminella summarized, safety mechanisms should function like vehicle brakes: their role is not merely to halt progress, but to allow systems to move quickly while retaining the ability to stop safely.

Andrey Rybka, Head of Architecture & Emerging Technologies in Bloomberg's Office of the CTO, shared real-world failure modes to illustrate why deterministic controls outside the model are essential. In one cited incident, an agent interpreted instructions inside a marketplace product review as an actual operational directive to stock inventory.

To prevent such vulnerabilities, Rybka outlined a defense-in-depth approach: input validation, defenses against prompt injection, provenance tracking, tool call restrictions, scoped credentials, outbound traffic filtering, strict budgets, and kill switches. His architectural directive is straightforward: monitor and log every call deterministically outside the model and always maintain a kill switch for high-value actions.

What it means for developers

For engineers building agentic workflows, the lessons from AIE New York mark a shift away from relying solely on system prompts for safety. As model capabilities expand, agent architectures must treat reliability and governance as external, deterministic software problems.

Key takeaways for application development include:

• Separate proposal from execution: In sensitive domains, design agents that prepare diffs, proposed trades, or draft modifications rather than executing tool calls directly.
• Prevent reviewer fatigue with classifiers: Do not force human sign-off on trivial steps. Deploy deterministic rules for clear-cut operations, automated classifiers for standard tasks, and reserve human review for ambiguous or consequential actions.
• Isolate the execution environment: Implement outbound network rules, strictly scoped credentials, tool execution limits, and external kill switches to counter prompt injection and unintended behavior.
• Maintain data provenance: Ensure that any number or insight delivered by an agent can be traced back to its raw source data.

As you test different models for tool calling, classification, and reasoning fidelity across your agent pipeline, developers can try top AI models cheaply through one API at https://apixoai.online to find the right balance between cost, latency, and reliability.

Source: Crossing the Threshold: How OpenAI and Anthropic Approach Autonomy and Safety in Financial Agents — Latent Space. Written by the Apixo team from that report.