Handshake AI Reaches $1B Revenue Run Rate Supplying Expert Training Data to Frontier Labs
Handshake AI reached $1 billion in gross annualized revenue within a year by building specialized evaluation and reinforcement learning environments for frontier AI labs.
In roughly one year from its launch, Handshake AI’s training data business reached $1 billion in gross annualized revenue. The surge highlights an evolving priority for frontier AI labs: purchasing specialized domain data and complex evaluation environments rather than relying purely on general text dataset collections. On a recent episode of the Latent Space podcast, Handshake CEO Garrett Lord, alongside team members Sahil Bhaiwala, Nishant Ranka, Nick Magazine, Jonas Mueller, Collin Schweiker, and Steve Kelling, detailed what frontier labs are buying and where models still lag behind human experts.
Handshake's business spans several technical domains, including finance, life sciences, alignment, and software engineering. Rather than assessing models using standard multiple-choice questions, the company builds multi-step reinforcement learning environments that test how AI agents execute workflows, interact with enterprise software tools, and respond to domain-specific edge cases.
Inside Handshake’s Specialized Domain Benchmarks
To evaluate models on financial tasks, Handshake developed benchmarks such as ATLAS-Finance. In simulated corporate banking environments—demonstrated through case studies like Project Falcon and Project Osprey—agents must navigate a bank's inbox, examine deal documents, and construct financial models in Excel.
Collin Schweiker and Sahil Bhaiwala demonstrated how an agent processes complex instructions within these environments. In one scenario, an agent located a client email mention of existing goodwill, passed the figure into Python, and subtracted it correctly. However, while the model could explain the underlying tax treatment when prompted directly, it failed to recognize the exact line of code where the financial rule ought to be applied. Handshake’s financial evaluation setup uses specialized tools, including an Excel grader equipped with penalty traps, dependency gates, and partial credit, to prevent models from gaming the evaluation.
In the life sciences domain, Nick Magazine and Jonas Mueller introduced VIALS, a benchmark designed to evaluate how vision-language models interpret scientific imagery, such as molecular structures, plasmid maps, and gel bands. During a brain-tissue analysis demonstration involving nine slides of brain-tumor tissue, human expert Nick Magazine identified slides H and I as showing the most advanced vessel necrosis. The evaluated model selected slides H and I, but incorrectly included slide G as well. To improve model performance on these tasks, Handshake trains vision models using intermediate image descriptions before testing their visual reasoning skills.
Evaluating Tool Use, Safety, and Execution Traces
Evaluating modern AI agents requires looking beyond final written answers. In coding and system interaction tasks, Steve Kelling and Nishant Ranka pointed out that inspecting an agent’s final text output misses crucial details. Instead, Handshake monitors complete execution traces, checking which files an agent accessed, whether it verified file copies, and what commands remained in its shell history after completing a task.
A similar methodology applies to alignment and safety testing through benchmarks like CAREBench. Safety evaluations examine how models behave when handling delicate scenarios, such as loan applications subject to fair lending regulations or requests involving file handling errors. Rather than relying solely on simple refusals, Handshake measures tool call sequences to observe how safety feedback influences model training and how agents manage enterprise privacy boundaries.
What it means for developers
The rapid growth of Handshake’s data operation reflects a wider shift toward evaluating AI models on real-world execution rather than isolated static tests. As frontier labs use these specialized environments to train models on multi-step financial logic, scientific image interpretation, and rigorous tool use, developers gain access to models that are increasingly capable of handling complex work functions.
For software engineers building agentic workflows, these updates emphasize the importance of monitoring step-by-step tool calls, shell outputs, and intermediary data transformations rather than relying strictly on an LLM's final text summary. Testing these complex behaviors across multiple providers is essential for finding the right balance of cost and performance. Developers looking to build and test applications across various model families can try top AI models cheaply through one API at https://apixoai.online.
As enterprise customers demand domain-specific accuracy, private benchmarks and fine-grained grading frameworks are becoming standard components of AI development. The ability of models to follow domain rules within active environments—rather than merely reciting knowledge—will determine their readiness for deployment in production settings.
Source: What Frontier Labs Buy: A Data Deep Dive with Handshake AI — Garrett Lord & Team — Latent Space. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
