Leash Bio Tackles Machine Learning Pitfalls in Drug Discovery with High-Throughput DELs

This benchmark also highlighted systematic biases within historical public datasets. For instance, in ChEMBL, an established public repository of bioactive molecules, models exhibited a "Clever Hans" effect. Rather than learning true chemical affinities, algorithms learned to predict the identity of the chemist who synthesized a molecule based on its chemical structure. Because a specific chemist's output correlates heavily with target activity, knowing "who made it" predicted experimental outcomes nearly as effectively as the underlying structural biochemistry.

To build models grounded in direct physical measurements, Leash trained Hermes, an in-house model relying solely on its internal datasets. Hermes matched the performance of open-source competitors like Boltz on public benchmarks, avoiding historical data memorization such as instances where Boltz had already encountered specific protein complexes (like BRD4) in the Protein Data Bank (PDB). When Leash detected Hermes memorizing protein "nicknames" instead of reading binding pockets, the team scaled up the production of engineered protein mutants to force models to learn authentic structural binding sites, occasionally spotting misfolded proteins through accidental Lumacaftor-like hits.

Turning Lab Data into an Agent Benchmark

Leash's data generation efforts have also led to Minos, an open dataset and simulation environment for scientific reinforcement learning. Minos turns years of laboratory observations into tens of thousands of lab-in-the-loop simulation steps, allowing large language models and autonomous agents to test their scientific reasoning.

Within this sandbox, agents evaluate structure-activity relationships (SAR) using natural language prompts. An agent is tasked with finding the most potent binders out of a candidate pool of 6.5 million small molecules, operating under a constrained budget of only 100 experimental selections. This turns biological discovery into an interactive optimization problem with verifiable outcomes.

What it means for developers

For engineers and researchers building machine learning models for the life sciences, Leash's findings highlight several practical takeaways:

• Beware of proxy signals in public data: Standard chemical benchmarks often contain latent human or institutional signatures. Models that perform well on static benchmarks may fail completely when tested on chemically novel libraries.
• Evaluation through physical perturbations: Overcoming memorization requires synthetic stress tests, such as protein pocket mutations, to ensure algorithms track molecular geometry rather than categorical target labels.
• The rise of agentic RL environments: Platforms like Minos show that high-throughput lab data can be converted into discrete environments for reinforcement learning. Developers working on automated reasoning agents can test iterative hypothesis generation against verifiable, ground-truth physical data.

As developer workflows shift toward agent frameworks capable of multi-step planning and tool-calling, benchmarking across multiple frontier models is essential. Developers building and evaluating agent pipelines can access leading language models affordably through a single API key at https://apixoai.online.

Ultimately, systematic data collection challenges the "Kosuri paradox"—the question of whether complex models are even needed if data collection can be scaled indefinitely. For Leash, coupling physical DEL data with computational screening has already yielded practical results, moving an internal inflammation program targeting cytokines from initial DEL hits to viable mouse PKPD data in six months.

Source: Lab-in-the-Loop at Compute Speed — Ian Quigley, Leash Bio — Latent Space. Written by the Apixo team from that report.