Microsoft and Hugging Face Release ThinkingBox-Training for AI Agents
Microsoft and Hugging Face have released ThinkingBox-Training on GitHub, using database states as reinforcement learning rewards to improve AI agent dependability.
Microsoft and Hugging Face have released ThinkingBox-Training on GitHub, a framework that converts database states into reinforcement learning rewards for post-training AI agents. This follows their previous work on ThinkingBox-Bench, a benchmark designed to catch agents that claim tasks are complete while leaving backend records incorrect. By transforming executable checks of terminal database states into direct reward signals, the research team successfully fine-tuned the Qwen3.8-27B model and significantly boosted its operational reliability.
Moving the Dependability Column
In the original evaluation of ThinkingBox-Bench, the researchers noted a gap between headline success rates and true consistency. Models like Kimi-K3 could solve a high percentage of tasks at least once (pass@20), but failed to complete tasks consistently across all 20 recorded attempts. Conversely, models with high headline accuracy did not necessarily guarantee better dependability for production database workflows.
To address this, the team used an offline policy-relative difficulty approach to curate a training pool from tasks outside the benchmark. By probing candidate tasks with the base policy and analyzing success probabilities using Jeffreys posterior probability, they filtered out tasks that were either too hopeless or already mastered. This left a balanced training pool that provided the contrasting outcomes required by Group Relative Policy Optimization (GRPO).
After 50 steps of GRPO fine-tuning using 24 H100 GPUs, the Qwen3.8-27B model demonstrated a sharp increase in repeatable success. Its observed 20/20 performance—representing tasks completed correctly on all twenty attempts—rose from 38 tasks (7.50%) to 107 tasks (21.10%), nearly tripling its consistency. Average success (pass@1) climbed from 51.70% to 60.78%, while estimated costs per dependable task dropped by 67.2% from $24.36 down to $8.00.
What it means for developers
For software engineers building production applications that interact with backend databases, headline accuracy metrics can be deceiving. As the ThinkingBox benchmark shows, high pass@1 scores do not automatically translate to reliable, repeatable task execution. Developers can easily test top AI models cheaply through one API at https://apixoai.online to evaluate how different systems handle multi-step tool calls and state changes.
The open-source release on GitHub provides the data-selection, rollout, reward, and Verl integration code needed to implement this training loop. By turning terminal database states into binary rewards, engineering teams can fine-tune open weights models to prioritize consistent side effects over superficial task completion, reducing unexpected errors in production environments.
Implementation and Training Stack
The released training stack requires Linux or WSL with Python 3.12, uv, NVIDIA GPUs with a compatible CUDA stack, a local Qwen3.8-27B checkpoint, and OpenAI-compatible endpoints for simulated users and judges. The validated reference setup utilizes a three-node configuration with eight H100 GPUs per node. The repository provides detailed guidelines for data hydration, policy probes, benchmark exclusion, and running the evaluation pipeline against the 507-task benchmark.
Source: Turning Database Grading into RL Rewards: Post-Training Agents with ThinkingBox — Hugging Face blog. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
