Fine-Tuning Laya on Free GPUs Hits 99.2% Accuracy in Institution Name Disambiguation
Researcher Jeet Mukherjee fine-tuned ConvAI’s lightweight decision model Laya on Kaggle's free GPUs, achieving 99.2% accuracy in resolving complex academic institution names.
Resolving messy, real-world text into standardized database entries has long been a quiet bottleneck in academic data science. In bibliometrics, the challenge is known as Institution Name Disambiguation (IND): determining whether variants like "IIT Delhi," "IITD," "Indian Institute of Technology Delhi," and "Dept. of CSE, IIT, Delhi, India" point to the identical campus. When multiplied across tens of millions of papers, inconsistent naming conventions quietly undermine analytics on research output and retraction tracking. While developing PostPub—a platform built with IRW founder Dr. Achal Agrawal to study Crossref’s Retraction Watch dataset—researcher Jeet Mukherjee demonstrated that fine-tuning a small, specialized decision model can outperform general-purpose alternatives while running on free infrastructure.
Reframing disambiguation as a fast pairwise judgment
Earlier efforts by Agrawal and Mukherjee explored unsupervised clustering using author-institution co-occurrence (documented in arXiv:2510.16407). While functional, that pipeline relied heavily on complete co-occurrence graphs and proved fragile against slight token shifts. Traditional string similarity also breaks down when entities share zero lexical overlap—such as colloquial abbreviations like "NIT Trichy" versus "National Institute of Technology, Tiruchirappalli"—or when distinct universities share identically named computer science departments.
Treating the problem as standard classification against comprehensive registries like the Research Organization Registry (ROR), which spans roughly 110,000 entities, is equally problematic. Large generative language models struggle to scale economically for continuous zero-shot labeling, and choice-based classifiers degrade when faced with thousands of potential classes.
Mukherjee addressed this by reframing the challenge as pairwise verification using Laya, an open-weight "System 1" decision model released by ConvAI. Designed for rapid, calibrated judgments in milliseconds rather than free-text token generation, Laya uses structured primitives including "choice," "score," and "noul" (a direct binary yes/no query returning a calibrated probability). Instead of querying an entire institutional registry at once, the task asks a single calibrated question: do string A and string B refer to the exact same organization?
Fine-tuning on free hardware with high precision
To train the system, Mukherjee constructed a dataset from raw affiliation strings resolved through Dimension's institutional resolution API, yielding 96,953 unique raw strings across 10,757 canonical entities in 163 countries. Independent verification of a random 900-institution sample using Claude confirmed a 97.2% label accuracy rate. The data was organized into strict, entity-stratified splits to ensure no institution variants crossed between train, validation, and test sets.
Training was conducted entirely within Kaggle’s free tier on two Nvidia T4 GPUs, taking 51.6 minutes (3,095 seconds) across four epochs. Using Laya's training protocol, all model parameters were updated using an RLCD/GRPO-style policy-gradient objective combined with soft cross-entropy guidance and discriminative learning rates (2.5e-5 for the encoder, 1.0e-4 for the decision head). Validation accuracy surpassed 99.07% after just the first epoch.
Evaluated on a held-out test set of 3,750 pairs spanning 1,415 previously unseen institutions, the fine-tuned model achieved 99.2% accuracy. Precision remained exceptionally high, with nearly all mistakes confined to 21 false negatives where severe colloquial shortcuts or sub-lab names lacked textual links to parent organizations. In comparative benchmarks against a 129-pair test set, the fine-tuned verifier demonstrated strong performance against alternatives:
- Zero-shot GPT-4o-mini: Produced high precision but suffered from poor recall, missing more than half of valid matches.
- ROR Affiliation API: Maintained strong precision but managed only 72.1% recall.
- Classical string baselines (RapidFuzz) and embeddings (all-MiniLM-L6-v2): Captured higher recall but suffered frequent false positives due to generic lexical similarities.
What it means for developers
This implementation highlights how reframing complex entity resolution into binary pairwise verifications can simplify high-throughput data pipelines. Developers do not necessarily need heavyweight LLM generation or costly hosted infrastructure to resolve ambiguous records at scale.
While developers can explore top AI models cheaply through one API at https://apixoai.online when building broad reasoning workflows or running candidate discovery, downstream verification tasks often benefit from small, highly calibrated models running locally or on minimal compute.
Architecturally, Mukherjee notes that this model is not designed to replace initial retrieval systems. Instead, it serves as an ultra-fast, single-digit millisecond reranker that sits directly behind standard vector searches or fuzzy retrieval layers, filtering out false positives before records enter production databases. Code and notebooks from the project have been made publicly available on GitHub.
Source: How I Fine-Tuned Laya to Solve an Important Problem in Scientometrics — With Claude Code — Towards AI. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
