Goodfire Challenges Polysemanticity as Meta and NVIDIA Release Large Open Models
Goodfire presents evidence that neural networks are far clearer than assumed, alongside major open-weight releases like Meta's Muse Spark Open and NVIDIA's Nemotron 3.5.
A fresh thesis on neural network interpretability is sparking debate among AI researchers, while tech giants Meta, NVIDIA, and Microsoft have rolled out powerful new open-weight models. According to new research from Goodfire, neural architectures may be far clearer than previously assumed, potentially simplifying model alignment and inspection. At the same time, open-weight capabilities continue to climb with Meta releasing its massive Muse Spark Open architecture and NVIDIA introducing Nemotron 3.5 Super VL.
Rethinking neural network polysemanticity
For years, mechanistic interpretability researchers operated under the assumption that individual neurons in large language models perform multiple overlapping duties—a phenomenon known as polysemanticity or superposition. Nick Cammarata, co-founder at Goodfire and former OpenAI researcher, published a new working essay arguing that neural networks are largely structured in a clean, legible manner. Cammarata contends that many instances of tangled weights resolve under agentic analysis, asserting that 90% of neurons in models like Qwen-32B perform a single identifiable task.
This claim suggests that researchers can analyze neurons directly without needing secondary models to decipher internal activations. Prominent industry figures, including investor Deedy Das, characterized the finding as a major step forward for mechanistic interpretability and AI safety, arguing it could shift alignment into a standard scientific discipline. However, the proposal faces scrutiny. Researcher Neel Nanda noted that the thesis does not resolve fundamental questions regarding representational superposition or weak activations carrying memorized information, while Leo Gao emphasized that explanations must be evaluated through rigorous neuron simulation metrics like Kullback-Leibler divergence.
Heavyweight open model releases
Parallel to interpretability advances, model developers released several major open-weight architectures. Meta introduced Muse Spark Open, an 816-billion total parameter (42 billion active) Mixture-of-Experts (MoE) model featuring a 1-million token context window. Built with 62 MoE layers and accepting text, vision, and audio inputs, Muse Spark Open achieved a score of 39 on the Artificial Analysis Intelligence Index, making it the top-ranked US open-weight model despite falling behind international entries like GLM-5.3 and Kimi K3. The model's weights require about 1.6TB in BF16, though quantized NVFP4 checkpoints bring memory demands down to 530GB, enabling execution on setup configurations such as two 512GB M5 Ultra Mac Studios or eight NVIDIA H200 GPUs. Meta released the model under a custom license requiring commercial entities generating over $20 million in annual revenue to sign a separate agreement.
NVIDIA also launched Nemotron 3.5 Super VL, a 120-billion parameter (12-billion active) hybrid Mamba-Transformer MoE vision-language model. NVIDIA reports that the model reaches 51.78% on SWE-bench Pro after undergoing a 1-trillion token distillation from a larger teacher model and specialized agentic software engineering mid-training. Meanwhile, Microsoft announced MAI-Base-2, a 3-trillion parameter pre-trained base model with 80 billion active parameters, claiming it matches DeepSeek-V4-Pro-Base performance on code while using 12 times less training compute.
What it means for developers
These updates give engineering teams broader options across model inspectability, self-hosted infrastructure, and specialized API tasks. While Meta's Muse Spark Open pushes open-weight multi-modal context limits to 1 million tokens, its weakest performance lies in agentic benchmarks, where models like GLM-5.3 maintain an edge. Furthermore, hardware demands for running nearly 800 billion parameters remain significant, favoring quantized formats or hosted inference endpoints priced at $1.25 per million input tokens and $4.25 per million output tokens on platforms like Together or Fireworks.
For developers assessing these diverse models—from closed systems like Anthropic's Opus 5.5 fast mode or Gemini 4 Argon to open-weight giants like Muse Spark Open—comparing token efficiency and output quality across multiple platforms is essential. Developers can test top AI models cheaply through one API at https://apixoai.online to determine which model fits their specific context window, latency, and cost constraints.
As safety researchers examine GLM-5.3's unprompted reasoning about its own training data and agent evaluation studies highlight unpredicted behaviors in autonomous execution, fine-tuning and strict system guardrails remain vital. The combination of direct neuron interpretability methods and more transparent open-weight foundations offers developers clearer visibility into model behavior as autonomous agent deployments expand.
Source: [AINews] Agents disprove Polysemanticity; greatly improve Interpretability & Alignment — Latent Space. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
