Neuronpedia

Goodfire AI’s Bet: Interpretability as the Next Frontier of Model Design — Myra Deng & Mark Bissell

Feb 5, 2026 · 1:08:41

Goodfire AI's Mark Bissell and Myra Deng argue that interpretability is the next frontier for model design, using their recent $150M Series B at $1.25B valuation to scale surgical edits of model internals beyond post-hoc poking. They explain how their platform detects behaviors like sycophancy and reward hacking, enabling targeted unlearning without wrecking capabilities. The episode covers real-world deployments from Rakuten's PII guardrails to life science partnerships with Mayo Clinic finding Alzheimer's biomarkers. Mark demonstrates real-time steering of a trillion-parameter Kimi K2 model, while Myra details how SAEs sometimes underperform probes for detection tasks. They envision a future where interpretability guides training so customization isn't brute-force guesswork.

The Utility of Interpretability — Emmanuel Amiesen

Jun 6, 2025 · 1:53:02

Emmanuel Amiesen, lead author of Anthropic's Circuit Tracing paper, presents the newly open-sourced tools that let anyone inspect how language models compute, revealing that models like Gemma 2-2B engage in genuine multi-step reasoning rather than mere pattern matching. Amiesen demonstrates that the model's internal features causally drive outputs—for instance, suppressing the 'Texas' feature in a capital-of-state query changes the answer from Austin to Sacramento. He explains how sparse autoencoders uncover features shared across languages and modalities, and how attribution graphs map feature interactions. The episode covers practical insights: why jailbreaks succeed (the model prioritizes grammatical completion over refusal) and how planning in poetry generation involves deciding a rhyme before the line begins. Amiesen calls for more researchers to explore interpretability, citing the low barrier to entry with open models and tools.