Google DeepMind

🔬 The Limits of AI in Science - Why We Need Self-Driving Labs — Joseph Krause, Radical AI

Jun 17, 2026 · 1:16:50

Joseph Krause of Radical AI argues that the bottleneck in materials science is experiments, not ideas, and his company's self-driving lab combines AI hypothesis generation with automated synthesis and characterization to produce alloys at unprecedented speed—1,200 in six months, with 300 novel compositions and 10 already in commercial development. Radical's closed-loop system runs research campaigns, not just automated tasks, overcoming challenges like sample manipulation at 3,000°C and tool vendors' software access. Krause details how their AI explores elemental families humans overlooked, why they open-source models like Matrix (the moat is experimental data, not models), and how they plan to compress discovery timelines from decades to 3–5 years for defense and space applications. He addresses the 10-year qualification process for aerospace, supply chain geopolitics (e.g., hafnium price up 10–15x due to Chinese dominance), and the need for public-private partnerships to accelerate U.S. R&D. Finally, he urges ML engineers to lean into their expertise rather than try to become material scientists.

Scaling Past Informal AI - Carina Hong, Axiom Math

Jun 3, 2026 · 1:33:04

Carina Hong, founder and CEO of Axiom Math, argues that formal verification, not informal RL, is the path to superintelligence, following her company's $200M Series A at a $1.6B valuation and a perfect 120/120 on the 2024 Putnam exam. Axiom's system uses Lean theorem prover data and reinforcement learning to produce verified proofs, achieving a 99% pass rate on the Verina code-with-proof benchmark (187 of 189 problems). Hong contends verification is about 'scaling brilliance' — not fixing hallucinations — and that only verified generation can compound AI reasoning. She explains Axiom's open-source Axle API for Lean at scale, addresses why frontier labs like OpenAI have deprioritized formal math (team departures, strategy shifts), and outlines a vision where verified reasoning transfers from math to code, hardware, and eventually AGI through self-improvement. She also discusses the Earth sciences challenge of autoformalization, the difficulty of search in mathematical literature (citing the Erdos controversy), and why fragmentation in the AI math field is a bottleneck.

Captaining IMO Gold, Deep Think, On-Policy RL, Feeling the AGI in Singapore — Yi Tay

Jan 23, 2026 · 1:32:05

Yi Tay, who leads Google DeepMind's Reasoning and AGI team in Singapore, explains how Gemini Deep Think achieved IMO Gold by abandoning symbolic AlphaProof for an end-to-end RL-trained model. He details the on-policy RL philosophy—models learn from their own generated outputs rather than imitating others—and the critical role of self-consistency through parallel sampling and internal verification. Tay describes the IMO effort: four co-captains in different time zones, a one-week training sprint, a live competition in Australia where researchers punched in problems as they were released, and the tension of waiting for human scores to determine the gold threshold. He discusses why the team believes one model must subsume everything for AGI, the data efficiency gap compared to humans, and his hiring focus on raw talent and research taste. Tay also shares his personal fitness transformation—losing 23 kilos and improving HRV—as integral to research productivity.

A Technical History of Generative Media

Sep 8, 2025 · 1:04:44

Fal.ai founders Gorkem and Batuhan detail their pivot from dbt pipelines to generative media inference, which now serves 2M developers and 350 models, crossed $100M ARR, and raised a $125M Series C. Key inflection points included Stable Diffusion 1.5 (company pivot), SDXL (first $1M revenue), Flux (jump from $2M to $10M monthly revenue), and Veo 3 (text-to-video with perfect lip-sync). They built a proprietary inference engine with 100+ custom kernels and a serverless GPU stack managing 10,000+ H100 equivalents across 6 cloud providers, typically delivering 1.5-10x speedups over stock PyTorch. Video models now drive 50% of revenue, up from 18% in February, fueled by open-source models like Hunyuan and partnerships with closed labs such as Play.ht and Google DeepMind. They argue advertising is the killer application for generative media and that image/video RL, specialized data pipelines, and cheaper conversational video models are underexplored startup opportunities.

The Magic of LLM Distillation — Rishabh Agarwal, Google DeepMind

Mar 23, 2025 · 46:42

Rishabh Agarwal from Google DeepMind discusses modern LLM distillation, arguing that while simple synthetic data distillation (generating outputs from a teacher and fine-tuning a student) gets 80-90% of the benefit, more advanced on-policy methods (sampling from the student and getting teacher logits) can close the remaining gap by fixing the train-test mismatch. He explains that logit-based distillation is more information-rich but often neglected due to infrastructure complexity, while online distillation is more expensive but crucial for long-horizon or agentic tasks. Agarwal highlights how flipping the KL divergence direction changes behavior (mode-covering vs. mode-seeking) and shows that smaller models can sometimes outperform larger ones in compute-matched distillation due to generating more filtered data. He also connects distillation to speculative decoding, where a distilled student speeds up the teacher, and encourages researchers to question current methods and mine RL literature for ideas.

Singapore: the AI Engineer Nation — with Minister Josephine Teo

Oct 19, 2024 · 56:40

Singapore Minister Josephine Teo outlines the city-state's refreshed national AI strategy and the Ministry of Digital Development's approach to balancing innovation, safety, and talent development. She details plans to triple AI practitioners from 5,000 to 15,000, the AI Verify testing tools and AI Safety Institute for governance, and internal government AI bots like AI Bot for RAG-based knowledge retrieval. On sovereign AI, she discusses hybrid cloud infrastructure and pragmatic data center expansion. Teo also explains Singapore's new law requiring transparency for AI-generated election content, emphasizing fact-based political discourse.

Answer.ai & AI Magic with Jeremy Howard

Aug 17, 2024 · 1:11:00

Jeremy Howard of Answer.AI argues that continuous pre-training should be treated as a continuum, not separate phases, and demonstrates how FSDP+QLoRA enables training a 70B model on just two NVIDIA 4090s. He reveals Answer’s non-hierarchical, manager-free R&D lab model that recruited unusual talent like Benjamin Warner and Ben Claviez, who independently launched BERT 24 to revive encoder-only architectures. Howard introduces FastHTML, a pure-Python web framework built on HTMX and Starlette, for creating modern SPAs without JavaScript. He previews “AI Magic,” a dialogue engineering system that moves beyond teletype-style chat interfaces and code editors, aiming to make AI-assisted development more interactive. The episode also covers Answer’s Public Benefit Corporation structure designed to resist hostile takeovers, and critiques decoder-only hype while advocating for encoder-decoder and state-space models.