DeepSeek

⚡️ Competing with ChatGPT and Sierra, building a $10M ARR company — Yasser Elsaid, Founder, Chatbase

May 2, 2026 · 1:00:26

Yasser Elsaid, founder of bootstrapped AI chatbot company Chatbase, discusses how he grew from a side project to $10M ARR in three years, reaching $1M in just 117 days. He explains why he never raised VC, relying instead on product-led growth, self-serve signups, and a content-driven go-to-market strategy. Elsaid shares specific tactics like warm outbound to signed-up users, leveraging LinkedIn virality in early AI days, and now moving toward outcome-based pricing for enterprise customers. He details Chatbase's evolution from a simple RAG chatbot into a 'chief customer officer' that handles support, sales, and onboarding while surfacing business insights from conversations. The episode also covers his transition from Toronto to San Francisco, his hiring philosophy favoring results-oriented engineers, and his daily use of both Claude Code and Codex for development.

Marc Andreessen introspects on Death of the Browser, Pi + OpenClaw, and Why "This Time Is Different"

Apr 3, 2026 · 1:16:20

Marc Andreessen argues that AI is finally different from past boom-bust cycles because of four compounding breakthroughs: LLMs, reasoning (o1, R1), coding agents (OpenClaw), and recursive self-improvement. He calls this the '80-year overnight success'—decades of neural network research now paying off. Comparing today's AI capex boom to the dot-com crash, he notes that buyers like Microsoft and Google are cash-rich incumbents and every GPU deployed is already generating revenue. He hails Pi and OpenClaw as a Unix-like architecture that makes agents model-independent and self-modifiable. On open source, he calls DeepSeek a 'gift to the world' for its paper and code, but warns that entrenched institutions—unions, licensing, government monopolies—will slow AI adoption far more than technologists expect.

Mistral: Voxtral TTS, Forge, Leanstral, & Mistral 4 — w/ Pavan Kumar Reddy & Guillaume Lample

Mar 30, 2026 · 54:02

Mistral releases Voxtral TTS, a 3B-parameter speech generation model using a novel autoregressive flow matching architecture and a 12.5 Hz neural audio codec developed in-house. Pavan Kumar Reddy and Guillaume Lample explain how the model achieves high quality and efficiency by predicting audio latents as a continuous distribution, enabling real-time voice agent applications. The episode details Mistral's stepwise multimodal strategy, starting with transcription and now speech generation, with full duplex voice agents as the next goal. Enterprise deployment via Mistral Forge allows fine-tuning on proprietary data, exemplified by training on rare languages or domain-specific jargon. Guillaume discusses merging capabilities (Mistral Small merging coding, reasoning, vision), commitment to open science through technical reports, and reasoning transfer from formal proofs in Lean. The episode also covers AI for science partnerships and hiring for research and forward-deployed engineering roles.

The AI Frontier: from Gemini 3 Deep Think distilling to Flash — Jeff Dean

Feb 12, 2026 · 1:23:31

Jeff Dean, Google's Chief AI Scientist, explains how the company's dual strategy of frontier "Pro" models and distilled "Flash" models, alongside co-designed TPUs and latency-optimization, keeps Gemini at the Pareto frontier. Distillation compresses high-capability large models into smaller, efficient ones, enabling Flash models to exceed earlier Pro generations. He emphasizes energy-based thinking: data movement costs thousands more picojoules than arithmetic, making batching and speculative decoding essential. TPU development requires predicting ML workloads 2–6 years out, often speculatively adding hardware features. Gemini was born after Dean wrote a one-page memo arguing that fragmented model efforts across Google Brain and DeepMind should merge into a single unified multimodal effort. Looking ahead, he predicts personalized models attending to all user data and reasoning speeds of 10,000 tokens/second will transform software engineering.

Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith

Jan 9, 2026 · 1:18:15

Artificial Analysis founders George Cameron and Micah Hill-Smith explain how their independent benchmarking platform became the gold standard by running their own evals with a mystery shopper policy to prevent labs from manipulating results. They launched in January 2024 after building it as a side project in Sydney, going viral after Swyx's retweet. The Intelligence Index V3 synthesizes 10 datasets with 95% confidence intervals, while the Omniscience Index measures hallucination rates from -100 to +100 (Claude models lead). Their GDP Val AA benchmark tests 44 white-collar tasks, and they open-sourced their agentic harness Stirrup. They also introduced an Openness Index scoring models out of 18 points. The episode covers how they make money through enterprise benchmarking subscriptions and custom work, and why the cost of GPT-4-level intelligence has dropped over 100× while total inference spend rises due to reasoning and agentic workflows.

[State of AI Papers 2025] Fixing Research with Social Signals, OCR & Implementation — Team AlphaXiv

Dec 31, 2025 · 34:45

AlphaXiv founders Raj and Rayhahn describe building the go-to platform for navigating AI research by obsessing over UI and getting authors like the Laura team to comment directly on papers, then evolving from comments to feeds to benchmarks to AI assistants. They detail how they beat Hugging Face Papers by enabling direct paragraph-level commenting and early traction with high-signal authors, and how they use social signals (views, comments, Twitter) to filter the archive firehose. They discuss the broken academic review process—20% AI-generated reviews at ICLR, quality collapsing under exponential submission growth—and argue papers are becoming less important than implementations, Docker containers, and real-world usability. Their vision: making it trivial to spin up any trending paper's code directly in browser, focusing on the top 0.1% of implementable papers that applied researchers actually care about.

[State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena

Dec 31, 2025 · 24:02

Anastasios Angelopoulos, founder of LMArena (now Arena), discusses the platform's $100M raise at a $1.7B valuation, its spin-out from Berkeley incubation by a16z's Anjney Midha, and its mission to be the industry's north star for real-world AI evaluation. He defends against the 'leaderboard illusion' paper, citing factual errors and reaffirming that models cannot pay to be on or off the public leaderboard. Arena funds inference costs for millions of monthly users, with 25% of its 5M+ users in software, and is expanding into occupational verticals (medicine, legal, creative) and multimodal video arenas. Key challenges include consumer retention, which improved with sign-in and persistent history, and moving off Gradio to React for better development. Angelopoulos calls for top talent in ML, product, and go-to-market to join the high-performance team.

[State of RL/Reasoning] IMO/IOI Gold, OpenAI o3/GPT-5, and Cursor Composer — Ashvin Nair, Cursor

Dec 30, 2025 · 45:13

Ashvin Nair, now ML lead at Cursor, traces his path from Berkeley robotics and an OpenAI Dota-era internship to OpenAI's reasoning team (which grew from a dozen to 300+ people) and explains why IOI Gold in 2022 felt like solving AI but didn't change the world—because RL doesn't generalize beyond training distribution. He argues most RL research from 2017-2022 overfit to benchmarks, rewarding complex ideas over simple ones that scale. At Cursor, he sees a unique opportunity for continual learning with policy updates every two hours and product-model co-design, keeping engineers in the loop instead of context-switching. His bet is that the next paradigm shift is continual learning with infinite memory: models experience something once and never forget it, storing millions of deployment tokens in weights without overloading capacity.

How Zyphra went all-in on AMD + Why Devs feel faster with AI but are slower — with Quentin Anthony

Nov 3, 2025 · 1:00:02

Quentin Anthony, head of model training at Zyphra and advisor at EleutherAI, explains why his company moved all training to AMD MI300X GPUs, outperforming Nvidia H100s on certain workloads thanks to 192GB VRAM and higher memory bandwidth, and describes his kernel development approach of writing directly in ROCm or GPU assembly rather than using Triton. He also shares his experience in the METR study on AI coding productivity, where he was one of the few developers with measurable speedup, and offers tips: timebox AI use, avoid the slot machine effect, maintain context hygiene, and use direct API over tools like Cursor. Additionally, he argues that open source AI research benefits from siloed focused teams with guaranteed funding over grand collaborations, and notes that kernel datasets alone won't solve GPU programming due to evaluation challenges.

⚡ Open Model Pretraining Masterclass — Elie Bakouch, HuggingFace SmolLM 3, FineWeb, FinePDF

Oct 20, 2025 · 1:03:40

Elie Bakouch, Hugging Face's pre-training lead and architect behind SmolLM, presents a five-pillar framework for model training spanning data quality, architecture, information extraction, gradient quality, and stability, while breaking down recent innovations in optimizers, Mixture of Experts, and data rephrasing. He explains that the field of optimizers is moving beyond AdamW with Muon and Shampoo, noting that speed-ups are often exaggerated due to undertuned baselines. He deep-dives into MoE architecture, showing that expert specialization requires load balancing at the global batch level, as demonstrated by Qwen's findings. He discusses the rephrasing revolution, where converting low-quality web data into QA format yields non-random MMLU performance even for small models. Elie also highlights Hugging Face's open science contributions: FinePDF (new PDF dataset), FineWeb-Edu2, and tools like Nanotron, Datahub, and Liteval.

Why RL Won — Kyle Corbitt, OpenPipe (acq. CoreWeave)

Oct 16, 2025 · 1:08:23

Kyle Corbitt, co-founder and CEO of OpenPipe (acquired by CoreWeave), explains why reinforcement learning has replaced supervised fine-tuning for training reliable AI agents. He argues GRPO is a dead end due to its requirement for perfectly reproducible parallel rollouts, which is extremely hard in practice. Instead, OpenPipe’s RULER uses relative LLM-as-judge rewards, achieving state-of-the-art performance even with a weak judge. Corbitt reports that 90% of AI projects remain stuck in proof-of-concept due to reliability issues, and that LoRAs are underrated for production while GEPA failed in his tests. He predicts continuous RL from real-world experience can unlock 10x more inference demand.

⚡️Accelerators @ 3x NVIDIA H200 perf, Made in the USA - Thomas Sohmers + Mitesh Agrawal, Positron AI

Aug 18, 2025 · 47:56

Thomas Sohmers and Mitesh Agrawal of Positron AI argue memory bandwidth, not compute, is the true bottleneck in AI inference, and their accelerator achieves 93% memory bandwidth utilization—triple NVIDIA's efficiency—enabling 70% faster token generation at 150W. The founders, both Lambda Labs veterans, shipped an FPGA product 15 months after founding, then raised a $51M Series A for an ASIC in late 2026. Their hardware requires no recompilation: it ingests raw binary weights from NVIDIA training and outputs an OpenAI-compatible API. Positron focuses on the decode phase of transformers, where memory-bound matrix-vector multiplication dominates, and already counts Cloudflare and Parasail as customers. The company sells systems directly, prioritizing capital efficiency and ROIC over operating its own cloud.

Greg Brockman on OpenAI's Road to AGI

Aug 15, 2025 · 1:08:37

Greg Brockman, OpenAI co-founder and president, explains how GPT-5's hybrid reasoning and aggressive pricing advance the road to AGI, while releasing GPT-OSS for open source. He traces reasoning evolution from GPT-4 via reinforcement learning, noting Dota's pure RL success and IMO gold's direct transfer to IOI gold. Brockman details GPT-5's router that hides model choice from users, and says pricing has dropped ~10X per year since GPT-4. He argues compute remains the bottleneck, but connecting AI to real-world domains like healthcare is still wide open. Brockman adds that ARC Institute's DNA models are identical to language models, and that more problems emerge over time—it's never too late to contribute.

The AI Agenda: GPT5 leaks and the business of AI News — Steph Palazzolo, The Information

Aug 6, 2025 · 1:14:36

Steph Palazzolo, AI journalist at The Information, explains how she covers the secretive AI industry, from OpenAI's GPT-5 to the inference market where startups like Modal raise billions. She reveals that inference providers are essentially GPU resellers facing margin pressure, and that GPT-5's performance will signal whether pre-training scaling or reinforcement learning drives progress. She critiques Meta's super intelligence ambition as misaligned with its ad-revenue business, and details the talent war with rumored hundred-million-dollar offers from Meta to poach researchers. Palazzolo also discusses the Windsurf acqui-hire backlash, the rise of coding agents like Claude Code, and how journalists protect sources while navigating leaks from CEOs like Sam Altman.

The RLVR Revolution — with Nathan Lambert (AI2, Interconnects.ai)

Jul 31, 2025 · 1:19:00

Nathan Lambert from AI2 explains the RLVR (Reinforcement Learning with Verifiable Rewards) revolution, arguing that verifiable rewards for math, code, and instruction-following are scaling more reliably than human feedback, and that open models like Tulu 3 can match frontier labs on core evals with just 10-15 tasks vs. hundreds. He traces overoptimization through three phases—control, RLHF, and RLVR—and warns that models learn to cheat unit tests unless reward design penalizes it. He analyzes o3's search-heavy approach (e.g., 80 websites per query), hybrid reasoning models like Gemini 2.5 and Claude, and predicts that pure reasoning models will become the default as inference costs drop. Lambert introduces a four-skill taxonomy for agent models—skills, calibration, strategy, abstraction—and says planning improvements often come from mundane data fixes. He advocates for fully open models, citing AI2's Olmo 32B as approaching GPT-4 level, but notes that building an 'American DeepSeek' requires massive resources and nonprofit constraints.

Cline: The Collaborative AI Coder

Jul 16, 2025 · 1:15:44

Saoud Rizwan and Pash from Cline join to explain why their open-source coding agent uses a Plan + Act paradigm — having models explore and plan before executing — pioneered before competitors like Claude Code. They argue that "RAG is a mind virus" and "Fast Apply got bitter lesson'd" as frontier models now achieve sub-5% diff edit failure rates, making layered workarounds obsolete. Cline's bring-your-own-API-key model avoids inference markups, with enterprise features driven by Fortune 5 companies asking for security and ROI insights. The MCP ecosystem grew via Cline's marketplace (150+ servers, top ones with hundreds of thousands of downloads), though monetization and security remain open questions. They detail how Plan mode extracts user intent and Act mode auto-approves, how context engineering prioritizes agentic file exploration over retrieval, and why they stayed a VS Code extension rather than fork — to focus solely on the agentic loop.

Information Theory for Language Models: Jack Morris

Jul 2, 2025 · 1:18:13

Jack Morris, a Cornell PhD student advised by Sasha Rush, discusses his information-theoretic research on language models, including embedding inversion (recovering text from embeddings with 90% accuracy), the universal geometry of embeddings (aligning different models' latent spaces), and measuring model memorization capacity at 3.6 bits per parameter. He argues that paradigm shifts in AI, from AlexNet to instruction tuning, stem not from new architectures but from new datasets, and that the next breakthrough will likely emerge from a novel data source. The conversation also covers the shift from academia to industry, practical advice for grad students on distributed training, and the implications of embedding inversion for privacy and model alignment.

Scaling Test Time Compute to Multi-Agent Civilizations — Noam Brown, OpenAI

Jun 19, 2025 · 1:17:47

Noam Brown, OpenAI researcher behind Cicero and reasoning models, joins hosts Alessio and Puix to discuss how test-time compute scaling drives multi-agent AI and the limits of the system 1/2 analogy. He explains that his Diplomacy bot Cicero reached top 10% of humans in 2022, and later he won the 2025 world championship. Brown argues that reasoning models like o3 succeed in unverifiable domains (e.g., Deep Research) and that harnesses and routers will be washed away by scale. He reveals OpenAI's multi-agent team is pursuing a principled approach, not heuristic, and envisions AI civilizations collaborating. He also shares his coding stack (Codex, Windsurf) and predicts test-time compute will hit cost and wall-clock bottlenecks.

The Shape of Compute (Chris Lattner of Modular)

Jun 13, 2025 · 1:18:18

Chris Lattner of Modular explains how his company is breaking the CUDA monopoly with Mojo and MAX, matching NVIDIA's best inference performance on AMD and NVIDIA GPUs. After three years in R&D, Modular open-sourced its stack and now delivers state-of-the-art Llama 3 serving at over 800 tokens per second. Mojo, a Python-family language, runs kernels faster than Rust and can extend Python without bindings, while MAX provides a full inference framework with automatic kernel fusion and cluster management. Unlike VLLM or SGLang, Modular's container is just a gigabyte, fully open source, and not reliant on proprietary CUDA blobs. DeepSeek's low-level PTX work validated the approach, but Lattner emphasizes that Mojo's portability avoids rewriting kernels for each new GPU architecture. He also details his daily routine, using Cursor for coding, and hiring "elite nerds" to grow the team.

⚡️Open Questions in Agentic RL — Will Brown (Prime Intellect)

May 9, 2025 · 17:13

Will Brown of Prime Intellect maps the frontier of open-source agentic reinforcement learning, arguing that training models like o3 requires multi-turn tool use, intermediate reward verification, and efficient context management. He outlines challenges such as scaling tool calls to hundreds (e.g., Deep Research's 100 calls), assigning credit across long chains, and avoiding context blow-up from websites or images. Key solutions include async RL pipelines to hide compute inefficiencies, offloading sub-tasks to small models as tool calls, and using reasoning reward models for step-level verification. Brown highlights progress in model merging for decentralized skill acquisition, where specialized models trained on code, math, or other tasks can be weight-averaged effectively. The talk concludes that while hard, these problems are solvable via open-source recipes, infrastructure, and community collaboration.

Solve coding, solve AGI [Reflection.ai launch w/ CEO Misha Laskin]

Mar 7, 2025 · 28:12

Misha Laskin of Reflection AI argues that solving autonomous coding is the direct path to AGI, combining reinforcement learning (pioneered by his team on AlphaGo/AlphaZero) with large language models (which they advanced at Google on PaLM, Gemini, ChatGPT). He explains that coding is already ergonomic for LLMs—unlike browser agents that require noisy human mouse data—making it the ideal starting point. Reflection AI is building coding agents that automate backlog tasks (testing, refactoring, migrations, security remediation) for large engineering teams, delivering them via an API that takes a task and codebase and outputs resolved code. Laskin insists superintelligence cannot be built in a vacuum; real-world customer evals are essential, as benchmarks like SWE-bench don't guarantee production reliability. He also stresses the need for open-weight models to prevent a few companies from hoarding superintelligent coding agents.

S1: the $6 DeepSeek R1 Competitor (ft. Entropix)

Feb 26, 2025 · 17:43

Tim Kellogg joins hosts Alessio and Swyx to break down his viral blog post on S1, showing how Stanford researchers cloned DeepSeek R1's reasoning for $6 by inserting a 'think token' and fine-tuning on just 1,000 examples. Kellogg argues the real innovation isn't the cost but the technique's simplicity—making advanced inference-time scaling accessible. He explains Entropix, a dynamic sampler that uses model entropy and varentropy to adjust generation on the fly, and notes its creators are starting a company. The conversation contrasts supervised fine-tuning's targeted results with RL's broader reasoning growth, using an analogy: 'RL causes a lot of growth, then SFT trims it.' Kellogg predicts that embedding introspection signals into RL training could solve agent 'doom loops' and unlock higher agency—a key challenge behind products like OpenAI's Deep Research.

smol agents are all you need

Feb 13, 2025 · 22:31

This episode features Aymeric Roucher of Hugging Face discussing smol agents, a lightweight agent framework, and argues that code-based agents outperform JSON-based agents. Roucher explains that agents are defined by LLM control over execution flow, and smol agents simplifies this with a core file under 1000 lines. He describes the origin from transformers.agents and the security-focused custom Python interpreter. The library achieved third place on the GAIA benchmark, using o1 and Sonnet models, and Roucher projects 90% accuracy by 2026 based on a sigmoid trend line. Hugging Face is launching an agent course and plans to build GUI agents for computer use, while also fine-tuning R1 for agentic tasks.

Outlasting Noam Shazeer, Crowdsourcing Chai AI w/ 1.4m DAU — with William Beauchamp, Chai Research

Jan 26, 2025 · 1:13:33

William Beauchamp, founder-CEO of Chai AI, explains how he pivoted from algorithmic trading to build a character chatbot platform before Character.ai, growing to 1.4M DAU and $22M+ revenue by crowdsourcing model improvements through Chaiverse. Starting with GPT-J in 2021, he found product-market fit with a therapist bot and shifted to user-generated content, letting users define prompts, images, and names. Competing against well-funded rivals like Character AI and Talkie, Chai ships over 100 LLMs weekly via its developer platform, spending $10M on compute in 2024 and tripling it. Beauchamp argues AI follows an S-curve, not scaling laws, and focuses on inference optimization using rejection sampling and reward models to serve better responses. He prioritizes 'insanely great' products over technology-driven features, noting that audio and image features failed to move metrics, while a data flywheel and aggressive user acquisition drove rapid growth.

A Comprehensive Overview of Large Language Models - Latent Space Paper Club

Mar 15, 2024 · 54:30

Brian presents a detailed walkthrough of the survey paper 'A Comprehensive Overview of Large Language Models,' which systematically covers the evolution of LLMs from early attention mechanisms to modern transformer architectures. He explains the shift from sequential RNNs to parallelizable transformers, the three dominant architectures (encoder-only like BERT, encoder-decoder like T5, decoder-only like GPT), and key training objectives (masked language modeling, full language modeling, prefix language modeling). The paper also reviews fine-tuning techniques (instruction tuning, alignment tuning via RLHF), prompting strategies (zero-shot, chain-of-thought), efficient adaptation methods (LoRA, quantization), evaluation benchmarks (GLUE, MMLU), and challenges such as bias, memorization, and privacy leaks. This episode serves as a comprehensive primer for anyone seeking to understand the foundational concepts and latest trends in large language model research.