Blackwell

After LLMs: Spatial Intelligence and World Models — Fei-Fei Li & Justin Johnson, World Labs

Nov 25, 2025 · 1:00:39

World Labs co-founders Fei-Fei Li and Justin Johnson argue that spatial intelligence is the next frontier beyond LLMs, with their product Marble generating editable 3D worlds from text or images using Gaussian splats. They claim language is a lossy channel for describing the rich 3D/4D world, while spatial intelligence handles everything from picking up a mug to inferring DNA's double helix. Marble already serves gaming, VFX, and film, with precise camera control and real-time rendering on phones and VR headsets. They discuss scaling compute from AlexNet to today's million-fold increase, why world models can 'soak up' modern GPU clusters better than language alone, and the gap between pattern-fitting and causal understanding in physics. Fei-Fei worries more about under-resourced academia than open vs. closed models, advocating for national AI compute clouds and open benchmarks like the BEHAVIOR dataset. Justin notes transformers are natively set models, not sequence models, opening architectural possibilities for new spatial data.

⚡️Accelerators @ 3x NVIDIA H200 perf, Made in the USA - Thomas Sohmers + Mitesh Agrawal, Positron AI

Aug 18, 2025 · 47:56

Thomas Sohmers and Mitesh Agrawal of Positron AI argue memory bandwidth, not compute, is the true bottleneck in AI inference, and their accelerator achieves 93% memory bandwidth utilization—triple NVIDIA's efficiency—enabling 70% faster token generation at 150W. The founders, both Lambda Labs veterans, shipped an FPGA product 15 months after founding, then raised a $51M Series A for an ASIC in late 2026. Their hardware requires no recompilation: it ingests raw binary weights from NVIDIA training and outputs an OpenAI-compatible API. Positron focuses on the decode phase of transformers, where memory-bound matrix-vector multiplication dominates, and already counts Cloudflare and Parasail as customers. The company sells systems directly, prioritizing capital efficiency and ROIC over operating its own cloud.

The Shape of Compute (Chris Lattner of Modular)

Jun 13, 2025 · 1:18:18

Chris Lattner of Modular explains how his company is breaking the CUDA monopoly with Mojo and MAX, matching NVIDIA's best inference performance on AMD and NVIDIA GPUs. After three years in R&D, Modular open-sourced its stack and now delivers state-of-the-art Llama 3 serving at over 800 tokens per second. Mojo, a Python-family language, runs kernels faster than Rust and can extend Python without bindings, while MAX provides a full inference framework with automatic kernel fusion and cluster management. Unlike VLLM or SGLang, Modular's container is just a gigabyte, fully open source, and not reliant on proprietary CUDA blobs. DeepSeek's low-level PTX work validated the approach, but Lattner emphasizes that Mojo's portability avoids rewriting kernels for each new GPU architecture. He also details his daily routine, using Cursor for coding, and hiring "elite nerds" to grow the team.

2024 Year in Review: The Big Scaling Debate, the Four Wars of AI, Top Themes and the Rise of Agents

Jan 1, 2025 · 1:51:07

In their 100th episode, hosts Alessio and Swyx recap 2024 in AI, arguing that pre-training scaling has hit a wall—backed by Ilya Sutskever and others at NeurIPS—and that inference-time compute (o1, o3) is the new frontier. They dissect the "four wars": data quality (lawsuits vs. synthetic data), GPU haves vs. have-nots (with the middle class dying), multimodality (Sora, Veo 2, Gemini 2.0's native image output), and the LLM OS/agents stack (LangChain, E2B, memory). Market share shifted from OpenAI's 95% to 50-75% as Anthropic and Gemini gained ground; prices dropped ~3 orders of magnitude for same ELO. The episode predicts 2025 as the year agents finally enter production, driven by models like o1 and tools like Devin, and warns that AI will set the skill floor for roles.