Mojo

The Shape of Compute (Chris Lattner of Modular)

Jun 13, 2025 · 1:18:18

Chris Lattner of Modular explains how his company is breaking the CUDA monopoly with Mojo and MAX, matching NVIDIA's best inference performance on AMD and NVIDIA GPUs. After three years in R&D, Modular open-sourced its stack and now delivers state-of-the-art Llama 3 serving at over 800 tokens per second. Mojo, a Python-family language, runs kernels faster than Rust and can extend Python without bindings, while MAX provides a full inference framework with automatic kernel fusion and cluster management. Unlike VLLM or SGLang, Modular's container is just a gigabyte, fully open source, and not reliant on proprietary CUDA blobs. DeepSeek's low-level PTX work validated the approach, but Lattner emphasizes that Mojo's portability avoids rewriting kernels for each new GPU architecture. He also details his daily routine, using Cursor for coding, and hiring "elite nerds" to grow the team.

Open Source AI is AI we can Trust — with Soumith Chintala of Meta AI

Mar 6, 2024 · 1:37:38

Soumith Chintala, creator of PyTorch and engineering lead at Meta AI, argues that open source AI is essential for distributing opportunity and trust. He details PyTorch's complexity—1,000 operators needed for generality—and explains synthetic data as a vehicle for imparting symbolic knowledge where humans already have good symbolic models. He highlights a coordination problem in open source: feedback is lost because frontends like Ooba and Ollama lack feedback buttons, and proposes a centralized sinkhole to collect high-quality feedback. Beyond text, he is excited about robotics, where hardware remains a bottleneck, and Osmo's work to digitize smell, which he compares to images in the 1800s.

Truly Serverless Infra for AI Engineers - with Erik Bernhardsson of Modal

Feb 19, 2024 · 1:08:58

Erik Bernhardsson, creator of Annoy and Luigi, founded Modal to build a 'postmodern data stack' with a self-revisioning runtime that eliminates container cold starts and lets developers attach GPUs with a Python decorator. His serverless platform can fan out to thousands of GPUs within seconds, becoming a natural fit for GenAI workloads like Stable Diffusion and fine-tuning (e.g., RAMP fine-tuning 100 models in parallel). Modal differentiates from Replicate by targeting custom models and workflows, and from Modular by offering a managed cloud service rather than licensed software. Erik argues that buying hardware is inefficient for startups and that infrastructure founders must be willing to spend years on load balancing, page faults, and DNS. He shares how his IOI Gold Medal background shapes Modal's talent culture, hiring competitive programmers for complex scheduling and bin-packing problems.

The End of Finetuning — with Jeremy Howard of Fast.ai

Oct 20, 2023 · 1:24:48

Jeremy Howard of Fast.ai argues that fine-tuning language models is essentially continued pre-training, not a separate process, and that the common practice of fine-tuning on a single task causes catastrophic forgetting. He recounts the discovery of single-shot memorization in LLMs, where models memorize entire datasets after one epoch, a phenomenon many practitioners ignore. Howard criticizes the current focus on zero-shot and few-shot learning, advocating for transfer learning and small models. He shares his journey from philosophy to founding Fast.ai, the creation of ULMFit (which inspired GPT), and his ongoing work on making AI accessible. He also discusses his involvement with Modular's Mojo language and the importance of democratizing AI technology.

Ep 18: Petaflops to the People — with George Hotz of tinycorp

Jun 20, 2023 · 1:23:24

George Hotz of tinycorp argues that a simplified, open-source ML framework (tinygrad) can democratize AI compute and replace NVIDIA, Google, and AMD. Tinygrad uses 25 ops (vs. XLA's 250) and fuses kernels automatically, achieving 2x speed on Qualcomm GPUs over their library. Hotz reports AMD kernel panics fixed after emailing CEO Lisa Su, but laments AMD's open-source as 'dumped on GitHub.' For hardware, tinybox is a six-GPU desktop at 350W per GPU, aiming for 5x better price-performance than NVIDIA H100 for 90% of training. He criticizes OpenAI's secrecy, revealing GPT-4 is an 8-way mixture of 220B models, and cites the Bitter Lesson. He also outlines FLOPcoin using hardware identity to prevent cheating, and his goal of an AI girlfriend as the third company, merging humans via data rather than implants.