Ethan He

Inside xAI: Building Grok Imagine in 3 Months, Videogen vs World Models, and Video Agents— Ethan He

Jun 1, 2026 · 1:44:43

Ethan He, former xAI and NVIDIA Cosmos researcher, explains how xAI built its first image and video models (Grok Imagine 0.9) from zero to one in three months, attributing rapid iteration to small teams with minimal meetings and strong infra that enabled fixing tiny data and training bugs for biggest quality gains. He argues that most improvements in video generation now come from language models and agents rather than diffusion technology, predicting that by end of 2025 video agents will produce production-grade content for ads. He defines world models as real-time, interactive, long-horizon videos, and details challenges like temporal compression, context management, and the high cost of storing and moving video data (e.g., tens of petabytes for a billion videos). Ethan also shares why he left xAI to focus on language model research, believing the next frontier is models that manage their own context length, similar to solutions already being explored in video generation.

Bee AI: The Wearable Ambient Agent

Feb 17, 2025 · 1:07:40

Bee AI's Maria and Ethan explain their wearable ambient agent, arguing that personal AI with continuous context is the future, and detail its $49.99 hardware, 7-day battery, and free inference. They trace their origin from a 2016 personal AI startup to Squad (sold to Twitter) and then to Bee, emphasizing that hardware was necessary because app friction (e.g., remembering to open it) killed the ambient experience. Privacy is addressed by not storing audio, only summaries, and offering geofencing and concept fencing to block sensitive topics. The software pipeline uses voice activity detection to reduce compute, speaker identification, and custom small models for memory retrieval instead of traditional RAG or knowledge graphs. Social features are in beta, with agent-to-agent negotiation—e.g., both agents coordinating a dinner reservation based on preferences and calendars. They are hiring AI engineers and see always-on AI as inevitable.

[Paper Club] Upcycling Large Language Models into Mixture of Experts

Oct 29, 2024 · 39:02

NVIDIA's Ethan He presents Megatron-Core MoE and the upcycling of dense LLMs into Mixture-of-Experts models, showing that upcycling a 15B-param dense model into a 64-expert MoE and training on 1T tokens yields 5% lower validation loss and 4% higher MMLU than continued dense training on the same compute. Key techniques include swapping the router order (softmax-then-top-K instead of top-K-then-softmax) with a 4× output scaling to preserve initial forward pass behavior, and initializing fine-grained MoE routers by duplicating half the weights so each shard group selects identically. A high learning rate matching the original pre-training peak is critical; constant low LR causes catastrophic forgetting. The upcycled model matches the compute of a 1.7× larger dense model per scaling laws, but data quality remains paramount—continued dense training still delivered a 20% MMLU jump. Megatron-Core's fused permutation, GroupedGEMM, and expert parallelism are available as a standalone library.

Personal AI Meetup - Bee, BasedHardware, LangChain LangFriend, Deepgram EmilyAI

Apr 6, 2024 · 58:54

This episode features Damien Murphy of Deepgram, Ethan of Owl/Bee, and Harrison of LangChain demonstrating how to build personal AI with real-time voice bots, wearable life-recording devices, and memory-enhanced journaling apps. Damien shows building a voice bot with subsecond latency using Deepgram, OpenAI, and open source code, costing about 6.5 cents per five-minute call. Ethan presents his Owl wearable that continuously records audio, triggers actions via hot word 'Scarlett,' and discusses challenges in adding vision and open source adoption. Harrison introduces LangFriend, a journaling app using conversational, semantic, and knowledge graph memory, referencing the Generative Agents paper for recency and importance weighting. The episode also highlights open source projects Whomane, Friend, and ADeus, arguing that hardware, voice, and memory are all necessary components for personal AI.