Genie

The Agent Cloud: Databricks’ Bet on the Future of AI — Matei Zaharia and Reynold Xin

Jun 24, 2026 · 1:10:06

Databricks cofounders Matei Zaharia and Reynold Xin argue the company is moving beyond the lakehouse into a full data-and-AI operating system, anchored by two new initiatives: Omnigent, an open-source meta-harness for combining coding and enterprise agents, and LTAP, a unified storage layer that gets most HTAP benefits without collapsing query engines. Omnigent provides a common API for agent sessions, files, streams, tool calls, and cancellation, solving portability, collaboration, and security issues across Claude Code, Codex, Cursor, and custom agents. LTAP writes transactional data directly in columnar Parquet format, eliminating brittle CDC pipelines—Reynold jokes CDC means 'continuous data corruption'—and enables instant analytics without overloading the source database. The episode details Databricks’ culture of rapid prototyping, where an engineer built the LTAP prototype without a formal design doc, and the thesis that traditional software will be rewritten once data is in the right place with agents on top. They also cover Mosaic’s shift from general frontier models to specialized fine-tuned models like document parsing, internal agent usage, and security features such…

Moonlake: Interactive, Multimodal World Models — with Chris Manning and Fan-yun Sun

Apr 2, 2026 · 1:06:48

Moonlake AI founders Chris Manning and Fan-yun Sun argue that interactive, multimodal world models require structured symbolic reasoning over pure scale, enabling indefinite multiplayer gameplay and causal consistency that video generation models like Genie and Sora cannot achieve. Their approach uses code engines and physics simulators as cognitive tools, producing reasoning traces that handle geometry, physics, and logic, while a separate diffusion model (Reverie) handles pixel fidelity. They aim to replace traditional rendering and empower creators by allowing human intent to be injected at a symbolic layer. Manning contrasts this with Yann LeCun's JEPA, emphasizing language and abstraction over pixel-level prediction. Moonlake is hiring engineers at the intersection of code generation, computer vision, and graphics.

World Models & General Intuition: Khosla's largest bet since LLMs & OpenAI

Dec 6, 2025 · 1:04:51

Pim de Wit, founder of Medal and General Intuition (GI), turned down a reported $500M offer from OpenAI to spin out GI with a $134M seed from Khosla Ventures — Vinod Khosla's largest bet since OpenAI — arguing that world models trained on peak human gameplay are the next frontier after LLMs. Medal's 12M users generate 3.8B action-labeled clips via retroactive recording, creating a privacy-preserving dataset of 'episodic memory for simulation.' GI builds fully vision-based agents that see only frames and output actions in real-time, using pure imitation learning without RL, and can transfer from arcade games to realistic games to real-world video. Pim explains why world models need actions, memory, and partial observability (e.g., smoke, camera shake) compared to video generation, and how they distill giant policies into tiny real-time models that navigate and hide like humans. He recounts his path from running the largest RuneScape private server to reverse engineering, cold-emailing the Diamond (world model) paper authors to assemble a top research team, and advises data founders to train models themselves before selling. GI's near-term customers are game developers replacing…

After LLMs: Spatial Intelligence and World Models — Fei-Fei Li & Justin Johnson, World Labs

Nov 25, 2025 · 1:00:39

World Labs co-founders Fei-Fei Li and Justin Johnson argue that spatial intelligence is the next frontier beyond LLMs, with their product Marble generating editable 3D worlds from text or images using Gaussian splats. They claim language is a lossy channel for describing the rich 3D/4D world, while spatial intelligence handles everything from picking up a mug to inferring DNA's double helix. Marble already serves gaming, VFX, and film, with precise camera control and real-time rendering on phones and VR headsets. They discuss scaling compute from AlexNet to today's million-fold increase, why world models can 'soak up' modern GPU clusters better than language alone, and the gap between pattern-fitting and causal understanding in physics. Fei-Fei worries more about under-resourced academia than open vs. closed models, advocating for national AI compute clouds and open benchmarks like the BEHAVIOR dataset. Justin notes transformers are natively set models, not sequence models, opening architectural possibilities for new spatial data.

Building AGI in Real Time (OpenAI Dev Day 2024)

Oct 4, 2024 · 2:09:35

OpenAI's DevDay 2024 unveils the Realtime API with WebSocket-based voice and function calling, Vision Finetuning, Prompt Caching, and Model Distillation. Product leads Olivier Godement, Romain Huet, and Michelle Pokrass detail design choices like zero-code caching and WebSockets for human-level latency. Alistair Pullen of Cosine explains how fine-tuning GPT-4o on custom reasoning traces beat o1 on SWE-bench Verified, while Sam Altman and Kevin Weil discuss AGI levels (o1 as level 2 reasoner), the coming agent era (level 3), safety through iterative deployment, and the importance of building for the frontier. The episode also covers moderation policy relaxation, plans for o1 system prompts by year-end, and the competitive admiration for NotebookLM.

Is finetuning GPT4o worth it?

Aug 22, 2024 · 1:01:16

Ali Pullen, founder of Cosine, discusses Genie, an AI software engineering colleague that achieved 30% on SWE-Bench and 43.8% on SWE-Bench Verified via fine-tuning GPT-4o on billions of tokens of synthetic data. Cosine built Genie by training on the process of software engineering—not just working code—using synthetic runtime errors and a self-improvement loop. The agent follows a four-stage workflow: code retrieval (66% accuracy), planning, writing diffs, and running CI tests. Pullen reveals that model performance degrades linearly beyond 60K tokens in the context window, and that OpenAI dynamically sizes LoRA adapters to handle Cosine's massive dataset. He explains why Genie isn't on the SWE-Bench leaderboard (refusing to publish trajectories to prevent distillation) and outlines plans to scale the dataset and fine-tune on customer codebases for personalized performance.