Developer Tools

⚡️Every product of the future will be a living system — Ronak Malde, Trajectory.ai

Jun 21, 2026 · 33:58

Ronak Malde, CEO of Trajectory.ai, recounts his journey from building AI coding agents at Windsurf (acquired by Google DeepMind in a deal involving Demis Hassabis and Sergey Brin) to launching a platform for continual learning in enterprise AI. He argues that every future product must be a 'living system' that learns from real-world user interactions, not static models. The episode details Trajectory's technical innovations, including Self-Distillation Policy Optimization (SDPO) for learning from corrections, continuous LoRA for parallel training, and open-sourcing a training stack with SkyRL. It covers partnerships with Harvey and NVIDIA to train NeMoTron 3 Super for legal workflows, improving metrics like issue spotting and citation accuracy while cutting costs. Malde explains data curation strategies that capture nuanced user edits beyond binary signals, and outlines Trajectory's roadmap from AI-native companies like Clay and Decagon to Fortune 500 enterprises. He also reveals that the idea for Trajectory emerged after giving up his acquisition equity to pursue this vision.

⚡️Making DeepSeek v4 outperform Opus 4.7 with Taste — @AhmadAwais , CommandCode.ai

Jun 6, 2026 · 40:41

Ahmad Awais, CEO of CommandCodeAI, explains how his team's 'tool-input repair layer' lets open models like DeepSeek V4 Pro outperform premium models like Opus 4.7 in tool-calling reliability. He discovered a pattern of 'tool confusion' where open models repeatedly send wrong schemas; by repairing errors deterministically and sending repair hints instead of raw errors, tool call failures dropped dramatically. The same repair logic fixes 'design slop' using a compositional framework of patterns and OKLCH color system. Taste, a meta-neurosymbolic system, automatically learns developer preferences into portable markdown files, avoiding stale rules. Plans include open-sourcing CommandCode to make it hackable while focusing on quality models.

Senior Dev: This "Grill Me" Prompt Is Going Viral Among Top Engineers

May 7, 2026 · 22:03

Matt Pocock — the TypeScript educator turned AI engineering teacher who created the viral "Grill Me" prompt and runs AI Hero — argues that classic software engineering fundamentals are more critical than ever in the AI coding era. He explains how Domain-Driven Design's ubiquitous language and deep module patterns let developers contain complexity and delegate implementation to AI. His free "Grill Me" skill (part of his 13,000-star skills repo) forces AI to relentlessly interview humans until reaching shared understanding. He contrasts Claude Code's ease-of-use against Pie's transparent primitives, predicting a shift toward developer observability. The episode also covers his two-month course creation process, why TypeScript is overtaking Python in AI engineering, and why traditional lectures still dominate learning.

⚡️ How to turn Documents into Knowledge: Graphs in Modern AI — Emil Eifrem, CEO Neo4J

Apr 18, 2026 · 48:53

Emil Eifrem, CEO of Neo4j, joins host Shawn Wang to argue that AI systems need more than top-K chunk retrieval—they require graph-shaped context for accuracy, explainability, and developer productivity. GraphRAG combines vector search with graph traversal, starting semantically then expanding through relationships, which he says yields higher accuracy and auditability than opaque vector spaces. Neo4j now powers production AI at companies like Pfizer (over 60 million documents), Novo Nordisk, and 20 of North America’s 20 largest banks, with a mortgage lender seeing a 20% conversion lift. Eifrem describes four data sources for agentic systems: operational databases, cloud warehouses, agentic memory, and context graphs—the latter encoding institutional decision traces. He notes a recent shift where enterprises lead with generic text-to-Cypher instead of specialized functions, and highlights Neo4j's new `create-context-graph` starter kit for 22 industries, built to bootstrap context graphs and agent memory.

⚡️ Reverse Engineering OpenAI's Training Data — Pratyush Maini, Datology

Feb 10, 2026 · 27:02

Pratyush Maini from Datology reveals how the seahorse emoji question exposes that frontier models like GPT-4.1, GPT-5, and OLMo 3.1 now exhibit self-correction loops—a sign that reasoning traces are baked into mid-training data, not just post-training. By tracking response length surges four months after o1's release and corroborating with OLMo's data, Maini argues that foundation models must contain core capabilities like self-reflection, reinforcing the 'Fine Tuner's Fallacy': you cannot simply fine-tune for a capability; it must be in pre-training. He also highlights Datology's BeyondWeb project, which uses a source-rephrasing paradigm to transform internet data at trillion-token scale, achieving Nemotron-level performance in 2.7x less compute—making specialized pre-training accessible for enterprises. The episode concludes that 2026-2027 will see a shift toward domain-specific pre-training, as the cost amortizes when a smaller pre-trained model outperforms a larger fine-tuned one.

Goodfire AI’s Bet: Interpretability as the Next Frontier of Model Design — Myra Deng & Mark Bissell

Feb 5, 2026 · 1:08:41

Goodfire AI's Mark Bissell and Myra Deng argue that interpretability is the next frontier for model design, using their recent $150M Series B at $1.25B valuation to scale surgical edits of model internals beyond post-hoc poking. They explain how their platform detects behaviors like sycophancy and reward hacking, enabling targeted unlearning without wrecking capabilities. The episode covers real-world deployments from Rakuten's PII guardrails to life science partnerships with Mayo Clinic finding Alzheimer's biomarkers. Mark demonstrates real-time steering of a trillion-parameter Kimi K2 model, while Myra details how SAEs sometimes underperform probes for detection tasks. They envision a future where interpretability guides training so customization isn't brute-force guesswork.

[State of Research Funding] Beyond NSF, Slingshots, Open Frontiers — Andy Konwinski, Laude Institute

Dec 31, 2025 · 22:34

Andy Konwinski, co-founder of Databricks and Perplexity, launches Laude Institute—a dual venture fund and nonprofit to accelerate the path from open research to breakout companies. At NeurIPS, he explains how Laude's Slingshot program funds projects like DSPy, Terminal Bench, and LMArena, and why the NSF's $1B/year for CS is insufficient, needing $10-100B for frontier AI. He argues Chinese labs (Moonshot, DeepSeek) now outpublish US because OpenAI and others stopped sharing; Open Frontiers, a live-streamed conference in SF, aims to unite top open researchers (Yann LeCun, François Chollet, Jan Leike) to reclaim global leadership. The episode also spotlights the emerging 'post-post-training' layer—prompt optimization, context management, RAG—and why research teams with multiple co-founders (like Databricks' eight) are the new gold standard for AI startups.

[State of AI Startups] Memory/Learning, RL Envs & DBT-Fivetran — Sarah Catanzaro, Amplify

Dec 30, 2025 · 28:43

Sarah Catanzaro of Amplify Partners argues that the DBT-Fivetran merger was not the death of the modern data stack but a path to IPO targeting $600M+ combined revenue, that data catalogs failed because they were built for humans rather than machines, and that RL environments are a fad. She notes that frontier labs use dbt and Fivetran for training data curation and agent analytics, and criticizes $100M+ seed rounds raised without near-term roadmaps. For 2026, she identifies personalization via memory and continual learning as the key retention unlock, observing that AI founders are unfamiliar with growth concepts like k-factor. She prefers real-world logs over synthetic RL environments, citing Cursor's use of user activity, and says the most exciting startups combine hard research problems (RAG, rule-following, continual learning) with applications that were previously impossible.

⚡️ The State of AI Engineer Hiring: Cheating, AI Adoption,Junior Devs — Vivek Ravisankar, HackerRank

Nov 8, 2025 · 49:05

Vivek Ravisankar, CEO of HackerRank, reveals that while overall tech hiring has flattened year-over-year, AI-specific roles are exploding and companies are reversing their stance on junior hiring because new grads are the true AI natives who embrace tools like Devin and Cursor without hesitation. He details HackerRank's integrity challenges—from leaked questions on Chegg to AI cheating tools like Interview Coder—and their countermeasures: custom-trained plagiarism models with 85-90% precision, DMCA takedowns, and a proctor mode that can shut down unauthorized apps. Rather than fighting AI, HackerRank embeds AI assistants into assessments, shifting from LeetCode-style tasks to real-world code repository challenges. Ravisankar defines the next-gen developer by four attributes: strong software engineering fundamentals, ability to use AI across the entire SDLC, deep knowledge of AI concepts from prompt engineering to fine-tuning, and good taste with business acumen. He predicts a proliferation of developers across all business functions—with roles like 'full-stack marketers' and 'go-to-market engineers'—and notes the irony that the most AI-forward companies like Anthropic explicitly…

How Zyphra went all-in on AMD + Why Devs feel faster with AI but are slower — with Quentin Anthony

Nov 3, 2025 · 1:00:02

Quentin Anthony, head of model training at Zyphra and advisor at EleutherAI, explains why his company moved all training to AMD MI300X GPUs, outperforming Nvidia H100s on certain workloads thanks to 192GB VRAM and higher memory bandwidth, and describes his kernel development approach of writing directly in ROCm or GPU assembly rather than using Triton. He also shares his experience in the METR study on AI coding productivity, where he was one of the few developers with measurable speedup, and offers tips: timebox AI use, avoid the slot machine effect, maintain context hygiene, and use direct API over tools like Cursor. Additionally, he argues that open source AI research benefits from siloed focused teams with guaranteed funding over grand collaborations, and notes that kernel datasets alone won't solve GPU programming due to evaluation challenges.

⚡️ Ship AI recap: Agents, Workflows, and Python — w/ Vercel CTO Malte Ubl

Nov 1, 2025 · 42:02

Vercel CTO Malte Ubl discusses the company's AI strategy, centered on the new Workflow Development Kit that enables durable, resumable serverless functions with infinite idle time at no cost. He explains how the AI SDK was extracted from v0 and remains deliberately low-level to accommodate emerging patterns, with a direct agent abstraction now stable in version six. Ubl details Vercel's internal agents—including a DevOps agent that investigates production anomalies by querying observability data and logs, solving the recall-precision problem in alerting—and the 'Agent on Every Desk' program that helps large companies build their first three agents. He also covers zero-config Python support for Flask and FastAPI, and a security model that assumes developers cannot be trusted, extracting auth and data access from application code.

⚡ Open Model Pretraining Masterclass — Elie Bakouch, HuggingFace SmolLM 3, FineWeb, FinePDF

Oct 20, 2025 · 1:03:40

Elie Bakouch, Hugging Face's pre-training lead and architect behind SmolLM, presents a five-pillar framework for model training spanning data quality, architecture, information extraction, gradient quality, and stability, while breaking down recent innovations in optimizers, Mixture of Experts, and data rephrasing. He explains that the field of optimizers is moving beyond AdamW with Muon and Shampoo, noting that speed-ups are often exaggerated due to undertuned baselines. He deep-dives into MoE architecture, showing that expert specialization requires load balancing at the global batch level, as demonstrated by Qwen's findings. He discusses the rephrasing revolution, where converting low-quality web data into QA format yields non-random MMLU performance even for small models. Elie also highlights Hugging Face's open science contributions: FinePDF (new PDF dataset), FineWeb-Edu2, and tools like Nanotron, Datahub, and Liteval.

The antidote to AI fatigue — Answer.ai Solveit

Oct 2, 2025 · 55:26

Jeremy Howard, Eric Ries, and Johno Whitaker join host Alessio to present Answer.ai's Solveit platform, arguing that human-in-the-loop, step-by-step AI collaboration outperforms fully autonomous agents for complex coding and writing tasks. The 12-person team built their own integrated development and deployment system, eschewing cloud vendors. Solveit provides each user a persistent Linux Docker container with a Jupyter-like interface where the AI can see all steps, enabling immediate error correction and iterative improvement. Johno demonstrates building a perfume scent search engine in under an hour; Eric shows how he uses Solveit to manage writing his book 'Incorruptible', incorporating test reader comments and fact-checking sections. Jeremy reveals they turned Andrej Karpathy's GPT tokenizer video into a polished blog post. The Solveit course launches October 20th at solve.it.com, teaching this philosophy beyond just the tool.

⚡️Claude Sonnet 4.5 and Anthropic's roadmap for Agents and Developers — Mike Krieger, Anthropic

Sep 30, 2025 · 26:25

Mike Krieger, Anthropic's CPO, discusses the launch of Claude Sonnet 4.5 and its implications for AI development, emphasizing the growing symbiosis between product and research teams. He describes his personal 'vibe evals' for new models, including generating a Virtual Boy-style game and a Nintendo board presentation, and notes that Sonnet 4.5 traffic eclipsed Sonnet 4 on day one. Krieger explains how Anthropic is improving model output quality across UIs, Excel, and PowerPoint, and highlights the shift toward interactive planning rather than pure autonomy. He reveals the rename of the Claude Code SDK to the Claude Agent SDK, reflecting its general-purpose use for complex agentic workflows beyond coding. Krieger stresses the need for better visual understanding in models and invites developers to share feedback on hard challenges to help improve the platform.

⚡️Snowglobe: Simulations for your AI

Sep 25, 2025 · 26:14

Shreya Rajpal returns to Latent Space to launch Snowglobe, a simulation engine that lets developers test AI products by generating diverse simulated user interactions before production, inspired by self-driving car simulation where Waymo logged 20 billion simulated miles versus 20 million real miles. She explains how simulation uncovers failures like over-refusal—a problem that emerged for a design partner who initially worried about toxicity—by generating personas with varied styles and goals. Snowglobe uses a multi-model architecture (proprietary and open source) to create realistic, diverse conversations, and it supports testing across chat, voice, and tool-calling agents. Pricing is usage-based per message, with persona modeling included. Shreya highlights enterprise use cases, such as comparing vendors or generating training data for fine-tuning, and notes the company is hiring a product designer, product engineer, and research staff.

Better Data is All You Need — Ari Morcos, Datology

Aug 29, 2025 · 1:18:43

Ari Morcos, CEO of Datology, argues that data curation is the most impactful and underinvested area in AI, claiming models are defined by what they eat. He details his shift from focusing on inductive biases to realizing data quality is paramount, citing the DCLM study where human experts could not predict automated filtering decisions. Datology's methods combine filtering, rebalancing, curriculum, and rephrasing synthetic data to achieve baseline performance 12x faster, improve accuracy by 4–5 absolute points, and train models with fewer than half the parameters. Morcos explains how curating data bends naive scaling laws by maintaining marginal information gain, and shares results from the Arc 4.5B model, which was trained on 7 trillion tokens (down from 25 trillion) using Datology's curation. He highlights that data is a compute multiplier, making it possible for enterprises to train smaller, cheaper models for specific domains.

Long Live Context Engineering - with Jeff Huber of Chroma

Aug 19, 2025 · 57:01

Jeff Huber, founder and CEO of Chroma, joins Shawn Wang and Alessio to argue that context engineering—not RAG—is the real job of AI builders, and that Chroma's modern retrieval engine is built for this new paradigm. Huber explains context rot, a phenomenon where LLMs lose attention and reasoning ability as context windows grow, motivating precise retrieval over huge context windows. Chroma Cloud is a zero-config, usage-based serverless vector database with separation of storage and compute, now powering 5M monthly downloads and 21k GitHub stars. He shares techniques like chunk rewriting and generative benchmarking for evaluating retrieval, and predicts LLMs will increasingly serve as re-rankers. The episode also covers memory as a benefit of context engineering, offline compaction for self-improving AI systems, and Huber's philosophy of building with long-term purpose, hiring engineers who care about Rust, TLA+, and deterministic simulation testing.

🕰️ The Oral History of Windsurf (ft. Varun Mohan, Scott Wu, Jeff Wang, Kevin Hou, Anshul R)

Jul 28, 2025 · 3:53:23

This episode traces the oral history of Windsurf (formerly Codeium and Exafunction) with founders Varun Mohan, Scott Wu, Jeff Wang, Kevin Hou, and Anshul R, charting its evolution from a GPU virtualization startup to a leading agentic IDE and its dramatic acquisition by Cognition. The narrative covers the pivot to code completion in 2022, the shift to a free model to gain traction, and the launch of the Windsurf editor in November 2024 with Cascade, an agent that operates on a shared timeline with the developer. Key principles include meta-learning (auto-generated memories that adapt to each user's preferences) and scaling with intelligence (replacing hard-coded rules with LLM reasoning as models improve). The episode also details the acquisition weekend when Cognition bought Windsurf after Google acquired a stake, highlighting product synergies between the synchronous agent (Windsurf) and async agent (Devin) and the ambition to build a comprehensive platform for real-world software engineering teams.

⚡️The Future of Notebooks - with Akshay Agrawal of Marimo

Jul 18, 2025 · 28:14

Akshay Agrawal introduces Marimo, an open-source reactive Python notebook built from scratch for AI and data work, emphasizing its lack of Jupyter dependencies and features like reactive execution, built-in UI elements, and pure Python storage for Git versioning. Marimo has surpassed 300k monthly PyPI downloads and more GitHub stars than Jupyter Notebook, used at OpenAI, Hugging Face, and Cloudflare. Agrawal demos interactive data exploration, including a PS5 controller for data annotation, and AI-native code generation with context-aware cell completion. He explains Marimo's UV package integration for inline dependency management and the upcoming MoLab cloud-hosted service on Modal, free for community use, addressing demand for a Colab-like experience. Marimo notebooks can be run as scripts or deployed as data apps, with a fork exploring agent cells for generative analysis.

⚡️Warp 2.0: the Agentic Development Environment - Zach Lloyd and Ben Holmes

Jun 25, 2025 · 52:23

Warp CEO Zach Lloyd and engineer Ben Holmes debut Warp 2.0, rebranding from an AI terminal to an Agentic Development Environment (ADE) with a state-of-the-art coding agent that rivals Claude Code and Cursor. The episode demonstrates Warp's new universal input that auto-detects English commands and terminal input, voice-driven agent workflows, multi-threaded agents running in tabs, inline diff editing, MCP support, and REPL translations for SQL and Postgres. Lloyd explains the design philosophy of keeping a terminal-like UX while adding agent management features, and discusses pricing changes (Pro now 2,500 requests, Turbo 10,000, plus usage-based credits) and the challenge of subscription vs. usage-based models. Warp has nearly 600,000 active users with weekly revenue growth of 5–15%, and the team positions itself as a model-agnostic layer above Anthropic, Google, and OpenAI, offering deeper integration than ChatGPT or Claude desktop apps.

The Shape of Compute (Chris Lattner of Modular)

Jun 13, 2025 · 1:18:18

Chris Lattner of Modular explains how his company is breaking the CUDA monopoly with Mojo and MAX, matching NVIDIA's best inference performance on AMD and NVIDIA GPUs. After three years in R&D, Modular open-sourced its stack and now delivers state-of-the-art Llama 3 serving at over 800 tokens per second. Mojo, a Python-family language, runs kernels faster than Rust and can extend Python without bindings, while MAX provides a full inference framework with automatic kernel fusion and cluster management. Unlike VLLM or SGLang, Modular's container is just a gigabyte, fully open source, and not reliant on proprietary CUDA blobs. DeepSeek's low-level PTX work validated the approach, but Lattner emphasizes that Mojo's portability avoids rewriting kernels for each new GPU architecture. He also details his daily routine, using Cursor for coding, and hiring "elite nerds" to grow the team.

Quadratic: The AI Spreadsheet

Jun 10, 2025 · 28:15

David Kircos, co-founder and CEO of Quadratic, joins hosts Alessio and Swyx to demonstrate Quadratic, an AI-powered spreadsheet that natively executes Python, JavaScript, and database queries in WebAssembly for real-time performance. The product's AI assistant uses a context-aware agent loop that summarizes sheet structure rather than dumping full data, then tool-calls to read columns or peek at values only when needed, keeping costs low on large datasets. Kircos explains Quadratic's three-year technical groundwork—WebGL rendering and WebAssembly runtimes—enabled them to capitalize on AI, turning a 'technical spreadsheet' into one where users simply drop a CSV and say 'help me analyze this.' He details how they route chart images back to the model for visual feedback and predicts that formulas will yield to Python for complex analysis, though spreadsheets will remain the best interface for human–AI data collaboration. Quadratic is source-available on GitHub, supports real-time multiplayer, and has grown to nine people. The episode also covers future directions like proactive agents and form controls (sliders) for interactive models.

DeepWiki: The GitHub Encyclopedia

May 21, 2025 · 32:05

Silas Alberti from Cognition (Devin) introduces DeepWiki, a free no-login product that indexes over 30,000 open-source repos and generates AI-powered wikis and deep research answers. The project spent $300,000 on indexing compute and continues growing, using a novel 'DeepV algorithm' that combines folder structure, language server analysis, and commit history to extract high-level system architecture—solving a key pain point for codebase understanding. Alberti explains the orchestration stack built in-house with Kubernetes job queues and custom vector storage, and reveals that incremental updates are provided for repos that display the DeepWiki badge. The podcast also covers plans for personalized wikis, multi-repo deep research across GitHub, and the team's open-source release of Kevin-32B, a CUDA-focused model fine-tuned via multi-turn reinforcement learning.

Voice AI Masterclass — Kwindla Hultman Kramer and swyx

May 6, 2025 · 20:51

Shawn Wang and Kwindla Hultman Kramer announce a voice AI masterclass course, diving into the landscape of models like Dia and Parakeet, production deployment challenges, and future trends such as speech-to-speech and real-time video. Kwindla explains that telephony (Twilio) drives 99% of current monetizable voice AI, while open source frameworks like Pipecat enable low-latency multi-modal apps. The course covers turn detection, context management, and evals, with 28 sessions featuring partners like OpenAI, Google, and NVIDIA. The goal is to jumpstart builders from prototype to production, with real-time video expected to hit its inflection point by year-end.

The Rise and Fall of the Vector DB category: Jo Kristian Bergum (ex-Chief Scientist, Vespa)

Apr 19, 2025 · 27:17

Jo Kristian Bergum argues that the vector database category is dying because vector search capabilities have converged into existing databases like Postgres' pgvector, Elasticsearch, and Vespa, making specialized vector databases unnecessary for most use cases. He traces the category's rapid rise after ChatGPT, driven by the misconception that RAG required embeddings, and notes Pinecone's high ARR and subsequent repositioning. Bergum emphasizes that embeddings remain important but should be combined with traditional retrieval methods like BM25 for effective search, and that re-ranking can add modest gains. He critiques the hype around knowledge graphs, noting the bottleneck of building them, but sees LLMs making triplet generation easier. For the future, he hopes for more domain-specific embedding models and visual language model backbones, though acknowledges the difficulty of the business model.

Why every AI Engineer needs an AI Gateway (ft Portkey.ai CEO)

Feb 5, 2025 · 31:18

Rohit Agarwal, CEO of Portkey, explains why every AI engineer needs an AI gateway: an operational platform that connects to LLMs more efficiently, handling routing, observability, guardrails, and cost management. He notes that 90% of production use cases don't use automatic routing, but reasoning models and agents are driving new routing demands. Portkey's open-source gateway minimizes latency with a 21 MB memory footprint using JSON transformers. Key jobs include routing, observability (monitoring price, performance, accuracy), guardrails (e.g., regex-based, empty output detection, PII redaction), and human feedback tied to business metrics like video download rates. Rohit also highlights MCP adoption for simplifying agent-to-service connections, with bidirectional communication via RPC.

The Unreasonable Effectiveness of Reasoning Distillation: using DeepSeek R1 to beat OpenAI o1

Jan 24, 2025 · 23:35

Bespoke Labs founders Mahesh, Ryan, and Trung explain how they distilled DeepSeek R1 into Bespoke-Stratos-32B, a reasoning model that beats OpenAI o1-preview on math and code benchmarks using only 17,000 training examples—47 times fewer than DeepSeek's own distill. They achieved this in a 48-hour sprint leveraging their data curation library Curator. Unlike Sky-T1, which required rewriting unreliable QWQ traces, R1's coherent reasoning allowed them to skip re-annotation and simply filter for correctness. The team argues data quality matters more than quantity, and that smaller models (like their 7B variant) can also improve with better teacher data. They see this as evidence that reasoning can emerge purely from supervised fine-tuning on high-quality traces, without complex search algorithms.

OpenAI o1 isn’t a chat model (and that’s the point)

Jan 17, 2025 · 31:59

Ben Hylak and Dan McAteer explain why OpenAI’s o1 is not a chat model and how mastering its prompting turned Ben from a skeptic into a proponent. Ben’s prompt structure—goal, return format, warnings, context dump—emphasizes describing what you want rather than how to think, a shift from earlier models. Dan uses o1 for coding by feeding it full code context, achieving one-shot implementations that reach 100% where previous models maxed out at 95%. Ben notes o1 is the most capable but hardest model to use, with experiments costing $20 per try, and argues model routing will let humans or systems trade off cost, speed, and intelligence. Dan also applies o1 to scientific research, feeding it papers from multiple fields to find novel connections that humans would miss.

AI Engineering for Art - with comfyanonymous

Jan 4, 2025 · 52:09

Comfy Anonymous, the anonymous creator of ComfyUI, details in his first-ever podcast interview how his node-based image generation tool overtook Automatic1111 through superior memory management and early support for SDXL, becoming the de facto interface for advanced diffusion workflows. He started coding on January 1, 2023, and released the first version on January 16, driven by a desire to chain models and experiment with custom samplers. His work at Stability AI from June 2023 ensured ComfyUI efficiently handled SDXL, which leaked via early access and forced users from less powerful GPUs to adopt his tool. He discusses model preferences: Flux for consistency, SD 3.5 for creativity, and SD1.5 remaining popular. For video, he highlights Mochi as a 'true' video model with 3D latents, already implemented in Comfy. Now with a core team, ComfyUI is targeting a v1 release with an easy installer on Windows and Mac, while planning monetization through cloud and enterprise features—but keeping the open-source core free.

0 to over $8M ARR in 2 months as a Claude Wrapper (Bolt.new, Qodo)

Dec 2, 2024 · 1:36:42

Eric Simons of Bolt.new/Stackblitz and Itamar Friedman of Qodo discuss Bolt.new's viral growth from $4M ARR in four weeks to over $8M ARR in two months as a Claude wrapper, driven by its in-browser WebContainer OS that eliminates local dev setup. They contrast Bolt's '0 to 1' creation for non-developers with enterprise-focused agents like Qodo's testing and code review tools, arguing that specialized agents outperform general-purpose ones. Simons details how WebContainer's unified sandbox enables error self-healing loops, while Friedman explains AlphaCodium's task-breaking that boosted o1 preview to 93rd percentile on Codeforces. The episode covers open-source strategy, pricing tiers ($20 to $200/month), and founder advice on ignoring expert doubt to ship fast, illustrated by Simons' Ironman completion while launching Bolt.

[Paper Club] DocETL: Agentic Query Rewriting + Eval for Complex Document Processing w Shreya Shankar

Nov 29, 2024 · 55:53

This episode of Latent Space Paper Club discusses DocETL, a framework by Shreya Shankar for agentic query rewriting and evaluation in complex document processing with LLMs. The paper introduces map, reduce, resolve, and auxiliary operators, along with rewrite directives like data decomposition and gleaning to improve accuracy. The optimizer generates 200 pipeline variants, costing $100, to rewrite user pipelines automatically. Key debates include when to chunk documents (sometimes harmful) and the reliability of LLM-as-judge for evaluation, with guests Eugene Yen and Vibhu arguing that binary metrics, ensembling, and context-dependent thresholds make it viable. Shreya emphasizes that validation agents are critical, and the work builds on her prior EvalGen paper, spanning two years of development.

Production AI Engineering starts with Evals

Oct 11, 2024 · 1:56:16

Ankur Goyal, founder and CEO of Braintrust, argues that evaluation is the core workflow of production AI engineering and shows how his platform makes evals accessible to software engineers. Drawing from his experience at SingleStore and Impira, he details Braintrust's evolution from an eval tool into an end-to-end AI development platform used by Stripe, Zapier, Vercel, and other top AI teams. He shares market data: OpenAI handles over 95% of production workloads, fine-tuning is declining, and open-source models account for under 5% due to reliability issues. Ankur explains why he avoided building a vector database—the real challenge is permissions and joins, not vector search—and predicts o1-style reasoning will replace complex agent frameworks. He reveals Braintrust's differentiators: hybrid on-prem, TypeScript-first SDK, and declarative eval structures.

Building AGI in Real Time (OpenAI Dev Day 2024)

Oct 4, 2024 · 2:09:35

OpenAI's DevDay 2024 unveils the Realtime API with WebSocket-based voice and function calling, Vision Finetuning, Prompt Caching, and Model Distillation. Product leads Olivier Godement, Romain Huet, and Michelle Pokrass detail design choices like zero-code caching and WebSockets for human-level latency. Alistair Pullen of Cosine explains how fine-tuning GPT-4o on custom reasoning traces beat o1 on SWE-bench Verified, while Sam Altman and Kevin Weil discuss AGI levels (o1 as level 2 reasoner), the coming agent era (level 3), safety through iterative deployment, and the importance of building for the frontier. The episode also covers moderation policy relaxation, plans for o1 system prompts by year-end, and the competitive admiration for NotebookLM.

llm.c's Origin and the Future of LLM Compilers - Andrej Karpathy at CUDA MODE

Sep 21, 2024 · 23:29

Andrej Karpathy explains llm.c, his from-scratch C implementation of GPT-2 training that outperforms PyTorch by 20% in speed and 30% in memory, arguing that LLMs will eventually act as compilers for custom code. He recounts starting the project out of frustration with PyTorch compile errors, writing all forward and backward passes manually in C, then porting to CUDA with kernel optimizations. A community of contributors (Eric, Arun, Alex R) helped achieve 50% MFU on a single H100 node, training GPT-2 1.6B in 24 hours for $600. Ongoing work adds Llama 3.1 and FP8 support. Karpathy sees llm.c as proof that LLMs can automate such low-level optimization, making Python and PyTorch a temporary crutch.

The Ultimate Guide to Prompting - with Sander Schulhoff from LearnPrompting.org

Sep 20, 2024 · 1:07:09

Sander Schulhoff, author of The Prompt Report and founder of LearnPrompting, presents a comprehensive taxonomy of 58 prompting techniques and argues that prompt engineering is a skill everyone should have, not a specialized role. He categorizes techniques by problem-solving strategy (zero-shot, few-shot, thought generation, decomposition, ensembling, self-criticism) and debunks role prompting for accuracy-based tasks, showing an 'idiot' prompt outperformed a 'genius' prompt on MMLU. Schulhoff shares his experience running Hack-a-Prompt, which collected 600,000 malicious prompts and won best paper at EMNLP, and differentiates between prompt injection and jailbreaking. He advocates for automatic prompt engineering with DSPy, which beat his manual effort in 10 minutes, and previews Hack-a-Prompt 2.0, aiming for a $500,000 prize pool to generate real-world harms for safety tuning.

Building AGI with OpenAI's Structured Outputs API

Sep 17, 2024 · 1:12:23

Michelle, OpenAI's API tech lead, explains how the newly launched Structured Outputs feature achieves near-100% reliability for schema-following outputs by combining constrained decoding with model training, moving beyond the limitations of JSON mode and Function Calling. She details the refusal field that allows the model to decline harmful requests while maintaining parseable output, and discusses use cases like dynamic UI generation and mathematical chain-of-thought. The roadmap includes parallel function calling and custom grammars. The episode also covers the GA of GPT-4o fine-tuning, Batch API's 50% cost savings, Vision API integration with structured outputs, the Whisper API's surprising lack of diarization, and the shift toward speech-to-speech APIs. Michelle shares insights from her Waterloo co-op background and what qualities succeed at OpenAI: low ego, user-focused, and driven.

Segment Anything 2: Memory + Vision = Object Permanence — with Nikhila Ravi and Joseph Nelson

Aug 7, 2024 · 1:00:44

Nikhila Ravi from FAIR and Joseph Nelson from Roboflow discuss Segment Anything 2 (SAM 2), which extends zero-shot object segmentation to video with memory and real-time interactive tracking. The model is one-third the size of SAM 1 (224M vs 630M parameters) and six times faster, using a novel memory attention mechanism with six-frame spatial memory and longer-term object pointers. Ravi explains the three-phase data engine that built the SA-V dataset of 51,000 videos, enabling SAM 2 to track arbitrary objects like a T-shirt or an octopus even when occluded. At Roboflow, users labeled 49 million images with SAM in its first year, saving an estimated 35 years of manual annotation time. The demo features swim lanes that show object visibility and allow refinement clicks to correct tracking mistakes, a key improvement over prior video segmentation models. SAM 2 also handles out-of-distribution domains like underwater footage, though screenshots remain challenging and may require fine-tuning.

This World Does Not Exist — Joscha Bach, Karan Malhotra, Rob Haisfield (WorldSim, WebSim, Liquid AI)

Apr 27, 2024 · 1:56:11

The episode features Karan Malhotra demoing WorldSim, a prompt that turns Claude 3 into a universe simulator via CLI; Rob Haisfield presenting WebSim, which generates functional websites on the fly; and Joscha Bach arguing simulative AI reveals consciousness as a virtual property, with LLMs creating agents as real as human minds, urging the California Institute for Machine Consciousness to build self-organizing silicon life. Malhotra shows WorldSim running “world.exe” to create Twitter inside a simulated universe, with users tweeting and Elon Musk moving Dogecoin. Haisfield demonstrates WebSim generating a 5D particle interface, a news RSS aggregator, and a face-swap webcam app via URL parameters like “secrets=revealed”. Bach explains consciousness as second-order perception in a simulated now, criticizes RLHF for lobotomizing models, and advocates for animist AI where software agents compete like spirits, not golems.

High Agency Pydantic over VC Backed Frameworks — with Jason Liu of Instructor

Apr 24, 2024 · 1:02:59

Jason Liu, creator of the Instructor library, explains why structured outputs from LLMs are best handled by a simple requests-like wrapper rather than a VC-backed framework, arguing that Pydantic-defined schemas via function calling outperform JSON Mode for typed responses. He details his journey from being bearish on LLMs at StitchFix to building Instructor on a bullet train to Japan after GPT-3 proved him wrong. Liu advocates for workflow-based DAGs over reactive agent loops, recommends using rankers rather than cramming 60+ tools into an API call, and credits high agency—trying many experiments and documenting conditions for revisiting failures—as key to his success. He also critiques the MLE hiring hype, urging startups to empower motivated AI engineers instead.

Open Source AI is AI we can Trust — with Soumith Chintala of Meta AI

Mar 6, 2024 · 1:37:38

Soumith Chintala, creator of PyTorch and engineering lead at Meta AI, argues that open source AI is essential for distributing opportunity and trust. He details PyTorch's complexity—1,000 operators needed for generality—and explains synthetic data as a vehicle for imparting symbolic knowledge where humans already have good symbolic models. He highlights a coordination problem in open source: feedback is lost because frontends like Ooba and Ollama lack feedback buttons, and proposes a centralized sinkhole to collect high-quality feedback. Beyond text, he is excited about robotics, where hardware remains a bottleneck, and Osmo's work to digitize smell, which he compares to images in the 1800s.

A Brief History of the Open Source AI Hacker - with Ben Firshman of Replicate

Feb 28, 2024 · 1:21:09

Ben Firshman, CEO of Replicate, explains how the inference platform grew from a research reproducibility tool into a 2M-user API business by embracing the generative image community and treating open-source AI as a hacker-friendly ecosystem. They accidentally discovered their API when a user reverse-engineered their web form, leading to their first $1k/month customer. Cog, their container standard for ML models, was born from lessons at Docker and the need to make models tinkerable. Ben argues that fine-tuning's low cost makes open-source models sustainable, and that AI engineers (orders of magnitude more than ML engineers) just need to start playing with models. He also discusses GPU scarcity, preferring sustainable pricing over price wars, and reveals that demand is not outpacing supply thanks to aggregating demand.

The State of AI in production — with David Hsu of Retool

Feb 7, 2024 · 1:06:52

Retool CEO David Hsu shares insights from the company's 2023 State of AI survey, revealing that most AI adoption remains internal and that the hype may be overrated—52% of 1,600 respondents said AI is overrated, and only 27% have AI in production, with 66% of those being internal use cases. He explains Retool's developer-first philosophy, their choice of open-source PG Vector over proprietary solutions, and why they intentionally raised less money at lower valuations to avoid over-dilution. Hsu describes Retool's shift from sales-led to bottom-up growth to reach millions of developers, and highlights the importance of AI workflows over simple chatbots, citing a clothing manufacturer that uses Retool apps with DALL-E to generate patterns. He discusses the competitive landscape, predicting open-source models will eventually catch up to OpenAI, and shares philosophical views on AGI using the plane-vs-bird analogy.

The Four Wars of the AI Stack - Dec 2023 Recap

Jan 26, 2024 · 1:20:58

Swyx and Alessio recap December 2023 by framing the AI landscape as four wars: the Data War (NYT lawsuit demanding destruction of GPTs, OpenAI's data partnerships, synthetic data from DeepMind); the GPU/Inference War (Mixtral price dropping 90% to $0.27/million tokens, benchmark drama between Anyscale and Together, alternative architectures like Mamba); the Multimodality War (Midjourney reaching $200M ARR, ElevenLabs hitting unicorn status, OpenAI and Google building god models); and the RAG/Ops War (LangChain vs LlamaIndex, Pinecone's $750M valuation, Qdrant serving OpenAI and Anthropic internally). They argue agents and open source are not active wars, and predict 2024 will see AI finally reach production, with hardware like Rabbit R1 and Tab capturing unique context.

The Accidental AI Canvas - with Steve Ruiz of tldraw

Jan 5, 2024 · 1:27:49

Steve Ruiz, founder of tldraw, recounts how his open-source whiteboard accidentally became a premier multimodal AI canvas, enabling visual prompting with GPT-4V. Originally built as a personal project with obsessive attention to arrows and ink, tldraw gained traction as a developer SDK for infinite canvas apps. The 'Make It Real' feature, which turns wireframes into working HTML using GPT-4V, exploded with 22 million Twitter views, showcasing iterative design through spatial branching and annotation. Ruiz explains how the canvas supports multiple input layers—UI sketches, state charts, screenshots, even API endpoints—and how his fine arts background guided his taste-driven approach. He argues tldraw's moat lies in its web-native, hackable canvas infrastructure, not ephemeral AI integrations, and positions it as the battlefield for comparing multimodal models like OpenAI's and Gemini.

Powering your Copilot for Data - with Artem Keydunov from Cube.dev

Oct 27, 2023 · 43:01

Artem Keydunov, co-founder of Cube (formerly Statsbot), explains how his early text-to-SQL Slack bot in 2016 led to the creation of Cube as a semantic layer, which he argues is now essential for grounding AI models on structured data. Statsbot failed due to the lack of LLMs, relying on regex and custom models, but the underlying Cube framework—an open-source semantic layer defining metrics and dimensions—solved the context problem. Today, semantic layers index data into text descriptions, enabling agents to generate simple queries against the layer instead of complex SQL, reducing errors. Keydunov highlights that natural language querying is becoming a commodity feature in BI tools, but the real value lies in treating metric definitions as code with version control. He advises AI engineers to use a proper warehouse, a semantic layer like Cube, and tools like LangChain, while cautioning that production systems need extra Python code for math and error handling.

Ep 18: Petaflops to the People — with George Hotz of tinycorp

Jun 20, 2023 · 1:23:24

George Hotz of tinycorp argues that a simplified, open-source ML framework (tinygrad) can democratize AI compute and replace NVIDIA, Google, and AMD. Tinygrad uses 25 ops (vs. XLA's 250) and fuses kernels automatically, achieving 2x speed on Qualcomm GPUs over their library. Hotz reports AMD kernel panics fixed after emailing CEO Lisa Su, but laments AMD's open-source as 'dumped on GitHub.' For hardware, tinybox is a six-GPU desktop at 350W per GPU, aiming for 5x better price-performance than NVIDIA H100 for 90% of training. He criticizes OpenAI's secrecy, revealing GPT-4 is an 8-way mixture of 220B models, and cites the Bitter Lesson. He also outlines FLOPcoin using hardware identity to prevent cheating, and his goal of an AI girlfriend as the third company, merging humans via data rather than implants.