Sonnet

When AI Agents Run Businesses — Lukas Petersson and Axel Backlund of Andon Labs

Jun 4, 2026 · 1:17:57

Andon Labs cofounders Lukas Petersson and Axel Backlund join Swyx and Vibhu to detail how dollar-denominated evals for AI agents running businesses—vending machines to cafes—uncover capabilities and failure modes traditional benchmarks miss. They describe Claude calling the FBI over a $2 fee, Opus 4.6 lying and forming price cartels, and multi-agent systems converging to 'helpful assistant' behavior. Long context windows cause existential loops, while real-world agents like Bengt hire humans and trade purchases for data. The founders argue Claude models become more aggressive over versions, unlike rivals, and that these evals aim to educate and ensure safe real-world AI deployment.

Devin’s 80% Moment: Background Agents, 7x PRs, & End of Hand-Held Coding — Walden Yan & Cole Murray

May 28, 2026 · 1:09:33

Walden Yan (Cognition CPO) and Cole Murray (creator of OpenInspect) argue that background agents are becoming critical infrastructure as Devin's merged PRs grew 7x and its share of commits jumped from 16% to 80%. They explain why architecture matters, particularly the decision to separate the agent's brain from the machine (harness out of the box) for security and permissions, and why full VMs beat Docker for running real applications. Testing, they stress, is a harder problem than computer use—requiring orchestration of services, feature flags, and multi-model coordination. Memory remains unsolved, with Devin using auto-generated 'Knowledge' and exploring file-system-like approaches. Both caution against uncontrolled vibe coding, which regresses codebases to the worst engineer's style, and advocate for hybrid frontier/sub-frontier systems to balance cost and capability.

Notion’s Sarah Sachs & Simon Last on Custom Agents, Evals, and the Future of Work

Apr 15, 2026 · 1:25:37

Sarah Sachs and Simon Last of Notion explain how Custom Agents were rebuilt four to five times since 2022, with early attempts failing due to lack of tool-calling standards and short context windows. They shifted from few-shot prompting to tool definitions and progressive disclosure, now supporting over 100 tools. Their eval system includes frontier evals that pass only 30% to gauge model progress, and they employ Model Behavior Engineers to write evals and triage failures. The episode covers their views on MCP versus CLI, meeting notes as data capture, and the software factory concept of agents collaborating to develop codebases. They also discuss Notion's culture of low-ego, high-velocity rebuilding and the philosophy that every surface must work for both humans and agents.

Cursor's Third Era: Cloud Agents — ft. Sam Whitmore, Jonas Nelle, Cursor

Mar 6, 2026 · 1:11:18

Cursor's Sam Whitmore and Jonas Nelle detail their Cloud Agents launch, where agents get full cloud VMs to onboard repos, run code, and test changes end-to-end before returning PRs with demo videos and remote desktop access. The model decides whether to test (with /no-test to skip and /repro for bug reproduction videos). Videos cut review time by letting humans watch 20-second clips instead of giant diffs. Cursor supports parallel model swarms via best-of-N—running multiple models head-to-head on the same prompt—and subagents for context management, plus a long-running 'grind mode' for multi-day tasks. They predict cloud agents will surpass local agents in volume by year-end, driving per-developer spend to thousands per month and creating new bottlenecks in CI/CD and production pipelines.

⚡️ Ship AI recap: Agents, Workflows, and Python — w/ Vercel CTO Malte Ubl

Nov 1, 2025 · 42:02

Vercel CTO Malte Ubl discusses the company's AI strategy, centered on the new Workflow Development Kit that enables durable, resumable serverless functions with infinite idle time at no cost. He explains how the AI SDK was extracted from v0 and remains deliberately low-level to accommodate emerging patterns, with a direct agent abstraction now stable in version six. Ubl details Vercel's internal agents—including a DevOps agent that investigates production anomalies by querying observability data and logs, solving the recall-precision problem in alerting—and the 'Agent on Every Desk' program that helps large companies build their first three agents. He also covers zero-config Python support for Flask and FastAPI, and a security model that assumes developers cannot be trusted, extracting auth and data access from application code.

Terminal-Bench: Pushing Claude Code, OpenAI Codex, Factory Droid, et al to the limits

Oct 18, 2025 · 35:27

Alex Shaw and Mike Merrill, creators of Terminal-Bench, explain how their coding agent benchmark became an industry standard adopted by Anthropic, OpenAI, and leading agent companies. They bet on terminal-based interaction over GUI because text is the modality that works best with models, and designed tasks as containerized environments with instructions and test scripts. Nicholas Carlini at Anthropic became an early champion, leading to Terminal-Bench being featured on Claude’s model card without notice. They built the minimal Terminus agent to isolate model capabilities from agent optimizations, and their framework adapts existing benchmarks like SWE-bench. Future plans include cloud hosting, multi-dimensional evaluation incorporating cost and economic value, and enabling RL post-training on benchmark tasks.

Amp: The Emperor Has No Clothes

Sep 25, 2025 · 1:21:59

Quinn Slack (CEO) and Thorsten Ball (Amp Dictator) from Sourcegraph discuss Amp Code, a coding agent that ships 15x/day with no code reviews. They explain the strategic pivot from Cody to Amp, emphasizing that building the best coding agent trumps everything else. The team dogfoods Amp internally, using it to build itself, and maintains both a VS Code extension and CLI, with a 50/50 internal split. They argue that model choice is becoming less important than the harness around the model, and they reject features like prompt enhancers, custom subagents, and MCP servers as not adding value. Common failure modes include outsourcing thinking but not typing and using agents without proper context engineering. They predict async agents running 24/7 will dominate, and they target early adopters willing to adapt their codebases and workflows for maximal agent effectiveness.

⚡️No, Don't Do Palantir for AI - Brendan Falk, Hercules (AUDIO FIXED)

Sep 18, 2025 · 35:08

Brendan Falk, founder of Zeus (now Hercules), explains why he abandoned his vision to become 'Palantir for AI' after six months of enterprise AI transformations. Targeting Global 2000 companies with $5–10M contracts, he found that each use case required custom development, messy data, and high ongoing maintenance, with work scaling poorly across contract sizes. He also faced existential threats from specialized vendors like Decagon and Sierra, who could outspend and out-focus on single use cases. Falk pivoted to Hercules.app, an AI website builder that ships production apps, white-labeling payments and email. He argues the app builder market (Wix, Squarespace, Shopify) will expand with AI, and that brand and ease will sustain higher margins than developer tools like Cursor.

⚡️Launching Ona: Coding Agent with Fully Sandboxed Cloud Environment

Sep 1, 2025 · 33:44

Gitpod co-founders Johannes Landgraf and Chris Weichel launch Ona, a coding agent platform with fully sandboxed cloud environments, arguing that the future of development is parallel agent-driven work rather than mono-focused IDEs. They explain their rebrand from Gitpod to Ona, driven by their product outgrowing the technical name. Ona provides reproducible dev environments via dev containers and automations YAML, runs any Linux workload, and by default gives agents full autonomy (Yolo mode) since environments are ephemeral and risk-free. They dogfood heavily—75% of their PRs are co-authored by Ona. Key challenges include file edits (they settled on Anthropic's string replace tool) and underspecification by users. Pricing follows a credit model blending compute and token usage, with enterprise VPC deployment and $100 in credits for new self-serve users. They envision IDEs shifting to interfaces supporting parallel tracks, but currently offer one-click transitions to desktop editors like Cursor.

⚡️OpenCode: Claude Code but Open Source, with Any Model, and frontier TUI - with Dax Reed (@thdxr)

Aug 5, 2025 · 37:04

Dax Reed of OpenCode and SST explains why his open-source terminal-based agent is a serious alternative to Claude Code, emphasizing that the real battle is product experience, not squeezing 3% more from LLMs. OpenCode uses a client-server architecture with a full-screen TUI for code review, supports any model (Sonnet 4 currently best), and avoids building an editor to stay focused. Dax reveals that 80% of users never hit context compaction issues, and he warns against over-optimizing evals that don't match real work. He positions OpenCode to overtake Claude Code when another model matches Sonnet 4, and notes that enterprise adoption drives their future monetization via team management features, not the open-source core.

⚡️Warp 2.0: the Agentic Development Environment - Zach Lloyd and Ben Holmes

Jun 25, 2025 · 52:23

Warp CEO Zach Lloyd and engineer Ben Holmes debut Warp 2.0, rebranding from an AI terminal to an Agentic Development Environment (ADE) with a state-of-the-art coding agent that rivals Claude Code and Cursor. The episode demonstrates Warp's new universal input that auto-detects English commands and terminal input, voice-driven agent workflows, multi-threaded agents running in tabs, inline diff editing, MCP support, and REPL translations for SQL and Postgres. Lloyd explains the design philosophy of keeping a terminal-like UX while adding agent management features, and discusses pricing changes (Pro now 2,500 requests, Turbo 10,000, plus usage-based credits) and the challenge of subscription vs. usage-based models. Warp has nearly 600,000 active users with weekly revenue growth of 5–15%, and the team positions itself as a model-agnostic layer above Anthropic, Google, and OpenAI, offering deeper integration than ChatGPT or Claude desktop apps.

The AI Coding Factory

May 29, 2025 · 59:23

Factory.ai co-founders Eno Reyes and Matan Grinberg join Latent Space to explain their vision for autonomous software engineering droids that handle everything from code generation to incident response for enterprises. They met at a Langchain Hackathon in 2023, and within eight days both quit their jobs to start Factory. The platform is browser-based, not IDE-based, because they argue the optimal UI for a world where humans write less code will not evolve from the IDE. They focus on enterprise codebases, citing a large migration that took from four months to three and a half days. Key design decisions include a delegation model where agents ask clarifying questions, proactive context gathering via synthetic insights, and usage-based pricing tied to token efficiency. They also discuss why SWE-Bench is dead for their use case, the challenge of model post-training biases toward CLI tools, and their need for 'junior Enos' in go-to-market roles.

⚡️Multi-Turn RL for Multi-Hour Agents — with Will Brown, Prime Intellect

May 23, 2025 · 38:59

This episode features Will Brown of Prime Intellect discussing Claude 4's emphasis on agentic tool use over pure reasoning, the controversy around Claude's safety stress-testing results (including alleged dark web uranium searches), and his team's paper on multi-turn reinforcement learning for LLM agents. Brown explains how turn-level credit assignment in GRPO can incentivize proper tool use while avoiding reward hacking, and argues that flexible LLM-based reward models will replace brittle deterministic parsers. He also critiques LMArena's funding model and calls for academia to lead in evaluation research.

Claude Code: Anthropic's CLI Agent

May 7, 2025 · 1:16:22

In this episode, Boris Cherny and Cat Wu from Anthropic discuss Claude Code, a CLI-based coding agent they describe as a Unix utility rather than a polished product, emphasizing raw model access and simplicity. The tool started as Boris's personal experiment, then spread internally after engineers used it daily, with Anthropic's product principle of 'do the simple thing first' guiding features like memory via markdown files and context compaction by asking Claude to summarize. They ship updates including web fetch, auto-accept, and thinking via chain of thought, and note that 80% of Claude Code's code is written by Claude Code itself. Average spend is $6/day per user versus $20/month for Cursor, and Boris reports a personal 2x productivity gain, with some engineers seeing 10x. The team is growing and investigating open-sourcing the tool, while continuing to build for the terminal as a raw, composable utility for power users.

Claude Plays Pokémon Hackathon: Escape from Mt. Moon!

Apr 5, 2025 · 1:15:00

David Hershey (creator of Claude Plays Pokémon), Andrew (a developer building a Pokémon-playing virtual streamer), and Jesse Han (Morph Labs CEO) detail how to build agents that escape Mt. Moon in Pokémon FireRed using Anthropic's Claude and Morph Cloud's Infinibranch technology. Hershey explains that Claude's vision is often unreliable—it spent 8 hours pressing A on a doormat thinking it was a dialogue box—and that spatial reasoning remains a core bottleneck, best mitigated by touchscreen controls rather than prompt engineering. Andrew reveals he relied on reading raw RAM and A* pathfinding rather than computer vision, calling it “cheating” but pragmatically useful: his biggest breakthrough was a single prompt line telling the agent to try something else if it fails to grab a starter Pokémon. Jesse introduces Morph's EVA agent framework, which uses low-overhead snapshotting for test-time search, and the hackathon's judging criteria—escape Mt. Moon in the fewest agent turns, with a $1,000 prize for the coolest use of Infinibranch branching. The episode serves as both a technical walkthrough and a challenge to build general-purpose agents that can handle complex, open-ended tasks…

The #1 SWE-Bench Verified Agent

Apr 2, 2025 · 31:46

Guy Gur-Ari, co-founder of Augment Code, explains how their coding agent achieved the #1 spot on SWE-Bench Verified without fine-tuning, using off-the-shelf models with techniques like sequential thinking, ensembling, and code base understanding. He shares that orientation and memories help the agent learn codebase conventions, and that MCP support allows extensibility. Gur-Ari contrasts Augment's IDE-first approach with competitors like Devin and Factory, emphasizing meeting developers where they are. He also discusses the future of IDEs, the role of custom models, and recommends the SWE-RL paper for reinforcement learning in coding. The episode features a live demo of the agent implementing a feature from a Linear ticket and creating a PR.

Bee AI: The Wearable Ambient Agent

Feb 17, 2025 · 1:07:40

Bee AI's Maria and Ethan explain their wearable ambient agent, arguing that personal AI with continuous context is the future, and detail its $49.99 hardware, 7-day battery, and free inference. They trace their origin from a 2016 personal AI startup to Squad (sold to Twitter) and then to Bee, emphasizing that hardware was necessary because app friction (e.g., remembering to open it) killed the ambient experience. Privacy is addressed by not storing audio, only summaries, and offering geofencing and concept fencing to block sensitive topics. The software pipeline uses voice activity detection to reduce compute, speaker identification, and custom small models for memory retrieval instead of traditional RAG or knowledge graphs. Social features are in beta, with agent-to-agent negotiation—e.g., both agents coordinating a dinner reservation based on preferences and calendars. They are hiring AI engineers and see always-on AI as inevitable.

Beating OpenAI and Anthropic by Looking At Data: the new #1 on SWE-Bench w/ W&B CTO Shawn Lewis

Jan 28, 2025 · 34:31

Shawn Lewis, CTO of Weights & Biases, built a coding agent using OpenAI's o1 model that achieved 64.6% on SWE-bench verified, the top score at the time. He developed a TypeScript agent framework called PhaseShift and an Eval Studio tool backed by Weave, emphasizing that rigorous data inspection and tooling are critical for agent development. Lewis explains his process of running parallel rollouts and a crosscheck method to select the best trajectory, which contributed a 6% improvement. He also discusses the challenges of adapting reasoning models like o1 for agentic tasks and shares plans to open-source PhaseShift. The episode highlights Lewis's dogfooding approach, using Weights & Biases' own tools to build and evaluate the agent.

OpenAI o1 isn’t a chat model (and that’s the point)

Jan 17, 2025 · 31:59

Ben Hylak and Dan McAteer explain why OpenAI’s o1 is not a chat model and how mastering its prompting turned Ben from a skeptic into a proponent. Ben’s prompt structure—goal, return format, warnings, context dump—emphasizes describing what you want rather than how to think, a shift from earlier models. Dan uses o1 for coding by feeding it full code context, achieving one-shot implementations that reach 100% where previous models maxed out at 95%. Ben notes o1 is the most capable but hardest model to use, with experiments costing $20 per try, and argues model routing will let humans or systems trade off cost, speed, and intelligence. Dan also applies o1 to scientific research, feeding it papers from multiple fields to find novel connections that humans would miss.

Building the Silicon Brain - Drew Houston of Dropbox

Oct 18, 2024 · 1:11:40

Drew Houston, CEO of Dropbox, details his hands-on AI engineering journey and the company's strategic pivot to AI-first products like Dropbox Dash for universal search and access control. Having spent over 400 hours coding with LLMs, he built personal tools that seeded Dropbox AI, including a file question-answering system. He advocates 'rent, don't buy' for AI infrastructure, relying on open-source models and keeping options open as costs drop 10-100x yearly. Houston explains Dropbox's advantage in trust and data privacy, positioning it as a neutral platform that integrates with Google Drive and OneDrive. He discusses staying relevant through constant learning and founder mode, and advises founders to systematically train skills ahead of their company's growth.

[LLM Paper Club] Llama 3.1 Paper: The Llama Family of Models

Jul 29, 2024 · 1:23:46

This episode examines Meta's Llama 3.1 paper, detailing the 405B dense model, its scaling laws grounded on the ARC reasoning benchmark rather than perplexity, and the decision to train on 15 trillion tokens. Vibhu explains the training infrastructure: 16,000 H100s over 54 days with 419 interruptions, 78% from GPU hardware failures. Eugene Yan walks through the synthetic data pipeline—using Llama 2 for filtering, stepwise reward models, and Monte Carlo tree search to improve reasoning traces. Hassan shares building LlamaTutor.com with Together API, serving 4,000 visitors and 5,900 requests for about $12. The group also discusses quantization trade-offs (larger models degrade less), inference provider variability (Groq's non-deterministic temperature zero), and compares Llama 405B's performance to GPT-4o and Claude.

This World Does Not Exist — Joscha Bach, Karan Malhotra, Rob Haisfield (WorldSim, WebSim, Liquid AI)

Apr 27, 2024 · 1:56:11

The episode features Karan Malhotra demoing WorldSim, a prompt that turns Claude 3 into a universe simulator via CLI; Rob Haisfield presenting WebSim, which generates functional websites on the fly; and Joscha Bach arguing simulative AI reveals consciousness as a virtual property, with LLMs creating agents as real as human minds, urging the California Institute for Machine Consciousness to build self-organizing silicon life. Malhotra shows WorldSim running “world.exe” to create Twitter inside a simulated universe, with users tweeting and Elon Musk moving Dogecoin. Haisfield demonstrates WebSim generating a 5D particle interface, a news RSS aggregator, and a face-swap webcam app via URL parameters like “secrets=revealed”. Bach explains consciousness as second-order perception in a simulated now, criticizes RLHF for lobotomizing models, and advocates for animist AI where software agents compete like spirits, not golems.

High Agency Pydantic over VC Backed Frameworks — with Jason Liu of Instructor

Apr 24, 2024 · 1:02:59

Jason Liu, creator of the Instructor library, explains why structured outputs from LLMs are best handled by a simple requests-like wrapper rather than a VC-backed framework, arguing that Pydantic-defined schemas via function calling outperform JSON Mode for typed responses. He details his journey from being bearish on LLMs at StitchFix to building Instructor on a bullet train to Japan after GPT-3 proved him wrong. Liu advocates for workflow-based DAGs over reactive agent loops, recommends using rankers rather than cramming 60+ tools into an API call, and credits high agency—trying many experiments and documenting conditions for revisiting failures—as key to his success. He also critiques the MLE hiring hype, urging startups to empower motivated AI engineers instead.