Notion

Notion’s Sarah Sachs & Simon Last on Custom Agents, Evals, and the Future of Work

Apr 15, 2026 · 1:25:37

Sarah Sachs and Simon Last of Notion explain how Custom Agents were rebuilt four to five times since 2022, with early attempts failing due to lack of tool-calling standards and short context windows. They shifted from few-shot prompting to tool definitions and progressive disclosure, now supporting over 100 tools. Their eval system includes frontier evals that pass only 30% to gauge model progress, and they employ Model Behavior Engineers to write evals and triage failures. The episode covers their views on MCP versus CLI, meeting notes as data capture, and the software factory concept of agents collaborating to develop codebases. They also discuss Notion's culture of low-ego, high-velocity rebuilding and the philosophy that every surface must work for both humans and agents.

Retrieval After RAG: Hybrid Search, Agents, and Database Design — Simon Eskildsen of Turbopuffer

Mar 12, 2026 · 1:00:32

Simon Eskildsen, founder of Turbopuffer, explains how his obsession with the cost of vector search at Readwise led him to build a search engine on object storage and NVMe SSDs, cutting costs by 95% for customers like Cursor. He outlines the three conditions for a major database company—a new workload (AI-driven search), a new architecture (object storage first, no consensus layer), and evolving to support every query plan. Turbopuffer's architecture uses S3's strong consistency and compare-and-swap for metadata, avoiding stateful systems. Eskildsen shares how they won Notion by absorbing latency issues with dark fiber, and discusses the shift from single RAG queries to agents making many concurrent searches. He also describes the "P99 engineer"—obsessive, trade-off minded, able to bend software to first principles. Future plans include expanding full-text search, reaching Common Crawl scale, and eventually adding simpler OLAP queries.

⚡️ Building the AI Hardware Engineer with Matthias Wagner, Co-founder of Flux

Nov 22, 2025 · 49:03

Matthias Wagner, CEO of Flux.ai, describes building the "AI hardware engineer" that turns product briefs into manufacturable PCB designs in under 30 minutes. After leaving Meta in 2019, Wagner built a browser-based collaborative CAD tool from scratch, designed as a reinforcement learning environment. The emergence of LLMs and reliable tool calling let Flux's agents search component libraries, check pricing and availability, and execute complex designs autonomously. In a live demo, Wagner designs a battery-powered Alexa-like device with ESP32, microphones, OLED display, and speaker in about 25 minutes. Flux now has 7,000 paying customers, growing 26x year over year, and Wagner envisions eventually letting users prompt full hardware products like smartphones into existence, claiming the cost of custom manufacturing is plummeting.

Production AI Engineering starts with Evals

Oct 11, 2024 · 1:56:16

Ankur Goyal, founder and CEO of Braintrust, argues that evaluation is the core workflow of production AI engineering and shows how his platform makes evals accessible to software engineers. Drawing from his experience at SingleStore and Impira, he details Braintrust's evolution from an eval tool into an end-to-end AI development platform used by Stripe, Zapier, Vercel, and other top AI teams. He shares market data: OpenAI handles over 95% of production workloads, fine-tuning is declining, and open-source models account for under 5% due to reliability issues. Ankur explains why he avoided building a vector database—the real challenge is permissions and joins, not vector search—and predicts o1-style reasoning will replace complex agent frameworks. He reveals Braintrust's differentiators: hybrid on-prem, TypeScript-first SDK, and declarative eval structures.