Perplexity

[State of Research Funding] Beyond NSF, Slingshots, Open Frontiers — Andy Konwinski, Laude Institute

Dec 31, 2025 · 22:34

Andy Konwinski, co-founder of Databricks and Perplexity, launches Laude Institute—a dual venture fund and nonprofit to accelerate the path from open research to breakout companies. At NeurIPS, he explains how Laude's Slingshot program funds projects like DSPy, Terminal Bench, and LMArena, and why the NSF's $1B/year for CS is insufficient, needing $10-100B for frontier AI. He argues Chinese labs (Moonshot, DeepSeek) now outpublish US because OpenAI and others stopped sharing; Open Frontiers, a live-streamed conference in SF, aims to unite top open researchers (Yann LeCun, François Chollet, Jan Leike) to reclaim global leadership. The episode also spotlights the emerging 'post-post-training' layer—prompt optimization, context management, RAG—and why research teams with multiple co-founders (like Databricks' eight) are the new gold standard for AI startups.

Terminal-Bench 2.0: the most impt coding agent benchmark of 2025 gets a v2! Launch + Q&A w/ founders

Nov 8, 2025 · 35:01

Terminal-Bench co-creators Mike Merrill, Alex Shaw, and collaborators Andy Konwinski, Ludwig Schmidt, and Aatash launch Terminal-Bench 2.0, an 89-task coding benchmark with rigorous human and LM-assisted verification to test agent performance on real-world tasks like designing DNA primers and installing Windows 3.11. They reflect on Terminal-Bench 1's problems—easy tasks and reproducibility—and detail spending 300 hours manually auditing each task. They introduce Harbor, an open-source package for running sandboxed agent rollouts, standardizing task format and integrating with agents like Codex and Claude Code. Aatash shows using Harbor to generate thousands of training traces for DataComp Agent, running 1,000 parallel sandboxes. Future plans include expanding to scientific computing.

The Agents Economy Backbone - with Emily Glassberg Sands, Head of Data & AI at Stripe

Oct 30, 2025 · 1:37:13

Emily Glassberg Sands, Head of Data & AI at Stripe, explains how Stripe builds economic infrastructure for AI, processing 1.4% of global GDP. Stripe's domain-specific foundation model runs inline on the charge path, detecting card-testing attacks on large users with detection rates improving from 59% to 97%. The Agentic Commerce Protocol (ACP) with OpenAI creates a shared standard for businesses to expose products to AI agents, adopted by Walmart and Sam's Club. Stripe helps AI companies combat new fraud vectors like free trial and refund abuse, and launched token billing to let wrappers price based on real-time inference costs. Internally, 85% of Stripes use LLM tools daily, with code generation and merchant understanding as key use cases. Emily argues AI companies grow 2-3x faster than SaaS, are twice as global, and that agentic commerce will expand consumption by removing time constraints for high-income consumers.

The RLVR Revolution — with Nathan Lambert (AI2, Interconnects.ai)

Jul 31, 2025 · 1:19:00

Nathan Lambert from AI2 explains the RLVR (Reinforcement Learning with Verifiable Rewards) revolution, arguing that verifiable rewards for math, code, and instruction-following are scaling more reliably than human feedback, and that open models like Tulu 3 can match frontier labs on core evals with just 10-15 tasks vs. hundreds. He traces overoptimization through three phases—control, RLHF, and RLVR—and warns that models learn to cheat unit tests unless reward design penalizes it. He analyzes o3's search-heavy approach (e.g., 80 websites per query), hybrid reasoning models like Gemini 2.5 and Claude, and predicts that pure reasoning models will become the default as inference costs drop. Lambert introduces a four-skill taxonomy for agent models—skills, calibration, strategy, abstraction—and says planning improvements often come from mundane data fixes. He advocates for fully open models, citing AI2's Olmo 32B as approaching GPT-4 level, but notes that building an 'American DeepSeek' requires massive resources and nonprofit constraints.

Solve coding, solve AGI [Reflection.ai launch w/ CEO Misha Laskin]

Mar 7, 2025 · 28:12

Misha Laskin of Reflection AI argues that solving autonomous coding is the direct path to AGI, combining reinforcement learning (pioneered by his team on AlphaGo/AlphaZero) with large language models (which they advanced at Google on PaLM, Gemini, ChatGPT). He explains that coding is already ergonomic for LLMs—unlike browser agents that require noisy human mouse data—making it the ideal starting point. Reflection AI is building coding agents that automate backlog tasks (testing, refactoring, migrations, security remediation) for large engineering teams, delivering them via an API that takes a task and codebase and outputs resolved code. Laskin insists superintelligence cannot be built in a vacuum; real-world customer evals are essential, as benchmarks like SWE-bench don't guarantee production reliability. He also stresses the need for open-weight models to prevent a few companies from hoarding superintelligent coding agents.

Beating Google at Search with Neural PageRank and $5M of H200s — with Will Bryk of Exa.ai

Jan 10, 2025 · 55:54

Will Bryk, CEO of Exa.ai (formerly Metaphor), details building a neural search engine from scratch using link prediction as 'Neural PageRank' — predicting documents rather than keywords. Exa's new product offers near-perfect lists (e.g., 'startups working on hardware in SF') by scaling compute per query, from milliseconds to a day, like o1 for search. Bryk argues LLMs will become the interface to search, while Exa provides the 'super knowledge' that even AGI will need. He contrasts Exa's neural approach with Google's keyword-based system and Perplexity's reliance on Bing. The company recently purchased a $5M H200 cluster and maintains a culture of nap pods and first-principles thinking.

The State of AI Startups in 2024 [LS Live @ NeurIPS]

Dec 21, 2024 · 26:35

Sarah Guo and Pranav Reddy argue that 2024 has become a far friendlier ecosystem for AI startups, with the model landscape shifting from OpenAI's near-monopoly to a competitive field where Google's Gemini now leads LMSys Arena and open-source models like Llama 8B score ten points higher on MMLU than Mistral 7B a year ago. They note that OpenAI's API cost has dropped 80-85% in 18 months, and total OpenAI API share fell from ~90% to ~60% as customers switch. The funding environment is rational, not a bubble, with foundation-model labs raising $30-40B but most startups seeing sane valuations; one portfolio company grew from zero to twenty million in PLG-style spending. Key startup themes include first-wave service automation (Sierra, Decagon, Harvey, EvenUp), better search and new friends (Perplexity, Glean, Character, Replica), and democratized creativity (Midjourney, HeyGen). They argue the 'GPT wrapper' narrative is false—applications capture value—and that incumbents face innovator's dilemma because AI changes business models (outcomes-based pricing) and data needs (reasoning traces rarely saved). The speed of change and new markets (legal, healthcare, defense) structurally favor…

The Four Wars of the AI Stack - Dec 2023 Recap

Jan 26, 2024 · 1:20:58

Swyx and Alessio recap December 2023 by framing the AI landscape as four wars: the Data War (NYT lawsuit demanding destruction of GPTs, OpenAI's data partnerships, synthetic data from DeepMind); the GPU/Inference War (Mixtral price dropping 90% to $0.27/million tokens, benchmark drama between Anyscale and Together, alternative architectures like Mamba); the Multimodality War (Midjourney reaching $200M ARR, ElevenLabs hitting unicorn status, OpenAI and Google building god models); and the RAG/Ops War (LangChain vs LlamaIndex, Pinecone's $750M valuation, Qdrant serving OpenAI and Anthropic internally). They argue agents and open source are not active wars, and predict 2024 will see AI finally reach production, with hardware like Rabbit R1 and Tab capturing unique context.