Computer Use

The new OpenAI Agents Platform: CUA, Web Search, Responses API, Agents SDK!!

Mar 11, 2025 · 25:39

OpenAI's Nikunj Handa and Romain Huet join host Swyx to announce their new Agents Platform, including the Responses API, Web Search, File Search, Computer Use tools, and an upgraded Agents SDK. The Responses API unifies Chat Completions and the Assistants API, offering free state storage for 30 days and acting as a strict superset for future agentic workflows; Chat Completions remains supported, but the Assistants API will sunset in first half 2026. Web Search is available both as a built-in tool in Responses and as a fine-tuned GPT-4o Search Preview model in Chat Completions, achieving 90% accuracy on simple QA compared to 30% for GPT-4o. The File Search tool gains metadata filtering, while Computer Use—powering OpenAI's Operator—enables browser automation via screenshots and action signals, described as the "GPT-2 of computer use." The Agents SDK builds on the Swarm experiment, adding type support, guardrails, handoffs, and tracing in the OpenAI dashboard, with plans to tie traces to evals and reinforcement fine-tuning.

The new Claude 3.5 Sonnet, Computer Use, and Building SOTA Agents — with Erik Schluntz, Anthropic

Nov 28, 2024 · 1:11:08

Anthropic's Erik Schluntz explains how Claude 3.5 Sonnet achieved state-of-the-art 49% on SWE-Bench Verified by using a minimal agent framework that gives the model full control via tools like bash and file editing, letting it self-correct without hard-coded workflows. He details the importance of tool design over prompt engineering, why XML tags work well, and how computer use reduces integration friction by letting models interact with any browser-based interface. Schluntz also shares his robotics experience, noting that while LLMs and diffusion models are promising for general-purpose robots, reliability at 99.9% and hardware variability remain major hurdles, and he expresses skepticism about self-driving as a business due to high vehicle costs versus driver-equivalent revenue.