Structured Outputs

Building AGI in Real Time (OpenAI Dev Day 2024)

Oct 4, 2024 · 2:09:35

OpenAI's DevDay 2024 unveils the Realtime API with WebSocket-based voice and function calling, Vision Finetuning, Prompt Caching, and Model Distillation. Product leads Olivier Godement, Romain Huet, and Michelle Pokrass detail design choices like zero-code caching and WebSockets for human-level latency. Alistair Pullen of Cosine explains how fine-tuning GPT-4o on custom reasoning traces beat o1 on SWE-bench Verified, while Sam Altman and Kevin Weil discuss AGI levels (o1 as level 2 reasoner), the coming agent era (level 3), safety through iterative deployment, and the importance of building for the frontier. The episode also covers moderation policy relaxation, plans for o1 system prompts by year-end, and the competitive admiration for NotebookLM.

Building AGI with OpenAI's Structured Outputs API

Sep 17, 2024 · 1:12:23

Michelle, OpenAI's API tech lead, explains how the newly launched Structured Outputs feature achieves near-100% reliability for schema-following outputs by combining constrained decoding with model training, moving beyond the limitations of JSON mode and Function Calling. She details the refusal field that allows the model to decline harmful requests while maintaining parseable output, and discusses use cases like dynamic UI generation and mathematical chain-of-thought. The roadmap includes parallel function calling and custom grammars. The episode also covers the GA of GPT-4o fine-tuning, Batch API's 50% cost savings, Vision API integration with structured outputs, the Whisper API's surprising lack of diarization, and the shift toward speech-to-speech APIs. Michelle shares insights from her Waterloo co-op background and what qualities succeed at OpenAI: low ego, user-focused, and driven.