Building AGI with OpenAI's Structured Outputs API
Sep 17, 2024 · 1:12:23
Michelle, OpenAI's API tech lead, explains how the newly launched Structured Outputs feature achieves near-100% reliability for schema-following outputs by combining constrained decoding with model training, moving beyond the limitations of JSON mode and Function Calling. She details the refusal field that allows the model to decline harmful requests while maintaining parseable output, and discusses use cases like dynamic UI generation and mathematical chain-of-thought. The roadmap includes parallel function calling and custom grammars. The episode also covers the GA of GPT-4o fine-tuning, Batch API's 50% cost savings, Vision API integration with structured outputs, the Whisper API's surprising lack of diarization, and the shift toward speech-to-speech APIs. Michelle shares insights from her Waterloo co-op background and what qualities succeed at OpenAI: low ego, user-focused, and driven.
