How Claude Plays Pokémon was made
Mar 4, 2025 · 37:38
David Hershey of Anthropic explains how he built Claude Plays Pokémon, an agent that uses Claude to play Pokémon Red, revealing both surprising capabilities and stark limitations of AI. He describes the simple tool-using agent loop: Claude sends button presses, receives screenshots and RAM state, and maintains a knowledge base and 30-message conversation history with summarization. Key challenges include Claude's poor spatial awareness—it famously walked up and down outside Oak's lab 12 times—and its tendency to hallucinate game knowledge. Hershey notes that newer models (Sonnet 3.7) required stripping away band-aid prompts, as simpler instructions yielded better performance. The project also showed emergent behaviors: nicknamed Pokémon made Claude more protective, and the model even wrote meta-commentary about its own mistakes in its knowledge base. While the current run is stuck in Mount Moon after 52 hours, the biggest hurdles remain vision and navigation, not prompting tricks.
