StarCoder

Amp: The Emperor Has No Clothes

Sep 25, 2025 · 1:21:59

Quinn Slack (CEO) and Thorsten Ball (Amp Dictator) from Sourcegraph discuss Amp Code, a coding agent that ships 15x/day with no code reviews. They explain the strategic pivot from Cody to Amp, emphasizing that building the best coding agent trumps everything else. The team dogfoods Amp internally, using it to build itself, and maintains both a VS Code extension and CLI, with a 50/50 internal split. They argue that model choice is becoming less important than the harness around the model, and they reject features like prompt enhancers, custom subagents, and MCP servers as not adding value. Common failure modes include outsourcing thinking but not typing and using agents without proper context engineering. They predict async agents running 24/7 will dominate, and they target early adopters willing to adapt their codebases and workflows for maximal agent effectiveness.

⚡️ Beyond Transformers with Power Retention

Sep 23, 2025 · 32:59

Jacob Buckman, CEO of Manifest AI, discusses their solution to AI's expensive computational bottleneck: the ever-growing KV cache for long context inference. Power Retention replaces attention in transformers with a fixed-size memory, achieving 10X training speedup at 64K tokens and 100X inference speedup over FlashAttention. Manifest releases Vidrial, a just-in-time CUDA kernel framework that finds optimal configurations, and Metamorphosis, a process to convert existing transformers like StarCoder-3B into Power Retention models in just two hours of mid-training. Buckman demonstrates PowerCoder, a 3B code model that matches baseline loss after 10K steps and handles contexts up to 32K and beyond. The open-source release includes kernels and tools, aiming to build community trust and encourage adoption by inference providers. He also highlights the need for genuinely long-context datasets beyond internet text, such as human trajectories.

The "Normsky" architecture for AI coding agents — with Beyang Liu + Steve Yegge of SourceGraph

Dec 17, 2023 · 1:33:26

Beyang Liu and Steve Yegge of Sourcegraph explain how their AI coding assistant Cody achieves a 30% completion acceptance rate by prioritizing high-quality context over agents. They introduce the 'Normsky' architecture—a blend of Norvig's data-driven models and Chomsky's formal systems—arguing that non-agentic, deterministic context retrieval (using parsers, trigram indexes, and the BFG code graph) outperforms multi-hop LLM-based approaches for code completion and chat. The duo details their data pre-processing moat, the 'bin packing' challenge of stuffing relevant code into context windows, and why they use open-source StarCoder for completions while leveraging Claude/GPT-4 for chat. They also discuss the death of DSLs, the limitations of LSP and LSIF, and how Cody's web version lets users query any public repo instantly.