# Amp: The Emperor Has No Clothes Latent Space · 2025-09-25 Quinn Slack (CEO) and Thorsten Ball (Amp Dictator) from Sourcegraph discuss Amp Code, a coding agent that ships 15x/day with no code reviews. They explain the strategic pivot from Cody to Amp, emphasizing that building the best coding agent trumps everything else. The team dogfoods Amp internally, using it to build itself, and maintains both a VS Code extension and CLI, with a 50/50 internal split. They argue that model choice is becoming less important than the harness around the model, and they reject features like prompt enhancers, custom subagents, and MCP servers as not adding value. Common failure modes include outsourcing thinking but not typing and using agents without proper context engineering. They predict async agents running 24/7 will dominate, and they target early adopters willing to adapt their codebases and workflows for maximal agent effectiveness. ## Questions this episode answers ### Why did Sourcegraph create Amp as a separate product instead of improving Cody? When Thorsten Ball rejoined Sourcegraph in February, he and CEO Quinn Slack hacked on a tool-calling agent with few constraints. The result was so different from Cody that they needed a new brand to reset expectations and pricing — it couldn't work with a $20 subscription. Existing Cody enterprise contracts would have slowed them, so Amp was born to ship 15 times a day, free from the cloud release cycle. [0:59](https://addtry.com/c6bc0a5a-b731-45b2-a6ff-1ac49794e2af?t=59000) ### Why doesn’t Amp let users pick which AI model to use? Quinn and Thorsten argue that what users actually want is the best coding agent, not model choice. They find that tuning the system prompt and tools to a model matters as much as the model itself, and offering a selector would slow their ability to optimize. Thorsten notes the game of rushing to support new models is 'kind of over' — users now care more about the overall experience than the underlying model name. [24:24](https://addtry.com/c6bc0a5a-b731-45b2-a6ff-1ac49794e2af?t=1464000) ### How does the Amp team ship software so quickly without code reviews? On the Amp team of about eight engineers, they push directly to main without formal code reviews — a practice Thorsten and Quinn started early on. They ship up to 15 times a day and rely on fast feedback from dogfooding their own tool. Thorsten says this beats established processes in a fast-moving environment; code is still reviewed because the agent writes 80–90% of it and the developer reviews before committing. [9:36](https://addtry.com/c6bc0a5a-b731-45b2-a6ff-1ac49794e2af?t=576000) ### Why is Amp skeptical about subagents and prompt optimizers for coding? Quinn calls prompt enhancers a 'bullshit feature' because LLMs need real information, not phrasing tricks. They avoid subagents because they bloat context with excess tokens, are hard to debug, and models aren't trained for such convoluted workflows. Thorsten adds that many agentic workflow experiments fail 2–5% of the time, making them unreliable for production use — what he calls a 'hangover' once the unreliability surfaces. [41:06](https://addtry.com/c6bc0a5a-b731-45b2-a6ff-1ac49794e2af?t=2466000) ## Key moments - **[0:00] Intro** - [0:30] Sourcegraph's Amp began as Thorsten Ball and Quinn Slack hacking on Claude 3.7 with no constraints - [1:46] Thorsten Ball: Amp required a new product and pricing, breaking from the $20 subscription model - **[3:18] Best Agent** - [3:21] Quinn Slack: "The only thing that matters is building the best coding agent. Nothing else matters." - [4:46] Amp enables Sourcegraph to ship 15 times a day, freeing them from the platform's release cycle - [5:31] Amp is growing >50% month-over-month, with small teams spending hundreds of thousands annually - [6:56] Thorsten Ball: the AI coding market changes every few months; survival depends on the ability to react quickly - [7:22] Thorsten Ball notes how quickly perception shifts: Copilot now seen as old and enterprise-y, illustrating the rapid churn - **[9:36] Dogfooding** - [9:36] Amp team ships 15 times/day with no formal code reviews, pushing directly to main and dogfooding the product - [10:17] Thorsten Ball describes balancing 'duct tape personal project mode' and enterprise practices to adapt quickly - **[12:35] CLI vs IDE** - [12:36] Q: Why did Amp build both a VS Code extension and CLI? Thorsten Ball: started with VS Code for speed, later CLI for flexibility and SSH use - [14:45] Thorsten Ball was surprised at developer adoption of CLI coding agents like Claude Code, which influenced Amp's CLI development - [16:55] Amp's internal poll shows a 50/50 split between CLI and VS Code usage, highlighting no clear winner in interface preference - [18:27] Quinn Slack nearly killed Amp's VS Code extension, but async agent and future plans reversed the decision - [21:39] Amp refuses to add an inline 'fix' button, as Quinn Slack believes that interaction won't matter in 6–12 months - [22:23] Quinn Slack: Amp is the only tool being honest about not chasing every user feature, to avoid losing speed and focus - **[24:10] Model Selectors** - [24:40] Thorsten Ball: model selection is becoming less important; the system prompt, tools, and harness matter more for coding agent quality - [25:29] Thorsten Ball: "We don't mention the model at all. We put it in the manual." - [27:45] Half of Amp's internal team now uses a non-Sonnet model, showing rapid model evolution and tool adaptation - [29:06] Q: Does Codex exclusivity for GPT-5 threaten tools like Amp? Quinn Slack: they still provide API access, and speed is understandable - [30:13] Amp uses multiple models (Anthropic, OpenAI, Google) in a single session, combining strengths - [30:49] Prediction: Thorsten Ball expects within 1–2 months a major shift where foundation model companies lose their lead to upstarts - **[32:39] Harness** - [32:39] Thorsten Ball: build 'wooden scaffolding' around models that can be torn down when models improve, per the bitter lesson - [34:28] Thorsten Ball: non-deterministic LLMs as product core are unprecedented; users can misuse them and get false confidence - [36:23] Thorsten Ball: Amp internal dogfooding puts them 1-2 months ahead of mainstream, allowing early pruning of ineffective patterns - **[37:19] Pitfalls** - [37:19] Q: What are common failure modes? Thorsten Ball: vibe coding — outsourcing thinking but not typing — leads to spaghetti code and wasted tokens - [38:18] Amp lost one deal to competitors offering 85-100% discounts for two years; Quinn Slack refuses to chase such uneconomic deals - [40:46] Quinn Slack calls prompt enhancers a 'bullshit feature' that misleads users - [41:07] Quinn Slack: subagents are token-heavy and hard to debug; MCP servers often flood tools, making agents worse - [42:20] Thorsten Ball: MCP servers often create a poor experience due to auth pain and low-level tools that confuse agents - [43:00] Amp may remove conversation compacting because it loses signal and data, raising doubts about its reliability - **[52:31] Subagents** - [53:40] Thorsten Ball: teams are unifying logs (client+browser+db) into one agent-readable log, optimizing for agentic consumption - [55:03] Thorsten Ball: agent-friendly logging can be verbose JSON lines, readable by agents but not humans - [57:06] Thorsten Ball references emerging frameworks that integrate coding agents directly into the development framework - [58:37] Quinn Slack: Amp explicitly targets users willing to adapt, which liberates the team from legacy constraints - [1:00:26] Thorsten Ball: For the first time in decades, the value of agents makes developers rethink their codebase practices - [1:02:14] Quinn Slack: building for laggards is a trap; Amp focuses solely on early adopters to avoid being trounced - **[1:03:13] Outer Loop** - [1:04:08] Q: How to manage multiple async agents? Alessio describes losing track of concurrent agents after breaks, highlighting a key UX gap - [1:05:31] Quinn Slack: future async agents need deploy previews and fast orientation to handle multiple background tasks - [1:06:32] Amp team writes 80-90% of code with agents, but each person reviews agent output themselves, replacing formal code reviews - [1:07:09] Q: How to handle merge conflicts from parallel agents? Alessio built a task manager for cross-agent file tracking - [1:08:35] Prediction: Alessio argues TypeScript may dominate because static types aid agent self-improvement - [1:09:41] Alessio predicts enterprise software shifting to user-generated, team-specific tools built via agents - **[1:10:36] Enterprise UGC** - [1:10:37] Quinn Slack: a board member with little coding experience replaced $250K/year software using Amp, proving power users matter - [1:13:34] Quinn Slack: vibe coding with poor feedback loops is not valuable; real value is empowering power users to build fast - **[1:14:39] Wrap-Up** - [1:14:39] Quinn Slack appointed Thorsten Ball 'dictator' of Amp to stay grounded and make fast decisions without bureaucracy - [1:16:48] Amp has no formal evals; Thorsten Ball relies on rapid dogfooding and 20x daily shipping for instant feedback - [1:19:57] Thorsten Ball: "Everything I learned over 15 years ... listening to Acquired podcast ... reading Hacker News ... is coming together" for Amp - [1:21:39] Quinn Slack: "We get to go build the future of coding. Feels like the final boss." ## Speakers - **Alessio** (host) - **Quinn Slack** (guest) - **Thorsten Ball** (guest) ## Topics Coding Agents ## Mentioned Sourcegraph (company), Amp (product), Bazel (product), Claude Code (product), Codex (product), Cody (product), Copilot (product), Cursor (product), GPT (product), Gemini (product), Git (product), GitHub (product), Linear (product), Playwright (product), Sonnet (product), StarCoder (product), VS Code (product), Vitest (product), Windsurf (product), o3 (product) ## Transcript ### Intro **Alessio** [0:04] Hey, everyone. Welcome to the Leading Space Podcast. This is Alessio, founder of Kernel Labs, and today there's no Swyx. He's in Europe with A engineer, but I'm joined by Quinn and Thorsten from Sourcegraph. Welcome. **Quinn Slack** [0:14] Thanks. Great to be here. **Thorsten Ball** [0:15] Great to be here. **Alessio** [0:16] So we already had origin of Sourcegraph with Beyang and Steve, um, so we'll put the, the link in there, and this was when you launched Cody. And today, I guess Cody is a brand, uh, that has passed, and now you have Amp. So l- let's maybe start there. Obviously, Quinn, you're CEO of Sourcegraph. Thorsten, what's your role, I guess, title? How, how do you describe what, what you do? **Thorsten Ball** [0:39] Well- **Alessio** [0:39] CEO is much easier. **Thorsten Ball** [0:42] I'm not gonna name my internal title, but I'm, I'm- **Quinn Slack** [0:44] The dictator of Amp. **Thorsten Ball** [0:45] Yeah, that's the, that's the internal title, yeah. But, uh, it's, it's, um, yeah, I'm the lead engineer and one of the creators of Amp, yeah. **Alessio** [0:53] So were you part of, like, the thumbs up, thumbs down on Cody brand? Like, how did you decide-- how did you get to Amp? Let's tell that story. **Thorsten Ball** [0:59] I mean, I'll start. You can jump in, but basically, I came back to Sourcegraph February, and then this was when Claude 3.5, 3.7 happened too, and then Quinn and I started hacking on, you know, what if we just take Claude 3.7, and what if we give it just tools and let it go nuts? You know, like, no constraints, no, um, a lot of the other stuff that we had in Cody, which works for Cody. Um, j-- let's just start trying this out, and we started a new project, and we were, you know... I remember first week in SF where I would stand up in the middle of the room like, "Quinn, you gotta, you gotta see this. Like, this is crazy." And then he was like, "Okay, let me try this." And then we went off from there, and then we realized relatively quickly that it's a different kind of product. Where Cody was very much, um, you know, one of-- first of its kind with Rag and assistant panels, assistant sidebar. But with, you know, a tool calling agent where I define an agent as a model, a system prompt, and tools and tool prompts that go along with this, um, that you give a lot of permissions for so it can actually, you know, see the file system, interact with the file system or your editor. It's a different thing, and we realized we gotta handle this differently. We gotta reset expectations. We gotta tell users that it's a different thing and they gotta use it differently in some sense, and also that we cannot make it work with a $20 subscription, which back then was seen as a, you know, offensive thing to say and- **Alessio** [2:28] You guys are charging money? **Thorsten Ball** [2:29] Yeah, yeah, exactly. **Alessio** [2:30] What the hell? **Thorsten Ball** [2:30] But now, you know, people are paying hundreds of dollars per month, which I've been saying this every day for the last two weeks, that's crazy to me still, like, how far we've come. So, you know, this is just how it started. Like, okay, this is a different thing. We were astonished, surprised, amazed by what these models can do, so we decided let's reset expectations. Let's tell a new story. We can-- We have enterprise customers for Cody, but they have expectations. We have contracts. These are large contracts, long-running contracts, and you can't just say, "Guys, you know, here's a new mode. It costs whatever, how many much dollars more. Um, it works completely differently. You need to hold it in a different way." So in order to avoid this and to avoid being disrupted, you create a new thing that kind of disrupts the business on its own, you know? ### Best Agent **Alessio** [3:19] Yep. **Thorsten Ball** [3:19] That's... I don't know. Wanna add? **Quinn Slack** [3:21] Yeah. The only thing that matters is building the best coding agent. Nothing else matters. Because if you can build that, that's way bigger than anything else that came before. And to be clear, nobody has built that yet. We are getting better and better, but I think you've seen this treadmill of tools that you use as a dev. First, it started with Copilot and then Cody. We were really good at Chat Rag, and then Cursor and Windsurf showed that kinda IDE forks and partially agentic things could get better and better. And then, you know, the next generation, Amp and Claude Code, and now you're already seeing people say, "Oh, well, Codex is better than Claude Code." And- **Thorsten Ball** [3:56] Mm. **Quinn Slack** [3:56] There's not been any tool that has stuck with devs for more than six or 12 months or something. **Thorsten Ball** [4:01] Six months, yeah. **Quinn Slack** [4:02] And we saw that firsthand. We're now on our second iteration, and we are able to move so much faster given that it has a totally different name, totally different brand, and some people don't even know that Sourcegraph or the people behind Cody made Amp. That has been so good. So I do not know how, if you had an AI tool that was relevant 9 or 12 months ago, how you can even bring the same brand and same customer contracts along with you- **Thorsten Ball** [4:27] Yeah **Quinn Slack** [4:27] ... and make a good product. It is so liberating to be able to say, "Totally different." **Thorsten Ball** [4:32] Yeah. To-- On a technical level, um, Cody was or is, you know, it's a Sourcegraph product, so it's kinda, it works with the Sourcegraph, uh, platform. That means you're tied to the release cycle of the Sourcegraph platform, and Sourcegraph is in the cloud. We have, you know, cloud versions of Sourcegraph, but also on-prem for some customers. Completely different game. And with Amp, we basically said, "Let's not do this. Let's build something that allows us to ship 15 times a day." And that's what we've been doing over the last six months. Like, we've, we're still doing this, and it's a game changer. Not just, you know, anybody who's done this knows this, but, you know, internally and externally, you need to reset expectations that this is a new way of how we build software. And having a new project with a new way to do it is, I think, a better way to do it than to try and get, like, the old to move in this new way because it would take longer. **Alessio** [5:24] Are there any numbers that you share about developers, like, you know, Amp usage overall? **Quinn Slack** [5:31] It's growing really fast. It's growing more than 50% month over month, a lot faster in, you know, some weeks. And really what we have seen too is there's a huge change in who's using it. So we have teams with, like, two or three people that are on annual run rates of, like, hundreds of thousands of dollars. **Thorsten Ball** [5:48] Mm. **Quinn Slack** [5:48] So that's it. We also made a decision to not try to go to every single dev in an enterprise, which we had done with Cody. We pick off the people that wanna move as fast as we wanna move, that wanna stay at the model product frontier like us. So it's all about just being able to move really fast. And I think that the way that agents work today, most of them are used in your editor or CLI interactively. You have one agent at most running with you at all times. That's gonna be blown up with async agents when they're running 24/7 concurrently in the background. Then you can have 10 or 100 times as many, and that's gonna dominate inference. That's gonna dominate the output you get. So it's really, you know, Amp is growing really fast, but it's, it's more about how do we get to be the first ones with that, like, 10 to 100X improvement? And everything is about how can we move fast and learn along the way? It just so happens that we are positive gross margins along the way. **Thorsten Ball** [6:44] I would say that's one of the biggest axioms that we have with Amp, is that we don't know where this ride is going. But what we do know is that it's changing every few months. And, you know, s- start of the year, right? Cursor was the king and the biggest, fastest growing start of all time. Now, if you were to ask a lot of developers, "What do you think is the dev tool king?" I don't think they would name Cursor as the first one. And then in, I think a couple months later or maybe a couple months before, somebody, uh, this was from somebody in sales, they said like, um, I don't know what it was, but, um, they were basically saying, "Blah, blah, blah, makes Cursor look like GitHub Copilot." You know? Like, makes it look old and boring and enterprise-y. And this is im- like, just think about this. Like, Copilot is not that old. Like, it was state-of-the-art, I don't know, maybe two years ago or something, and now the world has changed completely, and we know that this is not over yet. Like, the, the changes are still coming. So from an engineering and business perspective, this is priority number one. Position yourself in a way that you can react to these changes, and position your product and your expectations and your technical, the code base in a way that lets you react to these things as fast as possible. And then everything else flows from it. Everything else we've done is basically based on this. Like, that everything can change at, you know, release of another model or something. **Quinn Slack** [8:07] But how are you doing it internally from a team perspective? Because, you know, obviously you have a lot of customers already on the Sourcegraph product. There's kinda like this tension of, you know, going founder mode and kinda burning the bridge on maybe some of the old use cases, versus having a smaller team and a dictator, uh, for a new product. How does that look like from, like, a building the company perspective? When you have a really popular, successful product that's highly profitable, that funds a lot of this craziness, and we're able to do this also with the customer trust. So there's a lot of things on Amp that we do, like no consistent pricing, no user model choice, no checking off all the boxes that security and compliance and legal want, that, you know, takes nine months. We're able to get away without doing that stuff because we have that customer trust. So, you know, that, that has been a big thing. It requires you to totally change how you think about an existing business. It's not a way to sell through that same channel to those same users. It's a way to use that trust and that revenue to fund crazy stuff that you gotta do. But it's something that we deal with all the time, and we've got really smart devs, and yet it is hard for people to throw away everything that they have learned about how to build software development. And so in some cases, it's been really refreshing to have people that have only ever been at tiny, like, one-person companies. **Thorsten Ball** [9:25] Yeah. **Quinn Slack** [9:26] And they come here, and they have no preconceived notions about how you do planning or anything like that. And that is... It's great because you can throw all of that out of the window. **Thorsten Ball** [9:36] Yeah. We've had... This was radical in some sense that when we started, it was Quinn and I working on main, no code reviews, nothing, and just pushing, and it was like a personal project. And I think we're both experienced engineers, so it would be everybody owns their stuff. ### Dogfooding **Thorsten Ball** [9:53] You push, and if you break CI, you go and fix it, or if the other person is awake, you fix it or something. And it seems like when you move this fast and you ship this often, you have, um, you know, throughout the day, there's like 15 decisions you have to make, where you have to flip between the duct tape personal project mode, move fast, and the this is how they do it at Google mode. And it, you know, requires a certain expertise, or it requires also to be free from, like, the thinking of the last 15 years of, like, always do it like Google. Like, we always scale up. And the base assumption between, like, the whole Google thing was always that, oh, we found product market fit. Now we have a product. Let's scale this up, right? Every company I ever worked in was based on this assumption that this is the product. Let's make it proper and engineer it up. But now with these changes, what's ingrained in Amp is the understanding that, well, even if it scales up, we have to be prepared that somebody pulls the rug and a new technology comes out and it kinda shifts everything. So we have to be prepared for this. And again, it all flows from this, so now in our development mode, the team is super small, you know, compared to, I guess, other companies, but I think we're around eight people now on the Amp Core team. And we still don't do formal code reviews. We still push to main. We still ship 15 times every day. We dogfood this as much as possible, and it turns out that in a fast-moving environment like this, this beats a lot of other things, like fast feedback loops and using the product yourself and dogfooding it. Using the product to build a product beats a lot of established processes, you know? And we can get away with it because we can dogfood it. And how has it been internally received? I think we have the luxury of n- you know, making use of the infrastructure that we already have. For example, have a fantastic security team, right? Security team comes in, "Guys, let us take care of the security stuff for Amp," you know? So that is fine. And I'm like, "Cool." Like, I don't have to worry about this. Then we have infrastructure people. "Guys, let us take care of how to run this in the cloud." Cool. I don't have to worry about this. I can concentrate on the client or the UX or yeah. So this is a nice spot to be in where we can move fast, but use platform teams to kinda make sure that it doesn't break or it scales up or whatever, but still have, like, the, you know, the tip of the iceberg can melt and be rebuilt basically while the, the, the thing beneath the waterline- **Quinn Slack** [12:17] Mm-hmm **Thorsten Ball** [12:17] ... is stable, you know? **Quinn Slack** [12:18] Yep. **Thorsten Ball** [12:19] Not the greatest analogy But I think it's, it's a, there's a distinction between, you know, like platform stuff that does work, but on the UX or product application layer, you wanna be able to kind of tear the thing down and rebuild it as fast as possible. And I think that's what we're doing. ### CLI vs IDE **Alessio** [12:36] One thing is you get a separate team, and then the other thing is how do you put that team to work, right? Like, if you look at, like, the coding agent space, I mean, obviously you started with Cody, and I think there was maybe a thesis behind it, and then you had the rise of Claude Code, and you had Codex CLI, which is trying to catch up. I would say they're maybe a little behind on the UX- **Thorsten Ball** [12:56] Yeah **Alessio** [12:56] ... and all of that. But they obviously have, you know, billions of dollars to train a custom model, so that kind of weighs a lot of the option. **Thorsten Ball** [13:02] Yeah. **Alessio** [13:02] How did you decide about the structure? So you have both a plugin for IDs, so I use Amp Code in Cursor, but I can also go in the CLI and use Amp Code. Was that an easy choice? Like, was there a lot of discussion on we should just do one of the modes? Like, supporting both is obviously more work, right? **Thorsten Ball** [13:21] Yeah. **Alessio** [13:21] And a lot of these products don't support both. **Thorsten Ball** [13:24] Yeah. **Alessio** [13:24] So what was that initial design choice of, uh, the structure of the product, and then we'll dive into the models as well. **Thorsten Ball** [13:30] So we started with a VS Code extension because it was the easiest thing to get off the ground. Like, and when you have a VS Code extension, you have a marketplace, you can ship this, you can update it 15 times every day. You don't have to think about updating stuff. You also are next to the editor, and looking back, you know, it's been six months, the editor might be dying or you might do a lot of coding outside the editor. Back then, it sounded much more radical than it does sound right now. So we started with, like, let's explore this, and having the thing next to your editor is a good place to start. And we could, you know, you can see the cursor, you can do selection, whatnot. But we were really like, um, d- from the start, we didn't want to have, like, a deeply integrated thing. It was always like, "Ah, let's keep the features small. We gotta be able to move fast." And then we build up the CLI on the side as, like, a different client, which also gives us the ability to abstract, like, the core and the client stuff, so that's a nice boundary to have. But then to be 100% honest, we were also surprised by how many people were fine with using a CLI for Claude Code, for example. Like, if you had asked me half a year ago, I would've said, "No way, like a CLI tool?" And what we realized is, well, a CLI is not just, you know... It's a UI, sure, but also it's a CLI program. That means you can run it on SSH. You can run it in any other editor. You can run it in multiple split panes. You can run it in multiple tabs. If you wanna do this in VS Code, you have to rebuild a lot of stuff, and you have to rebuild the way you switch between conversation. You have to rebuild... I mean, SSH works out of the box in VS Code, sure, but still, like, you're tied to this. And we had an experiment, an internal one, about, um, a desktop application, um, so, like a standalone application, and turns out, yes, that's great to have multiple agents, but you also have to reinvent everything that right now a terminal gives you for free, right? If I use Ghost or iTerm or VS Term or whatever, I can Command+N, Command+T, I get tabs, split nu- you know, splits, different environments per tab. You can CD into directories, you can set envars. You get this for free, right? And if you do it in a desktop application, then you run into the issue of, you know, what people see with, like, a lot of the Async agents. Oh, you run a run a task, set the envars, which directory you have to be in, what's, you know, what's in the path, whatnot. Um, you have to do this beforehand, and in the terminal you get it for free. So that's kind of the short version of it, that, um, we started with VS Code because it was easy, and it gave a lot of feedback. We could concentrate on the stuff that matters and not worry about stuff like distribution, which VS Code takes care of. And then with the emergence of CLIs, we noticed that it's a big, big improvement, or th- there's other advantages to it. So now then we've rebuilt the CLI twice. Um, and now we have, like, a really nice GUI with our own framework. And one interesting thing is our VS Code extension has a lot of advantages over the CLI. For example, it's easy to display diagrams. It's easy to display images. It's easy to render a bunch of stuff. Like, um, we can do command, um, return to submit messages, you know, all of that stuff. And turns out we have, like, a internal poll last week at our company meetup where Beyang was asking, "Who of you uses the, the CLI and who of you uses VS Code?" And it was a 50/50 split. And it's very strange that it falls out, you know, that it comes out- **Alessio** [17:02] Right **Thorsten Ball** [17:02] ... like this, and there's not a clear winner, and both have advantages and disadvantages, and so right now we have both. **Alessio** [17:08] But do you cut the data based on the level of the engineer or maybe the specialty, you know, maybe front end versus back end- **Thorsten Ball** [17:15] Yeah **Alessio** [17:15] ... and, like, how do you segment that? Or do you just take it? **Thorsten Ball** [17:18] I mean, we haven't really segmented it. If I had to, you know, guesstimate here, there's also a generational divide where I would say the younger people, you know, younger than 25, their m- terminal seems old to them, and they were much more inclined to use the stuff in the editor. Um, but yeah, I don't, we don't have any fancy segmentation. I think, not to sound too dramatic, but, like, one of the other guiding principles that we've had from the start with Amp was whenever somebody is like, "Uh, what's the data on this?" Or, "Do we have, like, uh, analytics on this?" It's like, "Well, did you look for it yourself? Like, did you try it out? Because that o- did you talk to customers?" Like- **Alessio** [17:58] Mm **Thorsten Ball** [17:58] ... we constantly talk to customers. That beats a lot of other stuff. Um, so yeah, we don't have any segment analysis of who uses what and where and how. **Alessio** [18:07] Mm-hmm. **Quinn Slack** [18:08] I use both, and it-- This idea that everything is changing, it applies to this. We looked at this. We saw the way that things were going and how much more flexible s- a CLI was, and we, about three weeks ago, we said, "We think probably It's painful, but we will kill the VS Code extension for Amp. And we said that, I laid that out, and I didn't like it, but it seemed like that's how things were going. **Guest** [18:33] Mm. **Quinn Slack** [18:33] And then you think about async agents, which probably need to be on your phone and on the web, or maybe you use WhatsApp to interact with them. That's a whole other mode of interaction. Well, and if it's on the web, that's like the VS Code UI, not the terminal UI. And then there's this other thing that we're planning on doing that I can't share, uh, more about, but that also makes me think, "Well, actually, we really need to keep the VS Code UI in." And so this thing that seems so obvious, actually there's two other completely different things out of left field that totally overturned it. **Guest** [19:04] Yeah. **Quinn Slack** [19:04] So we're keeping it, and it's definitely adding some more complexity, but there's a lot of things we can do to reduce that and simplify it. **Guest** [19:13] But there's always a hand hovering over the button to... Can we get rid of this? Like, can we shed weight? Like, can we get rid of balla- Uh, can we reduce complexity so we're again in the spot of if a new model comes out, we can react quickly? And y- you know, sure, it's good engineering and there's not a lot of duplication, but still, updating one client is still faster than updating two clients. So there's this constant tension between what's the most minimal thing, the most minimal product that we can have. **Quinn Slack** [19:41] Yeah. **Guest** [19:41] And, and, you know, just to pick some other examples, there's a lot of niceties you can do in VS Code where, for example, uh, you have recent ex- Not recent, but a common example, you know how in VS Code you can hover over diagnostic and then you can say, um, you know, "Fix this," or whatever, and then people would ask like, "Can you add like a Let Amp Fix This button?" And it's like, you can ask. Amp knows about your selection, it knows about the diagnostics. It can see all of this, so you can just ask, like, "Fix this for me," and if you type three words, it will usually do it. So that's something where it's like, well, you can already do it. It's a nicety, but let's remove the surface area. Let's remove this other thing that we have to backport or keep working or whatnot. And d- you know, tiny example, but there's a- **Quinn Slack** [20:23] Yeah **Guest** [20:24] ... you know, 500 of these, what we say. **Quinn Slack** [20:25] But how do you think of that when the IDE is already a AI IDE? So I use Cursor, right? **Guest** [20:30] Yeah. **Quinn Slack** [20:30] There's already, like, fix in chat- **Guest** [20:32] Yeah, yeah, yeah **Quinn Slack** [20:32] ... that pops up, and they want obviously the button to go to their chat- **Guest** [20:35] Yeah, yeah, yeah **Quinn Slack** [20:36] ... versus like you guys are on the left side, and it's like- **Guest** [20:38] Yeah, yeah **Quinn Slack** [20:38] ... just do this here. Do you feel that in a way the IDE VS Code extension is more like for the people not using these like AI first tools, and using the features like most people, you know... I, I'm sure GitHub is, like, eventually gonna have something good to put in, in VS Code. How much do you think about VS Code extension just being, you know, maybe a stepping stone to the thing you cannot talk about- **Guest** [21:01] Yeah **Quinn Slack** [21:01] ... that you don't talk about, and then kind of the bifurcation of, of the TUI versus like the fully async, you're not looking at anything? **Guest** [21:08] I think we're not trying to maximize our revenue, our user adoption literally today with the state of today's models and today's tools because everything's changing so fast. So yeah, we're not trying to fight Cursor for who's gonna win the right to have users fix with our AI or their AI. Frankly, it doesn't really matter to us. I don't think that that interaction is a really important way that people are gonna be interacting with AI in 6 months or 12 months. I don't think we learn anything from that, and we just said we're not gonna do it. And users, some have definitely asked for that. And the other thing is we have to figure out what do users actually want, and they say they want a lot of things. And in the case of customers, a lot of times they'll say they want a lot of things. They'll say that they want bring your own key. They'll say that they want model choice. They'll say that they want a subscription for $100 a month or, you know, pricing to lock users out if they spend more than $30 in a day. But actually, what we've seen is they want the very best coding agent. Not everyone. Not everyone. But we're focused on the ones that want the very best coding agent. And when we tell them how that thing will slow us down, then that starts this conversation where they'd rather not have something they might use 2% of the time if that means that the tool is worse. **Quinn Slack** [22:22] Right, right. **Guest** [22:23] And we alone among the entire industry, it feels like we are being really honest and really bold with that. And I am really concerned just for the, the rate of progress overall, that a lot of these other tools that are great like Claude Code and Codex and Cursor and so on, that they've forgotten what made them great and what made them grow so fast, which is building the very best product. And they, they built it in a way that's too overfit on the current capabilities. And so they're just gonna peak, and then it's gonna be a slow fall. And zero of the software business model works if that happens. You need to have growth into the future. **Quinn Slack** [23:00] Yeah. **Guest** [23:00] So, you know, we're trying to... I, I think it's best for our business, but also I think that we're trying to push the whole industry to just, uh, be radical about the changes that are coming. **Quinn Slack** [23:10] Yeah, when you said the best coding agent, I'm always like, is there a market for like the mid coding agent? You know? Like, the- there's a... I think the model choice is a great example of like, why would you want a model choice? I think pricing, I guess, is like the only thing that people bring up, but I, I think to your point, it's like you already pay engineers a lot of money. **Guest** [23:26] Yeah. **Quinn Slack** [23:27] Like, the cost of like Sonnet 4 versus Sonnet 3.5- **Guest** [23:31] Yeah **Quinn Slack** [23:32] ... it's kind of like minimal compared to like 150, 200, 300K once you do taxes and benefits and all of that that you pay to employees. So, um, yeah, I think we're like in this part of the market almost where like people are not maxing these things. **Guest** [23:47] Yeah. There's absolutely a market today, literally today. Someone will pay a monthly fee for that cheaper AI product today, but they're not gonna be paying that in six months. They're... It's gonna be a different product or they're gonna be paying for something else. And if you have that much churn as a product, you simply cannot build software in that way. But a lot of people get tempted by that, and they hear a lot of users ask for it. Six months ago, it was still the game of, oh, a new model got released, and then everybody would tweet out, "It's already available in..." ### Model Selectors **Guest** [24:20] their editor or whatever it is, their extension, right? And I think- **Thorsten Ball** [24:24] That's kind of over. Like, it's just people realize that, well, the benchmarks are one thing, right? Oh, this is the best model. Turns out it's not in this editor, but it feels different in this editor. So the whole like, you know, the models are the thing. I, I don't wanna say that's over- **Alessio** [24:42] Mm-hmm **Thorsten Ball** [24:42] ... but it's becoming less important, and the people are now, now also waking up to the fact that it's not just the model, it's the system prompt, it's the tools- **Alessio** [24:51] Right **Thorsten Ball** [24:51] ... it's the, the harness, um, the scaffolding around the model. So I can give you the choice to use Gemini 2.5 in, in Amp, but without the system prompt- **Alessio** [25:01] Right **Thorsten Ball** [25:02] ... being tuned to it, without, you know, what I called before, like going with the grain of the model. The models are trained in different ways, so you wanna optimize the tool and all around it for this specific model. Without that happening, doesn't make a lot of sense. You get the wrong signal. I can drop you in a new model right now and have it available in 10 minutes, but that's not what you're after, right? You want the best possible version of this model in this tool. **Alessio** [25:24] Mm-hmm. **Thorsten Ball** [25:25] And, you know, that's, I think, become more important, less like the model selectors and whatnot. **Alessio** [25:29] Yeah. Why do you mention the models at all? So you have Sonnet 4 for the agent. You have o3 for the Oracle. **Thorsten Ball** [25:36] We, we don't. We, we don't show them in the product. We don't mention the model at all. We put it in the manual. We have like a owner's manual- **Alessio** [25:43] Right **Thorsten Ball** [25:43] ... because people kept asking us. **Alessio** [25:44] Yeah. Well, but even then it's like, why does it matter that they ask? Because you might... Now, now it's like- **Thorsten Ball** [25:50] Yeah **Alessio** [25:50] ... if you wanna change it tomorrow, then it's like you gotta- **Thorsten Ball** [25:53] Yeah **Alessio** [25:53] ... tell people you changed the model. **Thorsten Ball** [25:54] Yeah. **Alessio** [25:54] And it's like, where do you think we are on like the slope of like, hey, look, you guys should forget it all about what model is even running- **Thorsten Ball** [26:01] Yeah **Alessio** [26:01] ... what the difference is. **Thorsten Ball** [26:02] So I think we're going towards a future where the model will become an implementation detail to some sense, and we will end up on a different abstraction layer. And for example, um, you asked like when would I use a mid model, right? When you put it like this, it sounds obvious, like who wants to use the shitty version of the better version? But you know, we're thinking actively about this. There's models who might not be as smart, um, as Sonnet 4 as the main agentic driver, but it might be 10 times as fast. And that doesn't mean that you think, "Well, now I need to go fast. Let's use this." But I think there's different modes of working in your day-to-day work where this model in a different harness or in a different configuration can then be another way to do or get things done versus talking to a, you know, an agent in a back and forth. So in that sense, like we've seen this with like planning modes where people use different models, but it's still like pretty clear that it's a different model and whatnot, but I do think it will be pushed more and more in the background and that people will choose or have different ways to interact with models, and the specific model or its version will not be as visible anymore. **Alessio** [27:14] Yeah. And I know Cody was using StarCoder for inline edits, at least that's what- **Thorsten Ball** [27:19] Yeah **Alessio** [27:19] ... Beyangou said, uh, publicly. So I'm not leaking anything. Does this still seem interesting to you to figure out, hey, is there something in open source that we can use and maybe fine-tune to like make better? Or is it- **Thorsten Ball** [27:32] Yeah. **Alessio** [27:32] Are you still like- **Thorsten Ball** [27:33] Yeah, yeah, absolutely **Alessio** [27:33] ... or just, we just wanna be at the cutting edge, and, you know, that's maybe in the back burner. **Quinn Slack** [27:39] So first, it took people eight or nine months to figure out what 3.5 Sonnet was capable of from when it was released last June. **Thorsten Ball** [27:45] Yeah. **Quinn Slack** [27:45] And then once... This was around the time we were building Amp and Claude Code came out, and you realized that, wow, like a tool calling agent is incredible. And at that moment, everyone, all the smartest people in the world also realized that. Billions of dollars of money went in to training new models and, uh, harnesses based on that. And now it's, uh, September 2025, and we're reaping the benefits of all that investment, and you have so many more- **Alessio** [28:11] Mm-hmm **Quinn Slack** [28:11] ... models coming out. You have the open source models like Quinn 3 Coder and Kimi K2, and they're moving so fast. You have xAI's models. You have GPT-5 that came out, and we're still figuring out w- how to use these things. But it would actually be an incredibly pessimistic outcome if all those smart people and all that money were not able to build anything that was better than Sonnet. So we, in our internal team right now, and this could change, we have about half of our internal team using a different model other than Sonnet as their main way of using Amp, and that's a huge change. In the past, we had done that only to test and begrudgingly, but now we're using it, and there's a different way of interacting with an agent that's not the linear chat transcript. That actually means you don't feel like you're getting a cheaper mid model. You feel like this is a different way of interacting where that speed is really beneficial, and it's more constrained. So things are changing so fast. **Alessio** [29:06] Is the GPT-5 Codex only being available in Codex make you nervous about future availability of like cutting-edge models, and like does that put more emphasis in like figuring out maybe like an open source strategy? **Quinn Slack** [29:19] They make it available to API customers. It's delayed. And if they were doing that, I, I really think that for the most part, I take these model houses at their word, and they wanted to get it out to their first party product as quickly as possible because they honestly need to gather more data, and they're iterating in public. So yeah, I would love it if all the model houses perfectly coordinated with us before they released anything, but- **Thorsten Ball** [29:46] Yeah **Quinn Slack** [29:46] ... I know that would slow them down, and I don't wanna slow them down like that, in the same way that we want our customers to give us grace and help us iterate in public. **Alessio** [29:54] Yeah. I think there's an interesting just dynamic in the market. Like when Cursor switched from Sonnet to GPT-5 as like the default model, that was like, you know, 200 million of revenue for Anthropic that kind of went away and like- **Quinn Slack** [30:05] Yeah **Alessio** [30:05] ... moved on to GPT-5. So there's kind of like... Okay, we're all friends now- **Quinn Slack** [30:09] Yeah **Alessio** [30:10] ... you know, but maybe later that's gonna change. But yeah- **Thorsten Ball** [30:13] I- **Alessio** [30:13] ... it's an interesting- **Thorsten Ball** [30:13] The other thing also is that, you know, if you're building an agent and you're not at one of the model houses, you can use multiple models from different providers- **Alessio** [30:21] Right **Thorsten Ball** [30:21] ... right? So w- which is what we do. Like we... When you use Amp, you're using a model from Anthropic, you're using a model from, uh, OpenAI, and you're using a model from Google. And we also- **Alessio** [30:34] Very close to shipping like a fast open source model that we can use as a different subagent in there too. And you know, when you put it like this, it seems silly to say we only use one model of like this family r- because they all have different strength and weaknesses. **Thorsten Ball** [30:49] Yeah. I think we are one or two months away from a possible news cycle that is the foundation model companies have spent billions of dollars in CapEx and hired like crazy, and now, you know, they're no longer the best in this realm, and there's a huge stampede away from them. That's very possible, and I'm not saying anything new. Just imagine, you know, uh, last May when people were counting Anthropic out before Sonnet came out. Things change so fast here. **Alessio** [31:17] Yeah. **Thorsten Ball** [31:17] Yeah. **Alessio** [31:18] Yeah. And I think OpenAI obviously with Johnny Ive and some of that, it's moving more in a consumer fashion as well. So it's been interesting to see the big push on Codex. I would've imagined them to go more towards education, kind of like big en... I know they have a lot of big enterprise contracts for like ChatGPT for your enterprise- **Thorsten Ball** [31:36] Yeah, yeah **Alessio** [31:36] ... kind of thing. So, um- **Thorsten Ball** [31:38] Yeah **Alessio** [31:38] ... yeah, you guys I think are in a good, in a good spot because you have both like the Sourcegraph trust, like you said, um, but also like Amp. I see a lot of great stuff on Twitter. You know, people are like- **Thorsten Ball** [31:48] Yeah **Alessio** [31:48] ... "I just put all my Amp agents running. I come back. It's great." It's like, I, I think it's now on that wave of like, okay, this is like one of the best tools out there, and like if you're like a serious engineer, you should probably- **Thorsten Ball** [31:59] Yeah, yeah **Alessio** [32:00] ... use Amp- **Thorsten Ball** [32:01] Yeah **Alessio** [32:01] ... at least in some capacity- **Thorsten Ball** [32:02] Yeah **Alessio** [32:02] ... and then make your own choice. Um, how difficult is it to think about what goes in your harness versus like what people should build? **Thorsten Ball** [32:10] Mm. **Alessio** [32:10] So you have custom commands. You have... Y- you've done a great job on like the tooling where like people can put like, uh, executables as tools instead of- **Thorsten Ball** [32:19] Yeah, yeah, yeah **Alessio** [32:19] ... instead of having like an MCP servers. **Thorsten Ball** [32:20] Tools. **Alessio** [32:20] Like, yeah, how much of it you're like, "Hey, we're just giving you the tools," versus how much you wanna be opinionated? With things like, I mean, I think of like compacting conversation as like maybe- **Thorsten Ball** [32:30] Yeah **Alessio** [32:30] ... one of the key commands- **Thorsten Ball** [32:32] Yeah, yeah **Alessio** [32:32] ... that people have, and like in Clockcode you can give a custom prompt to compact. **Thorsten Ball** [32:36] Yep. **Alessio** [32:36] Like, uh, w- what's that discussion like? **Thorsten Ball** [32:39] Yeah. The main assumption again, everything's changing. We've gotta be able to move fast. That means what you want is... I don't use the picture of a harness often. What I use is like a, a scaffolding. Like you wanna build a scaffolding around the model, a wooden scaffolding that if the model gets better or you have to switch it out, the scaffolding falls away. ### Harness **Thorsten Ball** [33:00] You know, like the bitter lesson. Like embrace that a lot of stuff might fall into the model as soon as the model gets better, right? Because then it can remember more whatever, and it doesn't... Why invest three months in like a, a separate apply model when the next generation, the, the, you know, .7 version or .8 or whatever version of this model can now do all of the edits on its own. So that's again the bigger thing. And with that in mind, we really try to restrict a lot of the features that we add around the model. And you can do a lot of stuff. Like we could be busy all day adding stuff in our clients and whatnot, making the product more complicated, but we don't want to. So that's, that's the first thing. The other thing is we're living in strange times. We're living in strange times from a product development perspective, where basically I think the old triangle of design, product, and engineering, it's kind of changing. It's not a triangle anymore. I don't know what shape it is anymore- **Alessio** [33:58] Right. **Thorsten Ball** [33:58] ... with the blob, whatever. But it's not, it's not a triangle anymore. And the reason for this is, is because you can't build a roadmap. You can't say, "This is what we're gonna build in the next six months." People don't know yet how these models are... can be used to their full extent. Everybody's figuring this out on the go. That's another thing. The other third thing there is, we just talked about this while having coffee before coming here, is that we're now in, in a world where people... The, the only UI basically is like a text UI, and you can use this in the wrong way. And the example I used earlier was if, you know, you buy Jira, for example, but you use it for your shopping list, Atlassian is happy about this, but that's not what they built the product for, right? But you can use it in the wrong way and still get results. The problem with LLMs and a lot of the models is that you can use it in the wrong way, and it looks like you're getting results, you know? Like you can use OpenAI, ChatGPT to look up serial numbers or, uh, you know, technical specifications for a camera or something, and it will tell you this, you know? But the... it might be wrong, or 99% of the time or 98% or 95% of the time it might work, but in 5% it might not work. So having non-deterministic LLMs as the heart of your product is something unprecedented that we have in software, I think. So with that in mind, a lot of the features what we see, you know, where people build like elaborate workflows, like I have my custom slash commands and they trigger custom subagents, and they in turn trigger custom MCP tool calls on... behind which again another model is doing inference again and taking the input and blah, blah, blah. I think a lot of this will and has resulted in hangovers where people realize, oh, like this looks like it's a deterministic workflow. It looks like it does the thing that I want it to do, but actually I can't use it if it only does it in 98% of the time. So that's something we're really conscious of, where I think everybody's experimenting, everybody's sharing their experiences, you know, the Thread Boy tweets about what to prompt, where, and how. But you have to be super strict about, you know, not giving users a false sense of what the product can do and how reliable it is, because I think it's dishonest in some way, and it doesn't lead to good results. And, you know, just as an example, I think over the last three months, I would say we're, we're ahead of the curve, like using Amp internally. Like we're ahead of like the mainstream a- agentic adoption by like, say, a month or two, where we've tried a lot of this stuff and then realized, oh, this wasn't the best use of our time or the tokens- And now you see a lot of other people, um, waking up to this. Like famous on Twitter, Armin Ronacher, the y- the Python developer from Austria, he's done a lot of good stuff with Claude Code and shared a lot of his learnings. And you could see that the way he tweeted was super excited, like a lot of things, "I can now do this and this and this." And then a month later it's like, "Oh, maybe, you know, having eight remote control agents that I control with my phone and let them run for 20 hours, maybe that's not as productive as I thought it would be." And yeah, it's something that we're super conscious about. ### Pitfalls **Alessio** [37:19] What are those things? What are, like, the failure modes that you heard from customers where it's like, "Hey, we tried Amp and it just didn't work at doing XYZ"? Is there a collection of those that you guys use as almost like a North Star as you keep building, or? **Thorsten Ball** [37:34] But I think, like, one of the things is, um, the whole vibe coding stuff where people just use it and, you know, they're like, "Hey, I spent 10 bucks in tokens, and it didn't build me the full app", or something. Um, the failure mode of outsourcing the thinking but not the typing, which I think it should be the opposite. You still have to know engineering. You still have to know how to program. You still have to know your application and its architecture, how it's deployed, and then basically use the agent to do the work that you would have done, but you have to know what the desired outcome is and whatnot. Like, that's a common one where people just, you know, hands off the wheel, "Agent, you go and write this for me." And then turns out a couple hours later, oh, actually this... Nobody understands it. It's spaghetti code. **Quinn Slack** [38:18] Amp, it- it's different from the products it competes against. So we've had one head-to-head loss with Amp where we lost against the, you know, usual players, and the reason why is, uh, one of them discounted their other product 100% for two years. The other one discounted it 85% for two years, which is just crazy. And we wouldn't wanna do that because are we really gonna learn from that? And then how is it gonna be used? Is it gonna be used in a different way? Um, so usually the way that we w- might lose is there's some other product that would go to 80% of the devs in a company that is like the base layer. Sometimes that's Copilot or Cursor, and Amp is more expensive, it's more powerful, and they'll give it to that 20% of devs that they trust more. And in a previous world, any software company would say, "Oh no, we need to get 100%. That's... We don't want our competitor getting in there." But actually, that means that we're able to even more focus on being bold and crazy because all those devs can always fall back to a Cursor or a Copilot. So we actually really like that kinda deal. **Thorsten Ball** [39:22] Yeah. The other thing there, w- I think a bunch of questions already touched on this, is that, um, talking about segmentation or market or the ideal user, again, everything is changing. So what we try to do is we try to, you know, build a tool for people who are at the frontier or m- at least curious about it and want to figure out how to use these agents in the best possible way. And that's based on the assumption that if you build for the mainstream user who not... You know, mainstream sounds like... I don't know. It, it sounds bad, but what I mean is- **Alessio** [40:00] Yeah, yeah **Thorsten Ball** [40:00] ... what I mean is if you build a product for somebody who does not know what a good prompt looks like, you will fall behind right now because you will spend time and resources building stuff like the prompt enhancer and, like, blah, blah, blah, blah, blah. But then you will end up building this, and you miss the next step change that might happen. So the way we think about it is we build for the people who already get that a lot of stuff is changing, but we wanna leave the door open. If you're open to learning new things and you wanna learn how to use AI and agents in your workflow, please come with us. We're happy to have you. But if you're skeptical and you, you think, "Prompt engineering, that's a bullshit term. I don't care about this", we're not right now building a product for you because we would fall behind. **Quinn Slack** [40:46] Yeah, so prompt enhancer, that's a bullshit feature that doesn't actually work. The theory behind it is nuts because what helps LLMs is not tricks and phrasing your prompt in a certain way. It's fundamentally information that you have in your head that you can bring into the prompt, and if you don't have that in a prompt enhancer, a LLM cannot magically conjure that up. **Thorsten Ball** [41:06] Yeah, yeah. **Quinn Slack** [41:07] It cannot narrow the search space for you. Custom subagents. I... The way that we disqualify that as something we wanted to build at this point is because you look at all of the tokens that you're sending to the model, and it's so many more. It's, uh, you know, so much more convoluted. Um, it... We don't think that these models are trained in a way that would support this use case and, and the output of this going in here. It's so much harder to debug. And MCP is another thing. MCP has done a great job in getting products to expose the verbs that agents might wanna, you know, interact with, although in most cases, they do not actually get the right verbs exposed. But as a user-facing technology, it is such a common failure mode where a user will go and add in some MCP servers. Auth is a huge pain, but let's say they get over that hurdle. Then they have, I don't know, 50 tools exposed that often are too low level granularity, and it takes a ton of tokens in the model. It makes everything slower and more expensive. They're often misused, and it's just not a good experience. So, you know, there's all of these things that we've said no to, and other tools are bringing them in, and they're saying yes to all these things. **Thorsten Ball** [42:20] Yeah. **Quinn Slack** [42:20] I think it feels like they're making progress in the meantime, and people retweet, and people talk about how they're able to do these amazing things. But just the simplest example that seems so obvious, and frankly, it confounds me that pe- more people don't do this, you make it so that my Google Docs and Notion and Linear and GitHub Issues are all accessible to my agent. The vast, vast majority of developers who use Amp or Claude Code or anything else, they don't have all those context sources set up. That seems like such a slam dunk. So- Before we would do that, and we've, we built that, we ripped it out, before we would move forward with that, we'd have to get an answer even for our own usage, why are we not doing that? And it's frankly still puzzling to us- **Thorsten Ball** [42:58] Yeah **Quinn Slack** [42:58] ... but we're not gonna touch that until we get confident about that. **Thorsten Ball** [43:00] A- and to come back to the example, you mentioned compact. We have this in the product, but again, the hand is hovering over the rip it out button because I think compact is such a alluring thing where people think, "Oh, you know, I ran out of context. I hit that button, now I'm back to the start." But you lose signal, you lose data, and it's something where are the models really good enough? Is compacting good enough to really glance over this, that the user doesn't have to worry about it? Or is it something where you would have to somehow make it clear to the user that, "Hey, look, your conversation has 50 messages back and forth. If you hit compact, this is, you, this is all gonna become blurry. You know, you're gonna compress it, and you lose signal, you lose fidelity, and then you put it in a new context window. Are you sure this is the right trade-off?" And some users are, but again, like it's strange times because now we have like this thing at the heart of our software, this, this, you know, orb from outer space that can do sometimes whatever it wants. And it's, it's strange to build on top of this, and it's strange to educate your users about this, that this is the thing, right? Like imagine, you know, the end of the '90s PC era, you had to build Microsoft Word, and then you say like, "Well, at the heart of this new personal computer, the Pentium III whatever, there's a weird orb from outer space, and sometimes if you bold text in Word, it actually makes it italic, you know?" **Quinn Slack** [44:25] Right. **Thorsten Ball** [44:25] But that's the situation we're in. Like, that's the fact. Like, it doesn't always bold the text. I mean, it underlines it if you reach 150 tokens or 150,000 tokens or something. **Quinn Slack** [44:34] Yeah. **Thorsten Ball** [44:34] How do you teach this to the user? **Alessio** [44:36] Yeah, and you know, we're in the church of context engineering at the- **Thorsten Ball** [44:40] There you go. Yeah **Alessio** [44:40] ... Chrome office. **Quinn Slack** [44:41] Yeah, there you go. **Alessio** [44:41] Um, and when we had Jeff on the podcast, they talked about the context route paper that they did. **Thorsten Ball** [44:45] Yeah. **Alessio** [44:46] And they mentioned specifically in coding, for example, showing previous failures was like not helpful at all- **Thorsten Ball** [44:52] Mm-hmm, mm-hmm, mm-hmm **Alessio** [44:52] ... to the agent. And so I think when you're compacting conversation, there's almost like y- you know, if you have a long conversation, it usually means something went wrong along the way, and you had to like b- go back and forth on like- **Thorsten Ball** [45:03] Yeah **Alessio** [45:04] ... a bunch of things that didn't work, and you're keeping those in. Um, but I've been trying to figure out what's like, what's that gonna look like? In my mind, it's almost like if you take the idea of Linear, which I use, and I give to my agents just to get... Because then I have a canonic prompt for one issue, because often you have to restart- **Thorsten Ball** [45:22] Yep. **Alessio** [45:22] Because it's like it just goes too much wrong, down the wrong path. **Quinn Slack** [45:26] A lot of people don't restart. A lot of people- **Alessio** [45:27] Right, yeah **Quinn Slack** [45:28] ... just try to keep going. **Thorsten Ball** [45:29] Yeah . **Alessio** [45:29] Yes. That's bad . In that case, what can you take from that conversation as a learning and put it back in the upstream issue so that then the issue is like either more descriptive or has like more information, that it's not compacting, but it's almost like how- **Thorsten Ball** [45:43] Yeah **Alessio** [45:43] ... you would do as an engineer. It's like you're doing it in your mind, right? You, you get a issue, and then you start working, and then you kinda update your mental model. Like that doesn't really work for agents, but people are not doing this like small increment in the initial issue. **Thorsten Ball** [45:57] I would say in this case, it's still you cannot outsource your thinking, right? **Alessio** [46:00] Right. **Thorsten Ball** [46:00] Like in this case, I don't think you can expect right now a model to say, "Out of this conversation, this is the most important thing. Let me put this back in the Linear thing." I, maybe, you know, if you phrase it like this and automate it like this, and it's always a perfect conversation, maybe it works. But I think in this case, you still have to be mindful of the context. And what we encourage users to do, for example, in Amp is to start a lot of small threads and be really, you know, do context engineering and s- be really strict about what goes into context and what doesn't. And the other thing that I think touches this on is, um, you know, where a lot of CLI tools, for example, they have super verbose output, and Bazel, sorry to call this out, I'm not a big Bazel... But to just call Bazel out, super verbose output. So then the natural assumption is, "Oh, let's hide this from the user," you know? Like, "Let's, let's abstract this away and summarize the output or whatever- **Alessio** [46:53] Yeah **Thorsten Ball** [46:53] ... or whatever, just the exit code or something." And then you get into this dangerous territory where what you see is what you get is not true anymore, and in the context, what you see is like some other thing in the context, and that can lead to issues. But for me, the meta thing here too is, you know, everything is changing. That means we're seeing this. CLI tools right now are also adapting to being used by agents, so they're changing- **Alessio** [47:19] Right **Thorsten Ball** [47:19] ... their output too. So if you focus on the fact that Bazel will always be verbose and build something for this issue, you might be outdated in half a year, where somebody is like, "No, no, no, we have a Bazel agent wrapper now. This is- **Alessio** [47:31] Right **Thorsten Ball** [47:31] ... not an issue anymore." **Alessio** [47:32] Yeah, yeah. Interesting. Yeah. **Quinn Slack** [47:34] One model that I have is if you are relatively on the cutting edge of using agents, and there's some persistent problem like this that feels kinda out of band, like how the model itself, oh, will update its memory or will update the Linear issue, the model needs to be trained in order to do that better. If it's something like your own coding conventions, that's different, but if it's something fundamental that feels like about, uh, you know, out of band from the agent, the model needs to be trained to deal with memory better or to accept the fact that it might have a incorrect view of its own history if you go back and edit it. And we're feeling these pains right now because people have only been using agentic coding tools for a matter of months. Most people have been using them for like less than three months, and if we're only feeling them now, it takes a little bit of time for a team at a model house to go and do a fine tune of one of their really big models, or they've got other big models, the newer revisions that are being trained, and they can only fit a certain number of experiments like this in. They're probably gonna get half of their approaches wrong. So you can only do so much, and that's Thorsten's idea of going with the grain of the model. **Thorsten Ball** [48:36] And I mean, you've seen this, I'm sure, where a, a lot of users are going through this lesson where they, "Let me just add this MCP server that does everything I want it to do." And then two days later- It doesn't use it. Like, it never calls the tools, and it's like, yeah, that's... Y- it, it wasn't trained to do this. And you can kinda sense, you know, like, they have different philosophies in the model houses. I think Anthropic is, from what I can tell, working a lot or training a lot towards using memory, like storing information. While now ChatGPT o- obviously has this OpenAI, so if you give it a memory thing, yeah, it might use this. But then you have the issue of, well, if I give it this other custom-made MCP that we built internally, and our processes don't map to anything that OpenAI and Anthropic have seen or trained for, it won't be used, and you won't get good results. **Alessio** [49:29] Yeah. **Thorsten Ball** [49:29] And super strange, right? **Alessio** [49:32] Yeah. I read this article for the GPT-5 release about, um, models self-improving for coding. **Thorsten Ball** [49:37] Mm-hmm. Mm-hmm. **Alessio** [49:37] So I basically asked GPT-5, "What are tools that would be useful to you to be a better software engineer?" It's like, "Well, you know," and give a list of, like, 10 tools. **Thorsten Ball** [49:46] Yeah. **Alessio** [49:46] And I'm like, "Okay, implement them." Wrote all the tools, and then I asked it to do the same task I'd done before but with those tools, and then it goes through the whole task, and I'm like, "Which of the tools did you use?" And it's like, "Oh, I didn't use any of them." And I'm like, "Why did you not..." It's like, "You know, to be honest, I don't really need the tools." **Thorsten Ball** [50:03] Yeah. **Alessio** [50:03] "I can just do this task", you know? **Thorsten Ball** [50:04] Yeah. **Alessio** [50:05] And, and I think that's, like, a good metaphor just for, like, the trend of the models, which is like, hey, they're gonna use less and less of these, like, custom-made tools to fix today's issue. I think the things that we can bet on, and I'm curious to hear your thoughts, is like, they're always gonna have some sort of, like, test runtime. Like, I don't think there's gonna be a world in which the model is not gonna run tests and say, "I'm sure this is gonna work." **Thorsten Ball** [50:28] Mm. **Alessio** [50:28] The other one is there's always gonna be some sort of, like, infrastructure as code to then handle the deployment side. So I think whenever there's gonna be some runtime issue, they're gonna need to understand where they're running. **Thorsten Ball** [50:41] Yeah. **Alessio** [50:41] You know? So I think, like, you can put them in a box. Having an actual Docker file and whatnot, it's helpful for them to explain what they have access to. What do you think are, like, other things that you don't expect the model to, like, have in the model that you wanna still expose to it? So if we're gonna assume it's gonna test, we're gonna assume it's gonna have some definition of its environment, are there other things that come to mind? **Quinn Slack** [51:04] I think test is a big one, and there's many different kinds of tests. So we had subagents in Amp, you know, among the, the first to come out with this conception of subagents, which is a separate context window, separate cur- more curated set of tools. And I think there's a lot of potential to take a tool like Test, and right now you invoke it by the bash tool, and you have some complex invocation. Too often it'll run all of your tests, which is noisy, and it takes a long time. If you're in your editor and you've got something nice set up, you can hit, like, a hotkey, and then it'll only run the tests that you need, you know, at your cursor. So giving the LLM a tool like that seems to have a lot of potential, and then that could even potentially be a smaller model, a fine-tuned model for that task. It could be multiple based on what, uh, projects or, or stack you're using, and that, uh, could eliminate a lot of the confusion. Even with a good agents.md guidance about how to run tests, I still see with Amp, and I think, you know, we've, we tried to make this really good, it only gets it right maybe 90, 95% of the time. Sometimes it'll run the wrong test thing, or it won't escape it correctly, and I think we can eliminate that with a subagent. So there's so much more potential to go deep in areas like that. And then for every language it's a little bit different, so handle all those cases. **Alessio** [52:19] Do you feel like that will just be built by each company on their own, or do you think there's, like, a same default that you guys are gonna build, um, for that, that is gonna be effective for most, most code bases and test structures? ### Subagents **Quinn Slack** [52:31] This is where scale helps, and we have a lot of scale, so you know, increasingly we're able to see in this framework for this, you know, Go, the standard Go unit test, uh, package. That's easy. Uh, Vitest in JavaScript, that's easy. And once you start getting into more of the long tail, then, you know, it might have to just fall back to a really good model. But I think that we could probably, um, make something that's optimized for some of these more popular unit testing frameworks, and it's a combination of deterministic stuff and non-deterministic stuff. 'Cause right now in my VS Code, I can hit Apple T if I'm positioned in a test file inside of one of those test blocks, and it's only gonna run that one. **Alessio** [53:08] Yeah. **Quinn Slack** [53:09] So, you know, even that is a benefit. **Alessio** [53:11] And now I'm mostly bottlenecked by Playwright. **Quinn Slack** [53:14] Yeah. **Thorsten Ball** [53:14] Oh, yeah. **Alessio** [53:14] It just takes a long time, man. And- **Quinn Slack** [53:16] But the crazy thing is the vast majority of devs who are building web applications with coding agents do not have Playwright, and if they have it, it is set up in such a shitty way where it cannot really log into their app. They don't have any pattern for that. **Thorsten Ball** [53:28] Yeah. **Quinn Slack** [53:29] So even something like that, that's another example of a subagent that's go and try this basic end-to-end testing flow described in natural language with the running application. And wouldn't it be great if they could also do it in parallel? So, you know, there's all these ways that you can improve. That's a great example. **Thorsten Ball** [53:43] And I think touching on this, we have coding agents. They are productive. They add value. We cannot assume that everything around the agent in dev tooling or code bases will stay static. So I think people are already adapting their code base to be better used by agents, or they're adapting their tooling to be better used by agents. The more descriptive help text or whatever it is. So I think, uh, I don't know, we should have a counter, but everything is changing. I don't know, saying this again, but we cannot build right now with, oh, this is the tool that's gonna stick around, given that all of the code bases and all of the processes and all of the dev tools will stay the same. We have to assume that this stuff will change too, and we have to stay nimble, so we have to make, like, um, short bets or small bets and try and get us, you know, in small steps forward but al- always be reactive to this stuff, that, you know, if people... Again, hmm, I don't... Let's not use Bazel again, but I think, uh, Playwright is a good thing where, um- The feedback loop is incredibly important to working with these agents, like that the agent can see whether what it's doing is actually working. So what we've seen people now do is, well, instead of having the client log and having the browser log and having the database log, let's have one unified log because then it's easier for the agent to just look at this log and make sense of it. And then it turns out it doesn't have to be nicely formatted. It can be verbose. You can just have like JSON line outputs and whatnot because the agent can understand it much better than a human can. And I think that's just this, a little preview of f- more things that we will see where you're like, "Wait a second, this is not made for human consumption anymore. How can we optimize this for agentic consumption?" And then maybe the game changes. **Alessio** [55:27] And there's some things that now we get... For example, in my Vitest suite, I have a knock to record HTTP calls. **Thorsten Ball** [55:34] Mm-hmm. **Alessio** [55:35] So whenever- **Thorsten Ball** [55:35] Yeah. **Alessio** [55:36] Especially for inference, like you can't really mock- **Thorsten Ball** [55:38] Yeah **Alessio** [55:39] ... uh, you know. **Thorsten Ball** [55:39] Yeah, yeah. **Alessio** [55:39] We do classification, things like that. You just need to see what happens. Um, and then we just save the whole interaction, and then the model can actually see what the API returned, like in much detail. **Thorsten Ball** [55:50] Yeah, yeah, yeah. **Alessio** [55:50] And like it can reference it back in the future. **Thorsten Ball** [55:52] Yeah. **Alessio** [55:52] So when you add a new feature, it can look at the test, and it can see what the API usually returns. And it's like, oh, okay, it's gonna have that key and like the content and things like that. I think there's more of that to be done. I think there was maybe also a time in which having console logs was like really bad, and I think there's maybe now gonna be a console log that is like only funneling to like not the actual console in the browser, but like some way for like the agent to see- **Thorsten Ball** [56:18] Yeah. Yes **Alessio** [56:19] ... all of the details of like everything that is happening. What I haven't figured out is like how do you instrument that? Because you cannot put a whole bunch of console logs that go somewhere else in the code because then you're also polluting the context window of the model, right? So you need some other way to do it. But I think, yeah, the more you log in, the more the model can kind of like self- **Thorsten Ball** [56:38] Yeah **Alessio** [56:38] ... iterate, but I, I- **Quinn Slack** [56:39] And you just described like five approaches that seem absolutely worthwhile to go explore to improve how coding agents work. **Alessio** [56:47] Let somebody do it. We can do some of it at Kernel Labs, but we cannot do all of it. So- **Quinn Slack** [56:51] Yeah **Alessio** [56:51] ... let's, somebody help. **Thorsten Ball** [56:53] But, uh, again, like, uh, the world around us is also changing. Uh, uh, Jose Valim, the creator of Elixir and, you know, contributor, co-contributor to Rails, can't remember the name, but basically they have a new framework tooling out that is- **Alessio** [57:06] Phoenix. **Thorsten Ball** [57:07] Yep. Yeah, it's for Phoenix, right? But it's the, the name of the... I can't remember. **Alessio** [57:11] Okay. **Thorsten Ball** [57:11] But it's, it's about, well, what if you build a framework for an agent tool? What if the agent is integrated into the framework so that you can, if the application fails to run, you can ask the agent that has access to all of the context, and that's gonna be more and more, I think. Like, a lot of the, you know, developers will build stuff because they're fed up with like copy and pasting stuff around. So we're gonna see this in developer tools. **Alessio** [57:36] Yeah. Well, I mean, Rails was like one of the first frameworks that I know that in the error page, they had a CLI that you could like- **Thorsten Ball** [57:43] Yeah, yeah, yeah. Exactly **Alessio** [57:43] ... use the local context. **Thorsten Ball** [57:45] Yeah, yeah. **Alessio** [57:45] You know? And I think like more of that. Like in Next you have the copy to markdown- **Thorsten Ball** [57:51] Yeah. Mm-hmm **Alessio** [57:51] ... now button. **Thorsten Ball** [57:51] Yeah. **Alessio** [57:51] Whenever you have an exception, you can copy to markdown, put it in there. **Thorsten Ball** [57:54] That's the first sign, yeah. **Alessio** [57:55] Um, yeah, but I think there should... And in their docs too, you can like copy to markdown. **Thorsten Ball** [57:59] Yep. Yeah, yeah, yeah. **Alessio** [57:59] They say the thing, but then it's like you can only copy to markdown the whole page. **Thorsten Ball** [58:03] Exactly. **Alessio** [58:03] And it's like, well- **Thorsten Ball** [58:04] Exactly **Alessio** [58:04] ... you know, maybe I only wanna do this section or like I wanna do one, two, three. I, yeah, I, I don't know. I think that's why the Mentifys of the world, Stainless, all these company that do kind of like API docs and API generation from docs are like getting a lot of interest. I think you'll get more of that, but it's hard to get people to move over, you know? I'm sure you see it with like some of the Sourcegraph customers. It's like, how am I supposed to re-instrument this whole code base that is like 15 years old and like- **Quinn Slack** [58:31] Well, it's true, but what we have said is we explicitly are building Amp for the people that do want to move. **Alessio** [58:37] Right, exactly. **Quinn Slack** [58:37] And that's been so liberating. **Alessio** [58:38] And I think that that's the great thing about what you see in the market today, which is like you have all these companies that are like so AI first and like just use it and they're great, and then you go on Hacker News, and it's like, "I've never got a single good result from AI." **Thorsten Ball** [58:51] Yeah. **Alessio** [58:51] And I'm like, well- **Thorsten Ball** [58:52] Yeah. **Quinn Slack** [58:52] Yeah, yeah **Alessio** [58:52] ... obviously that's not true. And like maybe the extreme, the extreme is definitely true though. I think to me that's kind of like the thing is like the people that are spending $100,000 a year on Amp with two people, obviously they're getting value. It's not like they love burning money. **Thorsten Ball** [59:09] Yeah. **Quinn Slack** [59:09] Yeah. **Alessio** [59:09] But the people that are negative, to me, that's not always true because it's easy to be negative, and like it doesn't cost anything, right? **Thorsten Ball** [59:15] Yeah, yeah. **Alessio** [59:15] To put a comment- **Thorsten Ball** [59:17] Yeah **Alessio** [59:17] ... that is bad. And so what's gonna be the thing that forces the rest of the market to be, "Whatever, man. Let's just get on Amp and, like, make that work"? **Thorsten Ball** [59:27] They just have to see this work once or twice. I think, you know, we've been in developer tooling for a long time with Sourcegraph, and it's always been hard for the last, say, 10 years to get a company to adopt a developer tool that does not immediately fit into their code base. Because the code base, that's the standard. Everything else has to adapt to our code base and our processes and whatnot. What we're seeing now with agents is as soon as somebody has seen what it can do, they have such a multiplying effect, or they bring so much value that people are willing to adopt the code base for this. Like the first time in how many decades where people are like, "Maybe our code base is wrong." **Alessio** [1:00:12] Right. **Thorsten Ball** [1:00:12] Like maybe, maybe we should change the way we develop code to make more use of this. So I think people have to see this and, and, and then the agents will pull them along or like the, you know, the value this, this brings will pull it along. **Alessio** [1:00:26] Yeah. I'm curious, so I was on the board of a company called Launchable, which was founded by Kohsuke Kawaguchi, who built, uh, Jenkins. And the idea behind Launchable was like, well, instead of running all of your tests, we'll use machine learning to figure out what tests are impacted by your PR and just run the small subset of them. **Thorsten Ball** [1:00:43] Yeah. **Alessio** [1:00:44] And I think like what we found, then the company got bought by copies, but it was like in a lot of- Companies who go in there and they're like, "Oh, well, how can we trust it though? Let's do a POC." And then you do the POC, and it's like it works great for the subset. Well, you know, it worked for the subset, but, like, is it gonna work for, like, the whole test suite? **Guest** [1:01:00] Yeah. **Alessio** [1:01:00] And then you do a whole process. And I think with coding it's like for some companies it's like they see it work on one task and they're like, "It's worth trying on every task." And then there's another subset of companies that are like, "Well, you know, it works a little bit on the front end, but it doesn't work on, like, my Java service back there, so I'm not gonna use it at all." I haven't quite figured out what's gonna be the market pressure to make those people move along, you know? Uh, but it's like you said, it's like, you know, for some people it needs to work once. Maybe for some other people it's gotta be one task that always fail. My one task that I always use, we have built this KernelGem product which is like an MCP playground and tester, and I have a task which is like add YOLO mode, which is, you know, let a user toggle between autorunning. **Guest** [1:01:44] Yeah. **Alessio** [1:01:44] Which sounds easy, but it's actually quite hard without LLMs work to, like, stop inference to approve a tool and then run it again. And every model was failing until GPT-5 Codex and Codex CLI was, like, the first time I got it in, like, one shot it made the whole thing. And I wonder if everybody should build some sort of, like, four or five tasks that are like, "Okay, if you can actually do this end to end," then I'm like, "I'm in." But I feel like people are still in denial of, like, that's gonna work, you know? **Guest** [1:02:11] The same people- **Alessio** [1:02:11] So they don't wanna have... Yeah, they don't wanna have the conversation at all. **Guest** [1:02:14] Yeah. If you look at that early adopter, the, you know, laggards, that chart of technology adoption, there's a reason why the early adopters are the tiny little start of the curve, you know, 3%. And it feels like so many of these arguments are people saying, "Well, what if we made a product that was for the early adopters but somehow made the laggards also adopt it early? Why aren't we going after that big market? It's the vast majority of the area under the curve." And it's like because they fundamentally do not want what you are building, and maybe they should. Maybe they're gonna realize that, but you're not gonna make them realize it. Or if you waste your time trying to make them realize it, you're gonna be trounced by hopefully people like us that are only focused on the early adopters. It's a total mindset shift. **Alessio** [1:03:01] Yep. **Guest** [1:03:01] And if you are just focused on building something for early adopters and you literally do not care, and you set up your entire business and product to not care, not have to care about the people that are laggards, you can do a much better job, and that's what we're experiencing now. ### Outer Loop **Alessio** [1:03:14] Let's talk about the outer loop because I think that's kind of, like, the next step at least for me. **Guest** [1:03:18] Mm. Mm. **Alessio** [1:03:18] It's like I think the coding agents themselves do great on a task-by-task basis, but then there's like, you know, PR review, which GitHub is, like, so slow and so clunky, and it's so, like, order by file versus, like, I think we should get to a world which is, like, more semantic. It's like, hey, you know, these are really, like, the 50 lines of code that matter to look at, and everything else is, like- **Guest** [1:03:40] Yeah **Alessio** [1:03:40] ... it's fine. You can, like, skim through it. How do you think about that when you wanna... when especially when you think about async agents? You know, there should be an easy way to spin them up, which I think is fairly clear. But then I'm not sure if there's yet an easy way to catch up on what they're doing. You know? What I found when I use Conductor, like Vipecam Ban, it's like I spin out five, six of them, and I'm working on them, and I kind of jump between them, and then my wife is like, "Let's have dinner." And then we have dinner, and I go back and I'm like, "What the fuck is going on here again?" **Guest** [1:04:08] Mm. **Alessio** [1:04:08] It's like, which one is doing what? And it's like it's hard to, like, just at a high level see what each of them is working on, where it's getting blocked. Have you guys seen anything that works there? Have you been thinking about building any tools in that space? **Guest** [1:04:21] I agree, right? I, I feel this too. I think, you know, with our internal experiments, I think, you know, for example, this idea of well, I just spawn 10 agents and they work and I control them. I think Stevie's doing this- **Alessio** [1:04:36] Mm **Guest** [1:04:36] ... and he has like a whole workflow around this, and it seems to work for him. But for me, I guess I'm a one-tasker in my mind. Like, I need to... I can't do this. Like, I cannot control five agents at the same time. And then when I do it asynchronously, I realize that I need to be really strict about how I review what they've done, and that I also don't jump between them. And then it's also, you know, making sure that you don't miss anything. Like, I spun up so many agents and then haven't checked back on them because I forgot that they actually run, so that's something you need to build in the product. But yeah, I don't think it's figured out, you know? Like it's a... there's so much to do still. **Alessio** [1:05:13] Yeah. It's wide open. We think of it right now, like if you're playing chess, you can play one board at a time or the people in New York City, Central Park, who play against 10 different tables at once, and they go and they sit down in front of a table, they get oriented, they make a move, and then they go, and that's what we're trying to build. And, um, it turns out even if you've got a coding agent running in your editor in the CLI and then it makes a big diff, you've still gotta understand it. **Guest** [1:05:37] Right. **Alessio** [1:05:37] And it just becomes even more important when you have a lot running in the background. **Guest** [1:05:40] Yeah. **Alessio** [1:05:41] So we wanna make it easier to orient yourself with what's the change, and there's a lot of stuff that is not in the realm of coding agents that would help, like having a deploy preview consistently available so you could just click and click through it. And then we wanna make it fast for you to make a move and then, you know, get on with your next thing. **Guest** [1:05:58] Yeah. Or, you know, just UI so at a glance you can see... I don't know what it is yet, but at a first glance so you can see what the agent actually did without having to go- **Alessio** [1:06:08] Right. Play **Guest** [1:06:08] ... and read through, like, the emoji summary, finally we have it, and blah, blah, blah, stuff like this. But to come back to, like, your question of, like, the outer loop, I think, and, you know, if Beyond was here, he would talk for a long time about this because he's passionate about it, that the inner loop has changed a lot in that, you know, um, uh, write, test, review and whatnot, it's that you now review a lot more code. **Alessio** [1:06:32] Mm. **Guest** [1:06:32] And what effects does this have? For me, for example, we don't do any formal code reviews on the Amp team, but it doesn't mean that code isn't reviewed because we use, you know, Amp to write 80 to 90% of our code base. But that means everybody should review the code that the agent wrote, so it's reviewed by at least one person, right? And- **Thorsten Ball** [1:06:51] That's not reflected at all in GitHub yet **Alessio** [1:06:53] Right **Thorsten Ball** [1:06:54] Like GitHub is still based on this other mode where you tag somebody- **Alessio** [1:06:57] Right **Thorsten Ball** [1:06:57] ... but then it's like, well, I actually went through two agents to produce this code, and I reviewed it three times. Do I now tag five other people? And right now we're stuck in this mode where people would say yes, but I don't think it's gonna hold that much longer. **Alessio** [1:07:09] Yeah. The other thing I noticed is like merge conflicts. Like- **Thorsten Ball** [1:07:13] Yeah **Alessio** [1:07:13] ... I used to have very little because it's like, you know, I know what I'm working on- **Thorsten Ball** [1:07:16] Yeah, I know **Alessio** [1:07:16] ... and if I'm doing multiple task, I know how this is gonna impact that, and I'm gonna build towards it. **Thorsten Ball** [1:07:20] Yeah. **Alessio** [1:07:20] Versus the agents, w- especially when you run them parallel, it's like they just start to change whatever is convenient to them, and then it's like across them- **Thorsten Ball** [1:07:28] Yeah, yeah **Alessio** [1:07:28] ... they're like changing the same thing. And so one thing we've been thinking about building is like, you know, how do you do better cross-agent orchestration of like these changes? **Thorsten Ball** [1:07:37] Yeah. **Alessio** [1:07:38] So I built for like the GPT-5 post, this like task manager that's like CLI first, and basically any agent can like append what files they're touching, and then they can read what files other agents are touching- **Thorsten Ball** [1:07:49] Yeah **Alessio** [1:07:49] ... and see what those diffs are to like implement them back. But then I think the question is like, well, maybe what they're doing now doesn't end up being the final thing, and now you're wasting all these tokens, like reviewing- **Thorsten Ball** [1:07:59] Yeah, yeah **Alessio** [1:07:59] ... all these changes before review. I think at this point it's like, is Git well-designed for this future world that we're going into? You know, there's like, I think everything is back on the table. **Thorsten Ball** [1:08:08] Yeah. **Alessio** [1:08:08] I think maybe, you know, five years ago it was like, you know, there was like a couple YC companies doing, "Oh, we're like a new version control system," and I'm like, "Look, man- **Thorsten Ball** [1:08:16] Yeah, yeah **Alessio** [1:08:16] ... I'm not, I'm not really interested- **Thorsten Ball** [1:08:18] Yeah **Alessio** [1:08:18] ... in listening at, at this stage." And same with programming languages. It's like, you know, when, um, Chris Lattner even started working on Mojo, it's like, okay, because of AI, I understand why you need to build a superset of Python. And I think now with agents it's like maybe clear why TypeScript should win because of- **Thorsten Ball** [1:08:35] Yeah **Alessio** [1:08:35] ... type checking is like very good for the model- **Thorsten Ball** [1:08:37] Yeah **Alessio** [1:08:37] ... to like do self-improvement. What are like the other things that you sh- **Quinn Slack** [1:08:41] I think the interesting flex here is people assume that coding agents need to meet the bar of writing the exact same kinds of software to the exact same standard, and that is not necessarily an assumption that end users, consumers will apply. If they have software that's much faster, cheaper, much more personalized, if they can conjure it up on their own, then yeah, you're gonna tolerate if the loading state of this thing doesn't quite, you know, work correctly. So changing user demands and standards is an interesting thing that you can flex here. **Alessio** [1:09:13] Yeah. What do you think about that? You know, we've been thinking about enterprise software moving more towards user-generated content, which is like, hey, you know, expenses are like a great example of like, you know, all these expense tools, where, where there's so many companies when like the core action that you're doing is like take one line of expense and tag it with different things, but then you have to like set up all these categories and whatnot versus like just generate it for my company and for like each team separately because they have different things. And it's like, to me that feels like more and more of that will become true, and then the real value is like, you know, what's kinda like the underlying data store or like data stores that like you're feeding into this. And I know some enterprises are building, already built kinda like internal like lovables basically- **Thorsten Ball** [1:09:57] Yeah **Alessio** [1:09:58] ... where each employee can kinda like create a simple tool, and then they connect the tool to like internal data stores, and they might be the only users of it. There's nobody else that does it. And I'm curious how you guys think about... I know that Bold.new, for example, now has Cloud Code integration. **Thorsten Ball** [1:10:13] Mm-hmm. **Alessio** [1:10:13] Like, where do you see the line move between like software engineers build software, and like obviously Amp is like a great tool for that, versus going more upstream, which is like any non-technical people can also plug into the code and like build things on top of it. That feel, that feels in a way very different but also very similar in like the challenges that you need to solve for. ### Enterprise UGC **Quinn Slack** [1:10:37] I, I think this idea of non-technical is the wrong way to look at it. There are always gonna be people that are good at unambiguously specifying what they want out of a computer, and we've had non-coders, including one of our board members, who built something with Amp that replaced a like 250K a year piece of software that he used for a lot of their internal, uh, fund tracking. And he maybe took one computer science class. He hasn't really coded, but he's a really smart guy, and he knows how to unambiguously specify what he wants, uh, to his CEOs certainly, and now to a computer as well. So if you can get people like that a tool that's really powerful, they don't think of themselves as a non-technical person. I think that's, that's just such a, a bad mindset. So we wanna build for the power user, and if that person has not been a coder, but they can pick it up really quickly, that's great. Again, we're completely focused on the people that know how to and want to get the very best out of this and that want the agent to win, that aren't trying to be like, "Oh, you know, nada." **Thorsten Ball** [1:11:37] Yeah. **Quinn Slack** [1:11:37] "Hey, it didn't do this thing. Tell me when it does." **Thorsten Ball** [1:11:39] Yeah. Yeah, we had this at the, at the start a lot with people. Whenever you have like an AI tool, I think there's a natural, uh, tendency by engineers to get it in a gotcha moment. You know, like, "Oh, yeah, I asked it this, and it didn't know this." **Alessio** [1:11:53] Mm-hmm. Right. **Thorsten Ball** [1:11:54] And then it's like, are you trying to get something out of it, or are you trying to get it to fail? And you know, it's not, it's not worthwhile to build for- **Quinn Slack** [1:12:02] Yeah **Thorsten Ball** [1:12:02] ... somebody who doesn't want to fail. **Quinn Slack** [1:12:03] Yeah, and you know, actually, if you fast-forward how the world is going, you're seeing already over the last few years companies have really slowed down their growth in engineering headcount. This is a global phenomenon. You're seeing engineers like, uh, here on the Amp team and other companies that are using agents really heavily, they're cutting out the middlemen. They're putting the people who are building the product closer to the customer 'cause you can go and hear an idea from a customer. Literally in the meeting you can kick off an agent to go and build it, and then you have a first draft of it. So overall, the person who's using the coding agent is getting so much closer to the problem. They're also gonna share more in the rewards from solving the problem because without needing to share the profits with everyone else, there's naturally, you know, more to go to them. So I think if you fast-forward this, it's not that the firm or big companies are gonna completely go away, but you're gonna have people that have an incredible vision in their head and that are so close to the problem and have an incredible incentive to go solve that problem, equip them with a coding agent. If you're gonna build a coding agent that those people want, that is way better and more valuable, and you're creating more value. You're allowing more new things to be created in the world than if you were building a coding agent that is for the median developer that makes them 30% better. So that's who we're targeting, and I don't think that that will necessarily look like vibe coding. Vibe coding is this- Really unproductive thing to discuss 'cause everyone has a different definition of it. And too often it's having the agent write code with sh- with poor feedback loops and poor quality control, and I don't think that that's valuable. But it's giving that person the ability to build something truly great really fast when they're so incentivized, and they will have every desire for it to work well. **Alessio** [1:13:41] Yeah. And I know we're getting close to time, but a couple things I wanna touch on. So Thorsten, I was reading through your blog. You left Sourcegraph a year and a half ago, then you joined back. Good job, Quinn. Bring him at home. Uh- **Thorsten Ball** [1:13:53] Thank you, Thorsten **Alessio** [1:13:55] But when you wrote a post about leaving, one thing you wrote is that when you first joined in 2019, one thing that Quinn told you is like, "Hey, you know, Sourcegraph is your playground." **Thorsten Ball** [1:14:04] Mm-hmm. **Alessio** [1:14:05] "And you have skills and talents, and I want you to use those skills to like, you know, move the company forward." How do you take this idea of like the power user getting close to the customer and like how people are gonna build teams overall? Like there used to be engineering and product, like you were saying, the triangle. That's kinda going away. What are like the type of people that you think are gonna be most successful? Like how should people think about structuring teams? It's like obviously you're doing this with Amp in a way, right? You're like building a sub-team and sub-product within a larger company. Any tips that you have for other founders and executives? ### Wrap-Up **Quinn Slack** [1:14:39] Thorsten is incredible, and Amp would not exist in any way without him. He has a strong internal constitution of how he uses it and what's real and what's not, and it's so easy to get carried away with the hype, the possibilities, especially when you see other people, a lot of other smart people who are getting carried away by it. Thorsten has this incredible ability to stay grounded, and that, with everything changing so fast, with it being such a hype cycle right now, that's really important. Also, uh, you know, just these first principles thinking, like how we've completely rethought how we build everything in Amp based on how should we actually do it rather than what has come before. Thorsten is the rare person who's been at bigger companies, who's seen how Sourcegraph, how we build enterprise software and, you know, not the Google way, but in a different way, and has taken the parts of it that work and not, uh, the parts that don't. So all of that combined with someone who's an incredible engineer, incredible writer, communicator, that's a really powerful combination. So find those people. And then what I said when he rejoined is he is the dictator. That made him feel really uncomfortable- **Thorsten Ball** [1:15:50] It did **Quinn Slack** [1:15:50] ... as you can see. **Thorsten Ball** [1:15:51] Still does. **Quinn Slack** [1:15:51] I hope you cut to his face. But that's, that's exactly what you have to do, and have to put so much trust in people like that. And that also shows everyone else at the company that they can do crazy stuff, that they can, you know, go way beyond. They can take it to the extreme. They can make mistakes, and that's still okay because we're not trying to build something that's gonna, you know, go really big in the current state. Amp is growing incredibly fast, but the most important thing is we're building the, the coding agent god, that thing in the future, and that's something that we're all in search of. So in it, none of the mistakes, none of the successes in the month-to-month timeframe really matter. It's all about getting ourselves in the right trajectory, and you gotta do crazy stuff. So equipping Thorsten to do crazy stuff and to take the ideas that he has and make them scale up with all the, the reach that Sourcegraph has, that's been my goal. **Alessio** [1:16:48] On the first principle of thinking, how do you think about that? And so there's the world of evals and there's the world of vibes, right? **Thorsten Ball** [1:16:56] Yeah. **Alessio** [1:16:56] How do you approach it? Like, how do you look at the product and you're like, "Okay, this is good. This is bad. This is what we need to improve"? Is there something formal that you guys use internally, or is it mostly you as the dictator, uh, directing? **Thorsten Ball** [1:17:10] Well, two-part answer. I think the first part is, to also answer the other question a little bit, is what I've seen become more important or the shift I've seen is that, you know, I said the triangle, um, of, you know, PM, designer, engineer. I think as an engineer or any of the three, you now need to know a lot more about the other parts. Like as an engineer, you cannot see yourself as the person anymore who types out a spec or turns a product PRD into code. I think you need to be aware of business. You need to be aware of a product. You need to know and have some taste for software. Otherwise, I think, uh, the value of your work will diminish over time because the pure typing out of code, for most of the code, you know, exceptions being a John Carmack and, you know, whatever. For most of the code, I think the value will diminish. And we've already seen this. Like compare a GitHub contribution chart today, its value to- **Alessio** [1:18:09] Mm-hmm **Thorsten Ball** [1:18:09] ... say, two years ago, right? And, you know, to come back to the, the second part, like vibes and, you know, whatnot, I think we don't have any set evals. We don't, and this was controversial up until a week ago, I think, when, uh, I, I think- **Alessio** [1:18:25] Mm **Thorsten Ball** [1:18:25] ... uh, Boris from-- Or two weeks ago from Anthropic said they don't have evals- **Alessio** [1:18:28] Yeah **Thorsten Ball** [1:18:28] ... for the coding agent too. But, um, we don't, and we haven't had them. I've built evals before. I fine-tuned models before. I know that they're good. I love evals. I was addicted to LLM as a judge. I wrote about LLM as a judge. But for a coding agent who's supposed to work in, uh, many different code bases, who's supposed to work with many different types of prompt, who's supposed to work with many different type of tasks, it's a time investment that we cannot afford with everything changing and having to stay fast. And if you ship 20 times a day, you will get a lot of good feedback. I s- I swear you I could tune my system prompt a little bit now, and then I would say by this evening people on our team would go Why does it call this tool so often? Like, what's going on? What did we ship? **Alessio** [1:19:13] Right. **Thorsten Ball** [1:19:14] And that's incredibly valuable feedback, and that's incredibly valuable, you know, when people dock for the product and use it all day. And how do I make these calls? I don't know. Like, I think it's experience of, like, I think about software a lot. I l- love using software. I, I listen to a lot of business podcasts. I read a lot about business. I listen to a lot of software podcasts. I read a lot about software. And then I try to project, like, what does the business need? How can we get growth to 10X? How can we get our users to 10X? How can I use my engineering capabilities to serve as a function of the business to reach those goals? How can I organize the team or get the team to help me reach those goals, or together reach those goals? And, you know, it's hard to explain, but it's like- **Alessio** [1:20:00] Right **Thorsten Ball** [1:20:00] ... I, I feel like in this year truly here at Sourcegraph, like everything I learned over the last, say, 15 years of my career is, is coming together in the sense that all of the hours spent listening to the Acquired podcast to help me as much as, you know, reading Hacker News- **Alessio** [1:20:18] Right **Thorsten Ball** [1:20:18] ... for how many hundred hours- **Alessio** [1:20:19] Yeah **Thorsten Ball** [1:20:19] ... and writing code for how many thousand hours, you know? Um, and with code being now this tool that you can wield much easy or much fast or much more often, I think it's become much more important to how do you wanna wield it and when and for what reason. **Alessio** [1:20:35] I think the hard to explain is a great explanation why, you know- **Thorsten Ball** [1:20:39] Yeah **Alessio** [1:20:39] ... you just cannot one shot create these things because there's a lot of, you know- **Thorsten Ball** [1:20:43] Yeah, yeah, yeah **Alessio** [1:20:43] ... implicit preference. Awesome, guys. Anything to wrap? Call to actions? Are you hiring? Like, uh, who should reach out to you? Requests for startups? What should people build that is gonna be helpful to you guys? **Thorsten Ball** [1:20:55] I mean, all of the above. I don't ... Um, yeah, I don't know. I don't know. Um, we're always interested in talking to fellow engineers who are interested in agentic programming, figuring new stuff out. We wanna hear from them, like, what works and doesn't work. We're always willing to hire people with exceptional talents who are fully in this and realize that, you know, programming's changing a lot. And I don't know. What else? **Quinn Slack** [1:21:21] If you wanna come on this journey with us and see where coding agents are going, then come along. Yeah, use Amp, send us your feedback, and we are just so excited. We feel like kids in a candy shop. **Thorsten Ball** [1:21:37] Yeah, that's true. **Quinn Slack** [1:21:38] Just that- **Thorsten Ball** [1:21:39] Yeah **Quinn Slack** [1:21:39] ... we get to go build the future of coding. Feels like the final boss. **Thorsten Ball** [1:21:42] Yeah. **Alessio** [1:21:43] Nice. Thank you guys for coming on. This was fun. **Thorsten Ball** [1:21:46] Thank you. **Quinn Slack** [1:21:46] Thank you. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.