# Why Google failed to make GPT-3 -- with David Luan of Adept Latent Space · 2024-03-27 David Luan, co-founder of Adept and former early OpenAI leader, explains why Google failed to build GPT-3—its 'brain credit marketplace' prevented critical mass—and why Adept builds enterprise AI agents prioritizing reliability over generality. He recounts the GPT-2 demo that helped secure Microsoft's $1B investment and details Adept's goal: an AI teammate that can do anything a human does on a computer, targeting 'nines of reliability' for workflows like dispatching a physical truck. Luan contrasts Adept's vertical integration—training fast multimodal models (Fuyu) for charts and UIs—with pure-play foundation model companies that sell tokens, predicting commoditization. He notes Adept is sold out for Q1 and raised $420M, and explains how an augmentation focus creates a data flywheel from human oversight. ## Questions this episode answers ### Why didn't Google build GPT-3 before OpenAI? David Luan, who co-led Google Brain, explains that Google had all the resources—top researchers like Noam Shazeer, massive compute—but couldn’t concentrate them because of the "brain credit marketplace." Each researcher got credits to buy chips, so running a giant model required convincing many colleagues to give up their compute. This prevented the critical mass needed, while OpenAI could take focused big swings. [9:43](https://addtry.com/b2bbff7f-427f-4adb-8eb0-5e8bef841527?t=583000) ### What was the GPT-2 demo that helped Microsoft invest $1 billion in OpenAI? David Luan recounts the final pitch meeting where he, as OpenAI’s VP of Engineering, gave the GPT-2 demo to Microsoft CEO Satya Nadella and CTO Kevin Scott. Because the model had just been trained and its outputs were unpredictable, Luan was terrified of letting the executives type anything, fearing it would say something inappropriate, which kept him up all night before the meeting. [14:18](https://addtry.com/b2bbff7f-427f-4adb-8eb0-5e8bef841527?t=858000) ### What does Adept actually do, and why is it often misunderstood? David Luan clarifies that Adept is building an AI agent that can do anything a human does on a computer, focused on enterprise reliability. Unlike API or developer companies, they partner with large businesses to delegate complex workflows, making workers supervisors. Adept is not a chatbot or open-source tool; they prioritize high reliability and are currently sold out onboarding new customers for Q1. [15:51](https://addtry.com/b2bbff7f-427f-4adb-8eb0-5e8bef841527?t=951000) ## Key moments - **[0:00] Intro** - **[1:14] Career Path** - [2:07] OpenAI's Greg and Ilya handed their directs to David Luan on his second or third day - **[3:20] AI Eras** - [4:15] David Luan predicts AI progress's next driver is deep co-design of product, users, and technology - [5:00] Adept's David Luan defines AGI as 'a model that can do anything a human can do on a computer' - **[5:35] GPT Origins** - [6:10] David Luan: LLMs are behavioral cloning every word written on the internet - [9:05] David Luan: 'Every day we were scaling up GPT-3, I would wake up and just be stressed' that Google would beat them - [12:52] GPT-2's modeling section was only a paragraph long, and they feared 'the OGs in the field are gonna hate this' - **[13:38] Adept's Vision** - [14:18] David Luan recalls giving GPT-2 demo to Satya Nadella and Kevin Scott: 'that really kept me up all night' - [15:49] Adept is an enterprise company building AI agents that can do anything a human does on a computer, says David Luan - [17:52] Reliability is the most important thing for AI agents, says David Luan: enterprises need 'nines of reliability' - **[19:20] Adept's Playbook** - [19:46] Users become supervisors managing AI agents, not replaced, says David Luan: 'they're really excited about it' - [22:25] Adept is completely sold out for Q1 2024 (bandwidth for customer onboarding) - **[26:23] Interaction Layer** - [27:30] In early 2023, no usable multimodal models existed for agent tasks, says David Luan, so Adept built Fuyu - [29:07] 'The tyranny of chatbots as form factor' is facing pushback, says David Luan, and Adept is designing beyond it - **[30:37] Reliability Race** - [34:39] David Luan finds the self-driving levels analogy useful for agents, but wants to avoid self-driving's discontinuities - **[35:11] Competitive Edge** - [36:25] Good sensors and actuators are underrated for AI agent reliability, says David Luan, fixing many execution failures - **[37:30] Robot Agents** - [37:37] Q: Is there a trade-off between reliability and generality in AI agents? A: David Luan says it's only at the Pareto frontier - **[39:22] Join Adept** - [40:01] David Luan: the best part about defining AGI as computer tasks is having an environment to generate synthetic data - **[40:57] Compute Era** - [42:41] Open-source models like Llama will pinch pure-play foundation model companies, predicts David Luan, commoditizing general LLMs - [43:51] David Luan is fascinated by robotics and believes training a foundation model of robots 'is just gonna work' - [48:23] 'The rapid industrialization era of AI has begun,' says David Luan, and we must embrace it ## Speakers - **Alessio** (host) - **Swyx** (host) - **David Luan** (guest) ## Topics Agent Platforms ## Mentioned Adept (company), Axon (company), Dextro (company), Google (company), Microsoft (company), NVIDIA (company), OpenAI (company), A100 (product), Act One (product), DGX (product), Dota (product), Fuyu (product), GPT-2 (product), GPT-3 (product), GPT-4 (product), Llama (product), Persimmon (product), TPU (product) ## Transcript ### Intro Welcome to the Latent Space Podcast. We dive right in. Exploring a world of depths where new beginnings begin. David Luan, the founder. A visionary in his own right. Building autonomous agents. Taking us to new heights. Yeah. Yeah **David Luan** [0:32] Hey, everyone. Welcome to the Latent Space Podcast. This is Alessio, partner and CTO in residence at Decibel Partners, and I'm joined by my co-host Swyx, founder of Small AI. **Swyx** [0:41] Hey, and today we have David, David Luan, CEO, co-founder of Adept in the studio. Welcome. **David Luan** [0:45] Yeah, thanks for having me. **Swyx** [0:46] Been a while in the works. I've met you socially at one of those, uh, VC events, and, uh, you said that you were interested in coming on, and glad we finally were able to make this happen. **David Luan** [0:55] Yeah, happy to be part of it. **Swyx** [0:57] Uh, so we like to, uh, introduce the, the speaker and then also just, like, have you talk a little bit about, like, what's not on your LinkedIn, what, what people should just generally know about you. Um, you, uh, started a company in college, which was the first sort of real-time video detection classification API. That was, uh, Dextro, and that was your route to getting acquired into Axon, where you were director of AI. Um, then you were the, like, the thirtieth hire at OpenAI? ### Career Path **David Luan** [1:24] Yeah, thirty, thirty-five, something around there. **Swyx** [1:26] Something, um, something like that. Um, VP of Eng for two and a half years to two, two years and a bit. Um, briefly served as, uh, director of, um, or, or tech lead of large models at Google, and then in 2022, uh, started Adept. That-- So that's the, the sort of bri-brief CV, uh- **David Luan** [1:43] Is there anything else- **Swyx** [1:43] ... more or less. Yeah, is there anything else you, like, want to fill in the blanks or, like, people should know more, more about? **David Luan** [1:47] I guess, uh, broader story was, uh, joined, um, joined OpenAI fairly early, um, and then did that for about, yeah, two and a half to, to three years leading engineering there. It's really funny. I think second or, uh, second or third day of my time at OpenAI, um, Greg and Ilya pulled me in a room and were like, "Hey, um, uh, you know, the, uh-- you should take over, you should take over our directs and we'll go mostly do IC work." Uh, so that was fun. Just, um, you know, coalescing a bunch of teams out of, um, out of, uh, a couple of, like, early initiatives that had already happened. The company of the Dota effort was going, uh, pretty hard, and then more broadly trying to put some, uh, put some bigger picture direction around what we were doing with basic research. So I spent a lot of time doing that. And then at g- uh, Google, um, uh, so I led Google's LLM efforts, but also co-led Google Brain, was one of the Brain leads more broadly. Um, and I think, um, there's been a couple of different eras of AI research, right? And, like, if we count everything before t- uh, 2012 as prehistory, which people hate it when I say that, um, you kinda had this, like, you and your three best friends write a research paper that changes the world period from, like, 2012 to 2017. And then, um, and then from-- And I think the game changed in 2017, and, like, most labs didn't realize it, but we at OpenAI really did. I think in large part helped by, like, Ilya's constant beating of the drum that the world would be covered in data centers and, like- **Swyx** [3:05] And I think scale is what you need- **David Luan** [3:06] Yeah. Well, like, I think we had conviction in that, but it wasn't until we started seeing results that it became clear that that was where we had to go. But also part of it as well was, like, for OpenAI, like, when I first joined, I think one of the jobs that I had to do was how do I tell a differentiated vision for who we were technically compared to, you know, hey, we're just smaller Google Brain, or like we're Google-- Like, you work at OpenAI if you live in SF and don't wanna commute to Mountain View or don't wanna live in London, right? ### AI Eras **David Luan** [3:29] That's like not enough to, like, hang your technical identity as a company. And so, like, what we really did was, and I spent a lot of time pushing this, is just how do we get ourselves focused on, uh, a certain class of, like, giant swings and, and, and bets, right? Like, how do you flip the script from, uh, you just do bottom-up research to more about, like, how do you, like, leave some room for that, but really make it about, like, what are the big scientific outcomes, um, that you wanna show? And then you just solve them at all costs, whether or not you care about novelty and all that stuff. And that became the dominant model for a couple years, right? And then what's, uh, and then what's changed now is I think, like, like, um, the number one driver of AI progress over the next couple years is gonna be the deep co-design and co-evolution of, like, product and users for feedback and actual technology. Um, and I think labs that retool to go do that are gonna do really well, and that's a big part of why I started Adept. You mentioned Dota. Any memories thinking from, like, the switch from RL to transformers at the time and kinda how the, the industry was, um, evolving more in the LLM side and leaving behind some of the more agent simulation, uh, work? You know, I actually think that people, like, zooming way out, I think agents are just absolutely the correct long-term direction, right? You just go to find what AGI is, right? You're like, hey, like, well, first off, actually, I don't love AGI definitions that involve human replacement because I don't think that's actually how it's gonna happen. I think even this definition of like, hey, AGI is something that outperforms humans at economically valuable tasks is, like, is kind of a, you know, like, a implicit, like, view of the world about, like, how-- what, what are the-- what's gonna be the role of people. I think, um, I think what I'm more interested in is, like, a definition of AGI that's oriented around, like, a model that can do anything a human can do on a computer. Um, and I think, like, if you go think about that, which is, like, super tractable, then, like, agent is, like, just a natural consequence of that definition. And so, like, like, what, what did all the work we did on RL and stuff like that get us was it got us a really clear formulation, like you have a goal and you wanna maximize the goal and you wanna maximize reward, right? Like, natural LLM formulation doesn't come with that out of the box, right? So, like, I think that we, um, as a field got a lot right by thinking about, hey, how do we solve problems of that caliber? And then the thing we forgot is, like, like, de novo RL is, like, a pretty terrible way to get there quickly. ### GPT Origins **David Luan** [5:41] Why are we rediscovering all the knowledge about the world? Like, years ago, I had a debate with a, with a, with a Berkeley professor as to, like, like, what will it actually take to build AGI? And his view is basically that you have to reproduce all the flops that went into evolution- **Swyx** [5:56] Yep **David Luan** [5:56] ... in order to be able to get there, right? **Swyx** [5:57] The, uh, biological basis theory. **David Luan** [5:59] I think, like, we are, we are ignoring the fact that you have a giant shortcut, which is you can behavioral clone everything humans already know. And that's what we've solved with LLMs. We've solved behavioral cloning everything that humans already know, right? **Swyx** [6:10] Oh. **David Luan** [6:10] So, like, today, maybe LLMs is, like, behavioral cloning every word that gets written on the internet. In the future, you know, like, now that multimodal models are becoming more of a thing, we're behavioral cloning the visual world. But really what we're just gonna have is this, like, universal byte model, right? Where, like, tokens of data that have high signal come in, and then all of those patterns are, like, learned by the model, and then you can regurgitate any combination out, right? So, like, like, text into voice out, like image in to, I don't know, like, uh, to other image out or video out or whatever. Like, these, like, mappings, right? Like, are all just gonna be learned by this universal behavioral cloner. Um, and so I'm glad we figured that out, and I think now we're back to the era of, like, how do we combine this with all of the lessons we learned during the RL period? And that's what's gonna drive progress. **Swyx** [6:55] Interesting. Uh, I'm still gonna pressure you for a little- a few more, uh, early OpenAI stories before we turn to the Adept stuff. On your personal site, which I love, uh- **David Luan** [7:02] Thanks **Swyx** [7:02] ... 'cause it's, it's a, it's really nice, like, personal, you know, story context around, like, your, your history. **David Luan** [7:07] I need to update it. It is so old. **Swyx** [7:08] Yeah. Yeah, it's so out of date. Um, but you mentioned GPT-2. Um, did you overlap with GPT-1? I think you did, right? Uh, the- **David Luan** [7:16] Um, I actually don't quite remember. I think I was joining right around- **Swyx** [7:20] Right around then? **David Luan** [7:20] I was right around then, yeah. **Swyx** [7:21] Yeah. The, the canonical story was Alec, you know, just kinda came in and, uh, was, like, very obsessed with with, um, transformers and applying them to, like, Reddit sentiment analysis. **David Luan** [7:32] Yeah. **Swyx** [7:33] I- **David Luan** [7:33] Yeah, sentiment- **Swyx** [7:34] Take us through- **David Luan** [7:34] That's right, sentiment neuron- **Swyx** [7:35] What you know is just- **David Luan** [7:35] ... all the stuff **Swyx** [7:36] ... the history of GPT a- as far as you know, uh, uh, you know, according to you. **David Luan** [7:39] Ah, okay. History of GPT according to me. That's a pretty good question. So I think the, the real story of GPT starts at Google, of course, right? **Swyx** [7:46] Transformers. **David Luan** [7:46] Because that's where, that's where transformers sorta came about. And Bert- The number one shocking thing to me was that, and this is, like, a consequence of the way that Google's organized, where, like, again, like you and your three best friends write papers, right? Okay, so zooming way out, I think about my job when I was a f- a full-time research leader as a little bit of a portfolio allocator, right? So I've got, um, really, really smart people, and my job is to convince people to coalesce around a small number of really good ideas and then run them over the finish line. My job is not actually to promote a million ideas and never have critical mass. And then as the ideas start coming together and some of them start working well, my job is to nudge resources towards the things that are really working, and then start disbanding some of the things that are not working, right? That muscle did not exist during my time at, at, at Google. And I think had they had it, what they would have done would be say, "Hey, Noam Shazeer, you're a brilliant guy. You know how to scale these things up. Like, here's half of all of our TPUs." And then I think they would have destroyed us. Um, but- **Swyx** [8:45] He clearly wanted it too. He was talking about trillion-parameter models- **David Luan** [8:47] Yeah **Swyx** [8:47] ... in 2017. **David Luan** [8:48] Yeah. And so I think this gets to the core of the GPT story, right? Which is that, um, and I'm jumping around historically, right? **Swyx** [8:53] Mm-hmm. **David Luan** [8:53] But, like, after GPT-2, we were all really excited about GPT-2. I can tell you more stories about that. Um, it was the last paper that I even got to really touch before everything became more about just, like, bui- building a research org. You know, every day we were scaling up GPT-3, I would wake up and just be stressed, and I was stressed because, you know, you just look at the facts, right? Google has all this compute. Google has all the people who invented all of these underlying technologies. There's a guy named Noam who's really smart, who's already go- gone, gone and done this talk about how he wants a trillion-parameter model. And, and I'm just like, "You know, we're like, we're probably just doing duplicative research to what he's doing," right? He's got this, like, decoder-only transformer that's probably gonna get there before we do. Um, and I was like, "But like, please just, like, let this model finish," right? Um, and it turned out the whole time that they just couldn't get critical mass. So during my year where, um, where I led the Google LLM effort and, like, uh, and I was one of the brain leads, um, you know, it became really clear why, right? At the time, there was a thing called the brain credit marketplace. And do you guys remember the brain credit marketplace? **Swyx** [9:57] No, I never heard of this. **David Luan** [9:58] Uh, it was actually, it's a, it's a... You can ask any Googler. It's, like, just, like, a thing, but- **Swyx** [10:02] Yeah **David Luan** [10:02] ... that, that they do. **Swyx** [10:02] I mean, look, like yeah, in t- limited resources, you gotta have some kind of marketplace, right? You, you know? **David Luan** [10:07] You could, yeah. **Swyx** [10:07] Sometimes it's explicit, sometimes it's in, you know, just political favors. **David Luan** [10:11] You could. And so then, like, basically everyone's assigned a credit, right? So if you have a credit, you get to buy, you get to buy end chips according to supply and demand. So if you wanna go do a giant job, you gotta go convince like 19 or 20 of your colleagues not to do work. And if that's how it works, it's like, it's, it's, it's really hard to get that bottom-up critical mass to go scale these things. And like, um, and the team at Google were fighting valiantly, but, like, we were able to beat them simply because we, we took big swings and we focused. And I think, again, that's, like, part of the narrative of, like, this phase one of AI, right, of, like, this modern AI era to phase two. And I think in the same way, I think phase three companies can out, out-execute phase two companies because of the same, like, like, asymmetry of success. **Swyx** [10:56] Yeah. I think it's, um, underrated how much NVIDIA worked with you in the early days as well. I think, um, maybe-- I think it was Jensen, I'm not sure who, circ- circulated, um, um, a recent photo of him delivering the first, uh, DGX to you, to you guys. **David Luan** [11:11] I think Jensen has been a complete legend and mastermind throughout. **Swyx** [11:17] Yeah. **David Luan** [11:17] I have so much respect for NVIDIA. It is unreal. **Swyx** [11:20] But like, would, would OpenAI, like, kinda give their requirements, like co-design it, or you just work with whatever NVIDIA gave them, gave them? **David Luan** [11:27] So we work really closely with them. Um, there's-- I'm not sure I can s- share all the stories, but, like, I think, like, examples of ones that I've found particularly interesting. So, um, so Scott Gray is amazing, um, and, um, really like working with him. He was on one of my teams, um, the, the supercomputing team, um, which Chris Berner runs, and Chris Berner still does a lot of stuff in that. But, uh, a- as a result, like, we had very close ties to NVIDIA. Um, actually one of my co-founders at Adept, Eric Elson, was also one of the early GPGPU people. Um, and so he and Scott and, like, Bryan Catanzaro at NVIDIA and, like, um, and Jonah and Ian at NVIDIA I think all were very close, and were all sort of part of this group of just like, how do we push these chips to the absolute limit? Um, and I think, um, like, that kind of collaboration helped quite a bit. One interesting set of stuff is just like, you know, like knowing the A100 generation that, like, quad sparsity was gonna be a thing. Is that something that we wanna go, we wanna go look into, right? And figure out if that's something that we could actually use for model training. And I think more and more people realize this, but, like, six years ago, people-- or even three years ago, people refused to accept it. Like, this era of AI is really a story of compute. It's really the story of how do you more efficiently map, like, uh, uh, like actual usable model flops to compute, right? **Swyx** [12:37] Yeah. Cool. Is there another, you know, sort of GPT-2, 3 story that, like, you know, you- Love to get out there, um, that I think is, you- you think is, like, underappreciated for, like, the amount of work that people put into it. **David Luan** [12:49] So two interesting GPT-2 stories. **Swyx** [12:51] Love it. **David Luan** [12:52] I spent a good bit of time just, uh, sprinting to help Alec get the paper out, and I remember, um, one of the most entertaining moments. We were writing the modeling section, and I'm pretty sure the modeling section was, like, the shortest modeling section of any ML, like, reasonably legitimate ML paper to that moment. It was like, "Section three, model." Like, this is a standard vanilla decoder-only transformer with, like- ... these particular things. It was, like, a paragraph long, if I remember correctly. And both of us were just looking at this thing being like, "Man, like, the OGs in the field are gonna hate this. They're gonna, they're gonna say, 'No novelty.' Like, 'Why'd you guys do this work?'" Uh, so, um, so now it's, it's funny to look at in hindsight that it was kind of a pivotal, pivotal kind of, kind of paper. But I think it was one of the early ones where we just leaned fully into all we care about is solving problems in AI and not about, like, hey, like, is there, like, four different, like, really simple ideas that are cloaked in mathematical language that doesn't actually help, uh, move the field forward. ### Adept's Vision **Swyx** [13:48] Right. And it's, like, you, you innovate on maybe, like, dataset and scaling and, and not so much the architecture. **David Luan** [13:54] Uh, yeah. I mean, now, I mean, like, we all know how it works now, right? Which is that, like, there's a collection of really hard-won knowledge that you get only by being at the frontiers of scale. And that hard-won knowledge, a lot of it's not published. A lot of it is, like, stuff that, like, it's actually not even easily reducible to what looks like a typical academic paper. Um, but yet that's the stuff that, um, helps differentiate one scaling program from another. **Swyx** [14:16] Yeah. You had a second one? **David Luan** [14:18] Hilariously enough, the last meeting we, we did with Microsoft before Microsoft invested in OpenAI, Sam Altman, myself, and our CFO flew up to Seattle to do the final pitch meeting. And I'd been a founder before, so I always had, like, a tremendous amount of anxiety about partner meetings, which this basically- **Swyx** [14:36] Mm-hmm **David Luan** [14:36] ... this is what it was because it was, like, it was, like, Kevin Scott and Satya and, uh, and Amy Hood, and, um, it was my job to give the technical slides about, you know, what's the path to AGI, what's our research portfolio, all of this stuff, but it was also my job to give the GPT-2 demo. We had a slightly bigger version of GPT-2 that we had just cut maybe a day or two before this flight up. As we all know now, model behaviors you find predictable at one checkpoint are not predictable at another checkpoint. And so, like, I'd spent all this time trying to figure out how to keep this thing- **Swyx** [15:04] Mm-hmm **David Luan** [15:04] ... on rails, uh, to prevent it from saying anything bad. But, um, I had my canned demos, but I knew I had to go turn it around over to, like, Satya and Kevin and let them type anything in- **Swyx** [15:14] Mm-hmm **David Luan** [15:14] ... and that just, uh- **Swyx** [15:15] Mm-hmm **David Luan** [15:16] ... that, that really kept me up all night. **Swyx** [15:18] Nice. Yeah. **Alessio** [15:19] That must have helped you. Talking about partners meeting, you raised, uh, 420 million for Adept. Uh, the last round was a $350 million Series B, so I'm sure you do great in, uh, in partners meetings. **David Luan** [15:31] Pitching meetings. Nice. **Alessio** [15:32] No, that's a, that's a high compliment coming from a VC. **David Luan** [15:34] Yeah, no, I mean, you're, you're doing great already. **Alessio** [15:37] Let's talk about Adept. Um, and we were doing pre-, uh, pre-prep, and you mentioned that maybe a lot of people don't understand what Adept is. So usually we try and introduce the product and then have the founders fill in the blanks, but maybe let's do the reverse. Like, what, what is Adept? **David Luan** [15:51] Yeah, so I think Adept is, like, the least understood company in the, like, broader space of foundation models plus agents. So I'll, so I'll, I'll give some color, and I'll, I'll explain what it is, and I'll explain also, uh, why it's actually pretty different from what people would have guessed. So, um, the goal for Adept is we w- is we basically wanna build an AI m- AI agent that can do, uh, can basically help humans do anything a human does on a computer. Um, and so what that really means is, like, we want this thing to be super good at turning natural language, like, goal specifications, right, um, into the correct set of end steps, and then also have all the correct sensors and actuators to go get that thing done for you across any software tool that you already use. And so the end vision of this is effectively, like, I think in a couple of years, everyone's gonna have access to, like, an AI teammate that they can delegate arbitrary tasks to, um, at work, and then also be able to, you know, use it as a sounding board and, like, just be way, way, way more productive, right? And just, like, changes the shape of every job from something where you're mostly doing execution to something where you're mostly actually doing, like, these core liberal arts skills of, like, what should I be doing and why, right? I find this, like, really exciting and motivating because I think it's actually a pretty different vision for how AGI will play out. I think, like, uh, systems like Adept are the most likely systems to be proto-AGIs. Um, but I think the ways in which we are really counterintuitive to everybody is that we've actually been really quiet because we are, uh, we are not a developer company. We don't sell APIs. We don't sell open source models. We, um, also don't sell bottom-up products. Like, we're not a thing that you go and click and download the, the, the, the extension and, like, we want more users signing up for that thing. We're actually an enterprise company. So what we do is we have, we work with, like, um, a, a range of different companies, um, some, like, late-stage, like, multi-thousand-people startups, um, some Fortune 500s, et cetera. And what we do for them is we basically give them an out-of-the-box solution where, like, big complex workflows that their employees do every day could be delegated to the model. So we look a little different from other companies in that, like, in order to go build this full agent thing, the most important thing you gotta get right is reliability. I think over the last year or two, so initially zooming way back when, one of the first things Adept did was we released this, uh, uh, this demo called Act One, right? Act One was, like, pretty cool. It's, like, kinda become a hello world thing for people to show agent demos by going to Redfin and asking it to buy a house somewhere. 'Cause, like, we, we did that in the original Act One demo and, like, showed that, showed, like, Google Sheets, like, all this other stuff. Um, but, um, over the last, like, year since that has come out, um, there's been a lot of really cool, really cool demos, and you go play with them, and you realize they work 60% of the time. But since we've always been focused on how do we build an amazing enterprise product, like, enterprises, like, don't want-- can't use anything that isn't in the nines of reliability. And so we've actually had to go down a slightly different tech tree than what you might find in the prompt engineering sort of, um, sort of, uh, uh, place in the, in the agent space to get that reliability, and we've decided to prioritize reliability over all else. So, like, one of our use cases is crazy enough that it actually ends with a physical truck being sent to a place as the result of the agent workflow. And if you're like, if that works, like, 60% of the time, you're just, like, blowing money and- ... poor truck drivers going places. **Alessio** [19:09] Interesting. We had-- One of the-- our investment teams has this idea of services as software. Uh, I'm actually giving a talk at NVIDIA GTC about this, but basically, software as a service, you're wrapping user productivity in software, um, with agents and services as software is, uh, replacing things that, you know, you would ask somebody to do, and the software just does it for you. ### Adept's Playbook **Alessio** [19:31] When, when you think about these use cases, do the users still go in and, like, look at the agent kinda like doing the things and can intervene or, like, are these, like, fully removed from them? Like, the truck thing. It's like, does the truck just show up or, like, are there people in the middle, like, checking in? **David Luan** [19:46] Yeah. So actually, what's been really interesting is you could question whether they're fundamental, but I think there's two current flaws in the framing for services as software or I think what you just said. **Alessio** [19:55] Mm-hmm. **David Luan** [19:55] I think that one of them is, like, in our experience, as we've been rolling out Adept, um, the people who actually do the jobs are the most excited about it because they don't go from, "I do this job," to, "I don't do this job." They go from, "I do this job for everything, including the shitty rote stuff"- **Alessio** [20:09] Mm-hmm **David Luan** [20:10] ... to, "I'm a supervisor," and I literally-- like, it's pretty magical when you, when you watch the thing being used because, like, now it parallelizes a bunch of the things that you were o- you had to do sequentially by hand as a human, and you can just click into any one of them and be like, "Hey, I wanna watch the trajectory that the, the, the agent went through to go solve this." And, um, the nice thing about agent execution as opposed to, like, LLM generations is that, um, a good chunk of the time when the agent fails to execute, it doesn't give you the wrong result. It just fails to execute, and the whole trajectory is just broken and dead, and the agent knows it, right? So then those are the ones that the human then goes and solves, and so then they become a troubleshooter. They work on the more challenging stuff. They get way, way more stuff done, and they're really excited about it. I think the second piece of it that, um, we've found is, like, our strategy as a company is to always be a- an augmentation company, and I think, one, out of principle, that's something we really care about. But two, actually, if you're f- if you're framing yourself as an augmentation company, you're always gonna live in the world where you're solving tasks that are a little too hard for what the model can do today and, and still needs a human to, to provide oversight, provide clarifications, provide human feedback, and that's how you build a data flywheel. That's how you actually learn from the smartest humans how to solve things models can't do today. And so I actually think that being an augmentation company forces you to go develop your core, uh, AI capabilities faster than someone who's saying, "Ah, okay, my job is to deliver you a lights-off solution for X." **Alessio** [21:35] Yeah. It's interesting because we've seen two parts of the market. One is we have one company that does, uh, agents for SOC analysts. People just don't have them, you know? And just they cannot attract the talent to do it. **David Luan** [21:47] Mm. **Alessio** [21:47] And similarly, in, uh, software development, you have Copilot, which is the augmentation product, and then you have, um, Sweep.dev, and you have these products, which is like they just do the whole thing. Um, I'm really curious to see how, how that evolves. I agree that today the reliability is so important in the enterprise that, like, they just don't use most of them. **David Luan** [22:06] Yeah. **Alessio** [22:06] Um, yeah, no, that, that's cool. But, but it's great to hear the story because I think from the outside, people are like, "Oh, Adept, they do Act One, they do Persimmon, they do Fuyu, they do all these different models." **David Luan** [22:15] Yeah. **Swyx** [22:16] It's just the public stuff. **David Luan** [22:17] It's just public stuff, and so I think, um, you're gonna find-- So one of the things we, we haven't shared before is we're completely sold out for Q1. Um, and so I think- **Swyx** [22:25] Sold out of what? **David Luan** [22:25] Uh, sold out of bandwidth to go onboard more customers. I think we're, like, working really hard to go, like, make that less of a bottleneck, but you could, um-- But m- but our expectation is that, um, I think we're gonna be significantly more public about the broader product shape and, um, and the new types of customers we want to attract later this year. So I think that clarification will, will happen by default. **Swyx** [22:47] Why have you become more public? You know, if, if the whole push has-- You're sold out, you're, you're more enterprise, but you're also clearly putting effort towards being more open or releasing more things. **David Luan** [22:57] I think we just flipped over that way, uh, fairly recently. I think that, like, that's a good question. I think it actually boils down to two things. The public narrative is really forming around agents as being the most important thing. **Swyx** [23:07] Mm-hmm. **David Luan** [23:07] And I'm really glad that's happening because when we started the company in January 2022, like, we-- like, everybody in the field knew about the agents thing from RL, right? But, like, the general public had no conception of what it was. They were, like, still hanging their narrative hat on the, on the tree of, like, "Ah, everything's a chatbot," right? Um, and so I think now, I think one of the things that I really care about is that when people think agent, they actually think the right thing, right? Like, like, all sorts of different things are being called agents. Chatbots are being called agents. Things that make a function call are being called agents. Like, to me, an agent is something that you can give a goal, it can end step workflow done correctly in the minimum number of steps, right? Um, and so that's a big part of why. And I think the other part is because I think it's always good for people to be more aware of Adept as they think about what the next thing they wanna do in their careers. And I think the field is quickly pivoting in a, in a world where foundation models are looking more and more commodity. And if-- And I think a huge amount of gain is gonna happen from how do you use foundation models as, like, the, like, well-learned behavioral cloner to go solve agents, and I think people who wanna do agents research should really come to Adept. **Swyx** [24:10] Yeah. Excellent. Um, when, when you say agents have become more a part of the public narrative, uh, are there specific things that you point to? Um, so I'll, I'll, I'll, I'll name a few. Bill Gates in his, like, blog post, like, mentioning that agents are the future. Like, you know, "I'm the guy who made OSes, and I think agents are the next thing." **David Luan** [24:26] Yeah. **Swyx** [24:26] Um, so Bill Gates-- I'll, I'll call that out. And then maybe Sam Altman also saying that agents are the future for OpenAI. **David Luan** [24:32] And before that even, I think, um, there was some, like, New- New York Times-- Cade Metz wrote a New York Times piece about it. Right now, in a bit to differentiate, I'm seeing AI startups that used to just brand themselves an AI company but now brand themselves an AI agent company. It's just like, it's, it's a term I just feel like people really want. **Swyx** [24:46] From the VC side, it's a bit mixed. **David Luan** [24:48] Is it? **Swyx** [24:48] Uh, as, as in, like, I think there are a lot of VCs who are like, "Wait, I would not touch any agent startups," 'cause, like- **David Luan** [24:52] Why is that? **Swyx** [24:54] Well, you tell me. **Alessio** [24:56] Uh, I... I think a lot of VCs that are maybe less technical don't understand the limitations of the- **Swyx** [25:02] No, that's not fair. **Alessio** [25:02] No, no, no, no. I think, like- **Swyx** [25:03] You think so? **Alessio** [25:04] No, no. I think, like, the what is possible today and, like, what is worth investing in, you know? **Swyx** [25:08] Mm-hmm. **Alessio** [25:08] And I think, like-- I mean, people look at you and say, "Well, these guys are building agents. They needed four hundred million to do it." So a lot of VCs are maybe like, "Oh, I, I would rather invest in something that is, like, tacking on AI to an existing thing," which is, like, easier to get to market and kinda get some of the flywheel going. **Swyx** [25:23] Mm-hmm. **Alessio** [25:24] But I'm also surprised a lot of funders just don't wanna do agents. **David Luan** [25:28] Yeah. **Alessio** [25:28] It's not even the funding. Like, sometimes we look around and it's like, "Why is nobody doing agents for X?" And it's like- **David Luan** [25:34] Wow **Alessio** [25:34] ... I don't, I, I don't get it. **David Luan** [25:36] That's surprising. That, that's, that's good to know, actually. I, I never knew that before. I-- m-my sense from my, uh, limited perspective is there's the new agent company popping up every day, so maybe I'm ignorant to something. **Swyx** [25:45] They are, they are. Um, I, I-- but, like, I have advised people to take agents off of their title because it's so, so diluted as a term. **David Luan** [25:52] It's now so diluted, yeah. **Swyx** [25:54] So then it doesn't stand for anything. **David Luan** [25:55] Yeah. **Swyx** [25:56] You know? **David Luan** [25:56] That's a really good point. **Swyx** [25:57] So anyway, um, I do want to also cover, uh... So, like, you know, you're a portfolio allocator. You have, uh, pr- like, people know about Persimmon, p-people know about Fuyu and Fuyu Heavy. Uh, can you, like, take us through, like, how you think about the evolution of that, and what people should think about what that means for Adept's sort of research directions? **David Luan** [26:15] The critical path for Adept is we wanna build, uh, agents that can do higher and higher level abstraction things over time, all while keeping an insanely high reliability standard, because that's what turns us from research into something that customers want. ### Interaction Layer **Swyx** [26:28] Yeah. **David Luan** [26:29] And if you build agents with really high reliability standard but are continuing pushing the level of abstraction, you then learn from your users how to get that next level of abstraction faster, so that's how you actually build the data flywheel. That's the critical path for the company. Everything we do is in service of that. So if you go zoom way, way back to Act One days, right? Like, the core thing behind Act One is can we teach, um, a, a, a large model basically how to even actuate your computer? And I think we were one of the first places to have, to have, like, solved that and shown it and shown the generalization that you get when you give it various different workflows and text. Um, but I think from there on out, what we really realized was that, like, in order to get reliability, um, and also, like, companies just do things in various different ways. You actually want these models to be able to get a lot better at, um, having some specification of some guardrails for what it actually should be doing. Um, and I think in conjunction with that, a giant thing that was really necessary is really fast multimodal models that are really good at understanding knowledge work and really good at understanding screens, and that is-- needs to kinda be the base for some of these, for some of these agents. And so, like, back then, we had to do a ton of research basically on how do we actually make that possible. Well, first off, like, um, back in 2020, p- uh, forget exactly, well, month of '23, like, there were no multimodal models- **Swyx** [27:41] Mm-hmm **David Luan** [27:41] ... really that you could, that you could use for things like this, and so we pushed really hard on, on stuff like the Fuyu architecture. I think one big, uh, hangover from, uh, pr-primarily academic focus for multimodal models is, like, most multimodal models are primarily trained on, like, natural images, cat and dog photos, stuff that's come out of a camera. **Swyx** [27:59] Coco. **David Luan** [28:00] Y-yeah, right, and Coco's awesome. Like, I love Coco. I love TY. Like, it's like it's really helped the field, right? But, like, that's to build one thing. I actually think, like, like, it's really clear today multimodal models are the default foundation model, right? It's just gonna supplant LLMs. Like, why? W-you just train a giant multimodal model. Um, and so for that, though, like, where are they gonna be the most useful? They're gonna be most useful on knowledge work tasks. That's where the majority of the economic value's gonna be. It's not in cat and dogs, right? Uh, and so if that's what it is, what do you need to train on? You need to train on, like, charts, graphs, tables, invoices, PDFs, receipts, unstructured data, UIs. Like, th-that's just a totally different pre-training corpus- **Swyx** [28:35] Mm-hmm **David Luan** [28:35] ... and so Adept spent a lot of time building that. And so the, like, the, the public Fuyus and stuff aren't trained on our actual corpus. It's trained on some other stuff. But you take a lot of that data, and then you make it really fast, make it really good at things like, like dense, dense OCR on screens, and then now you have, like, the right, like, raw putty to go make a good agent. So that's kinda like some of the modeling side. We've kind of only announced some of that stuff. We haven't really announced much of the, much of the agents work. Um, but that if you put those together with the correct product form factor, and I think the product form factor also really matters. I think, like, um, we're seeing, and you guys probably see this a little bit more than I do, but, like, we're seeing, like, a little bit of a, of a pushback against, like, the tyranny of chatbots as form factor. **Swyx** [29:18] Hmm. **David Luan** [29:19] Um, and I think that the reason why the form factor matters is the form factor changes what data you collect in the human feedback loop. Um, and so I think we've spent a lot of time doing full, like, vertical integration of all these bits in order to get to where we are. **Swyx** [29:32] Yeah. Uh, I'll plug, uh, Amelia, uh, Wunberger's, uh, talk at our conference- **David Luan** [29:36] Yeah **Swyx** [29:37] ... where she, uh, gave a little bit of the thinking behind, like, what else exists oth-other than chatbots that if you could delegate to reliable agents, you could do. **David Luan** [29:44] Totally. **Swyx** [29:44] Um, and, uh, w-well, yeah. I mean, I, so I, I was kind of excited at Adept's experiments or Adept workflows. I, I don't know what the official name for it is. **David Luan** [29:53] Mm-hmm. **Swyx** [29:53] I was like, "Okay," like, "this is something I could use." Um, but it seems like it's just an experiment for now. It's not, it's not your product. **David Luan** [29:58] Yeah. We basically just use experiments as, like, a way to go push various, uh, ideas on the design side, uh, to some people and just, like, get them to play with it. And, um, the-- actually, the experiments code base, um, underpins the actual, uh, underpins the actual product, but it's, like, just the, the code base itself is, like, a kind of like a skeleton for us to go deploy arbitrary cards on the side. **Swyx** [30:21] Yep. Yeah, makes sense. **David Luan** [30:22] Um- **Swyx** [30:23] Yeah. **Alessio** [30:23] I was gonna say, I would love to talk about the interaction layer. So you train a model to see UI, but then there's the question of, like, how do you actually act on the UI? I think there was some rumors about OpenAI building agents that are kinda like they manage the endpoint, so the whole computer. ### Reliability Race **Alessio** [30:39] Uh, you're more at the browser level. Like, a-and I know I read in one of your papers you have, like, a different representation, kinda like you don't just take the dome and act on it. Uh, you do a lot more stuff. How do you think about the best way the models will interact with the software and, like, how the development of products is gonna change with that in mind as more and more of the work is done by agents instead of people? **David Luan** [31:01] There's so much surface area here, and it's actually one of the things I'm really excited about. And it's like, it's funny because, like, I've spent most of my, uh, time doing research stuff, but this is, like, a whole new ballgame that I've been learning about, and I find it really cool. Um, so I would say the, the, um, the best analogy I have to, to why Adept is pursuing a path of being able to, um, just C- use your computer like a human, plus of course being able to call APIs. Like being able to call APIs is the easy part. Like being able to use your computer like a human is the hard part. It's in the same way why people are excited about humanoid robotics, right? Like in a world where you had T equals infinity, right? You're probably gonna have various different form factors that robots could just be in and like all the specialization. But the fact is that humans live in a human environment, so having a humanoid robot lets you do things that humans do without changing everything along the way. **Swyx** [31:49] Mm-hmm. **David Luan** [31:50] Uh, it's the same thing for software, right? Like, um, uh, if you go itemize out the number of things you wanna do on your computer for which every step has an API, um, those numbers of workflows add up pretty close to zero. Um, and so then many points along the way, you need the ability to actually control your computer like a human. It also lets you learn from h- uh, human usage of computers as a source of training data that you don't get if you have to, uh, if you have to somehow, um, figure out how every particular step- **Swyx** [32:15] Mm **David Luan** [32:15] ... needs to be some particular custom private API thing. Um, and so I think like this is actually the most practical path. I think because it's the most practical path, I think a lot of success will come from going down this path. So what you're likely to see is you're gonna end up seeing agents that sort of like-- I kinda think about this early days of the agent interaction layer, uh, level is a little bit like... Do y'all remember, um, Windows 3.1? Like those days? Okay, this might be-- I, I might be, I might be too old for- ... for you guys on this. But like back in the day, Windows 3.1, right? Like the way, um, we had this transition period between like pure command line, right? Being like the default to this new, to this new world where the GUI is the default, and then you drop into the command line for like programmer things, right? The old way was you booted your computer up, uh, DOS booted, and then it would give you the C colon slash thing, and you typed Windows and you hit Enter, and then you got put into Windows. Um, I-- uh, and then the GUI kind of became a layer above the command line. Um, I think, um, the same thing's gonna happen with agent interfaces, is like today what we have in the GUI is like the base layer, and then the agent just controls the, the current GUI layer plus APIs. And in the future, as more and more trust is built towards agents and more and more things can be done by agents, and more UIs for agents are actually generative in and of themselves, then that just becomes a standard interaction layer. And if that becomes a standard interaction layer, like what changes for software is that like a lot of software is gonna be either systems of record or like certain customized workflow execution engines. And, um, a lot of how you actually do stuff will be controlled at the agent layer. **Swyx** [33:48] And you think so like the, the Rabbit interface is more like it would like you're not actually seeing the app that the model interacts with. You're just saying, "Hey, I need to log this call in Salesforce," and like you're never actually going on salesforce.com directly as the user. **David Luan** [34:02] I can see that being a model. I think, um, I don't know enough about how, what using Rabbit in real life will actually be like to comment on that particular thing. But I think the broader, um, I think the broader idea that like, that like, you know, you have a goal, right? The agent knows how to break your goal down into steps. The agent knows how to use the underlying software and systems of record to achieve that goal for you. The agent maybe presents you information in a custom way that's only relevant to your particular- **Swyx** [34:31] Mm-hmm **David Luan** [34:31] ... goal. Um, that all just really leads to a world where you don't really need to ever interface with the apps underneath unless you're a power user for some niche thing. **Swyx** [34:39] General question. Uh, so, so first of all, I, I think like this whole, um, the sort of input mode conversation, I wonder if you have an- any analogies that you like with self-driving because, um, I do think like it's, there's a little bit of like how the model should perceive the world and, um, you know, the, the primary split in self-driving is LIDAR versus camera. Um, and I feel like most agent co- companies that I'm tracking are all moving towards camera approach, which is like- **David Luan** [35:06] The multimodal approach that we're doing **Swyx** [35:07] ... you know, you know, multimodal vision- **David Luan** [35:09] Yeah **Swyx** [35:09] ... very, very heavy vision. All the Fuyu stuff that you're, you're doing, you're focusing on that, uh, pri-- i- including charts and tables and- ### Competitive Edge **David Luan** [35:16] Yeah **Swyx** [35:16] ... um, do you find like inspiration there from like, uh, the, the, the self-driving world? **David Luan** [35:23] That's a good question. I think sometimes the most useful inspiration I've found from self-driving is the, um, is the levels analogy. Uh- **Swyx** [35:31] Level one to five **David Luan** [35:31] ... and I think that's great. I think that's awesome. Um, but I think that, um, our number one goal is for agents not to look like self-driving, in that we wanna minimize the chances that agents are sort of a thing that you just s- uh, have to bang your head at for a long time to get to like two discontinuous milestones, which is basically what's happened in self-driving. Um, we wanna be living in a world where you have the data flywheel immediately, and that takes you all the way up to the top. But similarly, I mean, like compared to self-driving, like two things that people really undervalue is like really easy to get the like driving a car down Highway 101 on a sunny day demo, right? Like that actually doesn't prove anything anymore. **Swyx** [36:09] Mm-hmm, mm-hmm. **David Luan** [36:09] And I think the second thing is that, um, as, as a non-self-driving expert, I think one of the things that, um, that we believe really strongly is that, um, everyone undervalues the importance of really good sensors and actuators. And actually, a lot of what's helped us get a lot of reliability is like a really strong focus on like actually why does the model not do this thing? And the non-trivial amount of time, the time the model doesn't actually do the thing, is because if you're Wizard of Oz-ing it yourself or if you have unreliable actuators, you can't do the thing. **Swyx** [36:42] Yeah. **David Luan** [36:42] Um, and so we've had to fix a lot of those problems. **Swyx** [36:44] Yeah, makes sense. Um, yeah, I'm-- I was slightly surprised just because I do generally consider the Waymos that we see all around San Francis- San Francisco as the most, I guess, um, real case of agents that we have, you know? **David Luan** [36:56] Yeah. **Swyx** [36:56] In, in, in, in very material ways. **David Luan** [36:58] Oh, that's absolutely true. I think they've done an awesome job, but it has taken a long time for self-driving to mature, like from when it entered the consciousness and the 101, uh, s- uh, the driving down 101 on a sunny day moment happened to, to now, right? **Swyx** [37:11] To now. **David Luan** [37:11] And so I wanna see that more impressed. **Swyx** [37:12] I mean, and, you know, Cruise, you know, RIP, um, recently. Um, s- so and then one more, one more thing on just like, uh, just going back on this reliability thing. Something I have been holding in my head that I'm curious to get your commentary on is there's, I think there's a trade-off between reliability and generality Or I want to broaden reliability into just general like sort of production readiness, enterprise readiness scale, because you have reliability, you also have cost, you have less speed. ### Robot Agents **Swyx** [37:37] Speed is a huge emphasis for Adept. Um, all of that seems to-- tends towards wanting to reduce gen- y- the, the tendency or the temptation is to reduce generality to improve reliability and to improve cost, improve speed. Um, do you s- perceive a trade-off? Do you have any insights that, that solve those trade-offs for, for you guys? **David Luan** [37:59] There's definitely a trade-off, um, if you're at the Pareto frontier. Um, I think a lot of folks aren't actually at the Pareto frontier. Um, and I think the way you get there is basically like how do you frame the fundamental agent problem in a way that just continues to benefit from data? And, um, and I think that, um, I think like one of the main ways of like being able to solve that particular trade-off is like you, uh, you basically just wanna formulate the problem such that every particular use case just looks like you collecting more data to go make that use case possible. I think that's how you really solve it. Then you get into the other problems like, oh, okay, are you overfitting on these end use cases, right? But like you're not doing a thing where you're like being super prescriptive for the end steps and, uh, that the model- **Swyx** [38:42] Right **David Luan** [38:43] ... uh, that the model can only do, for example. **Swyx** [38:44] I mean, so th- then the, the question becomes kind of, uh, do you have one sort of house model that you then cu- that you then customize for each customer, um, and you're fine-tuning them on like each customer's specific use case? **David Luan** [38:55] Ah, yeah, we're not, we're not sharing that one. **Swyx** [38:57] You're not sharing that. **David Luan** [38:58] Yeah. **Swyx** [38:58] Uh, it, it's tempting because but like that doesn't look like AGI to me. You know what I mean? Like that, that is just you have a good base model and then you, you fine-tune it- **David Luan** [39:06] Yeah **Swyx** [39:07] ... to others. **David Luan** [39:07] Yeah. Yeah. Yeah. I mean, I think for what it's worth, I think there's like two paths to, um, a lot more capability coming out of the models that, that we all are training these days. I think one path is you figure out how to spend, compute, and turn it into data. ### Join Adept **David Luan** [39:24] I think the other path, and so like in that path, right, I consider search, RL, all the things that we, that we all, uh, that we all love in this era as part of that path, like self-play, all that stuff. Um, the second path is, is how do you get like super competent, high intelligence demonstrations from humans? And I think the right way to move forward is you kinda wanna combine the two. Like the first one gives you maximum sample efficiency for the second. Um, but I think that it's gonna be hard to, uh, be running at max speed towards AGI without actually solving a bit of both. **Swyx** [40:01] Yeah. Any, um, insights on-- You haven't talked much about synthetic data as far as I can tell. Um, probably, uh, this is a bit of a too much of a trend right now, but, uh, any insights on using synthetic data to augment the expensive human data? **David Luan** [40:15] The best part about framing AGI as, uh, being able to help people do things on computers is you have an environment. **Swyx** [40:22] Yes. **David Luan** [40:22] So. **Swyx** [40:24] So you can simulate all of it. **David Luan** [40:26] You could, you could do a lot of stuff when you have an environment. **Swyx** [40:28] Yeah. **Alessio** [40:28] We were having dinner for our one year anniversary the other night- **David Luan** [40:31] Congrats. **Swyx** [40:32] Thank you. **Alessio** [40:32] Yeah. Thank you. Um, Raza from Human Loop was there, and we mentioned you were coming on, on the pod. Uh, with-- This is our first- **Swyx** [40:38] So he submitted a question. **Alessio** [40:39] Yeah. This is our first- **Swyx** [40:40] He has a mailbag. **Alessio** [40:40] I guess, like mailbag question. Um, he asked, uh, when you started, GPT-4 didn't exist. Uh, now you have GPT-4 Vision, uh, which can help you building a lot of those things. How do you think about the things that are unique to you as, as Adept and like going back to like the maybe research direction that you wanna take the team and what you want people to come work on at Adept versus what has maybe now become commoditized that you didn't expect everybody would have access to? ### Compute Era **David Luan** [41:08] Yeah. That's a really good question. I think implicit in that question, and I, and I wish he were here too so he can push back on my assumption about his question. Um, but I think implicit in that question is like, uh, is a calculus of where does advantage accrue in the overall ML stack. Um, and maybe, uh, part of the assumption is that advantage accrues solely to base model scaling. Um, but I actually believe pretty strongly that the way that you really win is that you have to go build a-- you have to go build an, an agent stack that is much more than that of the base model itself. Um, and so I think like that is like always gonna be a giant advantage of vertical integration. I think like it lets us do things like have a really, really fast base model. It's really good at agent things, but is bad at cat and dog photo. Not-- It's pretty good at cat and dog photos. It's not like, it's not like SOTA at cat and dog photos, right? So like, um, so like we're allocating our, our capacity wisely, right? It's like one thing that you really get to do. I also think that the other thing that, um, is pretty important now in the broader foundation modeling space is like I feel, um, despite any potential concerns, um, about, you know, like how good is, uh, is agents as like a startup area, right, like we were talking about earlier, I feel super good that we're doing foundation models in service of agents and all of the reward within Adept is flowing from can we make a better agent? Um, because right now I think we all see that, you know, if you're training on publicly available web data, um, you put in the flops and you do reasonable things, then you get decent results. **Swyx** [42:41] Mm-hmm. **David Luan** [42:42] And if you just double the, the, the amount of compute, then you get predictably better results. And so like I think pure play foundation model companies are just gonna be pinched by how good the next couple llamas are gonna be and the next like what, uh, n- next good open source thing and then, um, seeing, um, the really big players put ridiculous amounts of compute behind just training these, these base foundation models. I think it's gonna commoditize a lot of the like, uh, r- regular LLMs and soon regular multimodal models. So I feel really good that we're just focused on agents. **Swyx** [43:13] So you don't consider yourself a pure play foundation model company? **David Luan** [43:17] No, because if we were a pure play foundation model company- **Swyx** [43:19] You train them **David Luan** [43:19] ... we would be training, um, general foundation models that do summarization and all and, uh, and, uh, all this other- **Swyx** [43:25] Right. You're dedicated towards the, the agent focus. **David Luan** [43:27] Yeah. And our business is an agent business. **Swyx** [43:29] Yeah. **David Luan** [43:29] We're not here to sell you tokens, right? **Swyx** [43:30] Mm-hmm. **David Luan** [43:30] And I think like, um, selling tokens, um, uh, unless there's like a- **Swyx** [43:35] We're not here to sell you tokens. **David Luan** [43:35] Yeah. **Swyx** [43:36] I love it. **David Luan** [43:36] It's like if you have a, if you have a particular area of specialty, right, then, um, you won't get caught in the fact that like everyone's just scaling to ridiculous, uh, ri- ridiculous levels of compute. But if you don't have a specialty, I find that I, I, I think it's gonna be a little tougher. **Swyx** [43:51] Interesting. Are you interested in robotics at all? **David Luan** [43:54] Personally fascinated by robotics. I always love, have always loved robotics. **Swyx** [43:58] No, but embodied agents as a business, you know, Figure is like a, a big also sort of OpenAI-affiliated company that raises a lot of money. **David Luan** [44:04] Yeah. I think it's cool. I think, I mean, I- **Swyx** [44:06] But you're not- **David Luan** [44:06] I, I don't know exactly what they're- **Swyx** [44:08] ... gonna go there **David Luan** [44:08] ... exactly what they're doing, but, um- **Swyx** [44:10] Robots. **David Luan** [44:11] Yeah, yeah. Well, I mean- ... that's, uh, yeah. **Swyx** [44:14] No, what question would you ask, like, if we had them on? Like, what, what, what would you ask them? **David Luan** [44:17] Oh, I just wanna understand what their overall strategy is gonna be between now and when there's reliable stuff to be deployed. Um, but honestly, I just don't know enough about it. **Swyx** [44:25] And if I told you, "Hey, re- fire your entire workforce, w- warehouse workforce and, you know, put robots in there," like, isn't that a strategy? **David Luan** [44:33] Yeah. Oh, oh yeah, yeah. Sorry. I'm not, I'm not questioning whether- **Swyx** [44:35] Okay **David Luan** [44:36] ... they're doing smart things. Uh, I, I hope it, I hope it didn't come off that way. **Swyx** [44:39] No, no, no. No, you didn't. **David Luan** [44:39] Uh, it's just, like, I genuinely don't know what they're doing as much. **Swyx** [44:42] Yeah. **David Luan** [44:42] Um, but I think, like, uh, look, I think there's, there's two things. One, I'm so excited for someone to train a foundation model of robots. Like, it's just ... I, I think it's just gonna work. Like, I will die on this hill. Like, I mean, like, like, again, this whole, this whole time, like, we've been on this podcast, we're just gonna continually saying, you know, like, these, uh, uh, these, these models are basically behavioral cloners, right? So let's go behavioral clone all this, like, robot behavior, right? **Swyx** [45:08] Mm. **David Luan** [45:08] And then now you figure out everything else you have to do in order to teach it how to solve new problems. Like, that's gonna work. I'm super stoked for that. Um, I think, um, unlike what we're doing with, uh, helping humans with knowledge work, um, it just sounds like a more zero-sum, like- **Swyx** [45:25] Mm-hmm **David Luan** [45:25] ... job replacement play, right? And, um, I, I'm personally less excited about that. **Swyx** [45:30] Yep. Got it. **Alessio** [45:31] Um, we had, uh, Kanjun from Imbu on the podcast. **Swyx** [45:34] Another guest. **David Luan** [45:35] Mm-hmm. **Alessio** [45:35] Yeah. We asked her why people should go work there and not at Adept. **David Luan** [45:39] Oh, that's so funny. **Alessio** [45:41] Um, um, her, her... Well, she said, you know, "There's space for everybody in this market. We're all doing interesting work" and she said they're really excited about building an operating system for agent and for her, the biggest research thing was, like, getting models better at reasoning and, and planning for these agents. The reverse question to you, you know, why should people be excited to come work at Adept instead of Imbu- ... and maybe what are, like, the, the core research questions that people should be passionate about to, to have fun at Adept? **David Luan** [46:12] Yeah. First off, I think, um, I think that, uh, I'm sure you guys believe this too, but, like, the, the AI space, to the extent there's an AI space, uh, and the AI agent space are both, like, exactly as, uh, oh, she likely said, like, I think colossal opportunities and, like, and, um, people are just gonna end up winning in different areas and people are all just gonna... A lot of companies are gonna do well. Um, so I, I really don't feel that zero-sum thing at all. I would say, like, to, like, change the zero-sum framing, it's like why do you, why should you be at Adept? I think there's two huge reasons to be at Adept. I think one of them is, like, is, like, everything we do is in the service of, like, useful agents. Like, we're not a research lab. Like, we do a lot of research in service of that goal, but, um, we don't think about ourselves as, like, a classic research lab at all. Um, and I think the second reason to work at Adept is if you believe that actually having customers and a reward signal from customers lets you build AGI faster, which we really believe, then you should come here, and I think the examples for why that's true is, like, for example, like, our evaluations, they're not academic evals. They're not like, um, uh, simulator evals. They're like, okay, like we have a customer that really needs us to do these particular things. We can do some of them. These other ones they want us to do, we can't do them at all. We've turned those into evals. Like, solve it, right? Like, I think that's really cool. Like, everybody knows a lot of these evals are, like, pretty saturated and the new ones that even are not saturated, you look at someone and you're like, "Is this actually useful?" Right? I think that's a, um, that's a degree of, like, of, like, practicality that really helps. Like, we're s- equally excited about the same problems around reasoning and planning and, um, and, uh, and generalization and all of this stuff, um, but it's like they're very grounded in actual needs right now, which is really cool. **Swyx** [47:57] Yeah. Uh, this has been a wonderful dive. Uh, you know, I wish we had more time, but, uh, you know, I, I would just leave it kinda open to you. I think you have broad t- thoughts, uh, you know, just about the agent space but also just the general AI space. Any, any sort of rants or, uh, things that you- they're at the top of mind for you right now? **David Luan** [48:11] Hmm. Any rants? **Swyx** [48:14] Mining you for just general- **David Luan** [48:15] Wow. Okay, so Amelia's already made the rant better than I have, but- **Swyx** [48:19] Chat bot thing? **David Luan** [48:19] ... but, like, not just, not just chat bots- **Swyx** [48:21] Okay **David Luan** [48:21] ... is, like, kinda rant one. **Swyx** [48:22] Yep. **David Luan** [48:23] Um, rant two is, like, is, like, AI's really been the story of compute, um, and compute plus data and ways in which you could exchange one for the other. Um, and I think, um, as much as, uh, as much as, uh, uh, our research community is really smart, like we have made many, many advancements, um, and that's gonna continue to be important, but, like, now I think the game is increasingly changing and, like, uh, the rapid industrialization era has begun, and I think we unfortunately have to embrace it. **Swyx** [48:55] Yep. Excellent. Awesome. **Alessio** [48:56] Awesome, David. Thank you so much for your time. **David Luan** [48:58] Cool. Yeah. Thanks, guys. This was fun. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.