Solve coding, solve AGI [Reflection.ai launch w/ CEO Misha Laskin]

Hey, everyone. Welcome to the Latent Space Podcast. This is another lightning episode, and I'm joined by my co-host Swyx, founder of Smol.ai.

Hey, and today we are very blessed to catch up with Misha Laskin from Reflection AI. Welcome.

Thank you guys for having me.

Uh, so today we're sort of recording this ahead of the announcement, but you have some very exciting news here. I, I think you're basically coming out of stealth. So, uh, yeah, I mean, if you want to sort of lead off with like...

I, I think the last that people might have heard of you was when you raised your round from Sequoia, uh, and you did a little podcast with the Sequoia team. But now you're actually ready to have your landing page say more than like two sentences about what it is you're doing.

So maybe you want to introduce the world to Reflection AI.

Yeah, definitely. And, uh, on the landing page, I think more than one sentence. I think we have kind of one highly undescriptive sentence that lives on there right now. So definitely have more to share than that. So I think, I mean, the, the main thing is that we're kind of ready to share the...

you know, what it is that this company's been working on, what our kind of sort of research approach is, you know, what our product approach is, and the team that's, uh, assembled around this problem. So I think that...

You know, maybe I'll just, I'll just start off, uh, say, you know, something short, and then we can take it from there. So I think that, you know, what we've come together to solve is... Maybe this was controversial ten years ago, but now not so much anymore.

I think that there are many multiple labs that are pursuing the problem of superintelligence. And this is kind of the, the mission of the company, our company as well. We're building reliable, superintelligent autonomous systems. I don't think that it's so much the difference in like the pursuit that, you know, makes us unique, but it's more around the team around it and a rather opinionated approach to how one should build a superintelligent system.

So first on, you know, the reason we decided to start this as a, as a company is we kind of realized that two building blocks, like on the technology side and the research side, have come together that even made this problem possible.

And the first one was reinforcement learning, which a lot of members of our, members of our team pioneered some of the largest advances in that, including deep Q-networks, AlphaGo, AlphaZero, so forth. And what was special about reinforcement learning is that it's actually...

it, it's already a blueprint to how to build superintelligence. Like, we, we've had super-- we've lived with superintelligent systems, but they are very narrow. So AlphaGo was superintelligent at the game of Go but couldn't really do much outside of that.

It was then retrained on chess and so forth and became superintelligent in other board games. And so while we kind of... we've actually been living with these superintelligent systems, we know what they look like, but they just haven't been very practical yet.

Uh, and the other core technology that a lot of the members of our team helped pioneer was large language models. So we pioneered projects like Google's PaLM, Gemini, ChatGPT. What happened with language models is that they are kind of the opposite.

They are these really general systems that are useful but require a user to drive most of the experience. Like, we kind of think about language models today as almost like cruise control, um, where it's helpful almost everywhere, but you're still kind of, uh, driving the AI and, uh, you know, doing most of the work as a user.

So we kind of thought these two ingredients have come together where you could combine them and build not just a narrow superintelligence but a general one on the computer. And I think that our particular approach to it is whereas, uh, maybe most people...

Or I, I, I won't-- I actually won't speak for other people. I think that it's kind of, uh, better to, you know... Different people have their different judgments. But I would say that rather than seeing autonomous coding or, you know, coding capability as one of many things that we think, uh, a superintelligence will do, our core belief is that if you solve this problem, you solve the autonomous coding problem and build a superintelligent coding agent, that that thing will lead to superintelligence more broadly.

So that kind of encapsulates, I think, uh, the kind of team and the core beliefs that we share as a company.

And just to do a quick historical timeline. So you mentioned AlphaGo. Um, OpenAI was also working on RL on games with Dota in the beginning, and then everybody just focused on language. And I think now the pendulum is shifting back to RL.

Is there anything specific, um, in the last six to 12 months that kinda made you decide, okay, now is the time to do it? Or is it just a matter of, you know, the team coming together at the right time and the, the market being ready?

It was really that... So Y-Yannis and I led a lot of the work for post-training and kind of RL chat for Gemini, and Yannis being my co-founder. And when we shipped Gemini One 1.5, we just realized that the models, like models that were basically at GPT-4 level or above, were capable enough as kind of starting points to then post-train or...

I mean, it's-- I wouldn't even call it post-training, just training with reinforcement learning. So it's really that I don't know if you've been able-- you would've been able to do the same stuff with an earlier model, like with a GPT-3 or a GPT-2, but GPT-4 had this kind of base of intelligence that you could actually go from there.

And by the way, this actually happened in AlphaGo as well and some of these systems previously, where before you did reinforcement learning, they first did imitation learning on human games. And it was important that the human playing ability was sufficiently high for you to be able to then bootstrap on top of that and then train with reinforcement learning.

That is, if you trained AlphaGo on very weak human gameplay, then for that first project, reinforcement learning wouldn't have taken you as far. Of course, AlphaGo then figured out transition to AlphaZero, where it was trained without human data whatsoever.

But I think that's where the analogy breaks. I think in the era of language models, uh, I, I don't think you'll be able to train superintelligence from scratch.

Yep. And then on the environment, you briefly mentioned the computer. Is the idea that, you know, you mentioned code, but do you wanna be very computer use-focused? You know, some people say the browser is kinda like the new OS anyway.

Uh, I'm curious your thoughts on where you want the agent to, to live in.

Yeah, it's, it's a really good question. I mean, if you start kind of an autonomous agent company today, you might pick from one of two categories, maybe some others. But I think broadly speaking, you could either say, "I'm gonna build browser agents or kind of computer use kind of agents more broadly, or build coding agents."

And I think that we're-- you know, our belief as a company is that the correct agent, the correct starting point to this entire problem is the coding agent because it's already... You know, software engineering is already what I would call kind of ergonomic for a language model.

That is to say, you have to design the problem that you're striving for to be compatible with sort of, um, what is intuitive for the intelligence that's working on it. And for humans, what's intuitive to us is, you know, using a mouse and keyboard and kind of geospatial reasoning because that's just how we evolved.

Uh, but language models never evolved. They were trained on the internet. And because they were trained on the internet, their priors for what they-- it's intuitive to them is completely different for what's intuitive to us. And so a language model, for example, has no prior for a mouse movement.

It, you know, it really never seen that on, on the internet. But it has a really strong prior for code. So out of the two categories, say web browsing and coding, coding is the only one that is actually already ergonomic to what the language model is good at.

And so it's much easier for you to amplify good behavior. Whereas if you wanna do a web browsing company or something like this, you have to go and collect this gigantic data set of human mouse interactions, which-- and, and figure out how to clean it up because I don't know if you ever tried capturing a human's mouse, but it's extremely noisy.

Oh, you know, nothing a Kalman filter cannot fix.

Yeah.

Yeah, so, so code is, you know, all you need, uh, it sounds like. But why so confident that it will lead to super intelligence? You know, I think you are one of the most strongly opinionated on this that I, that I've met, um, uh, because I, I guess, like obviously y- I think a lot of people understand that, you know, code can sort of augment the LLM in a sort of, uh, you know, like system one being deterministic, system two being a sort of more non-deterministic or sort of reasoning-focused, uh, thing that LLM can do.

But I don't really understand that leap to super intelligence that, uh, you mentioned. Like, I feel like this term means something special to you that we haven't really dived into yet.

I think that-- I mean, so for-- to me, um, it just means it's just a system that will be able to do most of the work that we want it to on a computer, and not just kind of do it, but do it more creatively than we could, right?

That it'll discover similar to how in AlphaGo, move thirty-seven was this kind of move that AlphaGo did that was more creative than what Lee Sedol, the kind of the Go champion could imagine at the time. Similar things I think will happen in, um, you know, in computer-based work where, you know, I think it's concrete in software engineering, where you ask a model to optimize some code for you, and it comes back to you with a solution that you didn't even know was possible but is amazing.

So I think that's kind of the, the world that we're headed into. But the reason coding is so fundamental for this broader vision is the belief that since code is already kind of the ergonomic form factor for, um, a language model, we believe that a lot of software that, you know, a, a lot of UIs are built for humans today, like the GUI mouse and keyboard was built for pe-- for a person.

Um, and over the coming years, there'll be more kind of AI-friendly or language model-friendly UIs. And what's friendly to a language model is, is code. So the way a model will be doing work, not just for coding and software engineering, is by basically making function calls or using SDKs or like programmatic interfaces that have been developed for it.

For example, I think the browser will fundamentally change. There'll probably be a user-facing browser that relies on the UIs that we use, but there will be, um, you know, more friendly language model interfaces for, you know, basically doing the same things that we do on a browser.

You can kind of think of this already happening in that, like the back end for like the OpenAI search products or Perplexity or some of these other companies. You can think about the back end for that is largely programmatic, and so they've effectively built these kind of AI-first browsers for themselves.

But I think that is more broadly coming.

Yeah, I think like abstractly, I think AI, AI-first browsers are more or less search APIs, maybe, maybe with a little bit more actions that you can take on, take on those things. One thing I, one thing I wanted to sort of dive into also as well is, is just the, the, the comment you made about sort of imitation learning and, you know, the interli-- uh, uh, you know, I dug up an old comment that you-- they had, uh, from twenty twenty-three about how ChatGPT interleaves code with text, and it was basically just imitation learning on contractor data.

Is that what you're seeing? I think like a-as you mentioned, like a lot of approaches typically end up with some f-version of self-play as well that, that scales up yourself. How, how are you approaching your data? Or like what are your strong beliefs on, on your data?

Well, you kind of, yeah, you kind of have to have both. So it's, it's really important to both get the right initial data mixture for-- from supervised fine-tuning, however it is that you gather it, that feeds the correct behaviors that you want to elicit out of the model.

And then you kind of ar-amplify those behaviors with reinforcement learning. So the, the key thing to reinforcement learning is that it only works-- I mean, it can potentially work otherwise, but practically it only works when the agent has interacted with a reward, right?

It's received a positive reward for what it's done. Maybe one out of ten times, one out of fifty times. But if it's getting zero reward, then you don't really know what behavior to amplify. And so that's why it's really important in the kind of initial mixture for it to already be doing things out of the box that are somewhat sensible, even if they're unreliable.

Yeah, good priors are very important. Yeah, so like, uh, and then I think one more thing that like ties to a previous conversation we had with, uh, Brett Taylor from OpenAI, which was like A lot of our code right now is just in languages that happen to be nice for humans to write, primarily Python and JavaScript.

And there's a question of do we, you know, just like you believe that there, there should be AI-native browsers, AI-native user interfaces, is there a different language that, like, will actually be better for AI to code in?

You know, that's actually-- that's a really fascinating question. You would think so. You would think that there's some better abstraction. Again, like, coding languages are mostly made for humans. You'd think there's probably a better abstraction that's more AI-native.

It's possible that something evolves that is different from what we have today, but it might also be... look somewhat similar because the things that language models understand best tend to be the kind of piece of code that are represented on the internet.

You know, I, I-- may-maybe, uh, I don't think a lot of people would be happy with this, that maybe the kind of fundamental language of AI becomes Python or something like this, like everything gets bundled in Python SDKs.

Um, it's... Yeah, it's, it's not s-- I'm not sure exactly where, where it will go, but it's probably going to be something close to what's already native to, to an AI or some abstraction of that thing.

What do you think, like, a Move 37 for a coding agent would look like? Is it... To me, it's kinda hard for me to think of, like, something in code that would, like, shock me when I write it.

Yeah.

Um, so I'm curious. Like, I'm sure you've given it a lot of thought.

You know, I, I think, uh, I think there are gonna be a lot of Move 37s. Uh, and, you know, one of them... So the-- Right, uh, for example, we're building, right, we're training these, uh, large language models and maybe, maybe kind of a collection of what some, some people might have perceived as, uh, close to Move 37 was, uh, you know, DeepSeek's recent, um, open sourcing of their, of their various code components, um, that they use to train that model.

Which I think outside of the big labs was not really well known to... Right, it was not really well known how to write kind of a kernel that's optimized for this particular p-type of attention. And that might be a glimpse of, you know, what that might look like.

Kind of i-imagine the kind of innovations that are happening, I mean, are at these big labs are just kind of invisible, um, and made visible by a company like DeepSeek. Imagine kind of an amplified version of that, where you ask an AI system to redesign, right, your neural network architecture or kernel structure to be much more efficient, and it comes back to you with basically something that is, is just much better than you would have expected.

Yeah. I, I think that's one of my, uh, issues with how people talk about deep research is like, can the model come up with new ideas? And to me, it's like, in what? You know, new ideas for whom?

You know? Like, yeah-

Right

... this is not a new idea for everybody in the world. Like, some people already knew it, but, like, you're kinda opening it up to everybody. So what, what do you think the role of open source will be in that?

It feels like these coding agents should, like, default try and be open source because I feel like it can compound the learnings so quickly. But I'm curious if, like, the external interactions kinda are negative in a way.

Well, uh, I think the... I think open source generally plays a really important part in the AI ecosystem because it allows for a plurality of research labs to exist. It allows, right, it's kind of if open source models didn't exist today or open weight models, then the starting point for any AI company that wants to train things would be to pre-train something.

That basically the starting point would be, you know, like it's a massive amount of capital required to just, you know, pre-train these models. So I think that the existence of open-- There, there are a lot of incentives, right, for open weight models to continue existing and improving because of how beneficial they are to the ecosystem.

And, you know, what I'm more worried about is a company developing a really powerful coding model and then keeping it to themselves. Because exact-- Like, at that point, it's not that the model itself has to be open sourced, just acc-- you have to have access to it.

So you can imagine, uh, you know, the world converging on a few companies have really powerful coding models. They basically release a nerfed version of that to the public at large and basically have a competitive advantage by having, you know, a, a super intelligent coding model that only they have access to.

So I think it's really important to have a player or more that, uh, basically continues releasing the product for, for others. And I think open weight is a really important part of the story. But in this case, it's kind of just having access, like the same model that you have access to internally that's incredible, that should also be the model that people externally are able to use.

Yeah. I mean, in some sense you are incentivized to do it or you have the ability to do that as a, as a foundation, uh, as a model lab that is, uh, working on coding models. So, like, you're, you're going to...

Uh, I mean, it sou- it sounds like, y-you know, I think we're gonna ask something about what you plan to release. Like, what is the shape of the products that you tend to release? It sounds like you will enable some kind of model access.

Are you going to, uh, you know, like Poolside AI is, is i-in a similar sort of category of code-focused model labs, and they've focused on their product being more of like a VS Code extension that people can use to access their models.

Like, how-- what is the shape of the product's thinking that you have so far? You know, I, I know you're not launching it yet.

Yeah. So the kinds of problems we wanna solve are maybe first just thinking about what are... There, there, there are multiple form factors for a coding product, and the form factor that is, um, most common today is w- you know, this kind of cruise control form factor, which is the Copilot.

GitHub Copilot or Cursor, which are incredible products. We use them internally. We're very happy with them. But again, this is-- these are products where the engineer is driving most of the work. Like, even in agent mode, really, the engineer is, like, driving most of the work, stepping the agent through and so forth, and really supervising it.

Um, what we're building is more of the autonomous vehicle. So, right, the thing that you give it a task and it takes you from point A to point B.

The form factor for that is, uh, is a bit different. I mean, one way to instantiate that form factor is through an IDE. Like, that's, that's definitely something that people have been doing. But another way is to provide access to it as an API.

And what I mean by API is different than an API that just streams tokens, right? Because if you need a, you need a model that's kind of coupled to a computer, right, that it's able to read, write, and run code.

So it's kind of, um, an API that rather than taking in tokens and outputting tokens, it takes in a task and a code base, and it outputs-- it does some work on its own in the background and then outputs the code that resolves the task.

So our product form factor is more towards kind of the latter to help mostly kind of large engineering teams is where we've been finding pull from. So these are teams that tend to have, like, large and pretty sophisticated code bases.

And when you're in that state, an engineer's work ends up being, like, it-- there-- a lot of this kind of backlog work accumulates, um, in terms of maintainability, refactoring, migrations, uh, you know, security vulnerabilities, and so forth, that as a company gets larger, the, like, engineer goes from spending most of their time on, you know, working on the features that matter and the kind of more, the more kind of value-driven work as a, at a startup to very large company where you have giant code base and a lot of your time is spent on, you know, kind of playing Whac-A-Mole on this backlog.

And so we help, uh, companies automate those backlog workflows. So kind of workflows that add tests or, you know, make sure the code follows the style guide, you know, remediate security tickets or infrastructure incidents, um, refactoring migrations, things like this.

We kind of, you know, provide an API that helps companies resolve some of these processes.

Yeah. Yeah. Very, very, uh, useful in, in, in any number of form factors once you expose as an API. I think one of the, uh, uh, one of the curiosities I have, I, I, you know, I just, I'm just gonna ask, um, personal curiosities now is for large code bases, like basically code indexing, uh, and one of the interesting approaches is what Magic.dev is apparently doing, which is training hundred million context windows.

Are you a believer in long context? Are you doing some kind of, you know-- what is the SOTA in code indexing these days?

Well, definitely I think, I think the answer may be not, not as clear-cut as just long context because, you know, on one end of a spectrum, you can just use a, you know, RNN or an LSTM, and you have infinite context, but, you know, the memory is really bad, right?

So it's not just having long context, it's whether your model truly understands what's inside the context. And needle in the haystack tests are, um, pretty crude and not very effective way of testing this kind of capability. The question boils down to either if you can put everything in a context, how deeply does a model actually understand it to attend to the right things?

So it's kind of what's the, what's the right attention mechanism over this long context? Or if you kind of make this problem more agentic and you have a limited context, then the problem is for an agent to go through a file system and kind of figure out what to stuff in the context.

I mean, you can land anywhere kind of along that spectrum. I would say it's, it's an open research problem. We have members of the team that, um, actually buil-built some of the-- I mean, it's called localization, and that have built some of the best localization systems before, uh, on pro-- on projects like Gemini.

But it's-- it doesn't have a clear-cut answer today. It's kind of a, it's a research problem.

Yeah.

My bet would be on long context, um, understanding, uh, and improving the attention mechanism over the long context. That has some interesting challenges that come up with it, right? The-- basically, the longer the context, the more memory intensive this becomes.

Becomes hard to fit on, you know, any reasonably sized cluster. Now, of course, there are tricks you can do to make your context longer, which is kind of like the stuff we did in Gemini 1.5. Um, but nevertheless, you need to kind of trade off, like, you still need to trade off kind of model size for context size, um, just by kind of the constraints of how valuable is it to right solve this problem.

Is it valuable having, uh, you know, a cluster of ten thousand GPUs that are just handling like a billion token context? You know, for one user, probably not.

Yeah. Yeah, it can be a really profound problem. Okay. And then, and then last thing I guess, uh, would be how do you believe that you should communicate or measure your coding agent abilities? Obviously, n-right now, people focus on Suitebench.

Um, do you have strong feelings on that? Do you have alternatives? How will you, how will you, I guess, encourage people to think about comparing your products versus like a Devin, like a Poolside, like a Magic? You know, there's a, there's a whole bunch of people all competing in this space.

Yeah. I think it's, I think it's kind of the heart of, of what's going on here. It's such an important question because it's kind of asking how do you evaluate the capabilities? And one of the core beliefs of the company is that the evals that matter most are the ones that are done in the real world.

So autonomous coding benchmarks, let's say like Suitebench, are useful. Not-- I'm not gonna discount them. They are useful. But let's say, you know, ninety percent on Suitebench could still mean something that just falls over flat within, with-within a customer setting.

So, you know, I think the... It's, it's a pretty simple answer. You have to work with users, you have to work with customers, and you have to set up evals that are representative of the kinds of problems that you're seeing kind of repeat across multiple different types of customers.

And kind of more importantly, I think that it's for this reason, like, because your customers define your evaluations, I don't think that superintelligence can be built in a vacuum. Because if you build it in a vacuum, you, you have no guarantees of-- you basically have no guarantees if it's superintelligence.

It might be superintelligence on the things that you trained it on that you then evaluate on, you know, a set of benchmarks. But how do you know that actually translates to the kinds of problems that people care about in the wild?

Now, if we think about superintelligence as You know, the system that helps us do kind of most work that we wanna accomplish on a computer, then it's kind of by-- that's a by definition, like a customer-centric approach to super intelligence.

How do you know if it's actually gonna help you with your work on a computer other than guessing if you don't co-develop it with the customer? And, and safety, I think, is another thing that, you know, that ties into this.

Like, how do you know it's, uh, safe and reliable for the things that your customers care about within their context if you didn't co-develop it with them? Um, you know, to date, how do, how do AI labs do safety?

The most practical safety method to date has been RLHF, which is the kind of area of Gemini that Yannis and I worked on really closely. And RLHF is this really simple idea that, well, if we want models to behave in, in a way that, you know, people would find pleasant and non-offensive and kind of not disclose harmful information, well, then you need to train them with human feedback in the loop.

And, you know, that human feedback comes from ultimately users, right? Ultimately, the users of a product like ChatGPT or, you know, Anthropic's products. Like, yes, there are, you know, uh, raters that are contracted independently of that, but a lot of the eval signal is, you know, comes from the product.

So I think it's no, no coincidence that the kind of most responsibly deployed, uh, AI systems in the world have, ha-have been through product. Like, it's always been kind of a model that's deeply coupled with the product.

Yeah. I know we're running out of time, but as we wrap, I'm sure you're hiring people, right? You're announcing a company. I'm sure you want people to work there. Who are the type of folks that are gonna thrive there?

What kind of work do they wanna like to do? What's kinda like the culture? And then, yeah, if you have specific roles, but I'm guessing it's across research and technical roles.

Yeah. I think across, like, uh, as far as roles go, it's mainly we have-- we're hiring across research, infrastructure, and product roles. In terms of, you know, in terms of a good fit, I think that, you know, let's assume that it's a good technical fit, right?

I think there's a certain technical bar that is required to work at a place like this. Um, but let's assume that the technical fit is there. You know, the things we value are we value agency. So kind of, um, you know, agency basically means like a sort of speed and intensity to how, like, someone conducts themselves, and they kind of go out and solve problems rather than waiting for hoping someone else will solve a problem for them.

We care a lot about craftsmanship. It's getting the details right in building these systems is really hard, and typically, people who take a lot of pride in getting details right and engineering systems in a robust way tend to do well.

And I think the last thing is, is just kindness. We like working with people who are kind. It's-- Yeah, it-it's really important. You know, it's sort of-- I think it goes without saying, but, you know, I think people have worked in a pretty diverse set of cultures, but, um, we don't really tolerate rudeness.

It's really important that people are kind, speak calmly to each other, are good at perspective-taking. Like, these seem like, you know, maybe kind of obvious things that people should be doing. But, um, but I think that sometimes we can get carried away with, you know, the mission is so important that you wanna achieve it at all costs, and this is kind of an important constraint that, that we take in the pursuit of super intelligence.

Awesome, Misha. Yeah, this was great. We're looking forward to doing another one once you have, uh, your first product too.

Yeah. It was, uh, yeah, really fun being here. Thanks for having me.