The AI Memory Problem: Why Long Context Isn’t Enough — Dan Biderman, Engram Co-founder & CEO

I'm too absorbed in the cooking, man.

Yeah.

Sorry, I'm fried, um, the entire show I pretended to be the cooking expert here.

Okay.

It has to be pure crazy.

Hey guys, welcome to the Latent Space Cooking Show, where we invite founders and researchers and let them cook. Today we have a very special guest, co-founder and CEO Dan Biderman of Engram.

Nice to be here.

Yes, thank you for coming. Um, I heard, or we got connected through Jack, who is saying that you're a very good cook. And again, congrats on the $98M seed round. Do you cook often? How frequent is it? I assume you're constantly researching and building and.

Yeah. So I would say since I started the company, there's a little bit less cooking of this type, uh, cooking other things. But cooking is something I've been doing since I, I was a kid. My parents cook, my grandparents used to cook, and it's, it's a way for me to kind of relax and digest things and think things through.

Great. Yeah. Well, I'm very excited. Today's a little different, where we're actually going to have Dan lead us through one of the recipes. Do you want to reveal to us what we're making?

Yeah. So I was thinking we can make, um, beef meatballs with some vegetables and have them swimming in some kind of, uh, white sauce with white wine and, and, uh, chicken stock. It's a pretty nice Mediterranean type, uh, meatballs that I've kind of stumbled upon recently when visiting my parents in Tel Aviv and cooking stuff with what they had at home.

So I thought it would be nice to, to redo this with you and see how it feels.

Great. Well, I'm very excited. Do you want to kick us off?

We're going to fry the onion. I'm going to chop it, and I'm going to chop some garlic, and then we'll, uh, make, make them into balls and, and fry them.

Fry them. Great. Okay.

Yeah.

Well, let's get started. We'll both do it in parallel.

Awesome.

Um, but yeah, I guess to start, I'm kind of curious about your background. So you wanted to become a professor, if I recall correctly, and you also had experience in the Israeli military or special forces.

Yeah.

Um, how did that turn you into becoming a founder of a research.

Yeah.

AI research company?

Yeah. So I guess, uh, it's, it's easy to connect the dots, uh, looking backward, but in real time, it was not. I didn't, uh, grow up planning to be this founder and a chef working on this hot topic in AI.

Mm-hmm.

Things came to it. Um, so I grew up in Tel Aviv and, uh, and as you said, uh, was an officer and, and working on special operations, specifically naval special operations, which was very entrepreneurial. It was, uh, basically finding and identifying crazy things we could do and, and prove to people that we should scale resources and, and people to actually do them.

So it was a lot of, a lot of researching, a lot of pitching, and then a lot of hard work to make things real.

Yeah.

Um, and then after I finished this, um, I went to university, um, and studied cognitive, uh, neuroscience in Israel, uh, in an interesting program where, um, actually a lot of Israeli novelists and professors and scientists and.

Mm-hmm.

And even chefs. You have Tamagotchi; the Israeli chef went to this program many years before me. Um, and so I went there, got really curious about the brain, curious about human behavior.

Yeah.

Curious about statistics. And then I went to, to New York to do a PhD in computational neuroscience, um, which is a community that's been one of those stubborn communities that cared about neural networks.

Mm-hmm.

For many decades before it was a hot topic.

Yeah.

Uh, here in the valley. Um, so I was there and studied with some statisticians and physicists and biologists trying to understand, um, how neural networks can be used to understand the brain.

Mm-hmm.

To understand animal behavior. And, uh, and then decided that I wanted to go deeper into the LLM world after seeing ChatGPT. So I started working at Mosaic, uh, where I worked on LoRA and met a bunch of great people.

And then from there, I went to work with Chris Ray at Stanford.

Chris Ray, yeah.

And Scott Linderman, who both are co-founders of my current company.

Yeah.

And where I met my co-founder Sabri and my other co-founders Jack and Jesse, uh, who worked on very similar topics from their labs in Cornell and Berkeley.

Yeah. No, it seems like a very, very research-focused and also serendipitous. Going back a little on your background.

Yeah.

Is there something about the Israeli special forces? Because I think it was the Wiz co-founders,right, who were also ex-special forces.

Yeah. Uh, we worked pretty closely with Assaf, uh, Rapa Poort, the Wiz CEO.

Mm-hmm.

Who's, uh, uh, a mentor to me. Um, and, um, yeah, there's interesting things going on. Uh, there, I would say the, the most important thing is actually, uh, when you're 18, it's joining the intelligence or taking on responsibility.

Uh, the interesting bit about it is that you're not just in a student mode. You're not just heads down doing exams and projects. Rather, you interact with, with grownups, and you have resources, and you argue your, your for your, uh, budgets and things like that.

Interesting. Oh, okay.

Um, and it forces you to kind of build a little bit of social skills, a little bit of maturity.

Mm-hmm.

Um, which, which I benefited from.

Yeah.

Um, and, um, you know, there's a reason many countries in the world choose not to have everyone go to the army. It's a long time, and it's, and it's, uh, uh, a package deal. And I was, uh, in the Israeli army in different years, not the ones that are the recent ones.

Gotcha.

Um, so it's not like I recommend everyone goes to the army, but I would say that it forces you to kind of think about other s other, other aspects of your personality, not just the intellectual ones. And also generally, I would say Israel as a culture is a place where, um, you can basically you get multiple shots at goal.

Mm-hmm.

If you're not the best in high school, um, you still might have a, a good position in the military. And if you're really good, more doors open up for you for university. And even if you didn't do anything super interesting in, in military and got to university and found out you have good and deep inclinations in science and engineering, you can be the most successful person.

And we have some of those Israelis who are, um, leaders, uh, in various, uh, academic and technological places.

Oh, that's great.

Um, so I would say, yeah, it's mostly like you can basically.

The culture.

The cards are dealt multiple times.

Yeah.

More emphasis on social aspects, more emphasis on being mature, on being a team player. Uh, but again, it comes at a price that you get, uh, you, you're older when you get to things. Um, so I started college when I was 24, which is, uh, a, a middle PhD in, in an American, uh, university.

But yeah.

That makes sense. I think it's great how it gives you a lot of shots on goal and is a very beautiful culture. I know fast forwarding to today, back to Engram.

Yeah.

What kind of prompted you to focus on this issue with context? Were there specific breakthroughs? Was it, you know, the performance of open source models or even just the issue of Context Rot, long horizon agents, or was it just that you're very passionate about this area as a whole?

Yeah. I'll answer that in a second. Can we.

Yeah.

Can we heat up some.

Yes. Let's start getting going.

Oil here to fry up the.

Great. So we can turn on the Impulse stove.

So yeah, I, my, my PhD was focusing on, on, on a field that's not super in vogue today, but I think will become super crucial again, which is semi-supervised learning.

Hmm. Okay.

Which is you have very little data. You don't have infinite trillion tokens of data. You have very infinite, you have a very small set of examples. And from them, you want to extrapolate and learn efficiently for, uh, and, and achieve things more, more generally.

Mm-hmm.

So how to be data efficient. So my whole academic work was basically all of my papers had the same plot. We're on the X-axis. I had.

Mm-hmm.

Some, some cost, some resource, and on the Y-axis, I had, um, I had, uh, accuracy, sort of Pareto curve. Uh, and that's been my, my upbringing.

Mm-hmm.

Efficiency, how to do more with less, um, salt.

Yeah.

Yeah. You can put some salt. Um, and I went to, to Chris Ray's lab, um, and.

At Stanford,right?

At Stanford. And there I worked on agents, on ideas called minions, uh, and, and asked kind of the question of how can we have agents interface with very large, uh, corpora of knowledge in a way that's economical, and when we introduce some patterns that involve sub-agents, uh, a bit before it was, uh, popular.

Oh, sorry. We can also put this side of your glass.

And from that perspective of efficiency and of cost, um, we started asking ourselves, actually.

Yeah.

Um, is there a more efficient way to interface with data that's not just agentic orchestration and things like this? Can we actually use the, the magic of training to, um, have models that are efficient, faster, and, and, and can operate with fewer tokens?

And some of these ideas kind of were developed in parallel by some of us, uh, Jesse, Sabri, and myself.

Yeah.

Uh, and the common theme was that if you have a very large corpus of knowledge, um, and you allow the model, you give the model time to study it in advance to basically ask itself questions, give itself quizzes, try to solve problems, and then train it using gradient descent in the same way you would pre-train a model.

Mm-hmm.

Um, then you can create very compact representations of your context that can later on, you can load them. We call those cartridges.

Mm-hmm.

These are like capsules of knowledge you can load in and out of the model that are like a brain state that describes the model's world and the corpus in a way that's going to be 1,000x more compressed. And when you use them, you can basically, uh, use far fewer tokens, be less confused, more accurate, and things like this.

So we got into.

And these are task-specific cartridges?

So they can be corpus-specific, like documents inside a company. They can be task-specific, like you want to learn a certain skill. But the idea is that in many cases, it makes sense to go beyond textual representations.

Yeah.

So when we're cooking now and we have all these, like, great textbooks.

Mm-hmm.

Our cookbooks here, Samina Swat and, you know, a book from French Laundry. These are amazing books that I.

Yeah.

I respect. But to say that because we have the books and we can read them, to say that we deeply know how to cook to the same level of Samina Swat or, or, or Thomas Keller.

Yeah.

It's incorrect,right?

Yeah.

We can read those things and we can repeat all their steps in a very robotic way. And we can come in, current LLMs are like coming into the kitchen first time every time, reading the textbook, uh, cooking the dish, measuring everything.

Uh, but they don't have the intuition of, uh, of a chef that's pinching salt and kneading, kneading dough and things like this. So the kind of thing we're after with this kind of, uh, training and creating those cartridges is this kind of intuition in the models that goes beyond notes and recipes to the kind of intelligence that allows then, allows you to come, come up with the next recipe, thing that hasn't been explored before to do the next, uh, move, the next extrapolation.

Yeah. I guess double-clicking on building this intuition, could you kind of contextualize it more on how it would be different from extracting, let's say, using the cooking example? If you get all the useful notes and useful sections from a cookbook, that will help you actually understand all the complexities of making a dish.

Just providing it, I guess that would be similar to RAG, getting just those chunks.

Yeah.

And then understanding that, I guess, in context and then providing an output.

Yeah. So this is, this is an excellent method. Um, we do not take the bet that I'm putting two eggs.

Yeah.

In here, if that's okay. Um.

And is this enough grated carrot and zucchini?

Yeah. Excellent. You can put it in here. Um.

Yeah.

So we're not taking the bet that no ne no notes need to be taken,right?

All of it.

Yeah. Put all of it inside.

Mm-hmm.

Um, all the greatest chefs in the world, they have notes and they have books and they.

Yeah.

They have diaries and they document what worked and what didn't work with their experiments.

Mm-hmm.

But they also have brains that they also have hands and, and, uh, and, uh, a tongue.

Yeah.

That can remind them how something tastes and what worked and what failed and what was easy and what was hard. So, uh, in all of our work, we never say that textual representations are useless.

Mm-hmm.

We basically use them all the time and we construct those wikis and knowledge bases and things like this.

Yeah.

Um, at the same time, what we say is that the layer above those, the learn the layer of intuition and learning, uh, that is in the form of numbers of parameters, um, is, is the, the full experience of the human chef.

The best chef in the world is all the notes combined with a nervous system that reads those notes.

Gotcha.

And can implement them and innovate them and maybe don't re-return to the same notes over and over again. There are some dishes where they don't need to, to reread them.

Mm-hmm.

And the current problem in, uh. So I was saying that, um, we want, we want the best of both worlds.

Mm-hmm.

Every knowledge worker, if they can't write notes and they cannot document the events of the day, they would be, uh, in a, a disadvantage.

Mm-hmm.

But, uh, if you wipe their brain every evening, they would also be at a severe disadvantage.

Yeah.

Um, so we want to have the best of both worlds. And, uh, the thing is that textual representations can take you, uh, a, a long way. Uh, but the thing we're thinking about is like, look, the, at the rate at which knowledge is being created.

Mm-hmm.

With now agents working on behalf of knowledge workers, creating artifacts, code, documents, uh, presentations, I think that people don't fully comprehend the size of the.

Yeah.

Of the knowledge workspaces they will deal with in 18 months.

Yeah.

Um, in 18 months, many companies would have, uh, maybe trillions of tokens, which.

Of internal company.

Of internal company data, proprietary data. I'm talking about like maybe trillions. It sounds exaggerated, but I don't think it's an impossibility if they're really AI native.

Yeah.

And I think, and, and what are trillions of tokens? Uh, like when I was at Mosaic two, three years ago, like we call this internet scale data, pre-training data.

Mm-hmm.

So imagine every company has data that it's basically internet scale data.

Um, yeah.

And I might need, uh, salt.

Yeah, we'll have tea.

And pepper and the bread crumbs.

Yeah.

For this. And we don't have a sink here, so hands will be a tiny bit dirty.

No worries.

The audience here is forgiving. Um.

Gotta get your hands dirty.

Yeah.

Awesome. I guess the principle here I once heard from a chef is that a ratio of one to one of meat with everything else is usually a healthy way to make meatballs. So I think we're, we're there.

Yeah, that's fair. We are on salt and pepper.

Um, nice meatballs. They're a bit warm with those fried, um.

Oh. We're back with the fix.

We're back.

We're back with the fixed meatballs.

With the fixed meatballs. It's, it's team building activity here.

Yes.

Oh, yeah. Let's put those spices that you brought.

Yeah.

I trust you on, on these ones. Cumin, let's put a little bit, not a ton.

Yeah. We won't dump it like last time.

Cumin is my wife's favorite spice.

Oh, yeah. Cumin's great.

I have mixed feelings about it, but.

Oh, okay.

Yeah. It's good. Yeah.

Good.

Good. So we have the meatballs. Now is the time to make the balls.

So we were talking about what were we talking about?

I'm.

I think I'm just.

I'm too absorbed.

Oh.

I'm too absorbed in the cooking, man.

Yeah.

I'm sorry. I'm.

No, it's good. You were.

You were pitching things here.

You're talking about how companies will have a corpus of data that will be at the scale of the internet?

Yeah. Yeah. And maybe today.

Maybe today.

Some they don't have it. And maybe today you can go relatively far.

Yeah.

With textual representations. They're also interpretable. They're very good.

Mm-hmm.

Um, but I just think at a certain scale, even those textual representations will be hard to make.

Okay.

If you have trillion tokens, how do you create exactly a wiki or an index of those that you keep updated all the time? How big is this?

Yeah.

Um, knowledge base that you create, how expensive will it be to process it with frontier models that know nothing about your, your company?

So is it just that using frontier models will be expensive because that's the start from scratch every time? Is that the main?

So there's the element of expensive because you reread more things, you consume more to tokens. That's one.

Yeah.

But two is like for the agentic tasks of 18 months from now, inside those major repositories of knowledge and asking the models more and more things and under specified ways.

Mm-hmm.

I suspect that the accuracy, uh, of the models would, would go down. The, the phenomenon of context rot,right?

Yeah.

The model has to read more. It will be less accurate. And we know this, and it will remain the same thing at even at the 10 million, uh, context window scale.

Okay. That makes sense. Okay. Let's go wash our hands real quick.

Yeah.

And we'll beright back.

Great.

And we're back. Hands are all clean. So now what's the next thing? Just frying the meatballs?

Um, yeah. Let's fry those meatballs.

Okay. Do you want to turn this one?

Yeah.

You just press it and then turn it. Very intuitive.

And we'll put some oil.

Okay. We could put these in and.

Yeah.

While we do that, I guess more so on the question of like long horizon agents.

Yeah.

Um, what's the main issue that Engram is trying to solve? Soright now, if I were to steel man the opposing view, um, maybe company, internal company data isn't actually big enough where I'd need to put it into the weights.

Like what's the issue with just having RAG or, um, having specific models or even cheaper models since open source models are also very performant to handle a lot of tasks?

Yeah. So I would say like, um, a thorny question in the research community is, can you come up with an example where only in weights training would work, where in-context learning will fail?

Mm-hmm.

And it turns out it's very hard to devise such examples. And for every example you give, someone can ask, well, what happens if the, the next, uh, the next fable has a 10 million context window?

Mm-hmm.

Um, and the way I see it in kind of my scientific upbringing, I see all of these questions of, of continual learning and memory as questions of long context, uh, in disguise.

Yeah.

Um, if the models could see a whole, whole company data and in principle would have this infinite context window, what then is the limitation?

Mm-hmm.

So the limitation is twofold. One is that we know even at very small scales that the more context you feed to the model, the more confused it gets.

Okay.

It's called the context rot. So you can feed in a certain, uh, number of tokens into the model and not get an error, but it doesn't mean that the model can reason in, in a, in a holistic way about them.

That's one thing. And.

And what's the problem with compaction? Is compaction not as useful, would you say?

So I think compaction is, is improving by the day.

Yeah.

And compaction for those in the audience who don't know what it is, it's like models actually managing their own context, uh, evicting certain tokens and, and keeping others. Uh, compaction works. Uh, that too, when you go into a longer horizon, compaction by definition is lossy.

Mm-hmm.

You discard some and keep other things. And I think it's, it's a, it's a correct way to, to go. Uh.

Yeah.

But it's, it's also like very deterministic.

Yeah.

Either you're in or you're out. And the current versions of compaction, uh, show these kinds of, uh, uh, issues when, when very deep into the session you can get, um, confused and you can get, um, forgetful.

Mm-hmm.

So we think compaction will be part of the story.

Okay.

We think another part of the story is some sort of neural memory trace, which too is a lossy thing. It evicts some and, and keeps some, but not in the text, uh, representation and the weights representation.

Okay.

But I would say the main thing we're trying to solve or the main thing for which continual learning is needed, one is token efficiency and cost, which is a major issue that wasn't actually an issue when we started the company late last year and became moreurgent now.

And problem number two is if you can do, uh, if you can do the same thing with, with fewer resources, when you scale up to very large resources, suddenly you can take on tasks that were previously not possible.

Yeah.

Way more long horizon.

Okay.

Way more, uh, adaptive. And we're not quite there yet. Uh, we're focusingright now on, on the first component, which is getting these models to reason on large context with fewer tokens and do it in a way that's, that's more, uh, uh, le-less confused.

Yeah.

But we think that eventually, uh, part of the.

Yeah. I'll take pictures. Thank you.

Part of the, part of the solution for very hard tasks in, in science and engineering and defense and all that stuff will involve some form of gradient-based updates.

Mm-hmm.

Um, during, uh, during, uh, doing these long horizon tasks.

Gotcha.

Uh, some people call this, uh, test time compute or test time training. Um, these are just different names for the same thing. I think we have a, a proof of existence.

Mm-hmm.

From pre-training that, um, you can pack a lot of information in very few numbers.

Yeah.

Very efficiently. Um.

Mm-hmm.

So the examples we like to give is that, uh, if you take a Llama 70B model and you load one article from Wikipedia, which is a few tens of kilobytes, and you have the model read this, uh, the brain state of the model when reading this few tens of kilobytes is like 80 gigabytes.

Mm-hmm.

80 gigabytes on, on the HBM of the GPU.

Yeah.

It's an insane amount. And the entire set of parameters of this model would be like 140 or, or, or so gigabytes.

Mm-hmm.

FP16. So those 100 or so gigabytes with some distortion represent the entire internet. And this one article about Taylor Swift is like same order of magnitude memory consumption on the GPU. So it's highly memory inefficient.

Yeah.

That's a systems problem. That's the, the KV cache monstrosity that the smartest people in the world are trying to solve from the chip side of things and from the, from the software and kernel side of things. Um, but yeah.

So I would say another way to look at what we're doing from that angle, from the systems angle, is basically, uh, if we, if we can get a bit more technical.

Yeah.

Instead of doing those, uh, prefills where the model's just reading and reading and reading a corpus, uh, we are kind of like the we're destroying prefill. We're scaling training compute in some other time so we can load the thing into the model and can just immediately start decoding.

Okay.

Or prefill a little bit. And this kind of goes hand in hand with trends in how data centers are built out, this aggregating prefill and decode and doing this on different specialized, uh, cards.

Mm-hmm.

And, uh, and this is part of our.

Yeah.

Part of our initial interest in the thing.

Mm-hmm.

Okay. So it seems like we're mostly ready in here. I think it's a good time for our.

Have the wine.

White wine.

Okay.

On both of them.

Great.

Uh, maybe.

Do you wanna do that?

Yeah.

Yeah.

Yeah.

And do you have any tangible examples on this when you're talking about.

Yeah.

Your approach.

So if you think about, um,

let's think, for example, about like an enterprise, uh, a firm.

Mm-hmm.

Or, um, investment banking, a law firm or investment banking, they can have many client matters, uh, many.

Like a lot of knowledge work.

A lot. Yeah. So many client matters. They have various clients, clients do, uh, financing, mergers and acquisitions and things like this.

Yeah.

And take loans and do deals.

You wanna put this on or not yet?

Um, no, not yet.

Okay.

Will you wanna let it, uh, let it.

Let it reduce a little? Okay.

Yeah. And so.

Yeah.

For example, these are the kinds of things we, we work with Harvey.

Mm-hmm.

Very, very large file systems. Uh, and there's many queries that agents might run into. Either humans ask them or the agents have to solve them.

Mm-hmm.

Which are these kinds of like ambient hard questions that are not easily searchable with RAG. For example, if you wanna ask like, which M&A deals haven't we completed this year? So to actually solve this problem, you have to go client matter by client matter.

Yeah.

Read all the files. You can't read in any place that it was not completed. You have to take the gist.

You have to understand.

The, the thing hasn't the loop hasn't been closed.

Yeah.

Thing hasn't been completed. And now you can solve these tasks with, with frontier models and compaction.

Mm-hmm.

And when you ask them to do so, they will consume thousands of dollars for queries that we think are harmless.

Mm-hmm.

That every employee in the company would be able to answer.

Mm-hmm.

So this is just like one example, but these kinds of holistic things that are, you know, you can get the gist if you read everything, but you can't find a thing in one thing. The kind of queries where the whole is greater than the sum of its parts.

So that's where, uh, this kind of magic of training, uh, comes in. And that's the, the magic that, that, you know, Ilya and others.

Yeah.

Have shown us with pre-training,right?

Yeah.

You learn the entire web in these sets of weights, and suddenly the model can infer things, can generalize, can interpolate and extrapolate to new things. And that's the kind of knowledge we wanna do.

Mm-hmm.

And it, and it's not a, not a coincidence that we're training on the entire web and we're not just doing RAG over the entire web.

Yeah.

Or putting it in the catalog and renewing it in time. Because we do think that learning from a lot of knowledge somehow creates these associations in the model.

Gotcha.

So some of it is concrete problems of now. Other parts of it are bets that in 18 months from now, the scale of the data will require the methods that we know from pre-training work.

Gotcha. And so are you doing parameter efficient fine-tuning for.

Yeah.

Specific companies like LoRA based off of the corpus?

Yeah. Yeah. So our, our ambition in the long term.

But also start to write?

Yeah.

Yeah.

So our ambition in the long term, uh, is that every person has a, has a model or, or a part of the model or a set of weights.

Mm-hmm.

Uh, that.

Yes, sir.

That represents their knowledge, their expertise.

Yeah.

Learns from them. That the more time they spend with the model, the better it gets for them. The more data they give the model, the better the model is for them.

Yeah.

They control those sets of weights. It's theirs, um.

And the rights?

Uh, and they're incentivized to let, let, let's let it boil over, you know?

Okay. Yeah.

Um, so they're incentivized to we can, we can put the spices up.

Mm-hmm.

So they're incentivized to, to make it better in the same way that you would work with, with a Tamagotchi. The more you nurture it, the happier it is. And that's the kind of thing we would like to create with the model.

So that's the ultimate continual learning. And we think on the extreme scale, on the single human scale.

Mm-hmm.

Um, turns out this is not just a research problem. That's a major infra problem. And in the long, long term, I do think these things will actually run on people's devices.

Yeah.

And, uh, we're seeingright now the, the new hardware on personal computers is already, uh, you know, soon approaching the ability to run inference on close to trillion parameter models.

Mm-hmm.

Uh, which will be very interesting for that kind of personalization. But in the shorter term, we do think that, uh, the kind of large corpora of knowledge, very dense, with a lot of expertise, uh, can be found in the enterprise.

Mm-hmm.

Um, that's where people spend most of their time. That's where AI is, is actually being used the most.

Yeah. And enterprises.

Um, so we, we, we go there and that's our bet. And there we're, we're our bet is that parameter efficient fine-tuning methods like LoRA, like cartridges, like memory layers, different things we, we contributed to as, as a research team, um, can actually represent knowledge and can be combined with other methods that are like context management, traceable methods that, that people can, can use and understand and, and audit.

Yeah.

So it's a combination.

Gotcha.

So again, it's like, it's like the chef that has the cookbook and it has the recipes, has the diary, but also has the nervous system that learns from every session. Uh.

Gotcha.

So it's boiling here. It's all.

Yeah.

Pretty efficient here.

Yeah. Very powerful stove.

Yeah.

Very powerful. And then medium, low, then you wanna put this on?

Yeah.

Salt?

Let's mix it up. Yeah.

Okay.

Did you put some salt?

Here, I'll put some salt in.

Um, we're gonna have yellow rice courtesy of Allen.

Yes.

Um, great.

Well, we're nearly there.

Yeah.

We'll let the these guys cook for a little bit.

With what you guys are attacking, how do you determine what, you know, should live inside of the weights? What should still kind of be handled with RAG? Um, where things should be orchestrated?

Yeah. So this is a major open question, I would say, for us and for everyone else. And it's not just an open question in, in, uh, AI and, and a startup. It's an open question in the study of human memory from its, its, uh, inception.

It's like what kind of knowledge should be internalized and what kind of knowledge should be externalized. Does it make sense for you to remember everything you've seen as a person? Uh, people who have that, uh, often are not enjoying that capability.

Yeah.

And it's, it can be very distracting. It can be, uh, at some times very scary. Um, so a certain amount of forgetting is healthy. Certain things you wanna remember in the weights and certain things you wanna put in, in text.

So I would say it's an open research problem. The way to work on it is to train models both to train models to manage it themselves. And that's an active area for us. Have the model know, like, without any explicit supervision signal to determine this kind of stuff I can pull from my brain.

Mm-hmm.

And that kind of stuff I rather keep in notes. And you can imagine these kinds of things, uh, relate to like the saliency of certain parts in the data, how, how often, the frequency at which they repeat, and the affordances of like what can you do when you know this fact from your brain.

Yeah.

Uh, you can imagine that, uh, remembering, I don't know, remembering, uh, the, the, your room number in a hotel for tonight is less important than remembering your partner's phone number or remembering your address,right?

Yeah.

So.

So it's higher signal data to.

Higher signal data. And now the thing is, if you start manually, heuristically saying this is in, this is out, then it becomes a whack-a-mole.

Okay.

Every, every person in every enterprise has different data, and you can't really very easily pick and choose what goes in, what goes out. So the holy grail is have the model learn for itself, have it operate with a notebook where it can take notes, have it operate with a, with a brain, associative parameter efficient thing that it can read from, and have it decide when to go to each and do this with training in an unconstrained way.

And we're working on it, and more breakthroughs are needed.

Yeah.

But I think that, that is the dream, that the model learns what, what comes in and what comes out, and we get out of the way.

Gotcha. Okay. And so that's the angle you wanna achieve where it's completely autonomous, so there's no human in the loop to kind of tell it what to fetch. Um.

The human in the loop can be the user.

Mm-hmm.

And if the user chooses to say keep this in or keep this out, we would like the model to listen to the user.

Yeah.

But we would lo not like to depend on the user,right?

Okay.

So user feedback is something we can learn from, and we can learn from implicitly, but we don't want a person supervising every step because that's not the way, uh, people enjoy using, uh, language models. But I would say the kind of models that we're building, uh, unlike other models where you, you do thumbs up and thumbs down, and you're basically like helping the provider maybe in the next version give you something that's more workable.

Yeah.

Here, if you give a thumbs up or thumbs down or you say something, you know that someone's gonna scale compute on what you said, and someone's gonna go and practice, uh, to get better at what you said. And this is kind of the thing we wanna get to, like building trust with the user that, uh, they're listened to, and they're making a model that's better.

And it's not generally better for everyone. It's better for them.

Gotcha. So the benefit is it being a tighter loop. Um, compared to the gen, like you said, a general model provider, the UI having a thumbs up, thumbs down, you don't know if that's actually gonna contribute to the feedback.

Yeah. There's tighter loop, and we use a different machinery.

Okay.

If, if we allow ourselves to use the machinery of training, we know we can, we can hammer that in. There's no uncertainty about it. And, and another important thing to say is not everything that a user tells you is ground truth,right?

Mm-hmm.

Not all of us, including myself, are Einsteins, and we can say things to the model where we think we'reright and the model is wrong. And increasingly, the models will get better, and increasingly, they'll know more things than we do.

So the model in some way has to learn and understand and kind of like discern what, which feedback is valuable and which feedback should be ignored.

Mm-hmm.

But there too, I think the holy grail is to get out of the way and have the model learn it if you define theright objectives for training.

Gotcha. Okay. So get out of the model's way. Have there been any moments that have really kind of shown you what is still needed, like, or what's still possible? 'Cause it seems very ambitious to be in an end state.

Yeah.

Where a model autonomously can handle this all. Um, and I think like multi-agent setups and evenright now too, just constrained to coding still has problems.

Yeah.

Um, and so I guess when you get to knowledge work and to specific enterprises, it seems a lot higher stakes.

Yeah.

Um, and you can't make as many mistakes. And so have there been moments or even research topics that you're kind of seeing great progress inright now that can get you there?

Yeah. Yeah. I would say like we're we, we will share more of our results.

Mm-hmm.

Like in the coming weeks and months.

Okay.

And we're kind of keeping, keeping the logs of it.

Still confidential.

Yeah.

Yeah.

But I would say the, the, the theme is and the kind of thing you're seeing, and we're not the only ones seeing it, but I would say like we're looking very closely into it is, is behaviors around token efficiency, the ability of the models to go where they need to go and solve things faster.

Um, and, and basically it's, it's this kind of view of intelligence where getting smarter means, um, exerting less energy, uh, to, to solve increasingly harder problems.

Yeah.

And these are the kind of things we're going for. Um, and, and a lot of it, it comes into, into life in the form of like, uh, efficiency and speed, but we will share more of that soon.

Okay.

Yeah.

I guess on the token efficiency point, do you also heavily consider like model routing? 'Cause I assume there may be some cheaper models that may do the same job for much less, but there may be some tasks that it may take, uh, much cheaper open source model, like $100 worth of tokens, when a much smarter model may be able to do it in one shot, like very quickly for much cheaper.

Yeah. I would say that, um, routing is an interesting thing.

Mm-hmm.

Uh, and, and it there's a reason so many, uh, enterprises and computer scientists are looking into it because, uh, we have models that are overkill for many things.

Yeah.

You, you don't need Fable to tell you how much like, um.

Water to put to.

Soft on to put in.

Yeah.

Rice and stuff. Um, at the same time, you do, you do need to know when to go to Fable when you are trying to crack something that's, that's, uh.

Mm-hmm.

That's above your pay grade. Um, so I think routing is a great direction. Um, routing too, it's not that easy as someone who's worked on it.

Yeah.

Um, you can train, uh.

Or what are the main challenges having the experience of seeing the difficulties of routing?

So I think routing will be part of the solution there for sure.

Yeah.

And I think many people, not just myself, say the solution is multimodal. It's not Engram taking over.

Yeah.

It is one model, and you teach it things.

Yeah.

And, and, uh, you can close Stargate. Um, that's not our approach here.

Yeah.

Um, the solution will involve some form of routing. Um, and in that part of routing, our, our thing would be like, you know, your best friend or your employee, you cannot fire someone who's been looking over your shoulder the whole time and can tell you where to look, where to focus, and can even go out and ask Fable things in a more targeted way with more context that Fable can then go and work on it for days and solve very high-stakes tasks for you.

Um, so I think routing is great. Uh, it's just, uh, it's just, uh, like any other research area, uh, it's an unsolved thing. There's been progress on it, but a lot of more work is required to actually route things to the, theright model, theright time, uh, theright cost.

Um, and models are continually updated. Uh, versions are coming out. So yeah.

Yeah. Okay. That makes sense. It seems like there's still a lot of open questions and even some of the work you're doing is confidential, which makes sense.

Yeah.

And has to release more. I guess now shifting towards a team.

Yeah.

Um, yeah. How is it like working with such a mixed group of folks, like some who had professors, um, while you were doing studying.

Yeah.

And some that you also met and have left their PhDs or finished their PhDs as well.

Yeah.

Like Jack.

Yeah.

Um, and yeah, having such like a diverse group.

So I would say, uh, there's many strengths to our group. Um, I'm not sure diversity is one of them. So if you're watching this.

Yeah.

We have a lot of researchers.

Yeah.

Uh, we could, we could diversify.

Research focused. Yeah.

We could diversify a little bit. Um, I would say, um, for this niche that we're trying to solve, which is memory and continual learning, taking knowledge, shoving it into weights, I think our team is the most specialized team in that, in that kind of thing.

Uh, and our team, our team is fun. You know, Jack.

Yeah.

Jack is a, is a fun guy.

Jack is a very fun guy.

Uh, and, and Jack is teaching us a lot of things about how to, how to think clearly, communicate ourselves clearly internally and to the external world, just see in the same way. Very complementary kind of like approaches. Uh, she thought a lot about how humans and AI work together, uh, in, in this closed loop.

Mm-hmm.

Of a machine and, and human. Um.

Yeah.

Sabrina and I were tiny bit more on the, on the systems, uh, side. And I was, Scott and I worked on statistics.

Yeah.

So it's all researchers and not diverse in this kind of way, but it is, as you said, diverse in our inclinations.

Yeah.

Some of us are more mathy, some of us are more systemsy, some of us are more like AI leaders. And so it's been interesting. The way we try to do this is to take those PhDs with a lot of experience and pair them up with like those kind of, uh, you know, up-and-coming, rising, uh, cracked types that.

Mm-hmm.

That have joined our company. Uh, for example, Shizza or Dhruv from Stanford and Berkeley respectively.

Yeah.

They're, they have research backgrounds and have written papers, um, but they're kind of, um, you know, entering this field with a lot of momentum. And we like to pair them up with someone who's been, who's been around the field for a few years, has some intuitions, can warn them from the rabbit holes.

And this, I think, powerful combination of different levels of expertise, uh, different levels of freshness of thought, uh, is making our place an interesting one. And I would say.

Yeah.

Culturally, all of us from day one, uh, in the, in the fall, winter of '25, which was a crazy time in terms of frontier lab.

Mm-hmm. Yeah.

Uh, recruiting and, and, and AGI anxiety. I would say all of us entered into the startup world in a very sober way, knowing that it's not just a research club in here.

Mm-hmm.

It's not just a journal club in here. The thing that's missing is products.

Yeah.

And products need to be distributed.

Gotcha.

You have to earn theright to play, um, by, by selling things that people love. So we're working very hard on that as well.

That makes sense. I guess more of a fun question.

Yeah.

Out of founders, who do you think has the best taste in food? I assume you guys sometimes like order food or even like go out. Like, are there some that stand out to you? 'Cause I'm the.

Yeah. I would say, uh, my co-co-founder Sabrina. He's, he's consistently, uh, he loves life.

Mm-hmm.

He loves, uh, good food. He's very well known in the company as someone who's the surf and turf guy.

Gotcha.

Uh, he will have it for lunch and dinner. And, um, yeah, being on DoorDash with him has been an inspiration for me. Um, and I would say, I would say, uh, Jack and Jesse have, have good taste as well.

Good taste as well.

Uh, yeah.

Great.

It's, it's fun to be with them. And we're now in like a small office, like an apartment. So we have all the lunches and then often dinners together.

Yeah.

And it's, it's, uh, a bit like a family, you know? We at home, you see your parents and siblings. Sometimes it's too much, but it's like, you know, you're, you're not never gonna forget.

Yeah.

That period.

No, it does sound very fun. Have you cooked for them yet?

Um, have I? Um, maybe not enough.

Okay.

Maybe I should.

Maybe this week.

Maybe this weekend I'll cook some.

Okay.

Um, they're cooking other things. They're cooking stuff on the science, but yeah.

That's true.

Um.

They're cooking on the research.

Yeah. On the research and there.

Yeah. That's great. Okay. The rice should be done.

Should be done.

Do we wanna give it a try? Let's see. But we'll see. First bite.

It's good. It's nice.

Oh, yeah. It is. Yeah. The bottom's definitely more cooked. Um, you can also probably let that sit.

Yeah. Now, these guys.

Yeah. Would you say this is ready?

These guys are, ooh, probably ready. Um, we can perhaps open it up a little bit.

Yeah.

Let it, uh, concentrate a bit. We can let those guys concentrate for a couple minutes here. And then, um, and then.

Yeah.

And then maybe we can cut those things with our handsright now.

Yeah. Do you have any call to actions? Are you guys looking for specific folks, hiring for types of researchers?

Yeah. Uh, I would say like, um, what we're trying to build.

Mm-hmm.

Which is those systems that continually learn. Obviously, there are many open problems on the research side, how you do this without destroying the model, how you do it.

Yeah.

In a cost-efficient way. Um, what data do you learn from, um, and stuff like that. But it's also an extremely ambitious, um, infrastructure problem.

Mm-hmm.

If you truly believe in the possibility that there's gonna be trillions of tokens of, of, uh.

Data within the company.

Enterprise data.

Yeah.

Or even personal data.

Yeah.

And if you truly believe that.

Yeah.

We can get to the level where we have those kinds of parameter-efficient adapters for.

Yeah.

Every person and team, you suddenly think about deployments that involve millions of different endpoints stored in different places.

Mm-hmm.

That need to be efficiently read from disk to HBM.

Yeah.

Uh, and then use that inference time.

Mm-hmm.

Um, and swapped and updated. It's gonna be if things work out for us, this thing ha will have a massive compute footprint and many new questions on, on systems and, and balancing of AI workloads in new ways. So.

Mm-hmm.

The kind of people that I think, um, could enjoy them and help us a lot are those, uh, one, uh, LLM kind of, um, performance engineers, research engineers who know how to make things go burr.

Yeah.

We have some of them.

Make things go burr.

Um, and we have, uh, Cade Daniel, who, uh, was one of the inference, uh, leads at Databricks.

Yeah.

And one of the core contributors of the LLM.

Mm-hmm.

And we are all kind of like, uh, systems inclined, but we think infrastructures engineers, uh, people who know how to work, uh, and, um, and build those large APIs and databases, I think would have a very fun time working on questions that they can't find in other places.

Mm-hmm.

Um, yeah. And, and generally, like, we always are, are open to, to smart and creative people who think out of the box and, and are committed to, to, to working on interesting problems.

Yeah.

Um, yeah.

No, that's, that's great. It seems like a very creative mix of researchers and people who also care about the infrastructure, as you said.

Yeah. Yeah.

Um.

And I also wanted to say something, I guess.

Yeah.

I was just thinking about it while I was talking before that a lot of the use cases for this kind of training and continual learning.

Yeah.

Uh, involve.

Mm-hmm.

Um, let's give it a teeny bit more.

Yeah.

A bit less fluids there.

Okay.

Uh, the quizzing.

Yeah.

And so.

Mm-hmm.

The, the point for me is the principle is any kind of like efficiency and intelligence, they cannot really be de-decoupled.

Okay.

Sometimes people think if you're building something that's more efficient that can save you dollars, therefore you're not in the premium category. You're in the, uh, you know, you're, you're making the, the cheaper product. And that and intelligence, this is just, um, you know, purely wrong,right?

Mm-hmm.

So the more you can do with less, the more ambitious tasks you can solve longer term. Um.

Gotcha.

So I think that, um, the current paradigm of scaling with AI has been doing more with more.

Mm-hmm.

Uh, and it took us extremely far, and it will keep being a valuable way to, to build intelligence.

Yeah.

But I think the next paradigm and many leaders of the labs are seeing it, uh, involves a certain element of doing more with less.

More with less.

Um, to take on longer horizon tasks and harder problems generally. So I think going beyond enterprises and going beyond efficiency, that's where I hope to go.

Yeah.

Uh, if we solve and, and, and the challenges that we're facingright now.

Mm-hmm. Okay. That makes sense. I think thinking of both coupled does seem very important, especially as you want to, like you said, do more ambitious tasks.

Yeah.

Okay. Should we mix in the spinach or just let it steam on top?

Yeah. You can try mix it up.

Okay.

And just try it. It will steam.

Let's soften a little bit. And I could cut this lemon.

Yeah. Not too big.

And then just squeeze the lemon in.

Yeah.

Okay.

The, the sauce,right?

Yeah.

Okay. Do you wanna squeeze this one? This one has a little less, but this one's up.

Amazing. And where can people find you?

Where can people find us?

Yeah. Are you guys.

They, they can find us at engram.com.

Engram.com.

Um, they can find us NSF. Um.

You guys are pretty fun.

They can write to me, uh, at dan@engram.com, um, to talk about things. Um, and yeah, I would say like as we're scaling up different parts of the company that involve engineering, that involve product and business, there's a lot of things to talk about beyond the, the frontier AI type research.

Uh, and there's a lot for us to learn from smart people.

Great. Awesome. That's exciting. Should we try it?

Yeah.

I, I challenge every, every leader of a NeoLab to come to my house in Noy Valley and cook things with me. And, uh, and I'm sure we can learn a lot from each other.

Cheers.

Good?

Mm-hmm.

I think it turned out well now.

Yeah. Very good. Wow. That's very good.

Uh, maybe a little bit Persian.

Mm-hmm.

I would say. Um, with the yellow rice. So great.

I'm a big fan. Okay.

Sean?

Rice is amazing. Rice and spinach is amazing on this one. Actually, I'm just gonna finish the whole thing after.

Yeah.

Mm-hmm. Or meatballs are a bit softer than that.

Yeah.

Mm-hmm.

It's way too heavy.

But it's so tasty.

The entire show, I pretended to be the cooking expert here.

Okay.

It has to be pure, pure crazy. Just kidding.

This is like a 8 out of 10, 9 out of 10. Mm-hmm. Great. But yeah, no, I think that's basically it. It turned out pretty well. I mean, how was it? Was it fun? Did you enjoy it?

It was the funnest, uh, podcast.

Okay.

I ever had.

Yeah.

Makes you feel at home.

That's true.

Um, easier to talk about things.

Thank you again for coming.

Yeah.

And hopefully it was a fun time.

It was super fun.

My weights have been updated.

That's good.

That's good.