Retrieval After RAG: Hybrid Search, Agents, and Database Design — Simon Eskildsen of Turbopuffer

I don't think I've said this publicly before, but I just called Loki and was like, "Look, Loki, like, if this doesn't have PMF by the end of the year, like, we'll just, like, return all the money to you.

Justine and I don't wanna work on this unless it's really working. So we wanna give it the best shot this year, and, like, we're really gonna go for it. We're gonna hire a bunch of people, and we're just gonna be honest with everyone."

Like, when I don't know how to play a game, I just play with open cards. Loki was the only person that didn't, that didn't freak out. He was like, "I've never heard anyone say that before."

Hey, everyone. Welcome to the Lit In Space Podcast. This is Alessio, founder of Kernel Labs, and I'm joined by Swyx, editor of Lit In Space.

Hello, hello. Uh, we're so, uh, we're recording in the Kernel studio for the first time. Very excited. And today we're joined by Simon Eskildsen, uh, of TurboPuffer. Welcome.

Thank you so much for having me.

TurboPuffer has, like, really gone on a huge tear, and I w- I, I do have to mention that, like, you're one of the-- you're now my newest member of the Danish Aarhus Mafia, where, like, there's a lot of legendary programmers that have come out of it like, uh, Bjarne Stroustrup, Rasmus Lerdorf, Anders Hejlsberg, and the V8 team a- and Google Maps team.

Uh, you're mostly, like, Canadian now, but isn't that interesting there's so many, so much, like, strong Danish presence?

Yeah, I was writing a post, um, not that long ago about sort of the influences. So I grew up in Denmark, right? I left, I left when I was 18 to go to Canada to, to work at Shopify.

Um, and so I, like, I've, I would still say that I feel more Danish than, than Canadian. This is also the weird accent. I can't say TH because it-- uh, this is like the-- I don't... You know, my wife is also Canadian.

Um, and I think, I think, like, one of the things in, in Denmark is just, like, there's just such a ruthless pragmatism, and there's also a big focus on just aesthetics. Like, they're, like, very-- people really care about, like, where, what things look like.

Um, and, like, Canada has a lot of attributes. US has, has a lot of attributes, but I think there's been lots of the great things to carry. I don't know what's in the water in Aarhus, though, um, and I don't know that I could be considered part of the Aarhus Mafia, Mafia quite yet- ...uh, compared to the-

You can't, you can't

...phenomenal individuals we just mentioned. But Rasmus Lerdorf is also a Danish Canadian.

Ah, okay.

Yeah, I don't know where he lives now, but... And he's the PHP.

Yeah. And, and obviously Tobi, German, but moved to C-Canada as well.

Yeah.

Like, there's this, like, import that, uh, that, that is this interesting, um, talent move.

I think I would love to get from you the definition of TurboPuffer because I think you could be a vector DB, which is maybe a bad word now in some circles. You could be a search engine. It's like, let, let's just start there, and then we'll maybe run through the history of how you got to this point.

For sure. Yeah. So TurboPuffer is, at this point in time, a search engine, right? We do full-text search, and we do vector search, and that's really what we're specialized in. If you're trying to do much more than that, like, then this might not be the right place yet, but TurboPuffer is all about search.

The other way that I think about it is that we can take all of the world's knowledge, all of the exabytes and exabytes of data that there is, and we can use those tokens to train a model. But we can compress all of that into a few terabytes of weights, right?

We can compress into a few terabytes of weights how to reason with the world, how to make sense of the knowledge, but we have to somehow connect it to something external that actually holds that, like, in full fidelity and truth.

Um, and that's the thing that we intend to become, right? That's like a very holier-than-thou kind of phrasing, right? But being the search engine for unstructured, unstructured data is the focus of TurboPuffer at this point in time.

And let's break down. So people may say, "Well, didn't Elasticsearch already do this?" And then some other people may say, "Is this search on my data? Is this, like, closer to RAG than to, like, a XR, like a public search thing?"

Like, how do, how do you segment, like, the different types of search?

The way that I generally think about this is, like, there's a lot of database companies, and I think if you wanna build a really big database company, sort of you need a couple of ingredients to be in the air.

We don't-- which only happens roughly every 15 years. You need a new workload. You basically need the ambition that every single company on Earth is gonna have data in your database multiple times. You look at a company like Oracle, right?

You will-- Like, I don't think you can find a company on Earth with a digital presence that it not, doesn't somehow have some data in an Oracle database, right? And I think at this point, that's also true for Snowflake and Databricks, right?

15 years later it's... Or even more than that, there's not a company on Earth that doesn't indirectly or directly is consuming Snowflake or, or Databricks or any of the big analytics databases. Um, and I think we're in that kind of moment now, right?

I don't think you're gonna find a company over the next few years that doesn't directly or indirectly, um, have all their data available for, for search and connected to AI. So you need that new workload. Like, you need something to be happening where there's a new workload that causes that to happen, and that new workload is connecting very large amounts of data to AI.

The second thing you need, the second condition to build a big database company, is that you need some new underlying change in the storage architecture that is not possible from the databases that have come before you. If you look at Snowflake and Databricks, right?

Commoditize, like, massive fleet of HDDs. Like, that was not possible in-- It just wasn't in the air in the '90s, right? So you just didn't, we just didn't build these systems. S3 and, and, and so on was not around.

And I think the architecture that is now possible that wasn't possible 15 years ago is to go all in on NVMe SSDs. It requires a particular type of architecture for the database that is difficult to retrofit onto the databases that are already there, including the ones you just mentioned.

The second thing is to go all in on object storage, more so than we could have done 15 years ago. Like, we don't have a consensus layer. We don't really have anything. In fact, you could turn off all the servers that TurboPuffer has, and we would not lose any data because we are all completely all in on object storage, and this means that our architecture is just so simple.

So that's the second condition, right? First being a new workload. That means that every company on Earth, either indirectly or directly, is using your database. Second being there's some new storage architecture. That means that the co- the companies that have come before you can't do what you're doing.

I think the third thing you need to do to build a big database company is that over time you have to implement more or less every query plan on the data. What that means is that you, you can't just get stuck in, like, this is the one thing that a database does.

It has to be ever-evolving. Because when someone has data in the database, they over time expect to be able to ask it more or less every question. So you have to do that to get the storage architecture to the limit of what, what it's capable of.

Those are the three conditions.

I just wanted to get a little bit of, like, the motivation right. Like, so you left Shopify. You were, like, principal engineer, infra guy. Um, you also head of Kernel Labs, uh, inside of Shopify, right? And then you consulted for Readwise, and that it kind of gave you that, that idea.

I just wanna You had to tell that story, um, maybe I- you've told it before, but, uh, just introduce the, the people to, like, the, the new workload, the sort of aha moment for TurboPuffer.

For sure. So yeah, I spent almost a decade at Shopify. I was on the infrastructure team, um, from the fairly, fairly early days around 2013. Um, at the time, it felt like it was growing so quickly and everything, all the metrics were, you know, doubling year on year.

Compared to the, what companies are contending with today, it's very cute growth. I feel like a lot, some companies are seeing that month over month. Um, of course, Shopify compou- has been compounding for a very long time now.

But I spent a decade doing that, and the majority of that was just make sure the site is up today and make sure it's up a year from now. And a lot of that was really just the, um, you know, uh, the Kardashians would drive very, very large amounts of, of data to, to, uh, to Shopify as they were rotating through all the merch and building out their businesses.

And we just needed to make sure we could handle that, right? And sometimes these were events with a million requests per second. And so, you know, we, we had our own data centers back in the day, and we were moving to the cloud, and there was so much sharding work and all of that that we were doing.

So I spent a decade just scaling databases, 'cause that's fundamentally what's the most difficult thing to scale about these sites. The database that was the most difficult for me to scale during that time, and that was the most aggravating to be on call for, was Elasticsearch.

It was very, very difficult to deal with, and I saw a lot of projects that were just being held back in their ambition by using it. And I mean-

Self-hosted.

Self-hosted.

Because it's-

Yeah

... obviously commercial also.

And this was, this was like 2015, right? So it's like a very particular vintage, right? It's probably better at a lot of these things now. Um, it was difficult to contend with, and I'm just like, I just think about it, it's an inverted index.

It should be good at these kinds of queries and do all of this, and it was... We, we often couldn't get it to do exactly what we needed to do or basically get Lucene to do, like expose Lucene raw to, to, to what we needed to do.

Um, so that was, like, just something that we did on the side and just panic scaled when we needed to, but not a particular focus of mine. So I left, and when I left, I, um, wasn't sure exactly what I wanted to do.

I mean, I'd spent, like, a decade inside of the same company. I'd, like, grown up there. I started working there when I was 18. Um-

You only do Rails.

Uh, yeah. I mean, yeah, Rails and-

He's a Rails guy.

Uh, love Rails. So good. Um-

We all wish we could still work in Rails.

I know. I know. I know. But some-

I tried learning Ruby. It's just too much, like, too many options to do the same thing. It's, that's my-

I-

I know there's a, there's a way to do it

... I love it. I don't know that I would use it now, like given Cloud Code and, and, and Cursor and everything, but, um, um, but still, it, like, if I'm just sitting down and writing a TSL code, that's how I think.

But anyway, I left, and I wasn't... I talked to a couple companies, and I was like, "I don't... I, I need to see a little bit more of the world here to know what I'm gonna, like, focus on next."

Um, and so what I decided is, like, I was gonna, I called it, like, angel engineering, where I just hopped around in my friends' companies in three months increments and just helped them out with something, right? And, and just vested a bit of equity and solved some interesting infrastructure problem.

So I worked with a bunch of companies at the time. Um, Readwise was one of them. Replicate was one of them. Um, Causal, I don't know if you've tried this. It's like a, it's a spreadsheet engine, yeah, where you can do distribution.

They sold recently, uh, um. We've been u- we used that in FP&A at, um, at TurboPuffer. Um, so bunch of companies like this, and it was super fun. And so when the ChatGPT moment happened, I was with, with Readwise for a stint.

We were preparing for the reader launch, right? Which is where you, you queue articles and read them later. And I was just getting their Postgres up to snuff, like, which basically boils down to tuning auto vacuum. So I was doing that, and then this happened, and we were like, "Oh, maybe we should build a little recommendation engine and some features to try to hook in the LLMs."

They were not that good yet, but it was clear there was something there. And so I built a small recommendation engine, just, okay, let's take the articles that you've recently read, right? Like embed all the articles and then do recommendations.

It was good enough that when I ran it on one of the co-founders of Readwise, like, I found out that I got articles about, about having a child. I'm like, "Oh my God, I didn't, I, I didn't know that, that they were having a child."

I wasn't sure what to do with that information, but the recommendation engine was good enough that it was suggesting articles, um, about that. And so there was, there was recommendations, and, uh, it actually worked really well. But this was a company that was spending maybe five grand a month in total on all of their infrastructure.

And when I did the napkin math on running the embeddings of all the articles, putting them into a vector index, putting it in prod, it's gonna be like 30 grand a month. That just wasn't tenable, right? Like, Readwise is a proudly bootstrapped company, and paying 30 grand for infrastructure for one feature versus fi- like, it just wasn't tenable.

So sort of in the bucket of this is useful, it's pretty good, but let us, let's return to it when the cost comes down.

Did you say it grows by feature? So g- for f- five to 30 is by the number of... Like, what's the, what's the scaling factor? Scale, it, it scales by the number of articles that you embed.

It does, but what I meant by that is, like, five grand for, like, all of the other, like, the Heroku Dynos, Postgres, like all the other, and this-

Then the storage is 30.

Yeah, and then, like, 30 grand for one feature, right? Which is like what other articles are related to this one. Um, so it was just too much, right? To, to power everything. Their budget would've been maybe a few thousand dollars, which still would've been a lot.

And so we put it in the bucket of, okay, we're gonna do that later. We'll wait, we'll wait for the cost to come down, and that haunted me. I couldn't stop thinking about it. I was like, okay, there's clearly some latent demand here.

If the cost had been a tenth, we would've shipped it. And this was really the only data point that I had, right? I didn't, I, I didn't, I didn't go out and talk to anyone else. It was just...

So I started reading, right? I couldn't, I couldn't help myself. Like, I didn't know what, like, a vector index is. I, I generally barely do about how to generate the vectors. There was a lot of hype about, this is, uh, early 2023.

There was a lot of hype about vector databases. They're raising a lot of money, and so I really didn't know anything about it. It's like, you know, trying these little models, fine-tuning them. Like I was just trying to get sort of a lay of the land.

So I just sat down. I have this, uh, GitHub repository called Napkin Math, and on Napkin Math, there's just, um, rows of like, oh, this is how much bandwidth, like this is how many... You know, you can do 25 gigabytes per second on average to DRAM.

You can do, you know- Five gigabytes per second of writes to an SSD, blah, blah, all of these numbers, right? And S3, how many you can do per... How much bandwidth can you drive per connection? I was just sitting down, I was like, "Why hasn't anyone built a database where you just put everything on object storage, and then you puff it into NVMe when you use the data, and you puff it into DRAM if you're, if you're querying it a lot?"

It's just like, this seems fairly obvious, and you... The only real downside to that is that if you go all in on object storage, every write will take a couple hundred milliseconds of latency, but from there it's really all upside, right?

You do the first query, it takes half a second, and it sort of occurred to me as like, well, the architecture is really good for that. It's really good for object storage, it's really good for NVMe SSDs. Well, you just couldn't have done that 10 years ago, back to what we were talking about before.

You really have to build a database where you have as few round trips as possible, right? This is how CPUs work today. It's how NVMe SSDs work. It's how S- um, S3 works, that you wanna have a very large amount of outstanding requests, right?

Like, basically go to S3, do, like, a thousand requests to ask for data in one round trip. Wait for that, get that, like, make a new decision, do it again, and try to do that maybe a maximum of three times.

But no databases were designed that way. With NVMe SSDs, you can drive, like, within, you know, within a very low multiple of DRAM bandwidth if you use it that way. And same with S3, right? You can fully max out the network card, which generally is not maxed out, and you can get very, like, very, very good bandwidth.

And but no one had built a database like that. So I was like, okay, well, can't you just, you know, take all the vectors, right, and plot them in the proverbial coordinate system, get the clusters, put a file on S3 called clusters.json and then put another file for every cluster, you know, cluster1.json, cluster2.json.

You know, that, like, it's two round trips, right? So you get the clusters, you find the closest clusters, and then you download the cluster files, like the, the closest end, and you can do this in two round trips.

You ran nearest neighbors locally.

Yes. Yes. And then, and you would build this, this file, right? Just, like, ultra simplistic, but it's not a far shot from what the first version of Turbopuffer was. Why hasn't anyone done that?

In that moment, from a workload perspective, you're thinking this is gonna be, like, a read-heavy thing because you're doing recommended... Like-

Yes

... is the fact that, like, writes are so expensive now, or with AI you're actually not writing that much.

At that point I hadn't really thought too much about... Well, no, actually, it was always clear to me that there was gonna be a lot of writes because at Shopify the search clusters were doing, you know, I don't know, tens or hundreds of cr- QPS, right?

'Cause you'd have to have a human sit and type in. But we did, you know, I don't know how many updates there were per second. I'm sure it was in the millions, right, into the cluster. So I always knew there was, like, a 10 to 100 ratio on the read/write.

In the Readwise use case, it's, um, even, even in the Readwise use case there'd probably be a lot fewer reads than writes, right? There was just a lot of churn on the amount of stuff that was going through versus the amount of queries.

Um, I wasn't thinking too much about that. I was mostly just thinking about what's the fundamentally cheapest way to build a database in the cloud today using the primitives that you have available. And this is it, right? You just...

Now you have one machine and, you know, let's say you have a terabyte of data in S3. You pay the $200 a month for that, and then maybe 5 to 10% of that data needs to be in NVMe SSDs and less than that in DRAM.

Well, you just, you're paying very, very little to inflate the data.

By the way, when you say no one else has done it, uh, would you consider Neon, uh, to be, uh, on a similar path in terms of being sort of S3 first and, uh, separating the compute and storage?

Yeah. I think what I meant with that is, uh, just build a completely new database. I don't know if we were the first. Like, it was very mu- it was... I mean, I, I hadn't... I just looked at the napkin math and was like, "This seems really obvious," so I'm sure, like, 100 people came up with it at the same time, like the light bulb in every invention ever, right?

It was just in the air. I think Neon, Neon w- was, was first to it and they're trying... They retrofitted it onto Postgres, right? And then they built this whole architecture where you have, you have it in memory and then you sort of, like, you know, mmap back to S3, and I think that was very novel at the time to do it for, for OLTP.

But I hadn't seen a database that was truly all in, right? Not retrofitting it. The database built, built purely for this. No consensus layer. Even using compare-and-swap on object storage to do consensus. I hadn't seen anyone go that all in, and I, I mean, there, there...

I'm sure there was someone that did that before us. I don't know.

I-

I was just looking at the napkin math.

A- and when you say consensus layer, uh, are you strongly relying on S3's strong consistency? You are.

Yes.

Okay. So that is your consensus layer.

It, it is the consistency layer. And I think also, like, this is something that most people don't realize, but S3 only became consistent in December of 2020.

I remember this coming out during COVID, and, like, people were like, "Oh," like, i- it was like, uh... It was just, like, a free upgrade.

Yeah.

They, they were just... They just announced it. "We saw consistency, guys," and like, "Okay, cool."

And I'm sure that they just... They probably had it in prod for a while, and they're just like, "It's done," right? And people are like, "Okay, cool." But that's a big moment, right? Like, NVMe SSDs were also not in the cloud until around 2017, right?

So you just sort of had, like, 2017 NVMe SSDs, and people were like, "Okay, cool. There's, like, one SKU that does this," whatever, right? Takes a few years. And then the second thing is, like, S3 becomes consistent in 2020.

So now it means you don't have to have this, like, big foundation DB or, like, ZooKeeper or whatever sitting there contending with the keys, which is how, you know, that's what Snowflake and others that have to do to-

So much of it are gone.

Exactly. Just gone, right? And so just pushed to the, you know, whatever, how many hundreds of people they have working on S3. Solved. And then compare-and-swap was not in S3 at this point in time.

By the way, uh, I don't know what that is, so maybe you wanna explain that.

Yes.

Yeah.

Yes. So, um, what compare-and-swap is, is basically you can imagine that if you have a database, it might be really nice to have a file called metadata.json, and metadata.json could say things like, "Hey, these keys are here, and this file means that," and there's lots of metadata that you have to operate in the database, right?

But it... That's the simplest way to do it. So now you have mi- you might have a lot of servers that wanna change the metadata. They might have written a file and want the metadata to contain that file, but you have 100 nodes that are trying to contend with this metadata.json.

Well, what compare-and-swap allows you to do is basically just you download the file, you make the modifications, and then you write it only if it hasn't changed- While you did the modification. And if not, you retry, right? You just have these retry loops.

Now, you can imagine if you have 100 nodes doing that, it's gonna be really slow, but it will converge over time. That primitive was not available in S3. It wasn't available in S3 until late 2024, but it was available in GCP.

The real story of this is certainly not that I sat down and, like, big brained it as, like, "Okay, we're gonna start on GCS. A- S3 is gonna get it later." Like, it was really not that. We started-- We got really lucky.

Like, we started on GCP, and we started on GCP because Tur- um, Shopify ran on GCP. And so that was the pr- platform I was most available with, right? Um, and I knew the Canadian team there 'cause I'd worked at, with them at Shopify, and so it was natural for us to start there.

And so when we started building the database, we're like, "Oh yeah, we have to build a con..." We really thought we had to build a consensus layer, like have a ZooKeeper or something to do this. But then we discover the compare-and-swap as like, "Oh, we could kick the can."

Like we'll just do metadata on JSON and just it's fine. It's probably fine. Um, and we just kept kicking the can until we had very, very strong conviction in the idea. Um, and then we kind of just hinged the company on the fact that S3 probably was gonna get this.

It started getting really painful in, like, mid-2024 'cause we were closing deals with, um, um, Notion actually, that was running AWS, and we're like, "Trust us, you, you really want us to run this in GCP." And they're like, "No, I don't know about that."

Like, "We're running everything in AWS." And the latency across the cloud were so big, and we had so much conviction that we bought like, you know, dark fiber between the AWS regions in, in Oregon, like, in the interexchange.

And GCP is like, "We've never seen a startup, like, do-- like, what's going on here?" And we're just like, "No, we don't wanna do this." We were tuning, like, TCP windows, like, everything to get the latency down 'cause we had so high conviction in not doing, like, a, a metadata layer on S3.

So those were the three conditions, right? Compare-and-swap to do metadata, which wasn't in S3 until late 2024. S3 being consistent, which didn't happen until December 2020-- uh, 2020, and then NVMe SSDs, which didn't land in the cloud until 2017.

I mean, in some ways, like, a very big, like, cloud success story that, like, you were able to, like, uh, put this all together, but also doing things like doing, uh, buying dark fiber, that, that actually is, uh, something I've never heard.

I mean, it's very common when you're a big company, right? You, like, connecting your own, like, data center or whatever, but it's like it was uniquely just a pain with Notion because the, um, the or- like, most of the-- Like, if you're buying in Ashburn, Virginia, right, like US East, the Google, like the GCP and, and AWS data centers are, like, within a millisecond on, on each other on the public exchanges.

But in Oregon, uniquely, the GCP data center sits, like, a couple hundred kilometers, like, east of Portland, and the AWS region sits in Portland, but the network exchange they go through is through Seattle. So it's like a full, like, 14 milliseconds or something like that.

And so a- anyway, yeah, it's, it's-- So we were like, "Okay, we can't. We have to go through an exchange in Portland." Yeah, anyway.

And you'd rather do this than, like, run your ZooKeeper and, like-

Yes. Way rather. It doesn't have state. I don't want state in two systems. Um, and I think I- all that is just informed by Justine, my co-founder, and I had just been on call for so long, and the worst outages are the ones where you have state in multiple places that's not syncing up.

So it really came from, from a, a, like, just a, a very pure source of pain of just imagining what we would be okay being woken up at 3:00 AM about, and having something in ZooKeeper was not one of them.

When you're, you're talking to, like, a Notion or something, do they care? Or do they just, they, they just-

They just cared about latency

... they, latency cost. That's it.

They just cared about latency, right? And we just absorbed the cost. We're just like, "We have high conviction in this. At some point, we can move them to AWS," right? And so we just, "We'll, we'll buy the fiber.

It doesn't matter," right? Um, and it's like five thousand do- And we-- Usually, when you buy fiber, you buy, like, multiple lines, and we're like, "We can only afford one." But we will just test it that when it goes over the public internet, it's, like, super smooth.

And so we did a lot. It's, anyway, it's... Yeah. It was-

That's cool. I can imagine talking to the GCP rep, and it's like, "No, we're gonna buy because we know we're gonna turn. We're gonna turn from you guys and go to AWS in, like, six months."

I mean-

"But in the meantime, we'll do this." It's, uh-

I mean, like, they, you know, this workload still runs on GCP for what it's worth, right? 'Cause it's so-- It was just, it was so reliable. So it was never about moving off GCP. It was just about, honestly, it was just about giving Notion the latency that they deserved, right?

Um, and we didn't want them to have to care about any of this. We also-- They were like, "Oh, egress is gonna be bad." And I was like, "Okay, screw it. Like, we're just gonna, like, VC- VPC peer with you in AWS.

We'll eat the cost. Yeah, whatever needs to be done."

And what were the actual workloads? Because I think when you think about AI, it's like 14 milliseconds is, like, really, doesn't really matter in the scheme of, like, a model generation.

Yeah. We were told the latency, right, that we had to beat.

Oh, right.

So, so we're just looking at the traces, right? And then sort of like hand dra- like, you know, kind of like looking at the trace and then thinking, "What are the other extensions of the trace," right? And there's a lot more to it because it's also, when you have, if you have 14 versus seven milliseconds, right, you can fit another round trip.

So we had to tune TCP to try to send as much data in every round trip, pre-warm all the connections and there was, there's a lot of things that compound from having these kinds of round trips. But in the grand scheme, it was just like, well, we have to beat the latency of whatever we're up against.

Which is, like, they-- I mean, Notion is a database company. They could have done this themselves. They, they do lots of database engineering themselves. How do you even get in the door? Like, yeah, just, like, talk through that kind of-

Last time I was in San Francisco, I was talking to one of the engineers actually who, who was one of our champions, um, at, at Notion, and they were, they were just trying to make sure that the, you know, per user cost matched the economics that they needed, you know.

Uh-huh.

Like, it's like the way I think about it is like I have to earn a return on whatever the clouds charge me, and then my customers have to earn a return on that, and it's, like, very simple, right?

And so there has to be gross margin all the way up, and that's how you build the product. And so then our customers have to make the right set of trade-offs that Turbopuffer makes, and if they're happy with that, that's great.

Do you feel like you're competing with build internally versus buy? Or buy versus buy

Yeah. So, sorry, this was all to build up to your question. So one of the Notion engineers told me that they'd sat and probably on a napkin, like drawn out, like, "Why hasn't anyone built this?" And then they saw Turbopuffer and was like, "Well, it- th- literally that."

So-- And I think AI has also changed the buy versus build equation in terms of it's not really about can we build it? It's about do we have time to build it? And I think they, like, I think they felt like, "Okay, if this is a team that can do that," and they, they feel enough of like an extension of our team, well, then we can go a lot faster, which would be very, very good for them.

And I mean, they put us through the t- through the test, right? Like we've had some very, very long nights to, to, to do that POC, and they were really our biggest-- our second big customer after Cursor, um, which also was a lot of late nights, right?

Yeah, that-- I mean, should we go into that story? The, the, the sort of Cursor story? Like a lot, um, they credit you a lot for, uh, working very closely with them. Uh, so I just wanna hear. I, I've heard this, uh, story from Swaleh's point of view, but like I, I'm curious what it, what it looks like from your side.

I actually haven't heard it from Swaleh's point of view. So maybe you can now cross-reference it. The way that I remember it was that, um, the day after we launched, which was just, you know, I'd worked the whole summer on, on the first version.

Justine wasn't part of it yet 'cause I just-- I didn't tell anyone that summer that I was working on this. I was just locked in on building it because it's very easy otherwise to confuse talking about something to actually doing it, and so I was just like, "I'm not gonna do that.

I'm just gonna do the thing." I launched it, and at this point, Turbopuffer is like a Rust binary running on a single eight-core machine in a Tmux instance. And me deploying it was like looking at the request log and then like Command+C-ing it or like Control+C-ing it to just like, "Okay, there's no request.

Let's upgrade the binary." Like it was like literally the, the, the, the scrappiest thing you could imagine. It was on purpose because just like at Shopify, we did that all the time. Like we'd like move, like we ran things in Tmux all the time to begin with before something had like at least the inkling of PMF.

It was just like, "Okay, is anyone gonna care about this?" Um, and one of the Cursor co-founders, Arvid, reached out, and he just, you know, the, the Cursor team are like all IOI, IMO like, um, contenders, right? So they just speak in bullet points and, and facts.

There was like this a-amazing email exchange just of, "This is how many QPS we have. This is what we're paying. This is where we're going," blah, blah, blah, and so we're just conversing in bullet points. And I tried to get a call with them a few times, but they were so-- They were like really riding the PMF bull here.

This is like late 2023. And one time, Swaleh emails me at like 5:00, no, what was it? 4:00 AM Pacific Time saying like, "Hey, are you open for a call now?" And I'm on the East Coast, and I-- it was like 7:00 AM.

I was like, "Yeah, great. Sure. Whatever." Um, and we just started talking, and something then, I didn't know anything about sales. It was something that just compelled me, I have to go see this team, like there's something here.

So I, I went to San Francisco, and I went to their office, and the way that I remember it is that Postgres was down when I showed up at the office. Did Swaleh tell you this?

No.

No. Okay. So Postgres was down, and so it's like they were distracted with that, and I was trying my best to see if I could, if I could help in any way. Like I knew a little bit about databases.

Back to tuning auto vacuum. It's like- "I think you have to tune auto vacuum, Swaleh." Um, and so we, we talked about that, and then, um, that evening just talked about like what would it look like, what would it look like to work with us, and I just said, "Look, like we're all in.

Like we will just do what we will do whatever, whatever you tell us, right?" They migrated everything over the next like week or two, and we reduced their cost by ninety-five percent, which I think like kinda fixed their per user economics.

Um, and it solved a lot of other things. And we were just-- Justine, this is also when I asked Justine to come on as my co-founder. She was the best engineer, um, that I ever worked with at Shopify.

She lived two blocks away, and we were just, "Okay, we're just gonna get this done." Um, and we did. And so we helped them migrate, and we just worked like hell over the next like month or two to make sure that we were never an issue, and that was, that was the Cursor story.

Yeah.

A- and is code a different workload than normal text? I, I don't know. Is, is it just text? Is it the same thing?

Yeah. So Cursor's workload is basically they, um, they will embed the entire code base, right? So they, they will like chunk it up in whatever they would, they do. They have their own embedding model, um, which they've been public about, um, and they find that on, on, on their evals it-- There's one of their evals where it's like a twenty-five percent improvement on a very particular workload.

They have a bunch of blog posts about it. Um, I think it works best on larger code bases, but they've trained their own embedding model to do this. Um, and so you'll see it. If you use the Cursor agent, it will do searches, and they've also been public around, um, how they've-- I think they post-trained their model to be very good at semantic search as well.

Um, and that's, that's how they use it. And so it's very good at like, "Can you find me other code that's similar to this or code that does this?" And just in, in just queries. They also use Grep to supplement it.

Yeah.

Um, of course.

It's been a big topic of discussion. Like is RAG dead because Grep, you know?

And I mean, like I just-- We j- we see lots of demand from the coding companies, right?

You mean semantic search in every part. Yes.

Uh, we, we, we see demand, and so I mean, I'm-- I like case studies. I don't like, like just doing like thought pieces on this is where it's going and like trying to be all macroeconomic about AI. That's, has turned out to be a giant waste of time because no one can really predict any of this.

So I just collect case studies and, I mean, Cursor has done a great job talking about what they're doing, and I hope some of the other coding labs that use Turbopuffer will do the same. Um, but it does seem to make a difference for particular queries.

Um, I mean, we can also do text. We can also do regex. But I should also say that Cursor's like security posture into Turbopuffer is exceptional, right? They have their own embedding model, which makes it very difficult to reverse engineer.

They obfuscate the file paths. They o- like you c-- it's very difficult to learn anything about a code base by looking at it. And the other thing they do too is that for their customers, they encrypt it with their encryption keys in Turbopuffer's bucket.

Um, so it's, it's, it's really, really well-designed.

And so this is like extra stuff they did to work with you because you are not part of Cursor.

Exactly.

Like, like, and this is just best practice when working in any database, not just you guys. Okay. Yeah, it makes sense. Yeah, I think for me, like the, the, the learning is kind of like you-- like all workloads are hybrid.

Like, you know, uh, like you, you want the semantic, you want the text, you want the regex, you want SQL. I don't know. Um, but like i-it's silly to like be all in on like one particular query pattern.

I think, like I really like the way that, um, um, that Swaleh at Cursor talks about it. Which is, um, I'm gonna butcher it here. Um, and, you know, I'm a, I'm a database scalability person. I'm not a-- I, I don't know anything about training models other than, um, what the internet tells me.

And what-- The way he describes it, this is just like cache compute, right? It's like you have a point in time where you're looking at some particular context and focused on some chunk, and you say, "This is the layer of the neural net at this point in time."

That seems fundamentally really useful to do cache compute like that, and, um, w- how the value of that will change over time, I'm, I'm not sure, but there seems to be a lot of value in that.

M-maybe talk a bit about the evolution of the workload, because even, like, search, like maybe two years ago, it was like one search at the start of like an LLM query to build the context. Now you have agentic search, however you wanna call it, where like the model is both writing and changing the code, and it's searching it again later.

Yeah, what are maybe some of the new types of workloads or, like, changes you've had to make to your architecture for it?

I think you're right. When I think of RAG, I think of, hey, there's an eight thousand token, uh, context window, and you better make it count. Um, and Search was a way to do that. Now, yeah, everything is moving towards the a- just let the agent do its thing, right?

And so back to the thing before, right, the LLM is very good at reasoning with the data, and so we're just the tool call, right? And that's increasingly what we see our customers doing. Um, what we're seeing more demand from from our customers now is to do a lot of concurrency, right?

Like Notion does a ridiculous amount of queries in every round trip just because they can. And I'm also now, when I use the Cursor agent, I also see them doing more concurrency than I've ever seen before. So a bit similar to how we designed the database to drive as much concurrency in every round trip as possible, that's also what the agents are doing.

So that's new. It means there's an enormous amount of queries all at once to the dataset while it's warm in as few turns as possible.

Can I clarify one thing on that?

Yes.

Is it-- Are they batching multiple users, or one user is driving multiple q-queries?

One user driving multiple-

Right

... one agent driving the-

It's parallel searching a bunch of things.

Exactly. Yeah, yeah.

So yeah, the, uh, Cognition also did, did this for the fast context things, like eight parallel at once.

Yes.

And, and, like an interesting problem is, well, how do you make sure you have enough diversity so you're not making the s- the same request eight times?

And I think, like that's probably also where the hybrid comes in, where that's another way to diversify. It's a completely different way to, to do the search. That's a big change, right? So before it was really just like one call, and then, you know, the LLM took however many seconds to return.

But now we just see an enormous amount of queries. So the, um, we just see more queries, so we've like tried to reduce query-- We've reduced query pricing. Um, this is probably the first time actually I'm saying that, but the query pricing is being reduced like five X.

Um, and we'll probably try to reduce it even more to accommodate some of these workloads of just doing very large amounts of queries. Um, that's one thing that's changed. I think the write, the write ratio is still very high, right?

Like there's still a, an enormous amount of writes per read, but we're starting probably to see that change if people really lean into this pattern.

Can we talk a little bit about the pricing? I'm curious, uh, because traditionally, a database would charge on storage, but now you have the token generation that is so expensive, where like the actual value of like a good search query is like much higher because they're like saving inference time down the line.

How do you structure that as like what are people receptive to on the other side too?

Yeah. I-- The, the Turbo Puffer pricing in the beginning was just very simple. The pricing on disc on-- for search engines before Turbo Puffer was very serverful, right? It was like, "Here's the VM, here's the per hour cost," right?

Great. And I just sat down with like a piece of paper and said like, "If Turbo Puffer is like really good, this is probably what it would cost with a little bit of margin." And that was the first pricing of Turbo Puffer.

And I just like sat down, and I was like, "Okay, like this is like probably the storage amp or whatever on a piece of paper." And-

Vibe, vibe pricing.

It was very vibe priced, and I got it wrong.

Oh.

Um, well, I didn't get it wrong, but like Turbo Puffer wasn't at the first principle pricing, right? So when Cursor came on Turbo Puffer, it was like, like I didn't know any VCs. I didn't know, like I just was like, I don't know, I didn't know anything about raising money or anything like that.

I just saw that my GCP bill was, was hi- was a lot higher than the Cursor bill. So Justine and I were just like, "Well, we have to optimize it." Um, and I mean, to the chagrin now of, of it, it, of, of the VCs, it now means that we're profitable because we had so much pricing pressure in the beginning because it was running on my credit card.

And Justine and I had spent like, like tens of thousands of dollars on like compute bills and like spinning off the company and like very like, like bad Canadian lawyers and like things like to like get all of this done because we just like, we didn't know, right?

If you're like steeped in San Francisco, you just like, you just know, okay, like you go out, raise a pre-seed round. I, I never heard of word pre-seed at this point in time.

When you had Cursor, you had Notion, you, you had no funding?

Um, with Cursor, we had no funding. Yeah. Um, by the time we had Notion, Locky was, Locky was here. Yeah. So it was really just we vibe priced it one hundred percent from first principles, but it wasn't it, it was not performing at first principles.

So we just did everything we could to optimize it in the beginning for that, so that at least we could have like a five percent margin or something, so I wasn't freaking out because Cursor's bill was also going like this as they were growing.

And so my liability and my credit limit was like actively like calling my bank. It's like, "I need a bigger credit." Like it was-- Yeah. Anyway, that was the beginning. Yeah. But the pricing was, yeah, like storage, writes, and query, right?

And the, the pricing we have today is basically just that pricing with duct tape and spit to try to approach like, you know, like a, as a margin on the physical underlying hardware. And we're doing-- This year, you're gonna see more and more pricing changes from us.

Yeah.

And like is how much does stuff like VPC peering matter because you're working in AWS land where egress is charged and all that, you know?

We probably don't-- Like, we have, like, an enterprise plan that just has, like, a base fee because we haven't had time to figure out skew pricing for all of this. Um, but I mean, yeah, you can run Turbopuffer either in SaaS, right?

That's what Cursor does. You can run it in a single-tenant cluster so it's just you. That's what Notion does. And then you can run it in, in, in BYOC, where everything is inside the customer's VPC. That's what, for example, Anthropic does.

What I'm hearing is that this is probably the best CRO job for somebody who can come in and-

I mean-

-help you with this.

Um, like Turbopuffer hired, like, I don't know what, what number this was, but we had a full-time CFO as like the twelfth hire or something at Turbopuffer. Um, I think I hear a lot of compa-- I don't know how they do it, like they have a hundred employees and not a CFO.

It's like having a CFO is like-

You're running a business, man, like, you know.

It's so good. Yeah. Like Money Mike, like he just, you know, just handles the money and a lot of the business stuff. And so he came in and just helped with a lot of the operational side of the business.

So like COO, CFO, like somewhere in between.

Just a quick mention of Lachy, just 'cause I'm curious. I've met Lachy, and like, he's obviously a very good investor now in ph-physical intelligence. Um, I call it a generalist super angel, right? He invests in everything. Um, and I always wonder like, you know, is there something appealing about, like, focusing on developer tooling, focusing on databases, going like, "I've invested for twenty years in databases," versus being like a Lachy, where he can maybe, like, connect you to all the customers that you need?

This is an excellent question. No-no one's asked me this. Um, why Lachy? Because there was a couple people that we were talking to at the time, and when we were raising, we were almost a little-- we were like a bit distressed because one of our, one of our peers had just launched something that was very similar to Turbopuffer, and someone just gave me the advice at the time of just choose the person where you just feel like you can just pick up the phone and not prepare anything and just be completely honest.

And I don't think I've said this publicly before, but I just called Lachy and was like, "Look, Lachy, like, if this doesn't have PMF by the end of the year, like, we'll just, like, return all the money to you."

But it's just like, I don't really-- Justine and I don't wanna work on this unless it's really working. So we wanna give it the best shot this year, and, like, we're really gonna go for it. We're gonna hire a bunch of people, and we're just gonna be honest with everyone.

Like, when I don't know how to play a game, I just play with open cards. And Lachy was the only person that didn't, that didn't freak out. He was like, "I've never heard anyone say that before." As I said, I didn't even know what a seed or pre-seed round was like before, probably even at this time.

So I was just, like, very honest with him. And I asked him, like, "Lachy, have you ever, have, have you ever invested in a database company?" He was just like, "No."

And at the time, I was like, "Am I dumb?" Like, but I think there was something that just, like, really drew me to Lachy. He is so authentic, so honest, like, and there was something just like, I just felt like I could just play, like, just say everything openly.

And that was, that was, I think that, that was like a perfect match at the time and, and, and honestly, still is. He was just like, "Okay, that's great. This is like the most honest, ridiculous thing I've ever heard anyone say to me," but like that, like that-

Why is it so difficult to say, "Competitor launch, this may not work out"?

It was more just like, "If this doesn't work out, I'm gonna close up shop by the end of the mon-the year," right? Like it was-- I don't know. Maybe it's common. I, I don't know. He told me it was uncommon.

Okay.

I don't know. Um, that's why we chose him, and he'd been phenomenal. The other people we were talking at the, at the time were database experts, like they, you know, knew a lot about databases, and Lachy didn't. This turned out to be a phenomenal asset, right?

I-- like Justine and I know a lot about databases. The people that we hire know a lot about databases. What we needed was just someone who didn't know a lot about databases, didn't pretend to know a lot about databases, and just wanted to help us with candidates and customers, and he did.

Yeah.

And I have a list, right, of the investors that I have a relationship with, and Lachy has just performed excellent in the number of sub-bullets of what we can attribute back to him. Just absolutely incredible. And when people talk about, like, no ego and just the best thing for the founder, I, like, I don't think that anyone-- Like even my lawyer is like, "Yeah, Lachy is like the most friendly person you will find."

Okay, this is my-- the most glowing recommendation I've ever heard.

He deserves it. He's very special.

Yeah, yeah, yeah. Okay, amazing.

Since you mentioned candidates, maybe we can talk about team building. You know, like especially in SF, it feels like it's just easier to start a company than to join a company. Um, I'm curious your experience, especially not being in SF full time and doing something that is maybe, you know, a very low level of detail and technical detail.

Yeah, so joining versus starting. I never thought that I would be a founder. I would start with-- Like Turbopuffer started as a blog post, and then it became a project, and then sort of almost accidentally became a company, and now it feels like it's, it's like becoming a bigger company.

That was never the intention. The intentions were very pure. It's just like, why hasn't anyone done this? And it's like, I wanna be the-- like I wanna be the first person to do it. I think some founders have this like, "I could never work for anyone else."

I, I really don't feel that way. Like, it's just like I wanna see this happen, and I wanna see it happen with some people that I really enjoy working with, and I wanna have fun doing it, and this, this, this has all felt very natural on that, on that sense.

So it was never a like join versus, versus, versus found. It was just this found me at the right moment.

Well, I think there's an argument for you should join Cursor, right? So I'm curious like how you evaluate it. Okay, I should actually go raise money and make this a company versus like this is like a company that is like growing like crazy.

It's like an interesting technical problem. I should just build it within Cursor, and then they don't have to encrypt all this stuff. They don't have to obfuscate things. Like was that on your mind at all or?

Before taking the, the small check from Lachy, I did have like a hard, like look at myself in the mirror of like, "Okay, do I really wanna do this?" And because if I take the money, I really have to do it, right?

And so the way I almost think about it is like you kinda need to ha-- like you kinda need to be like fucked up enough to wanna go all the way, and that was the conversation where I was like, "Okay, this is gonna be part of my life's journey to build this company."

And do it in the best way that I possibly can. Because if I ask people to join me, ask people to get on the cap table, then I have an ultimate responsibility to give it everything. And I don't th- I think some people-- it doesn't occur to me that everyone takes it that seriously, and maybe I take it too seriously, I don't know.

But that was, like, a very intentional moment, and so then it was very clear, like, "Okay, I'm gonna do this, and I'm gonna give it everything."

A lot of people don't take it this seriously, but... Uh, let's talk about, you have this concept of the P99 engineer. Uh, people are 10X-ing, everyone's saying, you know, uh, maybe engineers are out of a job. I don't know.

But you definitely see a P99 engineer, and I just want you to talk about it.

Yeah, so the P99 engineer was just a term that we started using internally to talk about candidates and talk about how we wanted to build the company. And you know, like everyone else, it's like, we want a talent-dense company .

And I think that's almost become trite at this point. What I credit the Cursor founders a lot with is that they just arrived there from first principles of, like, we just need a talent-dense, um, talent-dense team. And I think I've seen some teams that weren't talent-dense and, like, seen the counterfactual run, which if you've run in, been in a large company, you will just see that.

Like, it's just logically will happen at a large company. Um, and so that was super important to me and Justine, and it's very difficult to maintain. And so we just needed, we needed wording for it. And so I have a document called Traits of the P99 Engineer, and it's a bullet point list, and I look at that list after every single interview that I do and in every single recap that we do.

And every recap we end with, I end with, um, some version of, "I'm gonna reject this candidate," completely irregardless of what the discourse was, because I wanna see people fight for this person. Because the default should not be, we're gonna hire this person.

The default should be, we're definitely not hiring this person. And you know, if everyone was like, "Oh, I'll maybe throw a punch," then this is not the right-

Do, do you operate, like, if there's one ch- there must have at least one champion who's like, "Yes, I will put my career on, on, on the line for this," you know?

I'd say career on the line-

Maybe he's like a chair

... but yeah, you know, like, um, I would say so- someone needs to, like, have both fists up and be like, "I'd fight," right?

Yeah, yeah.

And if one person said it, then okay, let's do it, right?

Yeah.

Um, and it doesn't have to be absolutely everyone, right? And, like, the interviews are always designed and you're checking for different attributes, and if someone is, like, knocking it out of the park in every single attribute, that's, that's fairly rare.

Um, but that's really important. And so the traits of the P99 engineer, there's lots of them. There's also the traits of the P, like, triple nine engineer and the quadruple nine engineer, right?

Please.

This is like, it's a long list.

Okay.

Um, I'll give you some samples, right, of what we, what we look for. I think that the P99 engineer has some history of having bent, like, their trajectory or something to their will, right? Some moment where it was just, they just, you know, made the computer do what it needed to do.

There's something like that, and it will, it will occur to have them at some point in their career, and, uh, hopefully multiple times, right?

Give me an example of one of your engineers that, like-

I'll give an engi- uh, so we, we, we launched this thing called ANN V3. Um, we could-- We're also, we're working on V4 and V5 right now. But ANN V3 can search 100 billion vectors with a P50 of around 40 milliseconds and a P99 of 200 milliseconds.

Um, maybe other people have done this. I'm sure Google and others have done this, but, like, we haven't seen anyone, um, at least not in, like, a publicly consumable SaaS that can do this. And that was an engineer, the chief architect of TurboPuffer, Nathan, um, who more or less just bent this-- The software was not capable of this, and he just made it capable for a very particular workload in, like, a, you know, six to eight-week period with the help of a lot of the team, right?

It's been, been... There's numerous of examples of that, like, at, at TurboPuffer, but that's, like, really bending the software and x86 to your will. It was incredible to watch. Um, you wanna see some moments like that.

Isn't that triple nine?

Um, I think Nathan-

What's quadruple nine? That was only P99. I feel like this is too high for a P99.

Nathan, Nathan is, uh, Nathan is like, yeah, there's a lot of nines-

Okay

... after that P. So I think that's one trait. I think another trait is that, uh, the P99 spends a lot of time looking at maps. Generally, it's their preferred UX. They just love looking at maps. You ever seen someone who just, like, sits on their phone and just, like, scrolls around on a map?

Or did you not look at maps a lot? You guys don't look at maps?

I guess I'm not P99, I don't know, but...

You just disqual- What about trains? Do you like trains?

Uh, I mean, they're-

Not enough.

Yeah, okay, this is just, like, weaponized autism is what I call it. Like, like, j-

Um, I love looking at maps. Like, it's, like, my preferred UX and just, like, I, you know, I, like lots of-

Like instead of, like, random places?

So, like, you know... Yes. Okay, there we go.

So instead of, like, random places, like, how do you explore the maps?

No, it's, it's just a joke.

But it's autism. I'm laughing. It's like you are just obsessed by something, and you like studying a thing.

The origin of this was that at some point, I read an interview with some IOI gold medalist.

Uh-huh.

And it's like, "What do you do in your spare time?" It's just like, "I like looking at maps." And I was like, "I feel so seen." Like, I just, like, love, like, scrolling. I was like, "Oh, Canada is so big.

Where's Baffin Island?" Like, I don't know. I just, I love it.

Yeah.

Um, anyway, so the traits of P99, P-P99 is obsessive, right? Like, there's just, like... You'll, you'll find traits of that. We do an interview at, at, at, at TurboPuffer or, like, multiple interviews that just try to screen for some of these things.

Um, so there's lots of others, but these are the kinds of traits that we look for.

I'll tell you, uh, some people listen for, like, some of my DevRel stuff. Uh, I do think about DevRel as maps. Um, you draw a map for people. Uh, maps show you the, uh, what is commonly agreed to be the geographical features of what a boundary is, and it also shows you what it's not doing.

And I, I think a lot of, like, developer tools companies try to tell you they can do everything. But, like, let's, let's be real. Like, you, your, your three landmarks are here. Everyone comes here, then here, then here, and you draw a map, and, and then you draw a journey through the map, and, like, that to me, that's what developer relations looks like.

So I do think about things that way.

I think the P99 thinks in trade-offs Right? The P99 is very clear about, you know, hey, TurboPuffer, you can't run a high transaction workload on TurboPuffer, right? It's like the write latency is 100 milliseconds. That's a clear trade-off.

I think the P99 is very good at articulating the trade-offs in every decision, um, which is exactly what the map is in your case, right?

Uh, yeah, is, yeah. My, my, my world. My world.

How, how do you reconcile some of these things when you're saying you bend the will of the computer versus, like, the trade-offs, you know? I think sometimes it's like, well, these are the trade-offs, but the three nines is, like, actually, it's not a real trade-off because we can make something that nobody has ever made before and actually make it work.

The way I think about the bending trajectory to your will is, um, if you sit down and do the napkin math, right, where you're just like, okay, like if I have 100 machines, they have this many terabytes of disks, they have this bandwidth, whatever, right, and you sit down and you just do the f- like, high school napkin math on this is how many QPS we should be able to drive to it, similar to how I did the vibe pricing, right?

If you can sit down and do that, and then you observe the real system and you see, oh, we're off by, like, 10X. Bending trajectory to your will is, like, just making the software get closer and closer to that first principle line.

The P99 might even be able to cross the line, right, by f- finding even more optimizations than, than, than from first principle. So bending the software to your will is about that, right? Like, 100 millisecond P99 to, um, to S3, I mean, now you're talking, like, someone really high agency that, like, goes to Seattle, finds the S3 team and is like, "How are we gonna make this 10?"

You know? Like, it's... That, that's not quite what we talk about, right? But yeah.

What's the future of TurboPuffer?

TurboPuffer started out, act one of TurboPuffer was vector search. That's, was all we did to begin with. Act two of TurboPuffer is, is and was full text search. TurboPuffer today has a fairly start of the- state-of-the-art full text search engine.

Um, we beat Lucene on some queries. In particular, very long queries that we've optimized for because those are the text search queries we see today. They're generated by LLMs or augmented by LLMs, um, and we see them on web scale data sets, right?

Like someone searching for a very long text string on all of Common Crawl. We beat Lucene on some of those benchmarks, and we expect to continue to beat Lucene on more and more queries. Um, that's the performance and scale.

TurboPuffer does phenomenally now at full tech search performance and scale. What we work on now is more and more features for full tech search. People expect a lot of features with full tech search, and full tech search is still very valuable, right?

If you go in and you cr- press Command K and you search for SI, an embedding-based search might be like, "Oh, this is something agreeable," 'cause that's si, that's yes in Spanish, right?

Well, it's yes in Italian, too.

But in full text search, that's the prefix of maybe a document of like, you know, these are all the reasons I hate Simon, right? Like, these, this is like, that's a completely different... So that augmentation to, like, how the human brain works and mapping, like, data to user is very important, but it's a lot of features.

That feature grind is what we're firmly on, and you will see us just adding to the change log every month, just more and more full text search features. Um, so we're, like, fully compatible, and we see- we're seeing people move from some of the traditional search engine onto TurboPuffer, um, for that.

That's a big focus of TurboPuffer this year. The other, the other focus of, of TurboPuffer this year is just on scale. We're seeing more and more companies that wanna search basically Common Crawl level types of data sets, um, both internally at companies and, and externally at a, at a time, like query like 100 billion vectors or 100 billion documents at once.

This is tricky, and we wanna make it cheaper, and we wanna make it faster. Um, that's a big focus for TurboPuffer this year. That's, you know, we just released ANNv3, which we talked about before. We're working on ANNv- v4, and we also have planned what we're gonna do with ANNv5, right?

And then on full text search, we're working on a lot of these features will be like FTSv3, but it will all roll out incrementally. Um, those are some of the really big features. And then the other thing is, um, our dashboard.

I- have any of you ever logged into the TurboPuffer dashboard? There's not very much there. It almost looks like if, um, a founder two years ago just sat down and wrote enough dashboard that there was at least something there, and then other people just sort of added stuff on for the next two, like, the, the following two years, and then at some point, SSO and other things to just catch up.

And it may or may not be ha- what happened. But adding, like, I want PHP MyAdmin back. Like, do, do you guys remember? Like, it was, it was so good, right? And I think that that, like, software hardware integration between the, the, the dashboard of the console of the database and the database itself, um, I'm, I'm really excited for that.

There's lots of other things, um, that are gonna come out in the next sort of... Like, we talked a bit about some, some pricing and, and things like that, but those would be some of the big hitters right now.

You talk about eras of, like, TurboPuffer. I'm, I just, I have to ask, like, yes, there's the stuff that you're working on this year, but, like, I'm sure in your mind you already have the next phase that you're already thinking about.

Act three?

Yes.

Act four.

Yeah.

Act five. One whole thing about that-

What are the candidates? You don't have to decide-

Yeah

... now, but, you know.

I, I'll just say that if you wanna build a big database company, the database over time has to implement more or less every query plan. Because when you have your data in a database, you expect it to, over time, not just search, but also, hey, I wanna aggregate this column, I wanna join this data, all of that.

But when you're a startup, your only moat is really just focus. So you have to lay out the acts, and you have to not get overeager, and I think we've seen some of our peers get very overeager and overextend themselves.

And what I keep telling the team, I was just having breakfast this morning with our CTO and, and chief architect, and we were talking about, like, what we're most likely to regret at the end of the year is having tried to do too much.

Um, and so act three candidates could be, you know, a bunch of simpler OLAP queries, right? It could be, um, lending ourselves a little bit more into... We see some people who wanna do traces and logging and things like that, some very simple use cases.

Could be that, right? It could be maybe some time series. Some people are trying to do that, right? Like, there's lots of different things that you can do with Turbopuffer, but for now the- like, if you're trying, trying to do not search on Turbopuffer as the primary use case, you probably shouldn't.

But we see some customers that are like, "Oh, um..." Like at some point Cursor moved like 20 terabytes of Postgres data into Turbopuffer, 'cause like it's di- it's there, it works, and these particular query plans we know work well, and so they just moved it all to defer sharding.

Um, so we look for patterns like that in what future acts of Turbopuffer are going to be before firmly doubling down on them. But we wouldn't... If, today if you're using Turbopuffer, it should be because search is very important to you, and then we might do a lot of auxiliary queries to that, but that should not be the main reason to go to Turbopuffer at this point in time.

Yeah. Uh, you didn't mention, uh, one thing I was looking for was graph type queries, like graph database, graph, uh, queries. Can you basically trivially replicate this with what you already have?

We see some people doing that.

Right? 'Cause you have parallel queries and it's, it's the same thing.

Exactly. So we see some people doing that, right? Like, at the under- like Turbopuffer is just a KV, right? And then we expose things on top of it. So we are seeing people do that. And I think, you know, our roadmap is very much just the database that connects AI to a very large amount of data is what the path is to do that in the right order, which is what a good startup is around.

What is the order to do things in? Our customers are P99, and they will tell us what they care most about next. And so some of them are doing graphs now, and if they need more graph database features, they'll be banging on our door, and we'll prioritize accordingly.

Tea? Okay, give us the tea. Uh, this, you, you, uh, you kindly gifted us your favorite tea. This is Yabukita Kamairicha, uh, from the Green Tea Shop.

That's right.

Tell me about your love of tea.

Yeah. We, we were just talking beforehand about, um, um, um, caffeine, I think. Um, and, uh, especially when I'm on a trip like this to San Francisco, I consume a lot of caffeine. Um, but this is my preferred, uh, preferred caffeine.

It's this green tea. I have an Airtable with 200 teas that I've tried over time, over the past, like, 15 years, and this one is my favorite. Now, when you drink a tea, um, there's different, there's like six different types of tea.

I like green tea. In particular, I generally prefer Chinese green tea, and I don't really like Japanese green tea. But this little prefecture somewhere in Japan has specialized in, like, they're like Japanese but doing it the Chinese way, and it's just phenomenal.

But then the interesting thing about the tea world is that a- all of the different, um... Like, you can find this particular tea. There's probably, you know, hundreds of places that sell it, but they all go to a different family, right?

On whatever mountain that they have these, like, Camellia sinensis bush, bushes on. And this woman, Japanese woman in Toronto from the Green Tea Shop, um, I don't know, she just like has found a really good family, 'cause that's the best one.

The best time of year to get this is in a few months when they do the spring harvest. Uh, now it's, like, kind of old. Um, it's just, like, I love the spring for the fresh tea. So I hope you enjoy it, but it's not the right time of year.

It's out of season.

Yeah.

I, I, I actually didn't even know tea has seasons. This is how unsophisticated I am. But I, I think it, like, it ties in with, like, you know, loving maps and, like, being obsessed and- ... being P99 in everything that you do.

Um, yeah, but that's great.

Awesome. Well, as we were saying, we have instant hot water at Kernel, so, uh, any tea lover can come by any-

I, I have a little tea kit where I bring a, uh, where I bring, like, a little thermometer, too, like a little Thermoworks thermometer. Um, last Friday when we do demos, I have this thing where if there's not enough demos, then I fill the remaining time talking about something completely ridiculous as an incentive for people to actually demo.

And last ti- time, I spent 20 minutes ta- walking through my Airtable and going through my entire tea travel kit, including the, the, the temperature monitor. 'Cause, like, yeah, you show up, there's only a boiler. You can't get it to the right-

Yeah

... you know, you need this at 80 degrees for... Anyway, yeah, sorry.

Yeah. We have a, we have a electric kettle with the temperature thing at home.

No, I would watch this. You should start a company, YouTube, but, like, it doesn't have anything about search. It just has tea and, like, other rants.

I don't think I could talk, but something that I started doing... Do, um, do you two know Sam Lambert of-

Of course

... PlanetScale?

Of course.

Um-

Very outspoken guy.

I love the guy, and we just, um, we just, last, like, last week we just went on X Live and just sat and, like, shot the shit for, like, an hour, and I think we'll probably do that again.

Yeah, so we'll probably come up there. We'll... I don't know what we'll call it. Maybe P99 Live or the P99 Pod or something like that. Um-

P Pod.

P Pod.

Uh, cool. Well, thank you so much for your time. I know you have to go, uh, but this is a, a blast, and you're clearly very passionate and charismatic, so, uh, I, I bet you'll get some, uh, P99 engineers out of this podcast.

Yeah.

Thank you so much for having me. It was a pleasure.