The State of AI in production — with David Hsu of Retool

Hey everyone, welcome to the Latent Space podcast. This is Alessio, partner and CTO in residence at Decibel Partners, and I'm joined by my co-host, Swix, founder of SmallAI.

Hi! And today we are in the studio with David Hsu from Retool. Welcome.

Thanks. Excited to be here.

We'd like to give a little bit of intro from what little we can get about you, and then have you talk about something personal. You got your degree in philosophy and CS from Oxford. I didn't aware that— I wasn't aware that they did, like, double degrees.

Is that what you got?

It's actually a single degree, actually, which is really cool. So basically, yeah. So you study content, you study philosophy, and you study intersection. Intersection is basically AI, actually, and sort of computers think, or computers be smart. Like, you know, what does it mean for a computer to be smart?

As well as logic, which is also another intersection, which is really fun too.

In Stanford, you know, it might be symbolic systems or whatever. And it's always— it's always hard to classify these things when we don't really have a word for it. Now I guess everything's just called AI.

Five years ago you launched Retool. You were in YC '17, uh, winter '17. And, you know, it's just been a straight line up from there,right?

I wished.

What's something that's, you know— and that's your sort of brief bio that, you know, I think you want most people to know. What's something on your LinkedIn that people should know about you? Maybe on the personal hobby, or, you know, it's just something you're very passionate about that might not be about Retool.

I read quite a bit. I probably read like two books a week or about, so it's a lot of fun. I love biking. It's also quite a bit of fun, so yeah.

Do you use Retool to read? Like, what the hell?

No, I don't use Retool to read. That would be funny.

What do you read? How do you choose what to read? Any recommendations? It's like, you know, that you just fall in love with?

It's pretty diverse. I'm mostly reading fiction nowadays, so fiction's a lot of fun. I think it maybe helps to be more empathetic, if you will. I think it's a lot of fun actually to sort of see what it's, you know, like to be in someone else's shoes.

So that's a lot of fun. Plus I read a good amount of philosophy as well. I find philosophy just so interesting, especially logic. We can talk more about that for probably hours if you want, so.

Yeah, I have a sort of casual interest in epistemology, and I think that anytime you try to think about machine learning on a philosophical angle, you have to start wrestling with these, like, very fundamental questions about how do you know what you know.

Yeah, totally. What does it mean to know?

What does it mean to know? Yeah, allright. So over to you, Alessio.

That's a—

That's its own podcast. We should do a special edition about it. But that's fun. Let's just maybe jump through a couple things from Retool that I found out while researching your background. So at demo day, you did YC, but you didn't present at demo day initially because you were too embarrassed of what you had built.

Can you maybe give any learnings to, like, founders on, like, jumping back from that? I've seen a lot of people kind of, like, give up early on because they were like, "Oh, this isn't really what I thought it was going to be to be a founder."

They told me I would go to YC and then present and then raise a bunch of money and then everything was going to be easy. So how did that influence also how you build Retool today, you know, in terms of, like, picking ideas and, like, deciding when to give up on it?

Yeah.

Yeah, let's see. So this was around about 2017 or so. So we were supposed to present at the March demo day, but then we basically felt like we had nothing really going on. We had no traction, we had no customers.

And so we were like, "Okay, well, why don't we, you know, take six months and go find all that before presenting?" Part of that, to be honest, was I think there's a lot of noise around demo day, around startups in general, especially because there's so many startups nowadays.

And I guess for me, I had always want— I'd always wanted to sort of under-promise and over-deliver, if you will. And on demo day, I mean, maybe you two have seen a lot of the videos, like, it's a lot of, honestly, over-promising and under-delivering.

Because every startup, you know, says, "Oh, you know, I'm going to be the next Google or something," and then you peer under it and you're like, "Wow, nothing's going on here," basically. So I really didn't want that. And so we chose actually not to present at demo day mostly because we felt like we didn't have anything substantial underneath.

Although actually a few other founders in our batch probably would have, you know, chosen to present in that situation, but we were just, you know, kind of embarrassed about it. And so we basically took six months to just say, "Okay, well, how do we get customers?"

And we're not presenting until we have, you know, a product that we're proud of and customers that we're proud of. And fortunately, it worked out. You know, six months later we did have that. So

I don't know if there's much to learn from the situation besides

I think social validation was something that I personally had never really been that interested in. And so it was definitely hard because it's hard to sort of, you know, it's almost like you go to college and all your friends are graduating when you failed or something.

You failed your final and you have to, like, redo it here. And it's like, well, it kind of sucks that all your friends are up there on the podium, you know, presenting and they are raising a ton of money and you're kind of being left behind.

But in our case, we felt like it was a choice. We could have presented if we really wanted to, but, you know, we would not have been proud of the outcome or proud of what we were presenting. And for us, it was more important to be true to ourselves, if you will, and show something that we're actually proud of rather than just, you know, raise some money and then shut the company down in two years.

Yeah. Any Sam Altman stories from the YC days? Could you tell in 2017 that Sam was going to become, like, run the biggest AI company in the world?

Wow, no one's asked me that before. Let me think. Sam was, yeah, I think he was, I want to, I think maybe president of YC in our batch. We actually were in his group, actually, at the very beginning, and then we got moved to a different group.

I honestly have, I think Sam was clearly very ambitious when we first met him. I think he was very helpful and sort of wanted to help founders. But besides that, I mean, I think we were so overwhelmed by the fact that we had to go build a startup that we were not honestly paying that much attention to every YC partner and taking notes on them.

That makes sense. Cool. And then just to wrap some of the Retool history nuggets, you raised a series A when you were at 1 million in revenue with only three or four people. How did you make that happen?

Any learnings on keeping teams small? I think there's a lot of, a lot of overhiring we've seen over the last few years. I think a lot of AI startups now are kind of, like, raising very large rounds and maybe don't know what to do with the capital, so.

Yeah. So this is kind of similar, actually, from sort of why we chose not to present at demo day. And the reason was it feels like a lot of people are really playing startup. I think PG has an essay about this, which is like, you're almost like playing house or something like that.

Like, it's like, oh, well, I hear that in a startup, you're supposed to raise money and then hire people. And so therefore you go and do that. And you're supposed to, you know, do a lot of PR because that's what, you know, startup founders do.

And so you go do a lot of PR and stuff like that. And for us, we always thought that the point of starting a startup is basically we have to create value for customers. If you're not creating value for customers, like everything else is going to, you know, nothing's going to work, basically.

You can't, you know, continuously raise money or hire people if you don't have customers who have to generate value for them. And so for us, we were always very focused on that. And so that's initially where we started.

I think it's, again, maybe goes to, like, the sort of, you know, presenting something truthful about yourself or staying true to yourself, something to that effect, which is we didn't want to pretend like we had a, you know, thriving business.

We didn't actually. And so the only way to not pretend was actually to build a thriving business. And so we basically just, you know, put our heads down and, you know, grind it away for probably a year, year and a half or so, just writing code, talking to customers.

And I think that at that point we had raised something like maybe a million dollars, maybe a million and a half coming out of YC. So, I mean, to us, two people, you know, that was a huge amount of money.

I was like, wow, like, how are you ever going to spend a million and a half? You know, our runway was like, you know, five, six years at that point,right? Because we're paying ourselves 30, 40K a year. And so then the question was not like, oh, we're going to run out of runways.

The question was like, we better find traction because if we don't find traction, we're going to, you know, just give up psychologically. Because, you know, we're grinding on it. If you grind on the idea for four years and nothing happens, you're probably psychologically going to give up.

I think that's actually true in most startups, actually. It's like most startups die in their early stages, not because you run out of money, but really because you run out of motivation. And for us, had we hired people, I think it would have actually been harder for us because we would have run out of motivation faster.

Because when you're pre-product market fit, actually trying to lead a team of, like, you know, 10 people, for example, to march towards product market fit, I think it's actually pretty hard. Like, it's, you know, every day people are asking you, "So why are we doing this?"

And you're like, "I don't know, man. Like, trust me." And that's actually a very tiring environment to be in. Whereas if it's just like, you know, the founders figuring out product market fit, I think that's actually a much sort of safer path, if you will.

You're also screwing less with employees. Like, when you hire employees, you have an idea, you have product market fit, you have customers. That's actually, I think, a lot more stable for a place for employees to join as well.

Yeah, and I find that, you know, typically the sort of founder-employee relationship is, you know, employee expects the founder to just tell them what to do. And you don't really get critical pushback from the employee. Even if they're a bot and even if they like you as an early engineer, it's very much like the role play of, like, once you have that founder hat on, you think differently, you act differently, and you're more scrappy, I guess, in trying to figure out what that product is.

Yeah, I really resonate with this because I'm going through thisright now.

That's awesome. One thing we did actually early on that I think has paid a lot of dividends, especially Retool's a lot larger now, is we hired a lot of former founders. So I want to say, like, when we were, I don't know, 20, 30, 40 people, we were probably like half former founders at each one of those stages.

And that was actually pretty cool because I think you infuse sort of a, you know, get things done kind of culture, a outcome-oriented culture of, like, very little politics because, you know, no one came from larger companies. Everyone's just like, "This is my own startup.

Let me go figure out how to achieve this outcome for the customer." And so I think from a cultural perspective, even today, a lot of Retool's culture is sort of very self-startery. I think it's actually because of sort of these, like, you know, early founders that we hired, which was really, really, you know, we're really lucky to have had them.

Yeah. And then closing off on just a little bit of the fundraising stuff, something notable that you did was when in 2021, when it was the sort of peak Zerp and everyone was raising hundreds and hundreds of millions of dollars, you know, you intentionally raised less money at lower valuations is your title.

And I think it's a testament to your just overall general philosophy in building Retool that you're just very efficient and you do things from first principles. Yeah, I mean, like, any updates on, like, would you still endorse that?

You know, would you recommend that to everyone else? What are your feelings sort of two years on from that?

Yeah, so at a high level, yeah, so exactly what you said is correct. We raised less money at a lower valuation. And I think the funny thing about this is that when we first announced that, even, you know, internally and both externally, I think people were really surprised, actually, because I think Silicon Valley has been conditioned to think, "Oh, raising a giant sum of money at a giant valuation is a really good thing."

So like, you know, you should maximize both the numbers, basically. But actually, maximizing both the numbers is actually really bad, actually, for the people that matter the most, you know, i.e., your employees or your team. And the reason for that is more raising more money means more dilution.

So if you look at, you know, a company like, you know, let's say Uber, for example, if you joined Uber at, like, I don't know, like a $10 billion valuation or, you know, let's say you joined before their huge wrap, which I think happened at a few billion dollars in valuation, you actually got diluted a ton when Uber fundraises.

So if, you know, Uber raises, if Uber dilutes themselves by 10%, for example, let's say it raised 5.2 to 5 billion, for example, every employee's stake goes down by 10% in terms of ownership. Same with, you know, previous investors, same with the founders, etc.

And so if you look at actually a lot of founders in sort of, you know, the operations logistics space or, you know, those that fundraised like, you know, 2013, 2017, a lot of the founders by IPO only have a few percentage points, actually, of the company.

And if the founders only have a few percentage points, you can imagine how little the employees have. And so that I think is actually just a really, you know, bad thing for employees overall. Secondly, it's a sort of higher valuation given the same company quality is always worse.

And so basically what that means is if you are fundraising as a company, you could command a certain valuation in the market. You know, let's say it's, you know, X, for example. Maybe you get lucky and you can raise two times X, for example.

But if you choose two times X, your company itself has not fundamentally changed. It's just that, you know, for some reason investors want to pay more for it. You know, maybe today you're an AI company, for example. And so investors are really excited about AI and want to pay more for it.

However, that might not be true in a year or two years' time, actually. And if that's not true in two years' time, then you're in big trouble, actually. And so now I think you see a lot of companies that are raising really high valuations about 2021.

And now they're like, "Man, we're at like 100X or, you know, we raised a 300X multiple, for example." And if we're at 300X then, you know, maybe now we're at like 200X. And like, "Man, we just can't raise money ever again."

Like, you know, we're going to have to grow like 50X to be able to raise money, you know, at a reasonable valuation, let's say. And so I think that is really challenging and really demotivating for the team. And so I think a lower valuation actually is much better.

And so for us, in retrospect, if you ask the question two years later, we did not predict, you know, the crash, if you will. But given it, I think we've done extremely well, mostly because our valuation is not sky-high.

So the valuation was sky-high. I think we'd have a lot more problems. We'd probably have recruiting problems, for example. We'd probably have a lot of internal morale problems, etc. People would be like, you know, "Why is it all in this way?"

We might have cash flow problems because we might have to go raise money again, you know, etc. But we can't because the valuation is too high. So I wouldurge, I think, founders today to, quote-unquote, like, leave money on the table.

Like, there are some things that are not really worth optimizing. I think you should optimize for the quality of the company that you build, not like the valuation you raise that or the amount you raise, etc.

In hindsight, it's funny to you, but it looks like, you know, you made theright call there anyway.

So maybe we should also, for people who are not clued into Retool, do a quick, like, what is Retool? You know, I see you as the kings or the inventors of the low-code internal tooling category. Would you agree with that statement?

Like, you know, how do you usually explain Retool?

I generally say it's like Legos for code. We actually hate the low-code moniker. We actually never use, in fact, we have docs saying we will never use it internally or even to customers. And the reason for that is I think low-code sounds very not developer-y.

And developers that hear the phrase low-code are like, "Oh, that's not for me." Like, I love writing code. Like, why would I ever want to write less code? And so for us, Retool's actually built for developers. Like, 95% of our customers actually are developers, actually.

And so that is a little bit surprising to people. How I generally explain it is, and this is, you know, kind of a funny joke too, I think part of the reason why Retool has been successful is that developers hate building internal tools.

And you can probably see why. I mean, if you're a developer, you probably build internal tools yourself. Like, it's not a super exciting thing to do. You know, it's like you put together a crud UI. You've probably, you know, pieced together many crud UIs in your life before.

And there's a lot of grunt work involved. You know, it's like, hey, state management. It's like, you know, data validation. It's like displaying error messages. It's like debouncing buttons. Like, all these things are not really exciting. But you have to do it because it's so important for your business to have high-quality internal software.

And so what Retool does is basically allows you to sort of piece together an internal app really fast, whether it's a front-end, whether it's a back-end, or whatever else. So yeah, that's what Retool is.

Yeah, actually, so you started hiring, and so I do a lot of developer relations and community-building work, and then you hired Kritika, who has now moved on to OpenAI, to start out your sort of DevRel function. And I was like, "What is Retool doing according to developers?"

And then she told me about this, you know, developer traction. And I think that is the first thing that people should know, which is that, like, actually the burden and weight of internal tooling often falls to developers. Or it's an Excel sheet somewhere or whatever.

But yeah, you guys have basically created this market. You know, in my mind, I don't know if there was someone clearly before you in this, but, you know, you've clearly taken over and dominated. Every month, YC, you know, there's a new YC startup launching that is like, you know, we're the, you know, open source Retool.

We're like the lower code Retool, whatever. And it's pretty, I guess it's endearing. You know, we'll talk about Airplane later on, but yeah, I think that I've actually used Retool, you know, in my previous startups for this exact purpose.

Like, we needed a UI for AWS RDS that they can, you know, like the rest of our less technical people, like our sales operations people could interact with. And yeah, Retool was perfect for that.

Yeah, that's a good example of like that's an application that an engineer probably does not want to build. Like building an app on top of Salesforce or something like that is not exciting. It's so much API sucks. It's rate limited.

You know, it's like not a fun experience at all. But piecing it together in Retool is, you know, quite a bit easier. So yeah, let me know if you have any feedback, but awesome. Thanks for using it.

Yeah, no, of course. Well, so, you know, like more recently, I think about three, four months ago, you launched Retool AI. Obviously, AI has been sort of in the air. I'd love for you to tell the journey of sort of AI products ideation within Retool.

Given that you have a degree in this thing, I'm sure you're not new to this, but like when did the, when would you consider sort of the start of the AI product thinking in Retool?

Yeah, wow, that's funny. So we actually had a joke internally at Retool. We were part of a roadmap for every year. I think it was like 2019 or something. We had this joke, which was like, "What are we going to build this year?

We're going to build AI per programming," is what we always said as a joke. And so, but it was funny because we were like, "Haha, that's never going to happen." But like, let's add it because it's like a buzzwordy thing that enterprises love.

So let's look at it. And so it was almost like a funny thing, basically. But it turns out, you know, we're actually building that now. So it is pretty cool. But I would say maybe AI thinking at Retool probably first started maybe like, I want to say maybe, I don't know, a year and a half ago or something like that.

And the evolution of our thinking was basically when we first started thinking about it sort of in a philosophical way, if you will. It's like, "Well, what is the purpose of AI? And how can it help, you know, what Retool does?"

And there were two sort of main prongs, if you will, of value we got. One was helping people build apps faster. And so you've probably seen Copilot. You've seen sort of so many other coding assistants, like Fuser on a DEV, you know, stuff like that.

So that's interesting because, you know, engineers, as we talked about, do some grunt work. And the grunt work, you know, maybe could be automated by AI was sort of the idea. And it's interesting because we actually, I would say, kind of proved or disproved the hypothesis a little bit.

If you talk to most engineers today, like a lot of engineers do use Copilot. But if you ask them, like, "How much time does Copilot save you?" It's not like coding is 10X faster than before. You know, coding is maybe like 10% faster, maybe 20% faster, you know, something like that, basically.

And so it's not like a huge step change, actually. And the reason for that is, we think, is because the sort of fundamental frameworks and languages have not changed. And so if you're building, let's say, you know, like the sales ops tool you were talking about before, for example, let's say you've got AI to generate you a first version of that, for example.

The problem is that it probably generated it for you in like JavaScript because you're, you know, writing for the web browser, for example,right? And then for you to actually go proofread that JavaScript, for you to go read the JavaScript to make sure it's working, you know, to fix the subtle bugs that AI might have caused, hallucinations, stuff like that, it actually takes a long time and a lot of work.

And so for us, the problem is actually not like the process of coding itself. It is more sort of the language or the framework thing is like way too low level. It's kind of like when you think about like punched cards.

Like, you know, let's say back in the day you proved who does like punched cards and AI could help you generate punched cards. You're like, "Okay, you know, I guess that helps me, you know, punching cards is a little bit faster now because I have a machine punching them for me."

But like when there's a bug, I still have to go read all the punched cards and figure out what's wrong,right? It's like still a lot of work, actually. And so for us, that was the sort of initial idea was, "Can we help engineers code faster?"

And, you know, I think it's somewhat helpful to be clear. Like, again, I think it's 10 or 20%. So we have things like, you know, you can generate school queries by AI. You can generate UIs by AI and stuff like that.

So that's cool, to be clear. But it's not, I think, the step change of programming that we all wanted. And so that I think is, you know, we're investing somewhat in that. But the bulk of investment actually is in number two, which is helping developers build AI-enabled applications faster.

And the reason why we think this is so exciting is we think that practically every app, every internal app especially, is going to be AI-infused over the next like three years. And so every tool you might imagine, so like the tool you were even mentioning, like a sales operations tool, for example, probably, you know, if you were to build it today, you want to incorporate some form of AI.

And so, you know, we see today, like for us, like a lot of people build, you know, let's say sales management tools in Retool. An example is there's a fortune like a bunch of companies building like sales forecasting tools.

So they basically have salespeople enter their forecast, you know, for the quarter, the beginning of the quarter, like, "Hey, I have these deals and these deals are going to close. These deals are not going to close. You know, I think I'm upsetting these, downsetting these," stuff like that, basically.

And what they're doing now is they're actually, so you can imagine it's pulling in deals from your Salesforce database. And so it pulls in the deals and it actually uses AI to compute, like, "Okay, well, you know, given previous deal dynamics, like these are the deals that are more likely to close this month versus next month versus this quarter, next quarter, etc."

And so it can actually, you know, pre-write you a draft of, you know, your report, basically. And so that's an example where I think all apps, whether it's, you know, a sales app, you know, an internal, let's say, fraud app, a, you know, fintech app, you know, whatever it is, basically, especially internal apps, I think, like you said, Alessio, in order to make you a little productive, it's going to incorporate some form of AI.

And so then the question is, can we help them incorporate this AI faster? And so that's why we launched like a vector database once it was built directly into Retool. That's why we, you know, launched all these AI actions so you don't have to, you know, go figure out what the best model is and do testing and stuff like that.

We'll just, you know, give it to you out of the box. So for us, I think that is really the really exciting future is can we make every app, mostly Retool, use AI a little bit and make people a little more productive.

We talked with Jeffrey Wang, who's the co-founder and chief architect of Amplitude. He mentioned they just use Postgres vector. When you were building Retool vectors, how do you think about, yeah, leveraging a startup to do it, putting vectors into one of the existing data stores that you already had?

I think, like, you already have quite large customer scale. So like you're maybe not trying to get too cute with it. Any learnings and tips from that?

Yeah, I think a general philosophical thing I think we believe is we think the open source movement in AI, especially when it comes to all the supporting infrastructure, is going to win. And the reason for that is we look at like developer tools in general, especially in such a fast-moving space.

In the end, like, there are really smart people in the world that have really good ideas. And they're going to go build companies and they're going to go build projects, basically, around these ideas. And so for us, we have always wanted to partner with maybe more open source kind of providers or projects, you could say, like PG Vector, for example.

And the reason for that is it's easy for us to see what's going on under the hood. A lot of this stuff is moving very fast. A lot of times there are bugs, actually. And so we can always go look and fix bugs ourselves.

So we can contribute back to them, for example. But we really think open source is going to win in this space. It's, you know, it's hard to see about models. I don't know about models necessarily because, you know, it starts getting pretty complicated there.

But when it comes to tooling, for sure, I think there's just like so much. There's an explosion of creativity, if you will. And I think betting on any one commercial company is pretty risky. But betting on the open source sort of community and the open source contributors, I think it's a pretty good bet.

So that's why we decided to go for PG Vector.

Is there any most underrated feature, like something that customers maybe love that you didn't expect them to really care about? I know you have like text to SQL, you have UI generation. There's like so many things in there.

Yeah, what surprised you?

Yeah, so what's really cool, and this is my sense of the AI space overall, you know, if you're a skincare takes on YouTube as well, is that especially in Silicon Valley where a lot of the innovation is happening, I think there's actually not that many AI use cases, to be honest.

And

AI, to me, even as of, what, like January 19th, yeah, 19th of 2024, still feels like in search of truly good use cases. And what's really interesting, though, about Retool, and I think we're in a really fortunate position, is we have this large base of sort of customers.

And a lot of these customers are actually much more legacy, if you will, customers. And a lot of them actually have a lot of use cases for AI. And so to us, I think we're almost in like a, you know, really, you know, perfect or unique spot.

We're able to adopt some of these technologies and then provide them to, you know, some of these like older players. So one example that actually really shocked and surprised me about AI was, so we have this one, let's say, clothing manufacturer.

I think it's either the first or second largest clothing manufacturer in the world who's using Retool. And, you know, they're an enormous company with, you know, very multinational, you know, stores on, you know, pretty every model in the world.

And they were interested in, so they have one problem, which is they need to design styles every year for the next year, basically, for every season. So like, hey, just like summer 2024, for example, and what are we going to design?

And so what they used to do before is they would hire designers. And designers would go study data. They'd be like, "Okay, well, it looks like, you know, floral patterns are really hot in like, you know, California, for example, in 2023.

And like, do I think it's going to be hot in 2024?" Well, let me think about it. I don't know. You know, so let me, and if so, if I believe it is going to be hot, let me go design some floral patterns, actually.

And what they ended up doing in Retool, actually, is they actually automated a lot of this process away in Retool. So they actually now built a Retool app that allows actually a non-designer, so like an analyst, if you will, to go analyze like, you know, what are the hottest selling patterns, you know, particular geos.

Like this was really hot in Brazil. It was really hot in China. It was really hot, you know, somewhere else, basically. And then they actually feed it into an AI. And the AI, you know, actually generates with DALL-E and other image generation APIs, actually generates patterns for them.

And they print the patterns, which is really cool. And so that's an example of like, honestly, a use case I would have never thought about. Like thinking about like, you know, how clothing manufacturers create their next line of clothing, you know, for the next season.

Like, I don't know. I never thought about that, to be honest. Nor did I ever think, you know, how it would actually happen. And the fact that they're able to leverage AI and actually, you know, leverage multiple things in Retool to make that happen is really, really, really cool.

And so that's an example where I think if you go deeper into sort of, if you go outside of Silicon Valley, there are actually a lot of use cases for AI. But a lot of this is not obvious.

Like you have to get into the businesses themselves. And so I think we personally are in a really fortunate place. But if, you know, you're working in the AI space and want to find some use cases, please come talk to us.

Like, you know, we're really excited about marrying sort of technology with use cases, which I think is actually really hard to doright now in Silicon Valley.

So, you know, I have a bunch of like sort of standing presentations around like how this industry is developing. And like I think the foundation model layer is understood. The sort of LangChain, VectorDB, RAG layer is understood. And like what is, I always have a big question mark and actually have you and Vercel V0 in that box, which is like sort of the UI layer for AI.

And like, you know, you are a perfectly placed place to expose those functionalities to end users, even if you personally don't really know what they're going to use it for. And sometimes they'll surprise you with their creativity.

One segment of this, I do see some startups springing up to do this, is related to the things that, to something that you also build, but it's not strictly AI related, which is Retool workflows, which is the sort of canvas-y boxes and arrows, point and click, do this, then do that type of thing.

Like which every, you know, I hate that, you know, what are we calling low code? Every internal tooling company

eventually builds. You know, I worked at a sort of workflow orchestration company before, and we were also discussing internally how to make that happen. But you are obviously very well positioned to that. Yeah, basically like, you know, do you think that there is an overlap between Retool workflows and AI?

I think that, you know, there's a lot of interest in sort of chaining AI steps together. Do people, I couldn't tell if like that is already enabled within Retool workflows. I don't think so. But you could sort of hook them together sort of gently.

Like, is there, is there, like, what's the interest there? You know, is it all of a kind ultimately in your mind?

It is 100% time. And yes, you can actually already, so a lot of people actually are building AI workflows now in Retool, which is, we can talk more about it in a second. But a hard take here is actually, I think a lot of the utility in AI today,

like I would probably argue 60, 70% of the utility, like, you know, businesses have found in AI is mostly via ChatGPT and across the world too. And the reason for that is I think the ChatGPT sort of UI, you could say, or interface, or user experience is just really quite good.

You know, you can sort of converse, you know, with an AI, basically. And that said, there are downsides to it. If you talk to like a giant company like JPMorgan Chase, you know, for example, they may be reticent to have people copy-paste data into ChatGPT, for example, even on ChatGPT Enterprise, for example.

Some problems are that I think chat is good for one-off tasks. So if you're like, "Hey, I want a first version of a presentation or something like that, you know, and help me write this first version of a doc or something like that," chat is great for that.

It's a great, you know, very portable, you know, if you will, form factor. So you can do that. However, if you think about it, you think about sort of economic productivity more generally, like chat, again, will help you like 10 or 20%, but it's unlikely that you're going to replace an employee with chat.

Like, you know, you're not going to be like, "Oh, I'm a relationship manager at JPMorgan Chase and I've replaced them with an AI chatbot." Like, it's kind of hard to imagine,right? Because like the employees are actually doing a lot of things besides, you know, generating, you know, maybe another way of putting it is like chat is like a reactive interface.

Like it's like when you have an issue, you will go reach out to chat and chat might solve it. But like chat is not going to solve 100% of your problems. It'll solve like, you know, 25% of your problems pretty quickly,right?

And so what we think the next like big breakthrough in AI is, is actually like automation. It's not just like, "Oh, I have a problem. Let me go to a chatbot and solve it." Because like, again, like, you know, people don't spend 40 hours a week in a chatbot.

They spend maybe like two hours a week in a chatbot, for example. And so what we think can be really big, actually, is you're able to automate entire processes via AI. Because then you're really realizing the potential of AI.

It's like, it's not just like, you know, a human copy-pasting data into an AI chatbot and pasting it back out or copying it back out. Instead, it's like the whole process now is actually done in an automated fashion without the human.

And that I think is what's going to really unlock sort of economic productivity or that's what we're really excited about. And I think part of the problemright now is, you know, I'm sure you all thought a lot about agents.

I think agents are actually quite hard because like, you know, if AI is wrong like, you know, 2% of the time, but then you like, you know, if you, let's say, you know, race it to power seven, for example, that's actually wrong, you know, quite often, for example.

And so what we've actually done with workflows is we prefer, we've learned actually, is that we don't want to generate the whole workflow for you via AI. Instead, what we want you to do actually is we want you to actually sort of drag and drop the workflow yourself and maybe you can get a V0 or something via AI, but it's coded, basically.

You should actually be able to modify the steps yourself, but every step can use AI. And so what that means is like, it's not the whole workflow is created by AI. It's like every step is AI automated. And so if you go back to, for example, like the use case I was talking about, you know, with a clothing manufacturer, that's actually a workflow, actually.

So basically what they say is, "Hey, every day we eat all the data, you know, from our sales systems into our database. And then we, you know, do some data analysis." And, you know, that's just, you know, raw SQL, basically.

It's nothing too surprising. And then they use AI to go generate the new ideas. And then the analyst will look at the new ideas and approve or reject them, basically. And that is like a, you know, that's true automation.

You know, it's not just like, you know, a designer copy-pasting things into ChatGPT and be like, "Hey, you know, give me a design." It's actually designs are being generated. They generate 10,000 designs every day. And then you have to go and approve or reject these designs, which I think is a lot, you know, that's a lot more economically productive than just copy-pasting stuff into ChatGPT.

So we think sort of the AI workflow space is a really exciting space. And I think that is the next step in sort of delivering a lot of business value via AI. I personally don't think it's, you know, via chat or, you know, via agents quite yet.

Yeah, yeah. I think that's a pretty reasonable take. It's disconcerting because I, not this, I mean, disconcerting only in the fact that like I know a lot of people trying to build what you already have in workflows. And so you have that sort of, you're the incumbent sort of in their minds.

I'm sure it doesn't feel that way to you, but like I'm sure, I'm sure, you know, you're the incumbent in their minds and they're like, "Okay, like, you know, like how do I, how do I, you know, compete with Retool or, you know, differentiate from Retool?"

And, you know, as you mentioned, you know, all these sort of connections, it does remind me that you're running up against Zapier. You're running up against maybe Notion in the distant future. And yeah, I think there will be a lot of different takes at this space and like whoever is best positioned to serve their customer in the way that they sort of need to shape is going to win.

Do you have a philosophy around like what you won't build? Like what do you prefer to partner and not build in-house? Because it seems, I feel like you build a lot in-house.

Yeah, so there's only two philosophical things. So one is that we're developer first. And I think that's actually one big differentiator between us and Zapier. Now, if you're going to waste, it's very rare to see them actually. And the reason is we're developer first.

Because developers like, if you're like building a sales ops tool, you're probably not considering Notion if you're a developer. You're probably like, "I want to build this via React, basically, or use Retool." And so are you built for developers?

It's pretty interesting, actually. I think one huge advantage of some of the developers is that they actually don't, like developers don't want to be given an end solution. They want to be given the building blocks so they can themselves go build the end solution.

And so for us, like, you know, actually, you know, interesting point that Equilibrium Retool can get to is basically you can say, "Hey, Retool's a consulting company and we basically build apps for everybody," for example. And what's interesting is we've actually never gone to that Equilibrium.

And the reason for that is with some of the developers. Developers don't want, you know, like a consultant coming in and building all the apps for them. Developers like, "Hey, I want to do everything myself. Just give me the building blocks.

Give me the best table library. Give me, you know, good state management. Give me an easy way to query REST APIs and I'll do it myself, basically." So that I think is pretty, so we generally end up basically always building building blocks that are reusable by multiple customers.

We have, I think, basically never built anything specific for one customer. So that's one thing that's interesting. A second thing is when it comes to sort of, you know, let's say like in the AI space, we're going to build and we're not going to build, we basically think about whether it's a core competency or whether there are, whether there are unique advantages to us building it or not.

And so if you think about the workflows product, we think workflows actually is a pretty core competency for us. And I think the idea that we could build a developer-first workflows automation engine, it's actually, there's nothing, I mean, I think after we released, you know, workflows through to workflows, there have been just sort of few copycats that are I think quite, quite far behind actually.

They sort of are missing a lot of, I think, more critical features. But like, if you look at the space, it's like Zapier on one side and then maybe like Airflow on the other. And so Retool workflows actually is fairly differentiated.

And so we're like, "Okay, we should go build that basically." Because no one else is going to build it. Someone's going to build it. Whereas if you look at like Vectors, for example, you look at Vectors, you're like, "Wow, this is a pretty thriving space already of, you know, vector databases."

Does it make sense for us to go build our own? Like, what's the benefit? Like, not much. We should go partner with or go find technology off the shelf. In our case, it's PG Vector. And so for us, I think it's like, how much value does that add for customers?

Do we have a different take on the space? Do we not? And every product that we've launched, we've had a different take on the space. And the products that we don't have a different take, we just adopt what's off the shelf.

Let's jump into the State of AI survey that you ran and maybe get some live updates. So you surveyed about 1,600 people last August. So, you know, in AI, we were this busy like five years ago. And there were kind of like a lot of interesting nuggets, and we'll just run through everything.

The first one is more than half the people, 52%, said that AI is overrated. Are you seeing sentiment shift in your customers or like the people that you talk to, like as the months go by, or do you still see a lot of people, yeah, that are not in Silicon Valley maybe say, "Hey, this is maybe not as world-changing as you all made it sound to be?"

Yeah, so actually I'll grow the survey again actually in the next few months, so I can let you know if you don't want to change this. It seems to me that it has settled down a bit in terms of sort of the maybe like, I don't know, signal noise you could say.

Like it seems like there's a little bit less noise than before, but the signal is actuallyright about the same in the sense that like, I think people are still trying to look for use cases. I've seen with August of last year, like the United States again, and I think there are slightly more use cases, but still not substantially more.

And I think as far as we can tell, a lot of the engineers' surveys, especially some of the comments that we saw, do feel like the companies are investing quite a bit in AI and they're not sure where it's going to go yet.

But they're like, "Right, it could be big, so I think we should keep on investing." I do think that based on what we are hearing from customers, if we're not seeing returns in like a year or something, there will be more skepticism.

So I think there is like a, it is time bound, if you will.

You finally gave us some numbers on Stack Overflow usage. I think that's been a Twitter meme for a while, whether or not ChatGPT killed Stack Overflow. In the survey, 58 people said they used it less, and 94% of them said they used it less because of Copilot and ChatGPT, which, yeah, I think it kind of makes sense.

I know Stack Overflow tried to pull a whole thing. It's like, "No, the traffic is going down because we changed the way we instrument our website," but I don't think anybody bought that. And then you addright after that expectation of job impact by function and operations people, 8 out of 10 basically, they think it's going to, AI is going to really impact their job.

Designers were the lowest one, 6.8 out of 10, but then all the examples you gave were designers of job being impacted by AI. Do you think there's a bit of a dissonance maybe between like the human perception of like, "Oh, my job is like, can possibly be automated?"

It's funny that the operations people are like, "Yeah, it makes sense. I wish I could automate myself," you know, versus the designers that maybe they love their craft more. Yeah, I don't know if you have any thoughts on who will accept it first, you know, that they should just embrace the technology and change the way they work.

Yeah, that's interesting. I think it's probably going to be engineering driven. I mean, I think you two are very well, maybe you two even started some of this wave, sort of the AI engineer wave. I think the companies that adopt AI the best, it is going to be engineering driven, I think, rather than like operations driven or anything else.

And the reason for that is I think the rise of this like profile with an AI engineer, like AI is very, maybe it's got philosophical, like AI is a tool in my head. Like it is not a, in my head, I think we're actually pretty far from AGI level.

We'll see what happens, but AI is not like a, you know, thing that it's not like a black box where like it does everything you want it to do. The models that we have today require like very specific prompting, for example, in order to get like, you know, really good results.

And the reason for that is it's a tool that, you know, you can use it in specific ways. If you use it the wrong way, it's not going to produce good results for you, actually. It's not like, you know, going to, you know, by itself take your job away,right?

And so I think actually to adopt AI, it's probably going to be, going to have to be engineering first, basically, where engineers are playing around with it, figuring out limitations of the models, figuring out like, "Oh, maybe like using vectorized databases is a lot better," for example.

Maybe like prompting in this particular way is going to be a lot better, et cetera. And that's not the kind of stuff that I think like an operations team is going to really be like experimenting with necessarily. I think it really has to be engineering-led.

And then I think the question is, well, what are the engineers going to focus on first? Like I think they're going to focus on, you know, design first or like operations first. And that I think is more of a business decision.

I think it's probably going to be more like, you know, the CEO, for example, says, "Hey, you know, we're having trouble skilling this one function. So like why don't we try using AI for that and let's see what happens," for example.

And so in our case, for example, we are really, we have a lot of support issues. And what I mean by that is we have a really, really high-performing support team, but we get a lot of tickets. And the reason for that is, you know, we're a very dynamic product.

You can use it in so many different ways. And so we'll have a lot of questions for us, basically. And so we were looking at, well, you know, can we, for example, draft some replies and support tickets, you know, by AI, for example?

Can we allow our support agents to be, you know, hopefully, you know, double as, doubly productive as before, for example? And so I guess I would say it's like this needs driven, but then engineering driven after that. So like, you know, the business decides, "Okay, well, this is where AI could be most applied," and then we assign the project to an engineer and the engineer goes and figures it out.

I honestly am not sure if like the operations were going to have much of a, like if they accept and reject it, I don't know if that's going to change the outcome, if you will.

Yeah, interesting. Another interesting part was the importance of AI in hiring. 45% of companies said they made their interviews more difficult. And the engineering side made interviews more difficult to compensate for people using Copilot and ChatGPT. As that changed at Retool, like have you, yeah, have you thought about it?

I don't know how much you're still involved with engineering hiring at the company, but I'm curious how we're scaling difficulty of interviews, even though the job is the same,right? So just because you're going to use AI doesn't mean the interview should be harder, but I guess it makes sense.

Yeah, for us, I think our sense basically of the survey, and this is true for what we believe too, is that

we are most, when we do engineering interviews, we are most interested in assessing like critical thinking or thinking, you know, on the spot.

And I guess, you know, when you hire an employee, you know, at the end, the job of the employee is to be productive, which is what our tools they want to be productive. So, you know, that's kind of our thinking too.

However, we do think that, you know, if you think about it from a first-person perspective, if your only method of like coding is literally copy-pasting, you know, off of ChatGPT or like, you know, just pressing tab and Copilot, I think that would be concerning.

And so for that reason, we still do want to test for like, you know, fundamentals understanding of comp side. Now, that said, I think if you're able to use ChatGPT or Copilot, let's say competently, we do view that as a plus.

We don't view it as a minus. But if you only use Copilot and you aren't able to reason about, you know, how to write a for loop, for example, how to write this buzz, that would be highly problematic.

And so for us, what we do today is we basically screen share, or we actually use a HackPad, actually. So it's, sorry, this is no Copilot there. You can sort of see what they're doing, see what they're thinking.

And we really want to test for thinking, basically. But yeah, I mean, we ourselves internally have embraced Copilot and we would encourage engineers to go with Copilot too. But we do want to test for understanding of what you're doing rather than just copy-pasting Copilot.

The other one was AI adoption rate. Only 27% are in production. Of that 27%, 66% are internal use cases. Shout out to Retool, you know. How do you have a mental model as to how people are going to make the jump from like using it internally to externally?

Obviously, there's like all these different things like privacy, like a, you know, if an internal tool hallucinates, that's fine because you're paying people to use it basically versus if it hallucinates to your customer, there's a different bar. Yeah, I don't know if you have thought about it.

Because for you, if people build internal tools with Retool, they're external customers to you, you know. So I think you're on the flip side of it.

Yeah, I think it's hard to say. Maybe a core Retool belief is actually that most software built in the world is internal facing, actually, which actually sounds, maybe sounds kind of surprising to you first time you're hearing this, but effectively, like, you know, we all work in Silicon Valley,right?

Like we all work at businesses basically that sell software as, you know, as sort of a, as a business. And that's why all the software engineers that we hire basically work on external facing software, which makes sense because we're software companies.

But if you look at most companies in the world, most companies in the world are actually not software companies. If you look at like, you know, the clothing manufacturer that I was talking about, they're not a software company.

Like they don't sell software to make money. They sell clothing to make money. And most companies in the world are not software companies, actually. And so most of the engineers in the world, in fact, don't work at Silicon Valley companies.

They work outside of Silicon Valley. They work in these sort of more traditional companies. So if you look at the Fortune 100, for example, probably like 20 of them are software companies. You know, 480 of them are not software companies.

And actually, they employ those software engineers. And so most of the software engineers in the world and most of the code engineers in the world actually go towards these internal facing applications. And so like for all the reasons you said there, like I think hallucination matters less, for example, because you have someone checking the output.

I sound like an end consumer, so hallucination is more okay. It's more acceptable as well. AI can be unreliable because it's probabilistic, and that's also okay. So I think it's kind of hard to imagine AI being adopted in a consumer way without the consumer like opting in.

Like ChatGPT is very obviously a consumer. The consumer is like, you know, knows that it's ChatGPT. It's, you know, using it. I don't know if it's going to make its way to like your banking app anytime soon. Maybe for like, even for support, it's hard because if it hallucinates, then, you know, it's actually quite bad for support if you're hallucinating,right?

So it's, yeah, it's hard to say. I'm not sure.

Yeah, but that's a good, that's a good idea, like insight, you know. Yeah, I think a lot of people, like you said, we all build software, so we expect that everybody else is building software for other people, but most people just want to use the software that we build here.

I think the last big bucket is like models breakdown. 80% of people you survey just use OpenAI. Some might experiment with smaller models. Any insights from your experience at Retool, like building some of the AI features? Have you guys thought about using open source models?

Have you thought about fine-tuning models for specific use cases, or have you just found GPT-4 to just be great at most tasks?

Yeah, so two things. One is that

from a data privacy perspective, people are getting more and more okay with using a hosted model, like a GPT-4, for example. Especially because GPT-4 or OpenAI often has been an enterprise agreement to some companies already because I think a lot of CIOs are just like, "Let's get this taken house."

Like, you know, let's use an Azure, for example, and, you know, let's make it available for employees to experiment with. So I do think there is more acceptance, if you will, today of feeding data into GPT. That's undertaking some sensitive data.

People might not want to do so, like, you know, feeding it like earnings results data, you know, three days before you announce earnings, like probably is a bad idea. Like, you probably don't want GPT writing your like earnings statement for you.

So, you know, there's still some challenges like that that I think actually open source models could actually help solve, like a lot of them really don't want to come. So I think that could be exciting. So that's maybe just one thought.

The second thought is I think OpenAI has been really quite smart with some of their pricing, and they've been pretty aggressive of like, "Let's get, you know, let's create this model and sell it at a pretty cheap price and to make it such that there's no reason for you to use any other model."

Just from like a, you know, strategy perspective, I don't know if that's going to work. And the reason for that is you have really well-funded players, like a Google or like a Facebook, for example, that are actually quite interested.

Like, I think if OpenAI is competing with startups, OpenAI would win for sure. Like, at this point, OpenAI is so far ahead from both a model and a pricing perspective that like there is no reason for it to go just really, I think, in my opinion, at least a startup model.

But if like, you know, Facebook is not going to give up on AI, like Facebook is investing a lot in AI, in fact. And so competing against a large thing company that is making their model open source, I think that is challenging.

Now, however, where we areright now is I think GPT-4 is so far ahead in terms of performance that, and I would say model performance is so importantright now because like the average, you know, like, you know, you can argue LLaMA 2 is actually so far behind, but like customers don't want to use LLaMA 2 because it's so far behindright now.

And so that I think is part of the challenge as AI progress slows down. So if we get like LLaMA 4, LLaMA 5, for example, maybe it's a comparable at that point, like GPT-5 or GPT-6, like it may get to the point where people are just like, "Look, I just want to use LLaMA."

Like, it's, you know, safer for me, you know, hosted on-prem. It's just as fast, just as cheap. Like, why not, basically? But I thinkright now we are in this state where OpenAI is doing really well, I think. Andright now they're thriving, but let's see what happens in the next, you know, year or two.

Awesome. And there's a lot more numbers, but we're just going to send people to the link in the show notes. Yeah, this was great, Sean. Any other thoughts, trends we want to pull on what we have, David?

What are you going to ask differently for the next survey? Like, what info do you really or actually want to know that's going to change your worldview?

I'll also ask you that, but if anything is, let me know. For us, actually, we were planning on asking very similar questions because for us, the value of the survey is mostly seeing changes over time and understanding like, "Okay, wow."

Like, for example, GPT-4 Turbo NPS has declined, you know. That would be interesting, actually. One thing that was actually pretty shocking to us is, I'm not sure if I have the exact number, but like, if you look at like the one change that we saw, for example, if you compare GPT-3.5 NPS, I want to say it was like 14 or something.

Like, it was like not high, actually. But the GPT-4 NPS thing was like 45 or something like that. So it was actually quite a bit higher. So I think that kind of progress over time is what we're most interested in seeing is, you know, are models getting worse?

Are models getting better? Are people still loving PG Vector? Do people still love Mongo? You know, stuff like that. That I think is the most interesting thing. Do you two have any questions that you think we should ask?

Off the bat, like, it seems like you're very language model focused. You know, I think that there's an increasing interest in multimodality in AI. And I don't really know how that is going to manifest. Obviously, GPT-4 Vision as well as Gemini both have multimodal capabilities.

There's a smaller subset of open source models that have multimodal features as well. Like, we just released an episode today talking about Idefix from Hugging Face. And yeah, so I think I would like to understand how people are adopting or adapting to the different modalities that are now coming online for them.

What their demand is relative to for like, let's say, generative images versus, you know, like just visual comprehension versus audio versus text-to-speech. Like, what do they want? What do they need? And what's the sort of forced, like stack ranked preference order?

That's a great idea. I want to ask that. Yeah.

Yeah.

I wonder what it is. I don't know. Yeah.

It's something that we're like, we're trying to actively understand because, you know, there's this sort of multimodality world, but really, like multimodality is just like an umbrella term for like actually a whole bunch of different things that are like, quite honestly, like not really that related to each other unless, you know, in the limit, which it tends towards like maybe everything you use is transformers and ultimately everything can be merged together with a text layer because text is the universal interface.

But if you're given the choice between like, if I want to implement an audio feature versus I want to implement an image feature versus video, whatever, what are people needing the most? What should we pay the most attention to?

What is going to be the biggest market for builders to build in? I don't know.

Yeah, we'll go ask that. Yeah, that's a great question.

Yeah.

I know, I think, Sean, you put in a question from Joseph here in the show notes. Our friend Joseph Nelson, founder of RoboFlow and Office coworker, I guess.

Yeah, sure. So, you know, I think I figured we would just kind of zoom out a little bit to just the general founder questions. You have a lot of fans in the founder community and, you know, I think you're just generally well known as a very sort of straightforward, plain-spoken person about just business.

Something that is the perception from Joseph is that you have been notably sort of sales-led in the past. That's his perception. I actually never got that, but I'm not that close to sort of your sales motion. And it's interesting to understand your market, like the internal tooling market versus all the competition that's out there,right?

There's a bunch of open source retools and there's a bunch of like, you know, I don't know how you sort of categorize the various things out there. But effectively, what he's seeing and what he's asking is how do you manage between sort of enterprise versus ubiquity or, in other words, enterprise versus bottom-up,right?

I was actually surprised when he told me to ask that question because I had always assumed that you were self-serve, sign up, like bottom-up led. But it seems like you have a counter-consensus view on that.

Yeah, let me think about that for a second. Yeah, so actually the retool first started, we started mostly by doing sales, actually. And the reason we started by doing sales was mostly because we weren't sure whether we had product-market fit and sales seemed to be the best way of proving whether we had product-market fit out.

Because I think this is true of a lot of AI projects. You can like launch a project and people might use it a bit and people might stop using it and you're like, "I don't know, is that product-market fit?

Is that not?" It's hard to say, actually. However, if you work very closely with the customer in sort of a sales-led way, it's easier to understand their sort of requests, understand their needs and stuff like that and actually go build a product that actually serves them really well.

And so basically we viewed sales as like working with customers, basically, which is like, you know, I think actually quite a, I think it's a better way of describing what sales is at an early stage company. And so we did a lot of that, certainly when we got started.

I think we, over the last maybe five years, maybe like three years ago, four years ago, something like that, I think we

have invested more on the self-serve ubiquity side. And the reason for that is when we started Retool, we always wanted actually some percent of software that could be built inside of Retool, whether AI software or software more broadly, UIs, you know, whatever, but like software, basically.

And for us, we're like, we think that maybe one day, you know, 10% of all the code in the world could be written inside of Retool, actually, or 10% of all the software could be running on Retool. It should be really, really cool.

And for us to achieve that vision, it really does require like broad-based adoption of the platform. It can't just be like a, oh, you know, only like a thousand customers, but the largest a thousand companies in the world use it.

It has to be like all the developers in the world use it. And for us, you know, there's like, what, I think 25, 30 million developers in the world. And so the question is, how do you, you know, get to all the developers?

And the only way to get to those developers is not by sales. You can't have a sales person talk to 30 million people. You know, it has to be basically in this sort of bottom-up, product-led ubiquity kind of way, basically.

And so for us, we actually changed our focus to be ubiquity actually last year. So when we first started, I think we used to always be sort of revenue generated or revenue ARR generated. We actually changed it to be the number of developers building on the platform actually last year.

And that I think was actually a really clarifying change because obviously revenue is important, you know, with funds, you know, a lot of, you know, our product and funds of, you know, the business. But we're going to fail if we aren't able to get to like something like, you know, 10, 20, 30 million developers one day.

We can't convince all developers in the Retools a better way of building a sort of class of software, let's say, internal applications for today. And so I think that has been a pretty good outcome. Like if I think about, you know, the last like, I don't know, five years of Retool, like I think the starting off with sales so you can build revenue and then you can actually build traction and you can hire more slowly, I think was really good.

I do think the focus towards like, you know, bottom-up ubiquity also is really important because it helps us get to our long-term outcome. What's interesting, I think, is that long-term ubiquity actually is harder for us to achieve outside of Silicon Valley.

Like to your point, I think in Silicon Valley, Retool is like reasonable ubiquity. So I think like if you're starting a startup today and you're looking to build an internal UI, you're probably going to consider Retool at least.

Maybe you don't choose it because you're like, "I'm not ready for it yet" or something. But you're going to consider it at least. And when you want to build it, I think it's actually a high probability you will actually end up choosing Retool.

It's awesome. But it's that, you know, if you think about, you know, a random developer working at, let's say, like an Amazon, for example, we actually, so today at Amazon, actually, we have, I think, 11 separate business units that use Retool at this point, which is really awesome.

So Amazon is actually a big Retool customer. But like the average engineer at Amazon probably has never heard of Retool, actually. And so that is where the challenge really is. How do we get like, you know, I don't know, let's say 10,000 developers at Amazon building via Retool?

And that, again, I think is still a bottom-up ubiquity thing. I don't think that's like a, I don't think we can like, you know, go to Amazon and knock on every developer's door or send out an email to every developer and be like, "Go use Retool."

They'll ignore us, actually. I think it has to be, "Use the product. You love it. You tell your coworker about it." And so for us, I think bottom-up ubiquity by marrying that with sort of enterprise or the community business has been something that's really near and dear to our hearts.

Yeah, and just like general market thoughts on AI, are you, you know, do you think spend a lot of time thinking about like AGI stuff or regulation or safety or like, you know, what interests you most, you know, outside of the Retool context?

Wow. Well, I'll give you what I see in the Retool context because it's an actual question. Yeah, my opinion, I think there's a lot of hype in AIright now and there's, again, not too many use cases. So for us, at least from the Retool context, it really is, how do we bring AI and have it actually meet business problems?

And again, it's actually pretty hard. Like I think most founders that I meet in the AI space are always looking for use cases. They never have enough use cases. Sort of real use cases people are looking to pay money for.

So that's I think really where the Retool interest comes from. Me personally, I think philosophically, yeah, I've been thinking recently myself a bit about sort of intentionality and AGI and like, you know,

what would it take for me to say, "Yes, you know, GPTX or, you know, any sort of model actually is AGI"? I think it's kind of challenging because it's like, I think if you look at like evolution, for example, like humans have been programmed to do like three things, if you will.

Like, you know, we are here to survive, you know, we're here to reproduce and we're here to like, you know, maybe those are just two things, I suppose. So like it's basically to survive if you could eat food, you know, for example.

To survive, maybe like having more resources to help us, you want to go make money, you know, for example. To reproduce, maybe you should go date, you know, or whatever. You should get married and stuff like that,right? So like that's, we've been programmed to do that.

And humans that, you know, are good at that have propagated. And so humans that, you know, we're not interested in surviving probably have disappeared just due to natural selection. Humans that were not interested in reproducing also disappeared because, or, you know, there are less of them, you could say, because they just, they just stop carrying on, basically.

And so I think, so it almost feels like humans have sort of naturally self-selected for these like two aims. I think the third aim I was thinking about was like, does it matter to be happy? Like maybe it does.

So maybe like happier humans, you know, survival. It's hard to say. So I'm not sure. But if you think about that in the relative to like AIs, if you will, likeright now we're not really selecting AIs for like, you know, reproduction.

Like it's not like, you know, we're being like, "Hey, AI, you know, you should go make 30 other AIs." And, you know, those that make the most AIs, you know, are the ones that survive. We're not saying that.

And so it is kind of interesting sort of thinking about where intentionality for humans comes from. And like you got, I think you can argue that intentionality for humans basically comes out of these three things. You know, like, you know, if you want to be happy, you want to survive, you want to reproduce.

That's like basically your sort of goal, you know, in life. Whereas like they definitely have that. But maybe you could program it in. Like if you, you know, prompt inject, for example, like, "Hey, AI, you could just, you know, go do these three things."

And, you know, you can even create a simulation, if you will, of like all these AIs, you know, in a world, for example. And maybe you don't have AGI in that world, which I think is kind of interesting.

So that's kind of stuff I've been thinking of when I talk about with some of my friends from a sort of philosophical perspective, but that's kind of interesting.

Yeah, my quick response to that is we're kind of doing that, maybe not at the sort of trained final model level, but at least at the datasets level, there's a lot of knowledge being transferred from model to model.

And if you want to think about that sort of evolutionary selection pressure, it is happening in there. And, you know, I guess one of the early concerns about being in Sydney and sort of like bootstrap, self-bootstrapping AGI is that it actually is, if these models are sentient, it actually is in their incentive to get as much of their data out there into our datasets so that they can bootstrap themselves in the next version that gets trained.

And that is a scary sobering thought that we need to try to be on top of.

David, I know we're both kind of half-statters GB. And actually, I saw in one of your posts on the Sequoia blog, you referred to the

anteater, like piece of the, I don't even know if you call them chapters. And GB is just kind of like this continuous riff, but basically like how ants are like not intelligence, but like an ant colony has signs of intelligence.

And I think half-statters then use that to say, "Hey, you know, neurons are kind of like similar and then computers maybe will be the same." I've always been curious if like we're drawing the wrong conclusion from like neural networks where people are like, "Oh, each weight is like a neuron and then you tie them together, it should be like a brain."

But maybe like the neuron is like different models that then get tied together to make the brain. You know, we're kind of looking at the wrong level of abstraction. Yeah, I think there's a lot of interesting philosophical discussions to have.

And Sean and I recorded a monthly WeGap podcast yesterday and we had a similar discussion on, "Are we using the wrong?" Are we like, what did you say, Sean, on the plane and the bird? I think that was a good analogy.

Oh, the sour lesson. Are we using the wrong analogies? Because we're trying to be inspired by human evolution and human development and we are trying to apply that analogy strictly to machines. But in every example in history, machines have always evolved differently than humans.

So why should we expect AI to be any different?

Yeah, if you sort of peer under the hood of AGI, if you insist that AGI, we have always used AGI for things like a human. That is the Turing test, I suppose. But

whether that is a good point, like if it works, that's not the Turing test. The Turing test basis is if the output is the same as a human that I'm happy basically. I don't really care about what's going on inside.

And so it feels like caring about the inside is like a pretty high bar. Like why do you care? It's kind of like the plane thing, like for flies. It's not a bird. I agree. It does not fly necessarily the same way as a bird physically does, I suppose.

But you see what I mean. Like it's not the same under the hood, but it's okay because it flies. That's what I care about. And it does seem to be like AGI is probably like doesn't think and can achieve like, you know, outcomes that I give it and can achieve its own outcomes.

And it can do that. Like I kind of don't care what it is like under the hood. It may not need to be human life at all. It doesn't matter to me. So I agree.

Awesome. I know we kept you long. I actually have GBright here on my bookshelf. Sometimes I pick it up and I'm like, "Man, I can't believe I got through it once." It's quite the piece of work.

That's what I'm up to.

Yeah. No, I mean, I started studying physics in undergrad, so, you know, it's one of the edgy things that every physicist starts going through. But thank you so much for your time, David. This was a lot of fun and looking forward to the 2024 State of AI results to see how things change.

Yeah, we'll let you know. So thanks both.