⚡️ Ship AI recap: Agents, Workflows, and Python — w/ Vercel CTO Malte Ubl

All right. We are here in the remote studio. Thanks again to f.inc for lending us this space, with Malte Ubl, who's CTO of Vercel. Welcome.

Hey, how's it going? Glad to be here.

Did I get it right, Ubl? I've actually never pronounced it out loud until, like, just now.

Yeah, it was completely perfect. It rhymes with Google.

Ah, okay. Uh, so perfect that you, uh, worked on Search at Google and AMP and Wiz, which, like, uh, I think still people don't know enough about Wiz.

It is, like, no longer a secret, but yeah, you can't use it, so, like, unless you work at Google, in which case you probably know what it is. Otherwise, uh, there's no reason to really know.

Anyway, suffice to say that you are responsible for a lot of the web as it is today, so thank you for spending some time with us. You're also obviously now, uh, building the next web, as, as we say, with, uh, Vercel.

And we, we, we can cover framework-defined infrastructure. I, I think you, you probably saw I have a, I have a lot of interest in self-provisioning runtimes. Uh, we can cover v0, but here, really, this part is, is recorded right after you did Ship AI, which, uh, we're trying to sort of recap, right, for the general, uh, Lane space audience who may not be watching Vercel as closely as, uh, I do or, or, uh, you do.

So, like, so ba- basically, just generally, what, I guess, is your message to the broader AI engineer audience on what Vercel is doing with AI?

Yeah, I think the, the, the super high-level view is that what we're really trying to do is, like, we're, we're like the biggest fan of the AI engineering movement, and we are also fans of, you know, we're not, not just, like, going super hard on, on hype and the big ideas and, and talking about things, but, like, v- being very concrete about, like, you know, agents are very exciting, and you can actually build them, right?

And so, like, I think our entire conference was about both making that easier, right, and, and discovering the right abstractions as we're kind of figuring out what the, what people actually wanna do, right, which is emerging as we speak.

Then the way Vercel always does these things is by building things ourselves, right? And so that is both in terms of products, so agents that are products that you can purchase from Vercel, and stuff that we do basically in our back office to make our own operations more efficient.

And so this kind of building of apps lets us ground kind of what we do in, in that reality and then, and then, uh, you know, kind of extract the abstractions that we feel are really helpful to, to then put that on the road.

And I think the, the probably most talked about thing that we shipped at the, at the conference was our new Workflow Development Kit-

Right

... which really, really is just a way to make writing, uh, like, workflows, like, very idiomatic. It's something that just becomes kind of first class. It's something you do every day. You don't think about it. You don't write 15 design docs just because you want one of them.

It's just something you do literally every day. Uh, I think, like, since your, your audience is also more generally, like, into, I think, probably, like, listening to what people talk about, I think there's been a lot of talk about our Workflow Development Kit, but also, like, more generally, like, what is...

what are workflows, what are agents, how are they related? Do you use one or the other? I would actually love to talk about that as well but, like, um, obviously, like, in our, in our, uh, conference, we basically introduced just, like, what we hope is by far the easiest way to make your, you know, make your agents something that is easily embeddable into complex workflows and to make those workflows durable, resumable, streamable and so forth.

Yeah, I mean, as, as listeners might know, I have a long history with, uh, workflows, um, at, at Temporal. Um, and I think what's weird is a lot of people are discovering this for the first time. You don't really learn about this in CS classes.

You don't really learn about this in, like, boot camps or anything like that because it's not really a unit of compute and storage that is taught. It's kind of like emergent from, you know, Uber and, like, Stripe and, and everyone else.

I don't know if there's a version of this at Google.

I mean, there is. There's a version of this at every single company that has been doing anything in computer since 1950.

Yeah.

Like, but what's not necessarily the case is that it has been abstracted in any way, right? But, like, when I, when I run a, you know, bank transaction system in 1975, then I invent this, right? Maybe I'm in pure batch processing world, and I kind of avoided it, but, uh, but the reality is that I built this, right?

And so either I use something very productized, which, which obviously Temporal innovated on b- that being a thing, or I use something that's ad hoc, right? So I go and say, "Well, I need, I need some kind of queue that tracks the work, and, I mean, I need a database to store the state at any given point."

And I, I don't know, maybe I write a cron job that makes sure that the stuff on the queue doesn't get stuck, right? And so I... Like, I think what's extremely common in essentially every transaction processing system that has ever created is that there either is an explicit abstraction for workflows in it or someone made one ad hoc, because otherwise the thing just doesn't work.

Yeah, yeah. Totally. Uh, so the headline thing for people who maybe haven't dived into workflows enough is that you can sort of wait and, and resume code, so it- it's, it's as though the serverless function is kind of in- indefinitely long running.

Like, literally you can run an infinite loop inside of your service code, and that breaks a lot of people mental model if they don't really understand that the code pauses and resumes, and you can wait multiple days, and it doesn't matter.

It doesn't cost anything. Actually, I don't know if it doesn't cost anything. Maybe do... I don't know if you charge, uh, for that.

No, it, it literally does not cost anything. Um, so yeah, you can, you can run compute for infinite amount of time. You can then also, whenever, like, one of these steps fails, automatically retry them. Stuff like that that makes things more reliable.

Yeah. And so, like, it, it has a lot of parallels to long-running orchestration problems for agents. If you want to do human-in-the-loop as well, it's a simple task of waiting for... What's the API that you guys have? It's not-

Yeah, so-

I, I wanna say Signals, but you, you have something, like Resolve Webhook or something.

Yeah, I think we... It's a little bit... It's, mm, similar to a signal, but, like, the idea is that you basically, you make a webhook, like an ephemeral one, which is just a URL that you can ping, and then, so the, the realistic flow would be you reach that step where you want human approval.

You know, let's say that you, you write... You get the webhook URL, and you write it to some database. And let's say the user now, they log into their computer- Two hours later, they have a queue of things they need to approve.

That's from the database. They click on one, they say Approve. Now what just happens is that that system now calls that webhook, and then from the m- from the perspective of the workflow that you originally im- implemented, you were now able to await that webhook, and now it resolves, and you can just proceed with the program.

Yeah. It's, uh-

So-

It's very elegant. Uh, I would say that it eliminates some complexity that we introduced at Temporal, and then that's probably for the better. Um, uh, and the other thing I, I think i- it just, obviously as someone who is, like, in this space a lot and, like, I've seen all the solutions, you, you made, you made it open source, which is, uh, another above and beyond thing.

You could have made it proprietary, but you didn't.

Yeah, I think the, the way we think about Vercel, and I think we... I mean, I don't wanna go too deep into that s- tangent, but I think about open source as having ten- essentially three business model.

Okay.

The first one is Red Hat, where you just sell support. You think it's open source. The second one is open core, where you are the only one that gets to monetize it, but everyone else gets to run it if they want, right?

And, and Vercel maybe has not invented this, but certainly kind of is the most successful at a model where you say, "Okay, I have this software library, and it's g- truly open source. Everyone can run it. It comes with, like, adapters for every place on the planet," and that makes it really popular, and then we get a piece of the pie.

And so our strategy is to grow the pie, while what we actually see is that the, our p- our pie piece is a relatively constant size in proportion to the pie, right? And so we can drive the open source project.

And so that's why, like, you know, I don't wanna say we're like, you know, we're, we're in it for the, for the business model, but, like, I think it's a business model that has more winners than the, the alternatives.

I think also something that if people are seriously evaluating for production workloads they care about because, uh, and I ran into this in Temporal, like, these are gonna be extremely valuable workloads that you're gonna put on workflows, and so you wanna, you wanna, you wanna se- you want some ownership, you want some auditability.

In practice, like, who's gonna actually run it themselves? Probably not. But you want the option. You want the check, you wanna check the checkbox.

100%, yeah, and, and I actually do think people will run it themselves, and it's great.

Awesome. So there, that's workflows. By the way, I think the most disgusting is the use of directives, you know?

Yes.

Use cache, use no memo, use Nemo, use whatever. So fun. So fun. What's your... I don't know if you have a take on directives in general.

I don't f- I hon- hon- to be honest, I don't feel super strongly. I do find, in pa- particular the inside of the workflow dev kit, I find the use pretty elegant. The-

It's not-

I could imagine-

Yeah

... I could imagine other, other ways of doing it. I think we, we did post a blog post about, like, all the alternatives we considered.

Yeah.

Because there are some that are, like, that you think about after five minutes, and after two mo- hours of thinking about it, you realize, yeah, maybe this isn't such a good way to do it. But there could be other ways of doing it.

Like, I think we're working with C39 to bring decorators into more places-

Yeah

... which would kind of make this literally the same. The same thing would happen with-

You move the line above the function instead of below the function.

Yeah. So it's not a big difference, but it would become... You know, for example, you c- you could make it part, like, TypeScript be aware of it without a TypeScript plugin, which we already provide, right? And so I literally, I was vibe, vibe coding this thing on Sunday, and I said, "Okay, Claude, you have no idea what news workflow is," 'cause it came out on Thursday, "But here's the docs, and by the way, you know, put it on my site."

And yeah, so it did it, installed a TypeScript plugin for me, so I had, like, the perfect UX, and it, it just worked from scratch.

Wow.

So that, uh, yeah, was, was great.

Okay, awesome. So we can come back to the workflow anytime you want, uh, but I just wanted to keep, keep moving on, on, like, all the stuff you announced. Uh, we should probably also just touch on the AI SDK.

I know you're not, like, as closely involved to that team, but obviously one of the most successful open source projects. I mean, obviously Vercel is very good at frameworks, but I think it was not a given that AI SDK would be a winner because of LangChain, because of Mastra, because of, well, you know, everyone else trying to, to get that spot.

Except that you guys have the perfect package name, so, you know, I think that, that helps a lot.

I actually don't... I'm not sure how well, how much that helps, but it's great. Like, there's a, there's a fun background from what people thought AI was for 10 years ago. But, uh, yeah, I think we, you know, we, we announced version six beta, and I think the, the big...

I mean, it's not really news because, you know, these things are open source and you can follow them very closely, right? But, uh, what it does introduce as a stable feature, because it's already as kind of experimental in, in AI SDK five, is a direct agent abstraction, which so far wasn't there, right?

People would build agents with AI SDK, but they would have to do it in a bit more bare bones fashion. What I do wanna mention is actually, 'cause you mentioned it's very successful, which is true, and I think the, the, the reason why it's successful is because we could s- strain ourselves to be humble about what we know our users might wanna do.

The example I would li- I like to give, like, if when you built, like, a new web framework in 2025, you know exactly what people are gonna do. Like, the, the, the... it's such a well explo- uh, like, explored space.

Who... As the person doing it probably has done it three, four, five times in their life and failed and, and learned from that and, and tried the other things, right? There's... It's so mature. Like, it's the most mature thing.

Even 10 years ago, that was also true, right? That's why Next.js is so good, because when, when Guillermo started building it, Guillermo knew exactly what to do. He knew exactly what the app would be. Almost nothing has changed.

So the AI app space is the absolute opposite. Like, we know absolutely nothing, and we still know absolutely nothing. Things are emerging, but, like, but it's, but we're so early. And so if you put a very thick abstraction, then it's probably gonna be the wrong abstraction.

You have to... So you have to be humble and say, "Okay, I need to stay low level so that this can be flexibly used-

Mm

... as trends emerge," right? And so that's why we didn't have to rewrite AI SDK when everyone went from writing chatbots to writing agents. Because the, we stayed at a level where that- You know, almost look the same.

Right. It composes up.

Just stuff people would enter on top. And I think that's why, that's why it's successful because it doesn't... You know, we didn't say, "Okay, we know what the apps are gonna look like, and we're gonna do this, like, Hollywood principle, don't call us, we call you style framework, where, like, you just have to fill in the blanks.

It's super structured. You'll be happy," right? Like, we didn't do that even though that was, like, so in our DNA. Like, we did have to- ... like really restrain ourselves. But, like, that's why it's successful because it's so low level, right?

And so that's why, you know, on the other hand, you know, that's why we don't have an agent abstraction yet. Like, every other competing library, like, leads with that, right?

Yeah, like OpenAI's-

And, and-

... uh, SDK day one it was like this guy.

Exactly. Master, et cetera. And, you know, I mean, I'm not saying it's bad, and obviously that's, that's more accessible. Right now, you know, you have to understand, an agent's a tool with a loop. What do I do? I use the stream text function and gives it, give it tools.

Okay.

Added, we added all kinds of, like, control already in the AI SDK version five, where you can prepare the step, you can select the tools on every loop. Like, you can, you can, like, do all these things in a pretty advanced fashion.

Half of those other frameworks are built on top of AI SDK anyway, right? Like, so that it's, it forms the, the basis. And so what we're doing now is we're bringing what is emerging as the patterns that people built over and over again into as abstraction into the, into the library as these, as the usages kind of are solidifying.

Yeah. One, one I have interviewed enough agent framework builders and model and big lab model people that I actually find that I can push back on you. Uh-

Okay, go ahead.

It's, it's, it's really interesting because I think, uh, you are saying, basically you're saying we will be at the jQuery, where the... kind of at the jQuery era, right? Like, we are, we're... We don't know what we want yet.

We're building all these, like, uh, tools to make our, the smallest possible things easier. Uh, and then it composes up, and we're just starting the emerge of agents. I would say that the big lab people are obviously the opposite, but they're not, they're coming at it not from, like, a DX point of view.

They are, they're very big model pilled. They, they want everything to go through the model. The reason they want the Hollywood principle of we'll call you is because they want the model to control the tool calls, the reasoning, what have you.

And I feel like there is a s- there's a mentality... And obviously you can have frameworks that do both, but there's a mentality in the big labs, if you work at the big labs, that you always want to give the wheel to the model.

And then for you guys as framework developers and people who are software builders, it's more comfortable to build, like, the, the, the smallest possible thing instead of, like, the, the sort of AGI thing, if that makes sense.

Yeah. I, I actually don't think about it in, in, in those dimensions. I think the, like... 'Cause I 100% agree that, like, people, I think, have to be willing to let go and let the tool, or, like, let, sorry, let the model kind of take control, right, and to, to get emergent behavior, and certainly on coding agents, that works incredibly well.

But that, that I'm, I'm totally on, on, on, on point with, right? And that, that's AI SDK does this very well today. Like, all the agents that I've personally built work like this. That's not to saying the, it's...

But the other thing is, like, how do I now embed this, this into an application, right? Like, the model apps couldn't care less because they're not really building applications. And so that's, that's something that a company like Vercel thinks about a lot.

Like, again, what, what do, what does the developer actually want to express, and, and how do we let them do it? And so, like, I think one of the key things that, that people wanted and still want is they want streaming because these models are slow.

Yeah.

And so suddenly this almost obscure subgenre of programming where people are like, "You know, it's two, five milliseconds. I'm just gonna not ship it," right? Suddenly it's 30 seconds, and you, and it becomes absolutely important. And so we give people the tools to get, to build streaming applications in a way that feels intuitive, right?

And I think that unlocked a lot of, like, value there because that was, like, genuinely hard, and, and we made it easy. And so that, those are kind of the things that we're looking for but that, you know, are not obvious when you're kind of mostly concerned about the AI part.

Fair enough. And, and I think, like, the design space is, has more dimensions than what I tried to simplify it down to.

Yeah.

Uh, just, oh, just one more thing on AI SDK, and then we can move on to, like, the other agent stuff and obviously you guys announce so much. It's so hard to cover. Uh, so Vercel is a, like a house of frameworks, right?

You have so many framework authors, all of them legends in their own right. What's one, I guess, philosophy that you're also applying from your, all your years and all your people who work on frameworks, right, that, that, that is, that is informing you?

Um, I have one, and feel free to counter propose, uh, which is what Sebastian, uh, Mark Booga, who's, uh, obviously the tech lead of React, is used to say, which is have a small API surface area. I feel like that has maybe been, like, not as important or, like, there are other overwhelming priorities, but I just want to get a sense of, like, w- you know, what governing principles really resonate with you.

Yeah. I, I, I mean, Seb and I, I talk about this a lot, right? I think the, I'm often representing the, the kind of enterprise side where it's like, "No, but I actually wanna just control this," right? I wanna be-

Give me one API for this. Just one more, bro.

I, I, I wanna be, like, you know, in control, and I wanna be able to configure it, and I wanna, like, define the, the contr- the defaults the way I see it. So I think, I mean, th- but it's good to have, like, tension around these things.

I think the, the thing that, that, like, coming down from Guillermo is just the absolute founding pr-principle of Vercel is that we never give you an abstraction that we haven't used ourselves. Like, dogfooding is ultimately the thing, right?

Like, that, like, a- AI SDK was extracted from v0, and then we built it, and we kind of diverged a little bit, and then we took on the ch- the, the substantial work to bring back v0 actually fully hosted on AI SDK, and then we learned back, and we made sure that migration isn't too hard, which the users has appreciated as well, and so, and so forth, right?

And so there's this, there's an, this constant feedback loop where if you don't have that, which is like this sounds, like, so obvious, right? But the, the reality is that framework builders are usually not application des- builders. Yeah.

And so they built ivory towers that when they're hyper geniuses or they get lucky, they're happen to be good. But like, but, but if you want to do this in a reproducible fashion with a high hit rate, then the only thing you can do is you, you have to, you have to try that stuff out yourself.

And like that's what we do every day.

I, I, I really like that principle. Obviously, a good idea. It's just very hard to practice in, in, in real life, obviously, because when you're a maintainer of a framework, lot of bugs come to you and y- you know, they pile up and you have to, uh, spend some time working on framework level issues.

Happy to move on to Vercel Agent and maybe the Agent on Every Desk program, which, you know, I think you're, you're kind of, uh, also championing. So like, yeah, let's talk about the use cases like that you guys use, uh, internal agents at, uh, within Vercel and, um, what emerged.

Yeah. L- let's, let's structure this two ways because I do think there's a difference between kind of the agents that we're building internally versus the stuff that, that's, you know-

External

... essentially a Vercel product, right?

Yeah.

Uh, and which you can, which you can use today.

Okay.

We, we've been-

I thought they were the same thing.

No, they're not the same thing. That's actually, it's-- that, I think that's quite, quite important. Um, like we also, we're distinguishing between like agent as a service, right? And so the, the Vercel Agent, that's what it is, right?

It's ultimately a, a, a, an agent as a service product similar to, uh, you know, Codex in the cloud or the, the Cursor agent. Like not a, not, not as in like it's the same product, right? But as in like these are things you, where you go somewhere and you say, "I would like to use this agent."

And then, I don't know, maybe you give them a credit card and it works, right? Which is different from the stuff that we run internally. But let's talk about the Vercel Agent for a second. Like we, we've been...

Basically, I think our strategy overall is to have an agent that helps you build applications on Vercel. This is, you know, there is some overlap with coding agents, but like I think the, the thing that's unique about, about the Vercel situation is that because we have, we have your runtime data, we have, we see your error logs, we know where the preview deployments are, that they always will exist.

We know how to start the dev server. We already have the secrets, so there can be like a quite, quite integrated solution for something that otherwise can be quite hard, right? If you have a onboarded, for example, I mean, you have onboarded Devon a few times, but you probably have, you know, been in that situation where it's like, feels like onboarding a junior employee, right?

And so some of these things, like if you're within the Vercel ecosystem, become much more simple. And so in that world, we, we a- we've, you know, we've, we've been chipping away on different things, right? Like a while ago, shipped a code review agent, which I think is, is, is really good and well integrated.

And the thing that we announced last week is our broadly DevOps agent, which is actually tied to our anomaly detection system. So whenever there's anomaly that we detect on your, on your production side, um, it kicks off the agent and the agent does an investigation of what's going on.

From a technical point of view, what this agent has, it has several tools. It can ex- it can make any observability query against your project. So as a query builder, it can execute the queries. It has a way to read logs, obviously with another, with, with queries as well.

So what's, what's really magical is that it, it's just very good at this. Like it, like by the time you click on the anomaly, it will like almost all the time just very precisely tell you what happened. It shows you all the graphs that it looked at.

It just like, I don't know, man, it's just so much easier than doing it yourself. Like it, it takes away certainly minutes of work. But I think what I'm actually very excited about, and I think this is an overall pattern that, that, that we see with, with agents, is that there in many situations is this what we in search call a recall precision problem.

And that also happened with anomaly detection. With anomaly detection, you c- you have to tune it, right? And you either tune it to be very aggressive and then it fires and you, and the pa- worst case, it pages you in the middle of the night and nothing was wrong, right?

Just a team in Asia sent a newsletter. And, and so the traffic went up, right? Or you tune it not, um, like aggressive enough, and so you miss events. And with an agent, you can just say, "Okay, I'm actually gonna have this tuned very aggressively, and I'm not waking anyone up.

I'm, I'm telling the agent," and the agent can take two minutes to run. It's, um, actually fine with that because no one would have reacted in that amount of time in a very reliable fashion. And now it can actually look at a time series, can look what happened.

It can look at the IP addresses that, that are making the request. It can look at the type of error messages, right? And it can make a, like a call whether to escalate to on-call and wake someone up or to say, "Okay, um, this is completely fine for someone maybe to take a look next day."

Um, which is I think the perfect decision for agents to make. And so that's, that's something I'm very excited about, that you have this like essentially coworker that has no sleeping problems in the loop and, and they get woken up instead of you.

Yeah. I think, uh, the dream of AI SRE has been a long time coming, and I'm actually on the record at the start of this year, I made a podcast saying like, "Oh, I don't think anyone's gonna do AI SRE."

So I'm ex- very excited. I haven't tried it out personally. I, I have seen you tweet about it and I, I think yes, obviously, that is the goal. There's the dream. We should not... We should have, make Bryan Johnson happy and have good sleep.

Yeah.

But, uh, you know, we're, we're, we're not, we're not exactly there yet. And I think like the, the question is really fold, which is time series analysis is not exactly within the distribution for language models. There have been a lot of people doing like time series models.

There's a lot, there, there's a deep field of anomaly detection, b- which is basically what you're doing. And, you know, there's a question about like, is this a solved problem? Or, you know, like, how much can we trust it?

Uh, and then I think-

Yeah

... the other one is like aligning the, the human preferences, right? Like sometimes like I don't know until I've seen a few examples of like, oh yeah, this one you should wake me up, the other one you should not.

And then pretty much like when I solve the problem, you know, it goes away. So like may- the next problem, you're always fighting the last war in, in, in SRE.

Yeah.

So I, I give you, I give you a bunch of things you can take-

Yeah

... wherever you want.

No, I think the, the, the, the, like you have to try the product. It works, it works really well. The, the, so we don't do the anomaly detection in the LLM, right? The anomaly detection is as, you know-

It's a separate part of the-

You basically-

... Vercel product. Yeah

It's a completely separate part. It's our, it's a pipeline that works on our time series database, right? And, and launched independently of this. And so, but once you have this, the... So- Our experience is that, you know, if you give this agent a tool that does queries, it's really good actually at, at digging into individual parts of the time series.

And the other thing it has access to is logs, and logs are actually, they're just text, right? So if you say, "If you see from the time series, like, what, what happened, can I somehow s- figure out, like, what this is?"

And then you head over to, to logs and get, uh, do a deep- deeper dive. Now you're kind of more in the, in the world where the, where the model is comfortable. Um, one thing that we don't do today, but we'll do in the future, is that we also give the model ac- that particular agent access to your source code so that it can, first of all, figure out k- like, what does the error message mean, right?

And every time you can't actually make a PR to just fix it, but every so often that'll be possible, and so it would, uh, would then also do that.

Which is why I think, like, people like Datadog and Sentry are trying to do that, obviously because they're observability platforms, but they never own the code. And so, like, they, they're always limited in what they can do, you know?

Like, and-

100%. But also, like, I, I mean, um, I mean, uh, what I want to qualify, where, where I'm so happy about how, how it works is like, you know, you know, it's just a small part of the overall problem, right?

Like, we're actually not here to, like, build the AI SRE that replaces that job function. Like, that's another part of what I feel pretty passionate about, that it's at this moment there, like, agents are both extraordinarily effective and still very ineffective.

And you have to find, you have to find the right problems, and then when you find the right problems, they are super magical. And if you wander beyond, then they don't work. And so, like, that's the magic, right?

And what we see is that getting triggered on, like, a bunch increased error rate works well, and certainly making the decision what to change in the firewall, I would not let the agent do that yet, right? Like, it, it'll be...

It's too, it's just too dangerous.

Or do DNS migrations.

Exa- yeah, exactly.

That's-

That'll be-

That'll be, that'll be AGI. Yeah, it's, uh, i- it's interesting on, on all that stuff. Uh, yeah, I mean, I think that there's this, there's this growing consensus of, like, where agents are doing well and where agents are not.

I think, you know, for me, like, meeting notes are solved. You know, like, simple f- like, UI changes are solved. I guess, like, what, what else, you know, in that list of, like, things that are solved that, and, like, are reliable every day, what, what do you put in that bucket?

Yeah, I think the, the, what... So I had a section in my keynote about this, and, like, it boils down to this question. Like, basically where the idea is that you go around your company and you ask people, like, "What do you hate most about your job?"

And I, the, I really think it finds the sweet spot because it finds problems that are, they're boring because they're tedious and repetitive, but they're, you know, they would have already been automated if they were automatable without an agent in many cases.

They often, like, do require some kind of text, like, mini judgment, et cetera, right? So, like, people, people do these things, and so that, that's, I think that, th- the, that question that, thus yields a sweet spot where, like, the s- the, the problems are probably easy enough for, for a current generation agent to handle, and they're also often very high business impact because this is actually s- pretty substantial part of people's jobs.

Again, that's why they hate it, because it's like, takes so long. And so we, we ended up at our conference talking about three agents that we built internally, two of which we open sourced. Again, so people have a starting point because these are custom agents.

Like, they're not the, they're not software as a service things that you just install. So these are custom agents. But, like, the, the first one is one that handles processing of our incoming, uh, contact sales requests.

Yes, lead qualification.

Right.

Yeah.

And that, we, uh, there are obviously a lot of startups in that space, right? Like, so I think that's, that's very much in the, in the, in, like, the soft case, where, like, you give it a tool for LinkedIn, you give it a more generic tool for Google.

You give it a bit of an objective and, like, de-qualify, like, how, what do you, what do you care about, right? You give it a, like, a way to analyze, "Oh, this is really a support request. Okay, hand it over to the support team," right?

Like, there's, you, there's a few cases like that. It's not so complicated. So that one is, I think, is, is perfect, and we open sourced that so people can make their own. The other one that, uh, we falls into a similar category is abuse analysis.

So we get abuse re- like, reports. And so in this case, it's really the agent essentially doing the pre-work. So we still have a human person, like, look at the pre-work and then make the decision what's to hap- what happens in the end.

But what were they gonna do? They were gonna go to the reported website, right? They were gonna go look at the account and figure out how, what the age of the account is, and they were gonna see if they paid their bills and, you know, whatever, right?

Like, the, there's a s- there's a list of things they will do. And so what you can do is you can just make it so that when they eventually look at the ticket, it already has all this information.

And if the page looked like a Facebook login page, then a current-day LLM is also able to, to do that, like, make the adjustment call, and then you just quickly check if it's okay and then, and you move forward.

Yeah, amazing. Uh, and I think there's one more data ana- analyst agent. Um-

Yeah, yeah, exactly. I mean, this is also something we, we just wanted for ourselves, right? Is to do-

We have one, we have one too internally, yeah.

Yeah, like, um, and I think it's, it's... That one, that one's also open source. I think the, the idea is that, yeah, you wanna, you wanna ask questions against your data warehouse, and we were very unsatisfied with the current solutions because they ultimately didn't have access to enough information about the data model.

And, like, we are not promising that we have the magical tool that you give you, give you your, your prompt, and it spits out SQL, just access your s- your schema. But we get, we essentially developed just a structured way to document the semantics of your data so that then the agent is good enough, right?

And so we've been using that internally quite successfully.

Amazing. So yeah, the, for, for those who don't know, uh, Vercel also has a Agent on Every Desk program where you can sort of reach out and... Uh, it's, is it, like, a forward-deployed engineering situation where, like, you have, like, a SWAT team that comes in and helps people?

Yes, I think that's... But it's also not appropriate for every company, right? So I think the, like, my take is that it's, if I'm a large company, I have a lot of efficiency to gain, but it's also quite daunting to, to ship my first agent.

And so something like a forward-deployed engineer, which we are doing indeed, like, does help quite a bit in that scenario. I think, like, as a startup, I don't want a forward-deployed engineer in my office. I just want to see the open source project and, and feed it to Cloud Code, and then give it my own problem and say, "Build me something like that, but, like, here's what I want different," and, and that should also be successful, right?

Um, so I think we, we are kind of, with this particular program, really going for just unblocking people who feel that they just don't know what to do. They f- they hear the hype, right? They don't know how to pick the right project.

We talked about this, like, how do you actually find the project that's gonna be both successful and high impact? And, and then secondarily, okay, now that I have the project identified, how do I How do I do it?

And I think that their, like, forward deployment is effective. It's something that I, you know, as a framework engineer, I've thought about all my life, where, like, you need to have someone kind of guide you the first time you do something, and then the second time, maybe, you know, you build an agent yourself.

So we, we don't wanna stay there, right? We want... We basically sign contracts with companies saying, "Okay, you have to commit to building three agents, and if you do, we're gonna help you. Like, we're gonna build the first one for you.

And then the second one, we are gonna be there essentially by your side," you know, maybe not literally, but, like- ... on, on a phone rotation, right? And, and for the third one, the assumption is that you actually don't need any help anymore.

And you can still reach out obviously, but, uh, but if everything went well, now this is a company that's empowered to build its own custom agents.

Yeah, you, you know, you, you're gonna be helping your biggest customers, and obviously that's gonna be leading to a lot of good products, ideas, right? It's kind of dogfooding at scale, uh, with your, the people in the Vercel ecosystem and not just Vercel alone.

100%. You just discover things that probably the 500-person startup- ... would not have discovered.

Um, I think, you know, one, uh, one last thing, you know, just to, to leave, uh, off the wh- the whole topic, and we can, we can add in anything on the sh- on the Ship AI side. Actually, uh, anything else on, on Ship AI that you really wanna cover, get a s- soapbox about?

I don't know if we covered everything.

I think, like, one, one point that we, we, we haven't talked about is that we, like, Vercel as a company just has been investing in Python quite a bit. I think maybe the audience of this podcast might also be excited about this.

AI SDK obviously currently is, is a pure play TypeScript system. We d- we do find Python really interesting. What we have done over la- the last weeks is we d- have shipped, like, zero-config support on Vercel for all the popular Python fram- frameworks like Flask and, and FastAPI.

Zero-config means that you kind of get the Vercel experience where you throw your stuff over the fence and, and we're gonna run it for you, no questions asked. And then just as another thing, like we have, for example, also shipped a Python SDK for our API just to kind of, again, show that we are engaging with the, with that ecosystem.

It's very obviously something that, that is in a way n- new for Vercel, but, like, we have been making hires and, and infrastructure investments to, to, to make Python really well supported on Vercel. It is also, uh, on our Fluid Compute program, which gets you, for example, active CPU pricing.

So you get to run Python in production, and you only pay when you have compute, and otherwise it's, it's free, which is very nice if your back end takes 30 seconds to respond 'cause it's a AI model.

I w- I wa- I was also gonna make the observation that workflows are very nicely meshed. Like, it's almost like there's this fate that you're driving towards, uh, you know, workflows that you, you needed Fluid Compute in order to do all these, uh, fancy things or at le- at least make it easy to ship this kind of stuff.

Right. Like, I think there's some overlap, right? Like, I think the, the, um, Fluid... Like, with workflows, literally nothing's running between steps, right? So it's literally free. That's also why it's free, 'cause there's literally just nothing happening. Versus Fluid Compute, which, which does have the same property, except it's, it's more agile, right?

Like, usually in a workflow, I don't know, you're doing the FFmpeg thing, then you're doing the AI model, right? Like, you're not talking millisecond latency. There, there's some overhead on each step, versus Fluid Compute being like the VM being literally on, which does operate on a different kind of level.

Yeah, excellent. And then on the Python thing, what happened to always bet on JavaScript? You know, like, isn't that the Bret and Ike line? Basically, I think, I guess, the, the broader, the non-cheeky question is are we, like, 50/50 Python, JavaScript now?

Is that the future? There's no particular language that will win, or, you know, do we, do we not have an opinion?

I don't think I have an opinion. I saw today that, that, like, TypeScript is now, as of, as of today, is the biggest language on GitHub, right? Which last year you had to still cope that TypeScript and JavaScript is, like, drastically bigger than Python.

Honestly, I, you know, ob- obviously, I don't really care. I think both communities are very relevant, very large, and we are investing in supporting them. Like, the way Vercel's infrastructure works, because we essentially just run VMs, it is not actually hard for us to support Python.

We will eventually do PHP and Ruby, but we also, I think, care a lot about the details. And so the, essentially the reason why we haven't done it yet is because we do invest substantial amount of time to make the DX actually good and feel native to the ecosystem.

And so that's why it's, like, technically easy, but, like, in practice, actually very, very difficult for us to support these, these things. So we're, we're taking it careful, but, like, we are, uh, you know, there's no technical restriction on our system where we couldn't support all of these dist- different ways of running code.

Yeah, totally. I mean, I, that's a very fair response, and the fact is that I think when you're a serious AI cloud, you have to support Python. Um, so there's no, there's no way-

Exactly

... there's no way around it. Uh, okay, I wanted to zoom out. One other thing that we care about in AI engineering is AI leadership, which is leadership of AI engineers, and obviously your role as, as, as CTO has changed a lot since you joined.

Uh what are some, I guess, leadership principles that you've had to, I guess, create from first principles? Uh, when, when you're le- obviously you're in a very unusual type CTO role where you're in, in an infra company, you have frameworks, you have apps.

What comes to mind when I, when I say how has the CTO role changed for you?

Yeah. So I joined Vercel, like, a little bit less than three, uh, four years ago, and that was definitely, that was before ChatGPT. I, I mean, I came directly from Google, and I, I think I had some insights there of what was happening and...

But, like, but Vercel certainly wasn't living in this world. That was kind of the tail end of the last really big crypto wave and, and, and, you know, and everything else, right? And so, so then suddenly the, the AI revolution happened, and the thing that I definitely had to w- like, work through is, like, how do I transform the company into something quite different?

And I think the, the solution that we have come to, like, feels really good, and I think there's a lesson there to be learned for companies that haven't done this transition yet, is that you have to do something that feels native to your company.

And so the, the two big bets that we made early on, one being v0 and one being AI SDK, I think they felt, like, native to Vercel in the sense that v0 was especially originally designed as a, as a tool for, for making- ...

web pages, like the full stack stuff came later when the, with the Sonnet models. And originally it was very much a web developing tool, and the SDK was a framework for building AI apps, and because we're a framework company, it felt really native.

And I think you, you have to, like, make that, you know, you have to be honest with yourself, like, what product, even if I do have to, like, change to building something else, does kind of extend naturally from what I'm doing rather than being very kind of, like, being something entirely different that no one believes me that I would be the right place to buy that from.

That, that, I think that's just generally timeless advice. Have manager-to-IC ratios gone up? Do you know what I mean? Like, are-

Oh, that's a really good question. I don't, I don't track that very closely.

Okay.

Think that-

It's a thesis of mine, and it's coming up.

Um, um, I... Yeah. I would be supportive of it going up. It's a lo- it's a big question, like the role of the engineering manager. I have two reports.

Nice. Nice.

Right? So, like, the, that's the way we're organized. Our... We've... Are strongly-

I mean, it's a privilege to the CTO that you don't, you know, VPE is usually the people, people manager sort of-

Yeah, like, but it, but that's actually not, not the point. The point is that it's, like, the basic- the really, the one thing that Google really got right is that it has very strong ICs. It has IC levels all the way up, right?

There's, um-

To 11

... Sanjay at level 11.

Yeah.

And, and the... And so we're doing the same thing, and so we had to have someone as an IC at, at the, at the, at the top level, right? And so, um, that's why I think th- this is, I think, another thing that, that you have to be willing to, as a company, to live in this world where you don't make your strongest engineers have the choice of, like, not making more money or becoming a potentially very bad en- manager, right?

'Cause, like, there's little correlation from being the best engineer to being the right manager, right? You could be, but sometimes you have, like, these moments of, oh my God, like, glad we made this change, but, like, that's a, um...

it's essentially crap shoot whether that happens.

Yeah, that, that's the other thing that I find at... Obviously, Peter principle applies, which is the, the, the principle of promoting people to the level of incompetence. I think the other interesting thing that people are finding is that PMs and designers are now starting to contribute more to code because they feel they can just vibe code something, and maybe that's good, maybe that's bad.

Uh, I don't know if there's standards around this that have been established inside of Vercel where there used to be clear code owners, and now because we feel that we're coding agents, we, we can just, we feel we can do a lot more, and maybe that's caused some politics somewhere.

100%. Like, we wanna be at the avant-garde of doing this, right? Like, as makers of v0, like, we, we highly encourage all our employees to, like, contribute code. I think one thing we haven't talked about, and we might have time to go into, but, like, we are very deeply working on a way to build apps that follows the threat model that assume that the developer doesn't know what they're doing, and also they're using AI that also doesn't know what they're doing.

And so I wanna be able to build an app that is secure even if the developer is incompetent, right? But today, that's not the case, right? Today, I assume my developer-

Everyone has trust

... is competent.

Full trust.

Right? Like the... And, and, and so, um, and so we have, I think, very strong progress there, and we're working with large, m- lots of large companies that have-

Yeah

... data, for example, where you have the idea that you say, well, auth, auth cannot be part of the app because they're, they're not gonna get that right, right? So auth has to be extracted from the app. In fact, which data you can see, they also cannot be under control of the app because, again, you're gonna get it wrong, right?

And so we're, we're building these systems that, that try to have a minimum amount of security, fully independent of the quality of the app, and I think that is part of the future of how people will build stuff.

Okay, and this, this is-

So probably a good idea anyway

... this is Vercel level, not so much v0 or Next.js level. Like, uh, in front of-

Yeah, i- in a way, in an integrated fashion, right? Because-

Yeah

... I'm gonna build the app in v0-

Yeah

... and now I wanna deploy it to my fellow coworkers. Only the right people should be able to access it, and when they access it, they only should see the right data.

Yeah. I, I think this is very exciting. Uh, I think, like, the closest people that come to this is WorkOS, basically. Uh, and uh, yeah, I think there, there should be more, like, nat- a- agent native, if we can call it that, infrastructure that, uh, you know, lets people build and vibe code safely, uh, which we all, we all wanna enable them.

Exactly.

They just cannot be trusted. This is very insightful. I actually, I'm very excited to see that. Uh, yeah, ping me when it comes out. Uh, but otherwise, thank you for spending the time to, to recap Ship, recap all Vercel stuff.

Uh, just, I cannot imagine a better person to, to talk to. Uh, you, you're so, you're so generous and friendly and en- engaged. You know, I definitely don't feel like all CTOs are as engaged with, like, regular developers as you are.

I'm trying my best, and I'm spending too much time on X. But this is super fun. Thank you so have show, so much for having me.