⚡️Monty: the ultrafast Python interpreter by Agents for Agents — Samuel Colvin, Pydantic

Welcome back to the "Lean Space," uh, live studio. Uh, we are here with Sam Colvin from Pydantic, uh, who's recently launched Monty. Welcome.

Thank you so much for having me. Yeah, it's great to be back.

How have you been since the last time we saw you? How's, uh, Pydantic and Pydantic AI?

When did we last speak? Like, about a year ago, I'm guessing.

Yeah, exactly. Yeah. You just launched it.

Yeah. Um-

Uh, yeah

... it's-- Well, I mean, I don't wanna indulge in all the cliches of, like, it's been crazy-

No

... yada, yada, yada, but-

Brag a bit. Brag a bit, yeah.

But I mean, no, I, I meant as like everything has changed. Not, not so much... I mean, sure, we've had a great time, but, like, everything has changed in a year. I mean, at la- if we had spoken this time last year, and you told me...

You know, I, I basically was slightly laughing at anyone who said that they weren't reading all their code, and now here I am building this, like, if you wanna be pejorative, slop fork of Python, mostly with AI. I, I think it's incredibly powerful.

I'm proud of it. But, like, this is a crazy world where you can sit down over Christmas and write 30,000 lines of Rust that, that is actually powerful and useful. But yeah, the, uh, loads of stuff has changed.

I think, uh, Pydantic Logfire, our observability platform-

And what does that mean for observ- observability platform? What does that mean for, you know, an, an AI native observability platform?

So we're weird because we're the only... I guess we mostly consider ourselves against the other AI observability platforms, BrainTrust, LangSmith, Langfuse, Arize, those guys. But we're actually full observability, full open telemetry. You send us logs, metrics, traces. Our pricing is a bit cheaper than other general observability rather than 50X more expensive.

But we do lots of the AI stuff, the evals, prompt playground, LLM traces, all that stuff. And then, and I suppose two people mean two things by AI observability. They mean observability for AI and AI for observability. We do a bit of both, in particular because Logfire lets you write arbitrary SQL to query your data.

I don't think we've done anything particularly special in our MCP server, but because the AI can just go write SQL, it's way more powerful than I think on many platforms. We basically have, like, the AI SRE experience without having to do anything, 'cause the AI just goes and writes SQL, and it can, it can find a bug, or it can show you the five slowest endpoints by P95, or it can go do some random investigation on some attribute, which isn't possible if you don't happen to make this, like, weird esoteric decision back in 2023 to allow arbitrary SQL from your users.

Yeah, that is, seems like a nightmare if you allow that. So you started Monty over Christmas. What's the inspiration? What's the origin story?

So I actually had a, had a, like, very early version of this that I had done a couple of years ago and had completely abandoned. And then I spoke to maybe four different people at Anthropic, and each of them independently, I, like, did my standard thing about how important type safety is because I, you know, I think type safety is important for humans, but it's critical for AIs.

And I say this to people all the time, and some of them nod, and some of them don't nod. But, like, these four people I spoke to at Anthropic each independently said, "Well, yeah, type safety is super important if you're chaining tool calling or if you're, like, using, using code for tool calling."

And when the first f-fourth of these people spoke to me, I was like, "They're obviously thinking about something." I find Anthropic hilarious 'cause they're, like, the most secretive company, and yet everyone gets excited about whatever's going on inside Anthropic, and suddenly everyone-- and they all hint at you about whatever it is that's going on at the moment.

Um-

It's hard to resist, you know. But it also means you're speaking to the builders, not the marketers, right? The marketers won't say anything if you don't.

Yeah, yeah.

Yeah.

And so I kinda started building it, and then sure enough, in whatever it was, December, they came out with a programmatic tool calling piece, and then there was another one on, uh, using code to call MCP servers. Cloudflare kind of invented code mode or at least invented the term and have, like, pushed it hard.

And then actually, I was speaking to-- I spoke to an investor who had been looking at the sandboxing space, and he said his guess was that 70% of sandbox invocations are basically just, like, tool calling or glorified tool calling, whether it be, like, to render a chart or do a calculation, stuff that doesn't actually need, like, full computer use, but where writing code is very, very powerful.

So yeah, Monty attempts to slot in between simple tool calling, which is very safe and relatively easy to implement, doesn't require external infrastructure, but isn't that expressive for the LLM. And, like, sandboxes, which are much more expressive and do much more powerful things, but they require-- Well, the cost is actually the least of it, right?

It's the, it's the complexity of setup, which is not too bad if you're a startup and you can just go and put your credit card into Modal or E2B or Daytona or whoever. But if you're a massive financial institution and you need to self-host everything, those are not an option for you.

And so just add tool calling to add code mode to my agent is, like, basically not, not possible for them. Obviously, also true for people who can't afford it, but, like, from a customer point of view for us, the, the power is on the, the enterprise side where just use a, a sandbox is not a particularly easy solution.

And then the other massive win is latency, because if you can have a Python interpreter that runs inside the same process, you're looking at a boot time. I mean, in a hot loop, we can run-- we can go from code to execution result in under a microsecond, in, like, eight hundred nanoseconds.

In reality, it's like single-digit microseconds to run code, or single-digit microseconds to run the next step of a REPL, or single-digit microseconds to call a function on the host. Whereas if you're, like, creating a, um, Daytona sandbox for me was taking one second.

Obviously, it's not as bad as that for, like, resuming, but the, you know, these are, these are big differences in terms of time.

Yeah. Uh, I think this is a good point to bring up the, the screen just to show. I, I no- I noticed you have a really nice comparison table, and I like to... You know, obviously, it's a Python interpreter, so it can run Python, right?

I don't know what the, what else you would demo there.

Yeah. I, I can share my screen now, and I, I can just show that table and talk to it. And I'll be, I'll be blunt about the fact that there are trade-offs. It's not-

Yeah. I, I, I wanna see, like, what is the design space, right? Because, uh, this is all a design question. So this is the, the GitHub. Uh, obviously six thousand stars and, and still growing, which is in-incredible for an open source project.

But you're very good at that. You're-- You have a well-established reputation in open source these days.

We're generally better at getting downloads than stars, but yeah, it's definitely stu-struck a, struck a level of interest in people.

You track downloads for this?

Uh, yeah. I mean, it's nothing yet. I think it's, uh-

Okay. Okay

... I think we were, like, twenty-seven thousand downloads last week, which is obviously a nice number, but it's not-

Yeah, yeah.

Um, those of us who are maintaining Pydantic, it's a tiny number.

But anyway, so, so it's like, it's kind of like, you know, this is, to me, like a logical progression from type safety to just overall safety, I guess, of a code execution. But, uh, this is a crowded space.

The, the first thing that someone who's familiar with this mentions is Pyodide, right? Because you're, you're like, "All right, you want a WebAssembly sort of lightweight Python interpreter, you'll go to Pyodide."

Yes. And indeed, we did that. We made the classic mistake of using Pyodide. To be clear, I'm a massive fan of Pyodide. Hood, who maintains it, is a good friend of mine. I have nothing but respect for Pyodide as a project.

It turns out it's a very bad way of running Python in-- uh, uh, like not in the browser. So we maintained a thing called mcp-run-python, and one of the impetuses for this-- for us to build this is people just kept reporting security vulnerabilities with mcp-run-python.

And basically solving them got harder and harder because sure, Pyodide runs Python in, um, in, in WebAssembly, but WebAssembly is not inherently isolated, so you can't run it with Node. If you run it with Node, you can import the JS module and go and run arbitrary JavaScript code to access the host.

So you have to run it inside Deno. Now Deno, you can go and set restrictions on what, what file system, what networking it's allowed. But then you have to expose a bunch of files, because if you want it to be able to download packages, uh, it has to be able to write to the node modules directory.

Even if you don't allow that, Deno does not have any way of controlling memory. So even if someone can't run arbitrary code, they canoom your machine as often as they like. The other problem we had was someone pointed out that although you can't escape the Deno sandbox, you can go and run arbitrary code within that Deno sandbox and basically taint that server.

So every single invocation, you basically need to kill the Deno sandbox and create a new one. So your latency is actually worse than a full-on sandbox. It's like, yeah, I got two point eight seconds here for running basically one plus one in Pyodide.

You might be able to improve that slightly. You can work around some of the security problems, but like fundamentally it's pretty heavyweight. And of course, even if you get all those sources, you need to install Deno, then you need to download Pyodide.

You're looking at like, I think I might have a number here. Uh, yeah, so Deno is fifty megabytes. The Pyodide package is twelve megabytes. It's not trivial to just go and get it running. And one of the nice things about Monty is because it's just a single Rust binary ultimately that you can install with PyPI or with npm.

There's act- And there's actually PRs to have support for Dart and Kotlin at the moment. It should be very, very simple to install. Ultimately, you can run it anywhere where you can run Rust.

Amazing. What, what trade-offs do you make?

So the, the biggest downside of Monty is that it is not full CPython. We are implementing it all ourselves, and so I think I have some comparisons up here of what, what you can and can't do. We have a few standard library modules like asyncio and data classes that we support little bits of, but there's no thir- support for third-party libraries just to be installed.

There never will be directly, as in you'll never-- we'll never be able to speak the C Python ABI and like install Pydantic or install NumPy or something. You can't yet even define classes in, in Monty. You may-- We may at some point make that work.

We don't support m-match statements yet, but we probably will. Like, we can't just go, go use the standard library, so we have to work out which bits of the standard library we wanna manually go and implement.

This sounds like an enormous task, and also kind of like, uh, self-sacrificial that you don't-- This is a Pydantic runtime that doesn't allow Pydantic.

So I think... I, I, I hear you. I hear you. I think there's a fair bit of that. So, so what's, what's super powerful, I think we have a, like, very trivial example here. Uh, again, this, this is a kind of like joke example of running the, like, standard agentic loop within Monty.

But the point is you can call, in this case, call LLM, which is a external function on the host, which you can just go and call. So you can do-- you can make a network request by registering a fetch method.

You can do some Pydantic validation of some JSON data by basically having an external function call. You can do all of that stuff. It's just that those, those external packages don't run within the runtime. I mean, I, I think the other thing to say is like I, I wanted to implement the, like, twenty or so built-in functions available in Python.

I basically said to the LLM, "Go and implement all the standard library functions that you fi- that you need to implement and you need to use regularly." Did a bit of planning, and it sets off, and it goes and runs for two hours.

And what would've taken, uh, an experienced developer weeks to do was done in a couple of hours. My take is that there are four, four things where if you can, if you can cover all four of these things, LLMs are not like three X faster or five X faster.

They're like a hundred X faster. Internal implementation is well known to the model, as in it knows how to, in this case, implement a heap, uh, implement a bytecode interpreter. If the external API is well known to the model, you don't need to explain how Python should work, what the external interface should be.

It just knows it in its soul, in its weights. Thirdly, unit testing is really fucking simple. You're just like, "Does it match Python?" So it can almost Ralph itself into, into parsing, right? You just say the traceback needs to be bytecode, you know, to the byte identical.

And lastly, you don't have to bikeshed at all about what the, what the interface needs to be. You don't have to bikeshed, bikeshed about what the error message needs to be when you add a string to an integer, or what happens if you have the following sequence of arguments.

It's all just defined by Python, and so there's no need for the humans to go and argue. It just works. If you cover all four of those things, that's why everyone is, like, basically cloning Redis right now in Rust.

You know, that's the kind of meme right now, because those four rules all apply really well. There are loads of it, you know, the LLMs know what they need to do internally, they know what they need to do externally, the unit tests are easy, and there's no bikeshedding about the API.

If you can do those four things, like tasks get, like, I, I mean, you know, I would say ballpark 100X faster. And so, you know, I think there's a, there's a PR on Monty right now, which I profess I haven't actually read yet, that implements like 50 functions in the math module.

I don't know anything about this, this PR. I haven't read it yet. But the point is, someone has set off an LLM, and it's added 800 tests, or maybe that's the, the full test existing, and I need to look at it.

I'm not saying I know that much about the implementation. Other things-

You have a, you have a nice, uh... What, what is that? What is that orange thing at the top? What... Is that a chrome extension?

So it's a... It, it gets inserted here. Basically, this started off because one of the OpenAI co-founders created an issue on Pydantic, and we just closed it and said it was wrong.

Shit.

And so we have this that, like, injects itself and tries to summarize someone, and it, it gives them a, like, brutal score of one to five-

Yeah. Yeah, yeah

... on how important they are. So we basically don't go and like... Uh, when Dario next creates an issue, we're not just gonna like say, "That's dumb," and close it. Anyway.

Well, so, so, uh, I, I think there's, uh, there's more scope for, for AI reviews. I have worked on an AI review tool. There are many, many others, like, there. Now I wonder if you find this, those helpful at all.

We-- So in Pydantic AI, the guys are using Devin quite a lot, um-

Really? Okay

... and actually find it useful. We, we tried quite a few. Nothing else worked, but Devin... Well, not, not nothing else worked, but, like, having gone through a few different options, Devin seemed to be the best of the, like, off-the-shelf ones.

I think there are-- We actually now have our own one trained on basically all of our reviews that is, that is running alongside it. But yeah, I mean, like, if you look, I mean, if you look at the, for example, the date time implementation, which is here somewhere.

And I think date time is already merged, but, like, the date time implementation is an awful lot of code, but, like, the point is that, like, the API, the, the AI knows exactly what it's doing, and so it can implement it pretty, pretty quickly.

Yeah, I mean, like, uh, I've reviewed this, this a few times. I need to get on and merge it. It's a big PR. It's four thousand plus. But like, you can, you can go and look at the tests, and ultimately you can be like, "Yep."

"Sure enough." Tests in... You, you know, it's basically gonna go through and

test all of the stuff. And of course, the point is that every single one of these tests just runs with CPython and Monty and checks that the output is exactly the same. And so you, you know, these are the kinds of tasks that are much easier.

Now, I agree with you that implementing the full standard library would be a, like, a hellish task. I don't think that that's necessary. I think we need to do like three or four more modules, and you describe-- You just say to the AI, "You don't need to...

You can only use the following," and it's amazingly good at doing it. And for an enormous number of tasks, it just makes an enormous difference in, in terms of what's possible, in terms of how efficient the models are to, to perform the task, to have, uh, like this code mode available.

Yeah. Amazing. I think it's interesting how little moat software just generally has. 'Cause you're-- This is Pyodide. Pyodide's been around for, I don't know how many, 10 years? May-maybe less than that. But, you know, like, Redis has been around forever, and like, here we are trying to rewrite all these things on a whim.

And we can. Like, we, we actually can. Like, this is absolutely crazy. While we're on this topic, just generally, did you, uh, what... You used Devin. I'm sure you used Cloud Code. Yes? No? Um, or... Uh, wa-was that the, the main thing?

Any sort of coding tips? Anything, you know, if people were to try to attempt the same kind of thing, like implement a full runtime, what advice would you have?

So I use Cloud Code, Codex, and Gemini.

Oh.

I use mostly Cloud Code, but I use Codex-

Gemini CLI Gemini or Gemini in Cursor?

Gemini CLI, although... So my take is that, like, you can think of each of them as... If you think of them as a superhero, Cloud Code is like your clichéd Captain America, like Mr. Right. Like, pretty competent, does most things right.

A bit overconfident, but like, whatever, fine. Codex is this like neurotic, geeky, like the kind of Q guy in, in Bond. Like this neurotic, geeky person who's like very specific about little things that don't particularly matter. And then Gemini is like the Joker.

It's just this like unhinged lunatic who will occasionally do an incredible job, but half the time just delete all of your files. And I almost never run Gemini with the capacity to do anything other than... Actually, I can, I can show you now.

Yes, please. I don't meet enough Gemini CLI users, which obviously they're working hard on it, but they just don't have the market share.

So for example, this is my like very brutal way of like doing reviews. I just literally pipe that into Gemini, and Gemini doesn't... is not allowed to, by default, run, like edit any files, and it will just go off and like review a particular branch and write me out a report.

And then I basically just point Cloud Code at that report and say, "Implement the following things." I'll edit it if the stuff is wrong, and I do something fairly similar with Codex. Works fairly well. This is... I mean, look, you could, you could do lots of cleaner things with this.

I'm sure there are more elegant ways of doing it. I really wanna start using Code Puppy. I just haven't got around to it yet. For this, for-

I've never, I've never heard of Code Puppy.

Code Puppy is a coding agent built by an amazing guy called Michael Faffsburger at Walmart, built with Pydantic AI. And it's got like, hasn't got very many stars, hasn't got lots of like hype around it, but I don't know how much I'm allowed to say publicly, but it has got an enormous adoption amongst developers and non-developers to, to basically automate bits of their job, which is one of the reasons I find it really fascinating.

I, I won't say more about it because I'm not sure how much I'm allowed to say publicly. But yeah, it's, it's, um, but you know, I've heard really good things about it, and I want to try using it.

But like, you know, all open code-- I mean, I find open code's willingness to like get in the way of my scroll and be a full TUI is just like killer for me. So I, I can't... I'd rather honestly have Gemini CLI.

Yeah, you want, you want just like, just only do back and forth. Don't become a full u-user interface. Like that, like that's too much, right? Yeah, I, I, I hear that a lot.

So someone said to me, which makes sense, like the single biggest private code base in the world is Google, and supposedly Gemini is trained on it. So it makes sense that in the like very technical end of the spectrum, right?

Like Monty is not your average like build me a, build me a web app. Like it's, there are some very technical, like what's the best way of implementing a bytecode interpreter? How can I make this struct a few bytes smaller?

Whatever this might be, right? That stuff really credible that the things that Gemini knows how to do that other models don't. And so that, that's, th-I think that's my reason for trying it. Sometimes it comes up with something magic.

Obviously, I would need to run each of them three or four times to like do a scientific test of which one's best. I haven't done that. I just use, use a mixture of them.

I think it's a reasonable story to un- to think about Gemini, but like, uh, you know, I've asked actual Google people this. Uh, they use an internal thing called Jet Ski. And I mean, the simple reason is they have internal versions of everything else.

And so if they actually train the-- they had released the internal version to the external world, it would just make no sense. Like it would just reference Borg, and nobody has Borg.

Interesting. So you think it's, it's not actually trained on much of that internal-

It's not trained on, but it's probably informed by, which is close enough.

Yeah.

But trained on is a very specific thing where it's like it would have not, like the model would spit out names of internal Google names which you-- which would be meaningless to you because you don't have it.

Yeah, yeah. Makes sense.

So what's the point, right? Yeah. But, uh, but it makes sense that like it's informed by the same people that worked on them, therefore it has those like technical proficiencies, right? Which is just, it's hard, it's hard to articulate what those things are.

The other thing I will say about it in comparison to, to both Codex and Claude Code, it is so fast to review because, I mean, I, I don't know actually, but my in- my inclination is they basically wrap up the entire diff of your pull request if you're trying to review a branch, and just makes one request of it.

The model sits there and churns for a minute or two and returns you a report. Whereas Codex is gonna like go through and like agentically investigate all of the changes and try and link them up, and how does that relate?

And maybe it does a marginally better job, but one can take honestly half an hour, and one is gonna be done in like ninety seconds. And so quite often that like Gemini first review will find things that, that Claude Code and I have done wrong to, to, to fix more quickly.

Yeah. I think we need to standardize this. I think it's getting a little sloppy how people are incentivized to have more and more autonomy, but actually sometimes you just want a quick answer, and there's like sometimes no opt-out button to just do that.

And so people wanna just take over more and more things.

I was using, I mean, I use Opus 4.6 for the most part, but I have, I have honestly had tasks where I'm like, "I could do this faster than that." As in sure, I have a much better mental model of the code base than it does even after it's read Claude MD.

I'm not saying that it's a fair, fair fight in any, in other ways, but like... And of course, there are many situations where I would rather it churn for thirty minutes when I could do it in ten. But there are genuinely situations where I'm like, "I could have done that change faster."

And what I hear, I-- who knows whether this is true, is that for Opus 4.6 or the most recent Claudes, they basically took this feedback that Codex was more precise and like basically made it think harder, made it investigate for longer.

Fine. There are places, basically to your point, there are places where that autonomy is worthwhile, but there are places where I need to get something done where I would rather like, you know, move a bit more quickly.

Stuff. Well, so I, I think that is the state of, uh, of Monty. Uh, where is it going next? Is there a commercial sort of angle to this, uh, at all, or what are you thinking?

Yeah. Would it be interesting for me to show you a, like quick example of it? 'Cause I think that might, might be-

Okay

... interesting. I'll show you kind of what we're doing in this example, 'cause I think it will like hopefully give people a bit more compunction about what's going on. So fundamentally, what we're trying to do here is scrape the prices of LLM models from their websites.

As we all know, AI companies have a very well-developed sense of irony, and so having made their millions out of scraping the internet, they then make it incredibly hard- ... for us to get data back from their websites.

And we maintain a library called Gen AI Prices, where we, where we have the, the prices of all of the models. And at the moment, that's basically handwritten, as in we go and, you know, when a new model comes out, we go and read it.

And so what would it look like for us to go and download that, that stuff? So this is a Pydantic AI agent. In this particular mode, we're actually running it in a-- we've manually implemented the loop. In general, you wouldn't need to do that.

In general, you would just use the like code mode feature in Pydantic AI, but I wanted to do some slightly different things. But the interesting thing here is we, we take this, we take Playwright, and we have a like open page function when it has access to, to the browser, and it will return me an instance of this page type.

And coming back to your question about like how do I register stuff inside,

inside Monty, these data classes page here, which has a bunch of methods like, well, some attributes and a bunch of methods like go to and click and fill and everything. This is using real Playwright internally, but you can expose this data class inside Monty, and so the Monty code gets access to, to run all of these functions.

Uh, it's also worth saying Monty has the, the TY type checker built into it. So before it will go and run any code, it's running type checking with these type stubs. So the, the LLM should be confident that it's got the, the L- the code shouldn't run unless the LLM has got the typing right.

And then we have this beautiful soup function, which is gonna give us back this like- A basic thing that allows us to query the, the DOM. And that gives us this tag type where again, we do the same thing.

We have these like a bunch of functions that we can register with the LLM. And so ultimately, let me run this example and see if it will give you some idea of, of what we're doing. Does that, does that make sense, Swyx?

Yep.

Or is there anything there I should explain better?

I'm, I'm following so far.

It'll be probably more interesting to look at what's going on inside LogFire. Here we are. So you, you see here, like we have the system prompt where we've basically told it what Python, what, what, what code it can use.

And this is like, there's not that many instructions, right? And most of those things we will go and implement some subset of. So we haven't wasted thousands of tokens explaining what it can and can't do. And then we've given it the type hints of these are the functions you can go and call.

Um, here's the page open, here's the like these are the doc strings for the functions you're allowed to call, yada, yada, yada. It will then go and write, um, write the code that it, that it needs to go and exercise.

So here you see the code written by the LLM. So it's gonna go and get the pricing from, from Claude's docs. It's gonna, you know, uh, and it's gonna turn that into Beautiful Soup tag, and then it's gonna inspect that tag, and it's gonna use this to go and investigate, return some, some HTML about the tables, and it'll do a bunch of investigation.

And eventually, if the demo gods are with me, this is taking particularly long this time, but like sometimes it doesn't take this long. It'll come back, and it'll give me a summary of the prices of the model. You see it's run a bunch of...

It's run into a bunch of errors here. So in particular, it like it thought that asyncio didn't need to be imported because it assumed a REPL. Uh, it turns out the models are very strongly trained to assume a REPL wherever they're writing Python code.

Hmm.

So the, the big thing we're basically waiting for to have Pydantic AI like have code mode supported is we just, we know we need REPL, and we're just working through the final bits to solve it. But the point is that like, because it hadn't imported asyncio, the type checking failed and told it, "You need to go and do this," and so the next time it ran code, well, it got something else wrong this time.

But like eventually, once it had got it right, it was able to go and, and run the code. And if you look at the full agent run here, you'll see the, the code it wrote a number of different blocks of code, but ultimately came back and like was able to extract the prices pretty successfully.

Fully agentic web scraper.

But, but, uh, look, of course, you could do this with basically setting up Claude Code or your coding agent of choice, and that's fine if you're running it locally and you're either gonna watch it or you're gonna like YOLO have dangerously skip permissions.

But if you're running this kind of thing in the cloud and you, and you're gonna have ultimately untrusted people prompting the model, y-that is effectively the same as letting an unpr- trusted person write the code. And so you need, uh, this level of control where you can say, "This is exactly what you're allowed to do.

These are exact functions you can call." If you allow it, for example, to, to make a, to, to load a web page or to make a get request, you can be very specific because all of those function calls are going through the host about exactly what domains it can and can't connect to.

You can control how long it can execute for, how much memory it can use, and so you, you know. The, the ideal is we'll get to a situation in not too long where, um, you can basically go and run this and let any old code run, and the worst you're gonna get is, is an error.

Yeah. Beautiful.

But, but what I was gonna say in answer to your question is, I, I don't think I have an example of this here, but you saw this took 102 seconds and cost 30 cents to run. If you ask the LLM in its response, as well as the information about the prices to say, "Please also return the optimal Python code to run this task again," and then you give it that, that function code when it runs again, it gets it right first time because you've ef-effectively it's just gonna go and run that script.

And so I think that there is an enormous opportunity for effectively a new layer of state within applications. You could think of it as memory, but it's often a lot more than memory. It's the like current state of agent optimization, some of which will be code, some of which will be model choice and, and model settings, some of which will be system prompt, and that is the like big drive we have now in LogFire.

So basically, moving beyond evals to the agent improving over time, whether that be completely autonomously or with a user like looking at the changes and clicking Accept. And so, so Monty itself doesn't make sense to have a like hosted version of it because the whole point is you don't need to use a hosted service.

But I think there's an enormous space for basically services on top of it where you're like, "Yeah, of course you can go and use it in, in your language of choice, but by the way, we have the service that will make it that much, that much more performant."

I think a lot of people, a lot of observability people eventually find this sort of self-improving agent ideal. The problem is it requires a lot more scope than you currently have, including access to my code base which I don't know how you're gonna handle that, but it, it's gonna be interesting.

I, I, I agree with you. Yeah, I agree with you to some extent. I mean, one of the things we're doing now in Pydantic AI is we're, we're about to introduce... I think there's a PR out for this, so I think this is public, serializable agents.

So basically, you can define an agent entirely in a TOML file, everything from the model to the system prompt to all of the capabilities, whether they be code mode or compaction or registering an MCP server. There are lots of advantages to that, but one of the things is you have now one file where you can go and run that like optimization, and in theory, you can get to the point where that one file-

Mm-hmm

... can basically run untrusted, as in it only has-

Yeah

... the capabilities that you have registered with it, and Monty is running all of the like arbitrary code that's being executed in that process.

I think this is a good idea. I, I'm not sure about the TOML, uh, uh, but, uh, I-- because the, the, the main idea is that TOMLs are, uh, just like any specification language, any markup language, they are just a, a DSL, right?

Uh, and you would have to reinvent a DSL. You're gonna invent Branching and variables and looping, and then you, then you just have a, have poorly implemented version of a general language.

I agree with you, and I think if we get to that point, we, we resort to Python or we resort to Monty for some of that stuff. But this is the point is-

I think Monty is it. I think Monty is it. You're already there.

Okay. Okay. Uh, I hear you on that. I mean, I think the point is that, like, most... If you, if you think about what, what agents are, there are exceptions of course, but there's an awful lot of them that are somewhat formulaic.

You have a system prompt, you have a model, you have an output schema, you have some MCP servers that you register, you have some, some settings, like is it durable, compaction, yada, yada, yada. There's like 30 of, or so of these capabilities, and it's effectively a choice of like which ones do I switch on?

And sure, if I wanna go and register arbitrary tools, that's a whole different thing, and this, this thing breaks down. But often those tools are more and more, at least in enterprise, being packed up and put behind MCP servers.

And so again, it comes down to like the settings are the link to the MCP server and how we're gonna do auth.

Oh, yeah. Auth is a- another thing. But yeah, I think, I, I do think that this can work as a standard. I think, um, you have a shot at defining the standard and, um, serializable agents is like a really good name for it.

So, uh, I'm excited to see that spread. Excellent. I, I think that's, I think that's gonna be it for our, our, our pod. You're gonna come speak more at AIE in London, which is... We're, we're all, we're all very, we're all very excited.

I, I do think like this is kind of a big celebration of everything in like sort of European AI, particularly London, and, uh, yeah, I'm excited to see you there.

Yeah, really looking forward to it. I don't know when this podcast is going out, but we're also super excited for our conference next week. But I presume this will go out ... I think we're well sold out anyway, so I think we're good.

But yeah, really looking forward to, to London. That's gonna be great.

Presumably your, your conference is gonna be out on your YouTube, and, uh, we'll send people there.

Yeah. So, so we're co-hosting it with Prefect, so some stuff will be on our YouTube, some will be on Prefect, but it will all... there'll be links to all of it on, you know, you'll be able to find it relatively easily and it will all be recorded.

So yeah, we've got... we're so lucky to have people like Guido and Sebastian, uh, speaking, and Armin. Like, we have an amazing lineup of speakers, so I'm super excited for it.

It'll be a great day. Excellent. Well, thank you for your time and congrats on finding like a, a new, I guess, angle for open source to sort of penetrate the, the, the sort of runtime agent conversation. I do think like this is a...

Like, in a way that I guess Deno and Bun has revolutionized Node.js like pre-AI, I think you're kind of doing this in the AI native sort of era and building exactly what agents need. I do see that there's a gap in Python for that, so-

Yeah

... you guys are there for it.

As I, I spoke to my friend David Sora Pereira at Anthropic, showed him this and he was like, "Yeah, this is exactly what everyone needs." The question is, how are you gonna make money out of it? And I was like, "Yep."

I think that's, you know, from a... bluntly the biggest, the biggest question on Monty is like how do we make money from it? But like, hey, I think we're doing what we're not-

Bun didn't have to. Let's just put it that way, right?

Yeah. I- I've got a funny story on that, but I, I won't include it in, in the pod.

So not, not for public consumption. Okay. Okay. All right. Well, so stop the recording there and I'll see you in London.

Cool.

No-

Great. Thank you so much.