The Blueprint for Autonomous Work Agents | Gavriel Cohen, NanoClaw

We are here at The Engineer with Gavriel Cohen, founder of NanoClaw. How has it been?

It's been amazing. This has been such a great event. Yeah. Congrats for putting it on.

Yeah. Uh, it's, it's great to see you in person. Uh, I could see you, you on-online as well, and it sounds like you had a very eventful trip to Singapore. You met, you met the Minister of Foreign Affairs.

Uh, yeah, this is like, um, what did you do? Wh-what happened? Uh, like, tell the story of, like, the Singapore trip.

Yeah. So this really goes back maybe about a month now. I was actually on vacation, like, taking my first weekend off since, uh, putting NanoClaw out there, and finally took two days off at a nice hotel with my wife.

And then, uh, w-- I think it was Friday night, I was just scrolling on X, and I see this, somebody reposting a Facebook post from Minister of Foreign Affairs of Singapore, where he did this whole writeup of how he's using NanoClaw, his whole setup, memory system, uh, indexing, uh, deployment, running it on a Raspberry Pi.

And he called me out by name, which is like... I was like, "Okay, that's..." My name isn't even on the repost. He's clearly nerding out, and he went deep. Uh, and then I went and I found his post, and he had done a Facebook post.

Uh, and he posted this whole GitHub, uh, gist of his whole setup, or whole writeup of his whole, um, NanoClaw setup. And he's got this kind of second brain type of setup where he's, uh, got a Karpathi LLM wiki, uh, with embeddings and some memory system.

Which did you have that in your original design though, right?

Not at all. That's all stuff that he's brought in. The memory system that he's used, Nemon, which has kind of gotten a little... started to get a little bit popular now that he's posted about it.

Find that. It's, so it's like, it's a hundred and forty-one GitHub stars as I checked it. I was like, "What's the-- where is this from?" Like...

When I first checked it, I think it had, like, forty-three GitHub stars. So, and that was after he started using it. So he found this system, he liked it. He found NanoClaw, and he put it all together himself.

He, he codes, um, and he, he created this setup. He did a whole writeup. So apparently, he was migrating from the first version of NanoClaw to the second version, and he created this whole, uh, writeup of his setup as part of the migration, and then just decided to post it.

Yeah.

And it started to go a bit viral on X, and I did a retweet of it, and then he retweeted my ri-- retweet, and we started to talk.

Now you're best bros.

Best bros. And, and he said, you know, "When-- if you're ever in, uh, Singapore, we should, uh, you know, I'd, I'd love to kind of have you over and, and host you." And then I think it was two, three days later, you reached out and you said, "We're doing this event."

And I'm like, it just, it's, it's all perfect. I checked that he'd be around and, and he said yes. And then you managed to get him to come down and, and talk here, which is wild.

See, uh, I think what-- it did help that you were here, so you wanted to be. And I think also... No, my-- our intentions are parkour in the sense that we just want to promote the, the industry in Singapore.

I'm Singaporean. Uh, I live in, I live in the US, but, like, I think the government wants to support it, and it's under, we basically do visit with businessmen like this. Um, I'm curious, you guys talked for two hours.

Anything you can share about, like, surprising, uh, use cases or whatever. It's like you, you talk about the tech stack, but just like how does he think about it that maybe opened your mind as to how your users can use a little bit of both?

I think for me, his use case really helped for me to kind of crystallize what direction we wanna build in. Going back, I would say a month ago, there were these two overlapping but kind of distinct directions when we're thinking about adoption of claw type of agents, autonomous agents, in a business setting in a company.

Uh, so there's the one where it's the team manages agents, so it's agents that you build an agent factory or, or agents that, uh, automate workflows. I gave a talk about that, the agent factory that we built, and we had started to build that already for ourselves, and we've been building it for a while.

It's still kind of under construction. So that's one, this agent factory. It's the team as a team working together to build the agents and managing them as a team. And then you've got the other side, which is personal agents in, in a work setting, in a business setting, but individual people who have their agent, their assistant that's helping them do their job, and it's more one-to-one.

And we were going, working on both of those use cases, both internally, we were using agents in both ways within our team, uh, and started to work with design partners that were interested in both use cases. So we were working with a team where they wanted to give each person in their legal team their own personal assistant, and then we were working in-- with another, uh, design partner that wanted to give-- build agents that automate a workflow for their legal team.

So automate drafting first contracts. With the foreign minister, the Ministry of Foreign Affair-- Minister of Foreign Affairs', uh, use case, it really helped to crystallize that, from my perspective, the way to start in a business intro-introducing agents to a company is to give each person their own agent.

Uh, because there's a certain knowledge and expertise and learning curve on how do you work with agents? What are they good at? What are they not good at? How do you prompt them? How do you, uh, build the instructions, skills, adjust those over time, manage the, uh, context window?

And it takes some time to learn that. And initially, one of the biggest mistakes that people make is that they just wanna throw something at an agent and then walk away and expect to get a finished result at the end.

You gotta, you gotta put in the effort, right? It's the dream that we all have, and, and that's a natural instinct, right? It's good to be, to kinda be ambitious, but you have to work on it. You have to fine-tune...

not fine-tune a model, but, but fine-tune the output by adjusting instructions, adjusting skills, adjusting what you put in, and, and also iterating with an agent. From my perspective now, especially seeing his use case, the killer use case for claw type of agents, autonomous agents today is the second brain use case, where you're just kinda dumping in information-

Yeah

... and you're not expecting it to give you ready-made output, but it's just collecting that information, building up its internal memory or knowledge graph or wiki, wiki, LLM wiki, Wikipedia view, and then able to give you useful outputs at the end.

Yeah. Uh, I've been using Town as assistant for that- My last in AI Europe, I talked about how I'm using Devin from KnowledgeWork, uh, Town Assistants, even a little bit of, uh, Victor as well. These are all like some of the, the alternative personal assistants.

I use OpenClaw personally, but, uh, I, I think, like there's this general field of like knowledge management that needs to be capitals that I think you guys have done very set up. And like, I think you have the simplicity and the, uh, privacy focus that f- that drew him, right?

Like, I think that was... He mentioned that he looked at OpenClaw first. Uh, did he talk about the, the security side or like what, you know, the, so like OpenClaw comparison?

Yeah. The security side was the key thing that, that, that brought him in. There are a lot of people like him. I actually started to use OpenClaw, and we were using it for our business. At the time I was building a AI native marketing agency.

We had some customers we were ramping up, and then we set up OpenClaw and it started to manage our sales process. And very quickly, within like two days, it was doing the work of an employee, of a sales manager just managing our whole pipeline.

But then I started to dig in, and I started to see the size of the code base and the number of dependencies and some other things like logging all messages in plain text that just made me a, a bit apprehensive to use it for like production use cases to build a business on it.

So I felt fine using it personally, but when I was saying I'm gonna connect it to my customer's data and I'm gonna start to build all these workflows that are running and I'm trying to build my whole company on top of it, I didn't feel like it was stable enough for me.

So the key changes, first of all, made a really minimal code base with, uh, NanoClaw. So I didn't clone OpenClaw and change it, just started from the ground up from scratch. Used a lot more, uh, components out of the box.

So used Agent SDK instead of, instead of building our own agent. Uh, like OpenClaw uses Py, so then you gotta build a lot of session management and compaction and a lot of other things 'cause Py is really minimal.

Instead, we used Agent SDK, which comes with a lot of that stuff out of the box. Um, only supported one model initially, one agent initially, so saves a lot on that. Uh, used integrated with things for connections to messaging channels, like we integrated with, uh, Vercel's Chat SDK library that just gives you all of the messaging, different messaging apps out of the box.

So really minimal, uh, fewer dependencies. Uh, but then the biggest thing is the isolation model. So first of all, you run your whole NanoClaw in a VM or on its own Mac Mini, and that's great, but you still need to isolate the agent in its own agent runtime and separate that from the messaging bridge that's connecting to Slack or Discord or whatever, and the, uh, router and the other pieces that decide what gets pushed into the agent and what gets done with the agent's output.

So you need to have isolation of that, so we run each agent in its own container and then make sure that the agent doesn't have any credentials in its environment, and that's really big. I- if it's got credentials and it's touching on sanitized input, which for most useful, uh, use cases is gonna be, it's gonna be, you know, reviewing a PR, it's, it's got the whole PR input.

Anybody could open a, a pull request to an open source, uh, repo. So making sure that there are no credentials in the agent's environment, so even if it gets prompt injected, it can't leak credentials, it can't leak API keys.

And then the third key thing is separating, so the agent can use credentials. All requests coming out of the agent's environment get proxied through a vault, and the vault adds credentials if the agent is supposed to have access.

And then the third thing is having access policies and access control, including human in the loop approval. So you can give your agent access to emails, maybe set a policy that it can read emails without any approval process.

But if it wants to send an email, you get a message in whatever channel you've connected, so in Slack, and you've got buttons Approve, Reject. You can see what it's trying to send, and you decide if you should let it through or not.

Yeah. I feel like he may not have like the behind the scenes, uh, l- understanding that you have, but like the, the, the messaging does come across really clear your, your DOS, the IDs, what is. Um, yeah, I mean, like what else have you found in this trip to Singapore?

Like you met other users, like have you met other builders that inspired interesting conversations?

I met a lot of really interesting people out here. Um, you know, I, I, in my, uh, talk, I put my agent out there to, uh, book a, book, uh, coffee chats with.

I think there's sometimes a VPN proxy thing. I didn't think I was in Singapore because I, I have my US number. Uh, so it was doing something weird where like it didn't let me in.

Uh, the talks are streamed, uh, live on YouTube, right?

Yeah.

So yeah, I, I just wanted to make sure that people who were here would be able to access and it didn't get taken down by people, you know, massive people, uh, connecting around site. So I did set it that it's geo-blocked to Singapore.

Uh, sorry about that. I'll open it up for you later. So yeah, so I, I, I put my kind of agent, uh, out there and people chatted with it and then booked, uh, coffee chats and chatted with really interesting people building, uh, in especially memory, uh, students out here who are building really interesting things.

Actually, I did wanna ask, do you have your own memory solution that you use?

Personally, my, what I'm using personally is, is, uh, a kind of LLM Wiki, uh, type of solution. I just point my agents at Karpathy's, uh, posts, and I say, "Read this. Look at this." "Let's chat about it for a minute."

And then, uh... But NanoClaw has built in a kind of simplified version of that where it just has some basic instructions to the agent saying, "Create markdown files." Anything, any, anything substantive that the user shares with you, make sure to save it somewhere.

Either it goes in your cloud R&D, or it goes in a markdown file, or it goes in some other file, but it's gotta be saved f- somewhere.

Yeah. I, I think my, my problem with those things is just that even using all this, it would create a lot of duplicate files, which is like not know that it already has a similar thing, and it just creates another duplicate set of files.

And now I have like two sources of truth, right?

Yeah. You gotta have, uh, other instructions on top of that saying, "Create an index of all your files." And save that index in your, uh, in your instructions, see what files you have, then go and save it. And then you do need to have like a background process that runs daily or, or every few days looking over your, your memory files and finding the duplicates and flagging them.

Uh, yeah, there's definitely a lot more to do there that we're not doing yet. I, I know some of the other, uh, kind of claws are-

Anything off the shelf, so let's say, sounds like a beautiful hero.

I'm not sure that, like, retrieval-based solutions are, are ideal for, uh, like, personal assistant because it's very specific, and it isn't necessarily, uh, something that retrieval is really good at. Like, if you ask your assistant, um, "What are the most important things I should be focusing on this week?"

There's no retrieval-based search, semantic search, keyword matching search that's gonna be able to give your agent that information. But if you have a good LLM wiki that has like, "Here are the projects you're working on. Here's the timeline.

Here are the, you know, here's a log of the different calls you've had this week," it can go through a few different files and collect the things and give you that list.

Yeah. Amazing. Um, yeah, it's really cool. Uh, I think last question is about your company, right? Uh, when I first messaged you, you are still-- we still had a, the marketing stuff, but now you've started nanoCo. Uh, what is nanoCo?

What's your vision? Uh, is it gonna be a VC style startup or tell us about it.

nanoCo is-- We shifted gears. We were-- When I first spoke to you, when I first created, uh, OpenClaw, we were building a, an AI native marketing agency. Uh, we had some customers. We were ramping up. It was going well.

I built, uh, NanoClaw just for my own use as a side project on weekend, launched it on, uh, Hacker News, MIT license.

That's why I first found it. I tweeted about it.

Yeah, yeah. Uh, immediately, within like hours, I think, after it came out.

Okay. Like, I can read this code base. I cannot read OpenClaw code base.

That was the idea behind it 'cause I said I'm gonna build something that I, I have an idea of the right, right way to structure it to make it kind of sane and secure, but I, I don't quite trust myself with this, so I wanna make it readable and let other people look at it and validate it.

And since then, it, uh, I've had a lot of security experts review it and, and critique it and give feedback and point out small things, but what's come out of all of that is that the core approach seems to be sound because nobody's pointed out, uh, an issue with, uh, a general approach.

So we were building that agency. I built NanoClaw for myself. It started to get a ton of, uh, traction and attention, and then Karpathy, uh, tweeted about it, and it went to just a whole nother level. And for a little while, we were debating if we should build an agency or build NanoClaw, and then after Karpathy, it just became clear, like-

Yeah

... there's such a big community, so much energy behind this. So we went in all in on it. Um, we, we have a company now. We've got ten people on the team. Uh, and what happened was the early adopters, I think, not just of NanoClaw, but OpenClaw, Hermes, every, all the claws, have been a lot of VCs, CEOs, executives who are using it themselves and get really excited about it.

And we've had tons of CEOs and other executives approaching us and saying, "Hey, I built out this great setup for myself. I've got these three different agents, this memory system," just like, uh, the, the Minister of Foreign Affairs here.

Uh, and they say, "I wanna roll this out to everybody in my team, but I don't wanna become the IT guy who's now, like, fixing their agents and debugging the memory issues. Can you guys come and help me set this up for my team?"

Yeah. So you're not, you're not Agent Lab. Like, you're rolling it out. It's hosted and all fine.

Yeah. So we're gonna do deployments, uh, and for a lot of companies-

The thing I said in my talk that's gonna be very relevant here

I'm gonna go back and catch up on some of the talks.

Skipped the dev, the dev journey, but, like, more focused on manage work rather than coding.

Yeah, I'm gonna get-- ca-catch up on, uh, on your talk and a few others. So yeah, the, uh, gonna do the deployments. Uh, for a lot of companies it's gonna mean deploying on their infrastructure to their cloud.

Take, take notes from my thing. It is exactly those-- Like, people are gonna tell you about, like, SSO and, like, uh, uh, BBC peering and, like, on-prem deployments, all that stuff.

Yeah. Uh, we're gonna have to hook up to their credential management system, like getting their credentials from their vault.

Sure.

Yeah.

On AI. It's actually a lot of it's not yet.

It's not. But, uh, you have to understand the AI. I think that's what's missing for a lot of these companies. They have good engineers. They've got good DevOps. I spoke to a lot of these companies. Um, they've got good security teams.

But if you don't have the piece of the AI engineering, it's hard to put together the different pieces and to have confidence that what you built is secure, it works, a-and it's, uh, it's the right setup.

Yeah.

So having i-i-- I think of it more as a partnership between AI engineers coming in and working with the, their DevOps, their security team, their IT department, and helping them get set up, the initial setup, and then afterwards helping them manage it over time.

Uh, initially, you gotta connect it to their credential management, observability, their other security systems, um, and then help them get set up with their initial integration with their internal data sources. Um, but then over time, their IT team, their security team can pick up those pieces, and when there's a new request for integrating with a new data source, they can take that on their own.

But there's still management to do in terms of upgrading and maintaining it. Agents are different than normal software in that normal enterprise software, you can deploy it, put it on some server, and let it run for, like, five years, and as long as you never touch it, it just works.

Uh, agents don't really work that way. The core thing that you're building on is constantly changing over time. You can't just run 4.6 for the next three years and just leave it. You gotta upgrade to 4.8, 4.9, 5.

And every one of those upgrades changes things. Uh, and, and the labs are putting out new features, new capabilities. Uh, memory is getting baked into the LLM, getting baked into the agents, uh, and you have to be constantly updating in order to just stay kind of at the, at the front.

So that's what we're gonna be doing. We started to do our first deployments. Uh, we've got over 100 companies that have approached us, interested in rolling out agents. Um, and yeah, big announcements coming up.

Two f-closing questions. One, uh, for your Agent Factory, do you think that Git and GitHub will last?

I think it's gonna have to last for us because we're an open source project, and you've got the culture and the community. Um-

I'm trying to see what comes after GitHub.

Yeah.

I don't have it yet.

The- there is a weirdness there, right? Because the whole Agent Factory should be on GitHub and not happening in Slack.

Yeah.

But because it's an open source project, like, if it was closed source, we would just have the agents leaving responses, doing the reviews, test results. It would just all be in the, in the pull request on GitHub. Because it's open source, I don't wanna have an internal discussion back and forth, leaving feedback from my agent, meta, you know, feedback on the factory itself happening in a public, uh, GitHub thread.

It could be interesting, but-

Yeah

... it seems a bit off. Uh, but if this was internal, I would want it to be on GitHub, and then I think we'd be running into the issues where GitHub probably al- still isn't quite the right format of that.

Yeah. My, my people do all their stuff in Slack, and then they pay me to stamp the, the PR in GitHub. But like, why do I need to... Why do I need GitHub at all? You know, like, I should just be in Slack.

Okay, so maybe our approach is the right way.

Yeah.

So we have the whole thing in Slack, and then-

That's my money I use.

A- a- all the way till, you know, merge, and then you hit approve, and it gets merged. Um, and I think there's a lot to do. Slack has good, pretty good UI. You can, you can build out some good, uh, user experiences just in Slack.

Yeah.

So Slack over Whats- he uses WhatsApp, you use Slack. There's, like, a lot of Telegram. I like, I like Telegram's... Like, it's so much easier to set up bots with Telegram.

Yeah.

But you can do Dr- you know there.

I can do my, uh, my tiers of, uh, messaging apps. So I think, uh, S tier would be, uh, Slack, uh, and then Discord, Teams, and then goes, like, Telegram, and then WhatsApp. They're very, very limited in what, what you can do, and it's hard.

Like, you have to have your own phone number if you want, like, a decent experience. Uh, chatting, putting the bot on your own number, it makes sense if you wanted to just see your messages read only, but to chat with the bot on your own number, it's...

I mess- yeah, I haven't experimented with it enough.

Oh my God, it's, uh, terrible. Uh, you can't, I, I can't message myself because I need a separate number to, to receive it, so then I just give up.

Okay, so that's below WhatsApp on the tier list. We filled out our whole tier list.

See if I was obviously you have to buy, you have to buy your own number, but, like, it's annoying. I, I think there's some sort of service that do that. Anyway, uh, last, last question. What can people help you on?

What are you looking for help? What are you looki- like, what is a question that you want answered?

Managing open source projects today is, is a huge challenge. So, like, I- I would say, I, I won't even say for, for, for AI engineers, I won't even say, uh, contribute a pull request. It's like-

For me-

... it's part of the problem. Yeah, it's like triage. Help us figure out how to manage a, a open source project in a better way. Um, it's a big open challenge. It's like this, uh, uh, it's an arms race where coding agents have just made it exponentially easier for people to open pull requests.

Yeah.

And then it's so hard to, to triage, to review, to understand if it's aligned with-

The same as, uh, what Peace Berger is saying, right? No more pull requests, only prompt requests. Like, don't give me your code. Give me your, your, like, what, what are your use cases. And then, uh, something that comes to my mind after having this discussion is you have all these, like, oh, you know, my bug here, my feature request here, whatever.

It should just probably all go into a wiki of future developments, and then you pull from the wiki. So, like, the wiki is kind of a buffer. All the changes come in as a continuous stream of stuff, but it becomes a wiki, and then you pull from the wiki as you, as you need.

Yeah. We have started to build a wiki for our, uh, Agent Factory.

Yeah.

So as we're developing it, we're adding to the wiki kind of on each, uh, merge, on each commit. And then I think that is gonna become a standard part of building open source project. So what you guys are doing with, uh, is it DeepWiki?

Mm-hmm.

That's awesome. I think, though, that it's gonna become a standard for teams to build out the wiki as they're doing the development, uh, in real time. And then, yeah, the, the, the future development is in the wiki linked to the different parts of the project that it's related to, and the bugs are part of that wiki.

And then as you're developing it, each time you start working, you pull information from the wiki to get context, and then you finish the PR or finish the commit, and you push back to the wiki some more, uh, context.

Yeah. Great. Great. Um, well, thanks for coming. I so glad to meet you. Uh, I'm sure this is not the last time we will see you. Uh, but hopefully this been a nice trip support.

It's been awesome. Yeah.

All right. Be sure.

Thanks.