[AIEWF Preview] CloudChef: Your Robot Chef - Michellin-Star food at $12/hr (w/ Kitchen tour!)

Okay. We are in the remote studio with this very special podcast. Uh, we actually recorded, uh, a while ago a tour of CloudChef's kitchen, but we wanted to d- to record a little bit of an intro in our remote studio so that we at least get a nice audio podcast intro to the company.

And we're here with my friend and co-host, uh, Vibhu Sapra, and as well as Nikhil, who's, uh, founder of CloudChef. Welcome, Nikhil.

Thanks for having me this week.

Watch out.

Okay. So yeah, w-welcome back, Vibhu. But I think by the time this launches, like, people will have heard the, uh, Anthropic podcast that we did. But yeah, so I think the headline that people will s-see when people see CloudChef is that it is an AI chef.

You, you had this, like, pretty viral video on Twitter recently, you know, when you launched and, and told everybody. But also, people don't know that this is a real restaurant. Like, you actually run a real restaurant. You can order food on, I think, Uber Eats.

Yeah, Uber Eats.

And it's really good. Yeah. So like, what is CloudChef? Like, what is the scope of it? How do you pitch the company?

At a very high level, what we're trying to do is we wanna make high quality, nutritious food available to everyone. And the reason why it's possible to even think of a future like that is because you can automate practically all non-managerial work inside a commercial kitchen with culinary intelligent robots.

And culinary intelligent robots is basically just robots that act like human beings, learn like human beings, and work like human beings, or work like human chefs. So we actually have our first... The v- uh, the video that Shawn was talking about is the launch video for our first robot, and basically it's a robot that has a mobile base, has two hands, goes around and does work inside a kitchen.

So it's just like how you would hire a human employee or a human chef, you would hire our robot, and the robot will come to your facility, cook, and it come to your facility, learn recipes from the chefs inside the facility with one single demonstration and cook that dish over and over again, or participate in that workflow over and over again like a human employee would.

And then you pay the robot an hourly wage, like how you would pay a human. And so far our robots are used by, like, Michelin-star chefs. They're used by

fresh fast food restaurants, airline caterers, a whole bunch of, like, commercial facilities use our robots as hourly w-wage labor as compared to,

as, uh, hourly wage labor as compared to buying a robot. The thing that makes our robot special is the fact that it can execute at chef level or it, the fact that it has culinary understanding better than even the best chefs in any single cuisine.

Like, what that means is, okay, if you, uh, if a robot is going on, if, if a robot is cooking, it needs to know, okay, how brown the onions are, how far along you are in the cooking process, if you're cooking it in a slightly different appliance, like, uh, what state is the recipe in?

How much heat do you give it? All this, like, thermodynamics modeling of cooking, understanding visually what's going on, this is what we call culinary intelligence, and this is something that was not possible until, like, recently when, like, multimodal models got good enough and we basically built out some thermodynamics modeling to aid that.

And now the end result of that is a robot that can reason and make decisions in the real world in cooking processes like a chef would.

With more and more robot foundation models coming up, them getting better, these robots are finally also able to do real actions or real motions inside a kitchen. Right now they're good enough to only do stuff like gross manipulation where you, where you don't...

Like, if a human requires more than two fingers or three fingers to do a task, the robot's probably not able to do it. But the good part is most tasks inside a kitchen can actually be done with just two fingers.

So if you, uh, uh, go around any commercial kitchen, and if your two fingers had enough strength, you could probably do most tasks inside that kitchen. So we start with line cooking, which is, like, the biggest labor cost for restaurants and restaurants and other food, uh, producing facilities.

And our robot is able to do line cooking for about f- uh, forty to fifty percent of the world's commercially valuable cuisine to a point that if we put our robot against an expert chef in that cuisine, our robot is able to consistently make the food better than even the chef whose source recipe it is.

Like, it's something that computers just do inherently much better than, like, the, uh, human brain. So that's a quick overview on this. Basically, we've trained our in-house models to... We've trained our in-house models to do, like, thermodynamic perception, and we've, we leverage current, uh, VLMs and voice models to do perception and also to enable the robot to do tasks or, uh, for the robot to actually talk to human beings, interact with other coworkers in the facility to, like, course-correct its goals and whatnot.

So that's a quick overview on what we are. The high level goal, like I said, is to replace all non-managerial work inside commercial kitchens with culinary intelligent robots. And when that plays out, we think we'll all live in a future where we all have access to really high quality food at fast food price points.

So at McDonald's price points, you should be able to eat the tastiest food that you've ever had in your life. That's the thing that we want to create. And we think that now that the robots have started to work in the real world, we see a, a future in which-- we see a, a soon enough future in which we'll make that possible.

And to Shawn's earlier point, we actually started experimenting with these robots in our own facilities, and we basically just built an in-house delivery kitchen at our office in Palo Alto. We weren't expecting it to do this well. I mean, it just, like, picked up really well on DoorDash.

We just-- And we had moved from I-like m-me and, me and my co-founder, we had moved from India to Palo Alto, and we were actually just missing really high quality Indian food here. So we just went to our favorite restaurants in Bombay and Delhi and asked, "Uh, can you record your recipes?

We'll serve them in California, and we'll give, give you a royalty." So that's not the core business that we're focusing on. It's just something that we use to validate our technology, and the fact that it's doing so well and the ratings are so good is just, like, a testament to how the tech is and how good the robot is functioning right now.

I can confirm. We tried the food. We'll, we'll see later in the video. It's really, really good food. I like the term you used there, artificial cu-culinary intelligence. ACI has been attuned internally.

Yes.

It's interesting though, 'cause when you, when you frame it like that, you guys are doing something pretty different than, like, robotics, right? Like you mentioned that the robots are more so off-the-shelf parts. It's not like specialized robotics. It's actually the software underneath, right?

So yeah, if you could talk a bit more about that.

Correct. So when we started the company, we had one core ideology, which is that we will only solve problems that can be modeled as software problems. And culinary intelligence was the first, like, big-- like, culinary intelligence and decision-making was, like, the first big open problem that we could model in software and solve.

But when we had started robots, you-- robots were still, like, very much an electromechanical problem, like they weren't really a software problem. And now with, like, robot learning models and robot found-- like, uh, foundation, robot foundation models, it has gotten to a point where you can now start modeling the physical actions that somebody does also in the software term and solve it in software and have software iteration cycles.

And we didn't want to build any hardware. Like, we didn't want to be a hardware company because that was not our, like-- We, we, we weren't, uh, good at it. So we, we just didn't want to do hardware, and we didn't, uh, think that the hardware iteration cycles would be beneficial for a company like this.

Very recently, it has gotten to a point where you can just take off-the-shelf parts, put a bunch of, like, uh, robot intelligence, quote-unquote, I'll expand on that later, and get the robots to work. And that is a software iteration cycle.

You don't have to build your own hardware. You don't have to get into what motors, like how do you, how, uh, how do we manufacture our motors? How do we manufacture these robots? How do we desi-- Like all, all those open questions we rely on the ecosystem for us to solve, and then we basically source general purpose robot parts and-- or general purpose robots and these general purpose ro-- like, and write software on top of these general purpose robots.

So we take general purpose robots, we leverage all the, uh, general purpose intelligence like LLMs, ro-- uh, LLMs, VLMs, robot foundation models, and then build this proprietary culinary layer on top, which has e-everything to do with like the thermodynamics modeling of cooking, custom evals for like manipulation, understanding, like, uh, understanding through perception what stage of the cooking process you are in.

All of those things we've basically built, and we are hoping to surf the tide on both the, uh, advances in multimodal models and robot foundation models and use general purpose robots as, like, the vehicle to make that happen.

So that's, that, that's in a nutshell how the, uh, how our approach kind of works. And we are very focused on, like, like I said, modeling every part of the workflow as a software process and, and now that we are able to do it, we are able to basically do full stack work inside a kitchen and not just be a assistant robot that can be prompted by somebody on site or just a guidance system that tells humans what to do.

Like it's now able to do full stack work because the entire workflow can be modeled as a software process.

Uh,

I, I was gonna say, I know that like, you know, we, we see how the robots work and what they're doing under the hood later, but, uh, the one I guess like overall question I'm sure a lot of people have is like, what's the business model?

How do people kinda hear about this? How do you see like, you know, rent a robot for $12 an hour versus hire a chef? How do you, how do you come up to this hourly rental and all that stuff?

It's very cool to see and, you know, good to see it works. Yeah, I'm just curious how, how does that side of the business look?

So from a business perspective, the main thing to keep in mind here is that like food prep is the most labor intensive industry of all labor intensive industries here. And again, quantitatively, the way you measure it is how many full-time employees do you need per million dollars of revenue generated.

Food requires about 13 people per million dollars of revenue generated, and the second most labor intensive industry is hospitals, which require four people per million dollars of revenue generated. So food is like three times or like more than three times as labor intensive as the second most labor intensive industry.

Labor costs are just like going through the roof, and labor costs have been increasing year, year over year, and like depending on like what Trump does with illegal immigration could go even higher. And like the, the staff turnovers are really high.

Uh, the average restaurant is operating at like 130% staff turnover. By the, the end of 10 months, your like, practically your entire staff is new. So high turnover, very high cost, and the most labor intensive industry. And because the reason why we landed up at this price point or like this sort of pricing model is because food service is not a very profitable industry.

So they don't have free cash just lying around to do experiments, and there are no fixed, like there aren't like fixed budgets set out for buying new robots, testing things out. If it doesn't work, it doesn't work. They don't take that sort of an attitude.

Whereas there is a very readily available labor budget that we can tap into. Just like when you hire somebody, you don't pay for their college tuition, you just pay them a salary. We thought, like, why should that be any different for robot, uh, robotics and these robots are now cheap enough to a point where you are able to put that business model out there, not lose money every robot you sell.

So the robot costs have-- Basically, the point is the robot costs have gotten to a point where an hourly labor price-pricing model works, and the robots are also good enough to now do the entire chain of work so that it's possible.

And at twelve dollars an hour, it's like forty percent of what a loaded human co- uh, would cost, so our customers get their ROI on day one. The robot starts working from day one, and over time, these robots just get better.

The hope is that at some point they also even start making better, like, the food at any given facility that they're cooking substantially better, not just by cooking the same thing, but then enabling the facility to make, uh, recipes that they weren't able to do earlier.

Yeah. Awesome. I think the, the, the last, uh, part, um, you know, we'll, we'll go-- we'll cut right into the, the walk- the kitchen walkthrough video later. But the, the last part I think was... I think there's this general goal of demonstration learning, right?

Learning from experts, learning from the Michelin star chefs.

Mm-hmm.

How realistic it... You know, is this like a marketing promise or do you really just learn from one example? Because like, I mean, obviously food is messy, right? Food needs a lot of different demonstrations. How realistic is this?

So, so I, I want to clarify that the-- I want two-- I want to clarify two things. One, it is not a marketing thing, it's actually true. Two, the reason why it might feel counterintuitive is because our entire pipeline is not one end-to-end model.

Like if you had one end-to-end model and you had to like, uh, uh, train it to do, do a new thing, like being a one-shot learner is a, is a very big deal. Like, but in our case, we have many AI subsystems that work with each other.

Some are-- Like some of those are end-to-end neural networks, some of them are like hard-coded software pathways. So we basically use the best of both worlds to function. And sort of architecture choice means that we don't have to like-- We, we're not going in from like pixels of what a chef is doing and text or whatever to directly a generalizable recipe that can be cooked across any, uh, robot, any time scale.

It's... There, there are like software workflows and pathways before that, that take this chef demonstration, convert that into a chef demonstration, convert that into like an intermediate format that is easily digestible by different parts of our system. And yeah, I mean, one example for you would be, so say if you're like making an omelet and if you want to teach how-- teach it how to make an omelet.

What we are doing is we're not learning a new omelet making skill while we are, while we are showing the robot this. Like if we had that capability, we would be a robot foundation model and we would, uh, already be single shot learning.

It is that from a single demonstration, assuming that we have all the base skills for the robot to do it, it is extracting what kind of decisions the chef is making. Uh, is it visual? Is it thermal? What kind of skill the chef would be in-- uh, is invoking in themselves?

Is it like stir, sauté? And what are the parameters to those skills? So for us, learning is basically configuring this AI system and not going into a end-to-end model that's going directly from pixels to robot actions. We wouldn't be able to do a single shot recipe learning.

We would have to have the chef cook the recipe in va-various different backgrounds, various different sizes, various different appliances. Because we have these ethic engineered midpoint, like, uh, engineered, engineered midpoint go from one expert demonstration to, uh, uh, form a-- to a recipe form that can then be re-recreated across different kitchens, different appliances and, uh, in the future, also different robot mor-morphologies.

Ooh. Other, other robot morphologies. It's fun. Yeah. Now you only do the two arms, right? Okay. So we'll get, we'll get people to call to action and then we'll, we'll cut to the video. You are going to be at the AI Engineer World's Fair next week.

Um, well, if people wanna see the robot live, they can, they can see it there. Probably taste some food. Although I, I don't know how much food we can serve.

Yeah. We'll see.

We'll see . And, uh, obviously, I think part of the reason you're doing this is you're trying to hire, right?

Yes.

This is a inter-- like immediately applicable use case. Like w-- you know, what, what's, what's the, what's the pitch for engineers? What are they working on?

The pitch for engineers is that like there are only a handful of applied robotics companies that have a path to deploy more than a hundred robots in the next one year. And now that our robot is working and we have early signs of it being super helpful to customers, you will actually be working on a robot that is in production.

There are people in robotics who are working on more complex hardware, uh, more complex hardware problems, more complex software problems. But I think we are at the efficient frontier of value being delivered to the customer using cutting edge techniques and having a rapid scale-up pipeline.

I think that's not a lot of companies can say that they have all these th-three things. And cooking, like I said, it's a-- I think we have a very powerful mission in the sense that today the food that you're eating is fast food.

Like most cheap food is fast food. Whereas fast-forward ten years, you can eat very high quality food. And in fact, if you work with us, you can already eat very high quality food in our office, as Sean and Vibhu can, uh, confirm.

But I think coming back to it, I think the mission's very powerful. I think if you are excited by creating value in the real world while also doing it in a way that serves all the current capabilities of the state-of-the-art, uh, general purpose models, you're probably one of the maybe like two or three companies that are at the intersection of that.

And if that excites you, you should come talk to me.

Yeah. I, I think that's a actually a pretty, very, very strong pitch. I would say that you can a-- you can actually even just try the food on the-- on Uber Eats. Um, you can just kinda order here.

It's one of those like virtual cloud kitchens and it looks so good. Uh, we've tried it. We'll cut to the video later. But Yeah. Thanks for jumping on and sharing, uh, your journey with us. I think this is, um, very exciting.

I, I, I think you've, like, somehow found the way towards, like, the most immediate applicable industrial use case of robots and, like... And, and there's obviously a lot of scope for vision language models and solving a lot of hard engineering problems.

Like, and one thing you did say is, like, all this is within very tight engineering parameters, which I think, uh, is, is, is pretty hard. Like, you, you have to run it, run it a lot of frames per second and also, like, do a lot of that on device.

So, yeah. Cool. Um, well, I'm looking forward to see you next week at the, the conference, and, uh, we'll cut to the video now.

All the culinary decision-making is, like, 100% autonomous, and, uh, the actions are 90% autonomous.

The water has gone off, or, like, the probability of all water going off is more than 90%, and we have safety filters like that. That's what makes it deployable. Otherwise, it's not very deployable. So basically, how appliances are controlled is we go into, like, any...

So any appliance in any kitchen is controlled either using a knob or a touch screen. So we go in, remove the knobs that control the appliance, and then put these knobs in that can turn themselves.

Oh.

So every... So that gives us a actuation surface across all appliances. So we don't need to teach our robot to... For a salary.

Oh.

Yeah. So the-

Is it an hour?

Yes.

12 bucks an hour is what it looks like.

12... Yeah, so 12 bucks an hour is what you pay this. No CapEx. So-

What?

Yeah.

Oh. Is there anything else that's going on? Like, any other equipment?

I mean, for example, we... Basic... Uh, the other thing is all the ingredients that are required basically get measured in these weighing scales.

Okay.

And regardless of which kitchen we go to, all kitchens store their ingredients in boxes. We just, uh, slap, uh... Yeah, we, we just slap a bun- bunch of QR-