# Language Agents: From Reasoning to Acting — with Shunyu Yao of OpenAI, Harrison Chase of LangGraph Latent Space · 2024-09-27 Shunyu Yao of OpenAI and Harrison Chase of LangChain discuss the evolution of language agents, arguing that combining reasoning and acting through techniques like ReAct remains foundational, while tool design and cognitive architectures like CoALA will shape the next wave. Shunyu traces his work from text games to ReAct, showing how thinking as an action improves reliability; Harrison explains LangChain's adoption and the shift to LangGraph for stateful orchestration. They emphasize agent-computer interfaces (ACI) as a neglected area, noting that SWE-Agent's success came from optimizing tools for models, not humans. The conversation covers memory types (semantic, episodic, procedural) and applies CoALA's three dimensions to current systems. They highlight promising applications: customer support, coding agents, and spreadsheet-style UX for batch operations. Shunyu stresses that better benchmarks like SWE-Bench (which solved 30% of GitHub issues) and TauBench (best model 48%) are critical, and that data—not architecture—will drive future model improvements. ## Questions this episode answers ### What inspired Shunyu Yao's ReAct framework for language models? Shunyu Yao explains that during his PhD, he worked on text-based adventure games and noticed that RL agents just overfit to the game without language understanding. Humans solve these games by thinking step-by-step, so he wanted to give agents an 'inner monologue' to reason before acting. This idea, prototyped in 2021 before Chain of Thought, later applied to Wikipedia tasks, leading to the ReAct paper that combines reasoning and acting to interact with external environments. [6:27](https://addtry.com/43cad3b9-5322-4b14-9538-b59705bd5eec?t=387000) ### How does the Reflection paper extend agent capabilities, and what are its limitations? Shunyu Yao says Reflection replaces scalar rewards with language feedback, allowing agents to reflect on their performance and improve prompts. It works best when there is a reliable evaluator, such as coding errors, because the agent can reflect on bugs. However, it adds latency, making it unsuitable for real-time tasks, and is better used as a training-time technique to generate data for fine-tuning simpler methods. [15:35](https://addtry.com/43cad3b9-5322-4b14-9538-b59705bd5eec?t=935000) ### What is Agent-Computer Interface (ACI) design and why does it matter for AI agents like SWE-Agent? Shunyu Yao emphasizes that before optimizing agent prompts, one should design the environment interface for the agent. In SWE-Agent, they modified the terminal to provide feedback like syntax errors after edits, which standard terminals lack. This ACI approach treats agents as customers, and Yao argues that making the tool good and reliable is often 90% of an agent's success, simplifying the agent's logic. [40:31](https://addtry.com/43cad3b9-5322-4b14-9538-b59705bd5eec?t=2431000) ### What are the three key components of the CoALA cognitive architecture for language agents? CoALA organizes agents along three dimensions: information storage (working and long-term memories), action space (internal actions like thinking and external tool use), and decision-making (a loop with planning and execution). This framework provides a systematic way to understand agent design, and Yao notes it remains useful for analyzing even hypothetical powerful future models like GPT-10 by dissecting their components. [1:04:52](https://addtry.com/43cad3b9-5322-4b14-9538-b59705bd5eec?t=3892000) ## Key moments - **[0:00] Intro** - [2:00] Shunyu Yao's PhD advisor Karthik was second author of GPT-1, prompting Yao's shift to language models. - **[3:16] ReAct** - [4:23] Shunyu Yao: 'ReAct doesn't change the outside or the environment, but it does change the inside through the context.' - [5:20] Shunyu Yao frames AI's two branches: RL for agents, NLP for reasoning, and ReAct bridged them. - [6:27] Shunyu Yao's ReAct idea originated from adding a thinking step to text-based game agents. - [8:07] Shunyu Yao's ReAct prototype was built in November 2021, before the Chain of Thought paper. - **[9:09] Early apps** - [9:09] Harrison Chase built the first LangChain ReAct demo with a calculator and search tool. - [10:35] Shunyu Yao says ReAct's lasting contributions are general tool interaction and pairing inner monologue with tools. - [12:10] Harrison Chase notes OpenAI recommends including a 'thought' field in tool calling for improved results. - **[14:15] Reflection** - [14:15] Shunyu Yao reveals Noah Shinn chose Sierra over OpenAI based on his Reflection paper as a second-year undergrad. - [16:03] Shunyu Yao describes reflection as replacing RL's scalar reward with language feedback, akin to a boss's critique. - [18:10] Shunyu Yao says self-reflection needs a good evaluator, like coding errors, but is hard for pure reasoning. - **[19:35] Tree of Thoughts** - [19:35] Shunyu Yao: Tree of Thoughts uses search over thoughts, suited for tasks where latency is not critical like math proofs. - [24:03] Shunyu Yao: 'Thinking is kind of like taking an action, you can use action searching algorithms to think of thinking.' - **[24:21] SWE-Bench** - [25:40] Harrison Chase cites Jason Wei's five axes for evaluating prompting strategies: ease, compute, tasks solved, improvement, and future relevance. - [26:39] Shunyu Yao emphasizes that for practical prompting, simplicity is key because it requires fewer decisions. - [28:05] Shunyu Yao: 'I have never done those weird tricks. All the prompts that I write are kind of just talking to a human.' - [29:21] Shunyu Yao says modern language models are optimized for agentic use, reducing need for prompt engineering tricks. - [29:58] Swyx argues: 'You should not be a prompt engineer because it is the goal of the big labs to put you out of a job.' - [30:44] Shunyu Yao: MCTS was magical for Go but not for robotics; agent benchmarks lag behind methods. - **[36:21] CoALA** - [38:43] Shunyu Yao on SWE-Bench: 'The idea is super easy, but the engineering is super hard — a typical signal of a good work in the AI era now.' - [40:31] Shunyu Yao says Agent-Computer Interfaces (ACI) are a neglected part of agent design, as seen in SWE-Agent's terminal improvements. - [41:51] Shunyu Yao: 'We should treat agents as our customers, designing Agent-Computer Interfaces just as we do HCI for humans.' - **[42:24] ACI** - [43:42] Shunyu Yao suggests A/B testing agent interfaces serves both practical improvement and deeper understanding of agents. - [45:35] Shunyu Yao: 'Making the tool good and reliable is probably ninety percent of the whole agent; agent design can then be simple.' - [46:16] Swyx raises the tension: should frameworks be optimized for easy agent writing or human developer experience? - [47:22] Swyx notes Malte Ubl's observation that APIs easy for humans are easier for LLMs, but he still tweaks field names for LLM performance. - [49:36] Harrison Chase and Swyx discuss how LLM-optimized search APIs like Tavily provide cleaner, smaller outputs than generic Bing APIs. - **[50:52] UX Design** - [50:52] Swyx argues that error messages in agent tools should become more verbose to act as prompts for LLM self-correction. - [52:10] Shunyu Yao points out a fundamental ACI difference: human limited working memory vs. LLMs' ability to process 100 results at once. - [53:31] Swyx proposes a 'sour lesson' that AI will follow its own path, diverging from human-inspired designs. - [55:01] Shunyu Yao and Swyx debate whether intelligence can be separated from knowledge, with Yao citing Hinton's 'learning first' philosophy. - [56:12] Swyx suggests Apple Intelligence's on-device LoRAs may be an early step toward separating model intelligence from specific knowledge. - [58:13] Shunyu Yao says agent data is the main bottleneck for model improvement, as the internet only records final results, not process. - **[58:53] Data & Models** - [59:36] Shunyu Yao's FireAct paper proposes training models on diverse correct agent trajectories to learn when to use simple or complex methods. - [1:02:00] Shunyu Yao: 'The simplest way to achieve AGI is literally just record the ReAct trajectory of every human being.' - [1:04:46] Shunyu Yao's CoALA framework organizes agents into three dimensions: memory storage, action space, and decision-making loop. - [1:06:44] Shunyu Yao: 'A very first principle way of thinking of agents is pretty much a neural network plus the code to call and use the neural network.' - [1:08:02] Shunyu Yao credits ReAct with introducing internal actions (thinking) and says CoALA's three dimensions fully describe an agent. - [1:09:09] Harrison Chase explains that because LLMs struggle with planning, LangGraph allows encoding the decision-making procedure in code. - **[1:16:10] TauBench** - [1:16:11] Alessio and Shunyu Yao discuss TauBench, where GPT-4o scores only 48% on customer service tasks with simulated users. - [1:19:09] Shunyu Yao explains TauBench's design relies on information asymmetry: the simulated user expresses needs, but the agent handles complex reasoning. - [1:20:03] Harrison Chase identifies customer support, coding, and research agents like AI SDRs as the most promising AI application areas. - **[1:20:09] Future** - [1:22:10] Harrison Chase highlights a spreadsheet-style UX for agents, where each cell runs an agent and the first rows provide few-shot examples. - [1:23:06] Harrison Chase reveals he built an email assistant with LangGraph that triages his inbox and proactively asks for input on ambiguous emails. - [1:24:10] Harrison Chase introduces LangGraph Studio, an agent IDE with graph visualization and time travel debugging inspired by Devin. ## Speakers - **Alessio** (host) - **Swyx** (host) - **Harrison** (guest) - **Shunyu Yao** (guest) ## Topics Agent Platforms ## Mentioned Cognition (company), Meta (company), OpenAI (company), Sierra (company), Code Interpreter (product), Devin (product), GPT (product), LangChain (product), LangGraph (product), LangMem (product), LangSmith (product), Llama (product), SWE-Agent (product), SWE-Bench (product), TauBench (product) ## Transcript ### Intro **Alessio** [0:03] Hey, everyone. Welcome to the Latent Space Podcast. This is Alessio, partner and CTO in residence at Decibel Partners, and I'm joined by my co-host Swyx, founder of Small AI. **Swyx** [0:12] Hey, and today we have a super special episode. I actually almost wanted to take like a selfie and go like, you know, POV, you're about to cha- revolutionize the world of agents because we have, uh, two of the most awesome pioneers in agents in the house. So first, we're gonna welcome back Harrison Chase. Welcome. **Harrison** [0:28] Excited to be here. **Swyx** [0:30] Uh, what's new with you recently in, in sort of like the ten, twenty-second recap? **Harrison** [0:34] LangChain, LangSmith, LangGraph, pushing on all of them. Lots of cool stuff related to a lot of the stuff that we're gonna talk about today, probably. **Swyx** [0:42] Yeah, we'll mention it in there. Yeah. **Alessio** [0:43] And the Celtics won the title. **Harrison** [0:45] And the Celtics won the title. **Alessio** [0:45] So you got, you got that going on for you. **Swyx** [0:47] I don't know. Is that like floorball, handball- ... baseball? **Alessio** [0:52] Basketball. **Swyx** [0:53] Basketball. Basketball. **Alessio** [0:53] I'm joking. **Harrison** [0:53] Patriots aren't looking good though, so that's, uh- **Swyx** [0:56] Right. And then Shunyu, y-you've also been on the pod, but only in like a sort of oral paper presentation capacity. But welcome, uh, officially to, the Latent Space Pod. **Shunyu Yao** [1:04] Yeah, I've been a huge fan, so thanks for the invitation. **Swyx** [1:07] Uh, thanks. Uh, thank-- well, it's an honor to have you on. You're one of like... you're maybe the first PhD thesis defense I've ever watched in, in like- ... this, this AI world because most peop- most people just publish single papers, but every paper of yours is a banger. So congrats. **Shunyu Yao** [1:22] Thanks. **Swyx** [1:24] Yeah, maybe we'll just kick it off with, um, you know, what was your journey into using language models for agents? I f- I like that your thesis advisor, I didn't catch his name, but he was like, you know, Karthik. Yeah, like, it's like this guy like just wanted to use language models, and it was such a- ... a controversial pick at the time. **Shunyu Yao** [1:38] Right. The full story is that in undergrad, I did some computer vision research, and that's how I got into AI. But at the time, I feel like, you know, you're just composing all the GAN or 3D perception or whatever together, and it's not ex-exciting anymore. And one day, I just see this transformer paper, and that's really cool. But I, uh, I really g-got into language model only when I entered my PhD and, uh, met my advisor, Karthik. So he was actually the second author of GPT-1 when he was like a visiting scientist at OpenAI. **Swyx** [2:09] With Alec Radford? **Shunyu Yao** [2:11] Yes. **Swyx** [2:11] Wow. **Shunyu Yao** [2:12] That's what he told me, is like back in Ap- OpenAI, they did this GPT-1 together, and Ilya just said, "Karthik, you should stay because we just solved the language." But apparently, Karthik is not fully convinced, so he went to Princeton and start his professorship. And I'm really grateful, so he, uh, accepted me as a student, even though I have no prior knowledge in NLP and, you know, we just met for the first time and he's like, you know, "What do you want to do?" And I'm like, "You know, you have done those task games things. That's really cool. I wonder if we can just redo them with language models." And that's how the whole journey began. **Swyx** [2:46] Awesome. **Alessio** [2:46] And that was GPT-2, was that- **Shunyu Yao** [2:48] Yes **Alessio** [2:48] ... at the time? **Shunyu Yao** [2:48] That was 2019. Yeah. **Alessio** [2:50] Way too dangerous to release. **Shunyu Yao** [2:52] Yeah. **Alessio** [2:52] And then I would guess the first work of yours that I ca-came across was ReAct- **Shunyu Yao** [2:57] Sure **Alessio** [2:57] ... which was a big part of your, uh, defense. But also, Harrison, when you came on the podcast last year, you said that was one of the first papers that you saw when you were getting inspired for LangChain. So maybe give a recap of why you thought it was cool because you were already working in AI and machine learning, and then yeah, you can kind of like intro the paper formally. But why was that interesting to you specifically? ### ReAct **Harrison** [3:17] Yeah, I mean, I think the interesting part was using these language models to interact with the outside world in some form and, and I think in the paper you mostly deal with Wikipedia, and I think there's some other datasets as well, but the outside world is the outside world, and so interacting with things that weren't present in the LLM and APIs and calling into them and thinking about it and the, yeah, the, the ReAct reasoning and acting and kind of like combining those together and getting better results. I'd been playing around with LLMs, been talking with people who are playing around with LLMs. People were trying to get LLMs to call into APIs, do things, and it was always, how can they do it more reliably and better? And so this paper was basically a step in that direction, and I think really interesting and also really general as well. Like, I think that's part of the appeal is just how general and, and simple in a good way, I think the idea was, so that it really appealing for all those reasons. **Shunyu Yao** [4:07] Simple is always good. **Harrison** [4:09] Yeah. **Alessio** [4:10] Do you have a favorite part? Because I have one favorite part from your PhD defense, which I didn't understand when I read the paper, but you said something along the lines, "ReAct doesn't change the outside or the environment, but it does change the inside through the context." Putting more things in the context, you're not actually changing any of the tools around you to work for you, but you're changing how the model thinks, and I think that was like a very profound thing when I-- now that I've been using these tools for like eighteen months, I'm like, I understand what you meant, but like to say that at the time you did the PhD defense was not trivial. **Harrison** [4:41] Yeah. **Shunyu Yao** [4:41] Another way to put it is like, um, thinking can be an extra tool that's useful. **Harrison** [4:48] Makes sense. **Shunyu Yao** [4:48] Yeah. **Harrison** [4:49] Checks out. **Swyx** [4:49] Who would have thought? I think it's also more controversial within his, his world because everyone was trying to use RL for agents. **Shunyu Yao** [4:57] Right. **Swyx** [4:57] And this is like the first kind of zero gradient type approach. **Shunyu Yao** [5:01] Yeah, I, I think the, the bigger kind of historical context is that, uh, we have these two big branches of AI, right? So if you think about RL, right, that's pretty much the equivalent of agent at the time, and it's like agent is equivalent to reinforcement learning, and reinforcement learning is equivalent to whatever game environment they're using, right? Atari game or Go or whatever. So you have like a pretty much, you know, you have a biased kind of like set of methodologies in terms of reinforcement learning and represents agents. On the other hand, I think NLP is like a historical kind of subject. It's not really into agents, right? It's more about reasoning. It's more about solving those concrete tasks. And if you look at SAL, right, like each task has its own track, right? Summarization has a track. Question answering has a track. So, so I think really it's about rethinking agents in terms of what could be the new environments that we came to realize, not just Atari games or whatever video games, but also those text games or language games. And, uh, also thinking about could there be like a more general kind of methodology beyond just designing specific- ... pipelines for each NLP task. That's like the bigger kind of context, I would say. **Alessio** [6:14] Is there a inspiration spark moment that you remember, or how did you come to this? We had Tree down on the podcast, and he mentioned, uh, he was really inspired working with like systems people to think about Flash Attention. What was your inspiration journey? **Shunyu Yao** [6:27] Yeah. So actually before ReAct, I spent the first two years of my PhD focusing on text-based games or, or in other words, text adventure games. It's a very kind of small kind of research area and quite ad hoc, I would say. And, uh, there are like, I don't know, like 10 people working on that at the time. And, uh, have you guys heard of Zork one, for example? So basically the idea is you have this game and you have text observations. Like you see a, you see a monster, you see a dragon and- **Harrison** [6:57] You're eaten by a grue. **Shunyu Yao** [6:59] Yeah. You're eaten by a grue. And you have actions like kill the grue with a sword or whatever, right? And that's like a very typical setup of a text game. So I think one day after, you know, I've seen all the GPT-3 stuff, I just think, think about, you know, how, how can I solve the game? Like why those AI, you know, machine learning methods are pretty stupid, but we are pretty good at solving the game relatively, right? So for the context, the predominant method to solve this text game is obviously reinforcement learning, and the idea is you just trial and error in those games for like millions of steps and you kind of just overfit to the game, but there's no language understanding at all. And I'm like, "Why can't I solve the game better?" And it's kind of like, because we think about the game, right? Like when we see this very complex text observation, like you see a grue and you might see a sword, you know, in the right of the room and, and you have to go through the wooden door to go to that room. You, you will think, you know, oh, I have to kill the monster, and to kill that monster, I have to get the sword. Like to get the sword, I have to go right. And this kind of thinking actually helped us kind of throw shots to solve the game. And it's like, why don't we also enable the text agents to think? And that's kind of the prototype of ReAct. And I think that's actually very interesting because the prototype, I think, was around November of 2021, so that's even before like Chain of Thought or whatever came up. So we did a bunch of experiments in the text game, but it was not really working that well. Like this text game are just too hard. I think today it's still very hard. Like if you use GPT-4 to solve it, it's still very hard. So the change came when I, uh, started the internship in Google, and apparently Google care less about text game. They, they care more about what's more practical. So, uh, pretty much I just reapplied the idea, but to more practical kind of environments like Wikipedia or- **Alessio** [8:51] Ah. **Shunyu Yao** [8:52] Or, or, or like simpler text game like Alford, and it just worked. It's kind of like you first have the idea and then you try to find the domains and the problems to, to demonstrate the idea, which is, I would say, different from most of the AI research, but it kind of worked out for me in that case. Yeah. For Harrison, when you were implementing ReAct, like what were people applying ReAct to in the early days? ### Early apps **Harrison** [9:15] I think the first demo we did probably had like a calculator tool and a search tool, so like general things. We tried to make it pretty easy to write your own tools and plug in your own things, and so w- this is one of the things that we've seen in LangChain is people who build their own applications generally write their own tools. Like there are a few common ones. I'd say like the three common ones might be like a browser, a search tool, and a code interpreter. **Shunyu Yao** [9:36] Yeah. **Harrison** [9:36] But then other than that- **Shunyu Yao** [9:37] Like thesis, the LMOs. **Harrison** [9:38] Yep. Yeah, exactly. It matched up, it matched up very nice with that. Um, and we just, we actually just redid like our integrations docs page, and if you go to the tool section, we like highlight those three, and then there's a bunch of like other ones. **Shunyu Yao** [9:48] Yeah. **Harrison** [9:49] And there's such a long tail of other ones, but in practice, like it-- when people go to production, they generally have their own tools or maybe one of those three, maybe some other ones, but like very, very few other ones. So yeah, I would, I think the first demos was a search and the calculator one, and there's... What's the data set? **Shunyu Yao** [10:04] Hotpot QA. **Harrison** [10:05] Yeah. Oh, so there's that one and then there's like the celebrity one by the same author, I think. **Shunyu Yao** [10:10] The Olivia Wilde's boyfriend squared- **Harrison** [10:12] Yeah. **Shunyu Yao** [10:12] ... to the power of zero point two three. **Harrison** [10:15] Yeah. **Shunyu Yao** [10:15] Right, right. **Harrison** [10:16] There's... I'm forgetting the name of the author, but there's- **Shunyu Yao** [10:18] I was like, we're gonna over optimize for Olivia Wilde's boyfriend, and it's gonna change next, next year or something. **Harrison** [10:22] There's a few data sets kind of like in that vein that require multi-step kind of like reasoning and thinking. So one of the questions I actually had for you in this vein, like the ReAct paper, there's a th- I think there's a few things in there, or at least when I think of that, there's a few things that I think of. There's kind of like the specific prompting strategy, then there's like this general idea of kind of like thinking and then taking an action, and then there's this even more general idea of just like taking actions in a loop. Today, like obviously language models have changed a lot. We have tool calling. The specific prompting strategy probably isn't used super heavily anymore. Would you say that like the concept of ReAct is still used though, or like do you think that tool calling and running tool calling in a loop, is that ReAct in your mind? **Shunyu Yao** [11:03] I would say like, uh, it's like more implicitly used than explicitly used. To be fair, I think the, the contribution of ReAct is actually twofold. So, so first is this idea of, you know, we should be able to use calls in a very general way. Like there should be a single kind of general method to handle interaction with various environments. I think ReAct is the first paper to demonstrate the idea, but then I think later there are tool former or whatever, and this become like a trivial idea. But I think at the time, that's like a pretty non-trivial thing. And I think the second contribution is this idea of what people call like inner monologue or thinking or reasoning or whatever to be paired with tool use. I think that's still non-trivial because if you look at the default function calling or whatever, like there's no inner monologue. And in practice, that actually is important, especially if the tool that you use is pretty different from the training distribution of the language model. So I think that's the, like those are the two like main things that are kind of Yeah, inherited. **Harrison** [12:10] Yeah. On that note, I, I think OpenAI even recommended, like, when you're doing tool calling, it's sometimes helpful to put, like, a thought field in the tool a-along with all the actual required arguments- **Shunyu Yao** [12:19] Right **Harrison** [12:20] ... and then have that one first, so it fills out that first and then-- and that's-- they've shown that that's yielded kind of, like, better results. The reason I ask is just, like, the same concept is still alive and it-- and I don't know whether to call it, like, a ReAct agent or not. Like, I, I don't know what to call it. Like, I think of it as ReAct. Like, it's the same ideas that, that were in the paper, but it's obviously a very different implementation at this point in time, and so I just don't know what to call it. **Shunyu Yao** [12:40] I feel like people will sometimes think more in terms of different tools, right? Because if you think about a web agent versus, you know, like a function calling agent calling a Python API, you would think of them as very different. But in some sense, the methodology is the same. It depends on how you view them, right? And I think people will tend to think more in terms of the environment and the tools rather than the methodology. So-- or in other words, I think the methodology is kind of trivial and simple, so people will try to focus more on the different tools. But I think it's good to have, like, a single underlying principle underlying all of those things. Yeah. **Swyx** [13:18] How do you see the surface of ReAct getting molded into the model? So function calling is a good example of, like, now the model does it. What about the thinking? Now most models that you use kind of do chain of thought on their own. They kind of produce steps. Do you think that more and more of this logic will be in the model, or do you think, like, the context window will still be the main driver of, like, reasoning and thinking? **Shunyu Yao** [13:39] I think it's already default, right? Like, like you do some chain of thought, and you do some tool call. Like, the cost of adding the chain of thought is kind of relatively low compared to other things, so it's not hurting to do that, and I think it's already kind of common practice, I would say. Yeah. **Swyx** [13:57] Is this a good place to bring in either tree of thought or reflection? Uh, your pick. **Shunyu Yao** [14:01] Maybe reflection, like to respect the time order, I would say. **Swyx** [14:05] Yeah. Any backstory as well. Like, you know- **Shunyu Yao** [14:07] Yeah **Swyx** [14:07] ... the people involved with Noah and, like, the Princeton group- **Shunyu Yao** [14:09] Yeah, yeah, yeah. Yeah **Swyx** [14:09] ... I think, uh, you know, we talked about this offline, but people don't understand how these- ... research pieces come together and this ideation. ### Reflection **Shunyu Yao** [14:15] Yeah, yeah. I think reflection is mostly Noah's work, like a more, like, advising kind of role. So the story is, I don't remember the time, but, like, one day we just see this preprint that's like reflection and autonomous agent with memory or whatever, and it's kind of like a extension to ReAct which uses this self-reflection. I'm like, "Oh," somehow it become very popular. **Swyx** [14:39] Yeah. **Shunyu Yao** [14:39] And Noah reached out to me and was like, "Do you want to collaborate on this and make this from, like, a archive preprint to something more solid, you know, like a conference submission?" I'm like, "Sure." We start-started collaborating, and we remain good friends today. And, uh, I think another interesting backstory is, like, Noah was like, I think, contacted by OpenAI at the time and was like, "This is pretty cool. Do you want to just work at OpenAI?" And- **Swyx** [15:02] Wow **Shunyu Yao** [15:02] ... I think Sierra also reached out at the same time and was like, "This is pretty cool. Do you want to work at Sierra?" And, and I think Noah chose, uh, Sierra, but- ... i-it's pretty cool because he was, like, still, like, a second year undergrad, and he's a very smart kid. **Swyx** [15:17] Based on one paper. **Shunyu Yao** [15:18] Based on one paper, yeah. **Swyx** [15:19] Oh, my God. **Shunyu Yao** [15:20] He's done some other research based on, like, programming language or, or chemistry or whatever, but I think that's the paper that got the attention of OpenAI and Sierra. **Swyx** [15:28] Okay. For those who haven't gone too deep on it, the way that you present the inside of ReAct, like, can you do that also for re-reflection? **Shunyu Yao** [15:35] Yeah. I think one way to think of reflection is that the, the traditional idea of reinforcement learning is you have a scalar reward, and then you, you somehow back propagate the signal of the scalar reward to the rest of your neural network through whatever algorithm, like policy gradient or A2C or whatever. And if you think about the real life, you know, most of the re-reward signal is not scalar. It's like your boss to-told you, you know, "You should have done a better job in this, but good job on that," or whatever, right? It's not like a scalar reward, like twenty-nine or something. I think in general, human do more-- deal more with, you know, long scalar reward, or you can say language feedback, right? And the way that they deal with language feedback also have this kind of back propagation kind of process, right? Because you start from this, "You did a good job on job B," and then you reflect, you know, what could I have done different to, to change to make it better? And you kind of change your prompt, right? Basically, you change your prompt on how, how to do job A and how to do job B, and then you, you do the whole thing again. So it's really like a pipeline of language where instead of gradient descent, you have something like text reasoning to, to replace those gradient descent algorithms. I think that's one way to think of reflection, yeah. **Harrison** [16:47] One question I have about reflection is how general do you think the algorithm there is? And so for context, I think at LangChain and at other places as well, we found it, like, we and others found it pretty easy to implement, kind of like ReAct in a standard way. You plug in any tools, and it kind of works off the shelf, you know, can get it up and running. I don't think we have, like, an off-the-shelf kind of like implementation of reflection in kind of like the general sense. I think the concepts, like absolutely we see used in different kind of like specific cognitive architectures, but I don't think we have one that comes off the shelf. I don't think any of the other frameworks have one that comes off the shelf, and I'm curious whether that's because it's not general enough or it's complex as well, 'cause it also requires running it more times. Maybe that's not feasible. Like, I'm curious how you think about the generality, complexity. Why, yeah, should we have one that comes off the shelf? **Shunyu Yao** [17:36] I think the algorithm is general in the sense that it's just as general as, like, other algorithms if you think about, like, policy gradient or whatever. But it's not applicable to all tasks, just like other algorithms, right? So you can argue PPO is also general, but it works better for those set of tasks but not on those set of tasks. I think it's the same situation for reflection, and I think a key bottleneck is the evaluator, right? Basically, you need to have a good sense of the, the signal. So for example, like, if you're trying to do a very hard reasoning task, say mathematics For example, and you don't have any tools, right? It's you operating in this chain of thought setup. Then reflection will be pretty hard because in order to reflect upon your thoughts, you have to have a very good evaluator to judge whether your thought is good or not. But that might be as hard as solving the problem itself or even harder. The principle of self-reflection is probably more applicable if you have a good evaluator, for example, in the case of, like, coding, right? Like, if you have those errors, then you can just reflect on that and how to solve the bug and stuff. So I think another criteria is that it depends on application, right? Like if you have this latency or whatever need for like actual application with a end user, right? End user wouldn't let you, you know, do like two hours of tree of thought or reflection, right? You need something as soon as possible. So in that case, maybe this is better to be used as like a training time technique, right? You do this reflection or tree of thought or whatever. You get a lot of data, and then you try to use the data to train your model better. And then in test time, you use, still use something as simple as ReAct, but that's already improved. **Alessio** [19:11] And if you think of the Voyager paper as like a way to store skills and then reuse them, like how would you compare like this like reflective memory and like at what point it's just like ragging on the memory versus like you wanna start to fine-tune some of them or like what's the next step once you get a very long kinda like, uh, reflective corpus? **Shunyu Yao** [19:30] Yeah. So I think there are two questions here. The first question is what type of information or, or memory are you considering, right? Is it like semantic memory that stores, you know, knowledge about the word, or is it the episodic memory that stores, you know, trajectories or behaviors, or is it like more of a procedure memory, like in Voyager's case, like skills or code snippets that you can use to do actions, right? ### Tree of Thoughts **Shunyu Yao** [19:54] That's, that's one dimension. And the second dimension is obviously how you use the memory, either retrieving from it, using it in the context or, or fine-tuning it. I think the cognitive architecture for language agents paper have a good kind of categorization of all the different combinations. And of course, what-- which way you use depends on the concrete application and, and the concrete need and the concrete task. But I think in general, it's good to think of those like systematic dimensions and all the possible like options there. Yeah. **Swyx** [20:25] Harrison also has in LangMem, I think you, you did, you did a presentation in my meetup, and I think you've done it at a couple other venues as well. User state, semantic memory, and append-only state I think kind of maps to what you just said. **Shunyu Yao** [20:38] What is LangMem? Can you give it like a quick- **Harrison** [20:40] One of the modules of LangChain for a long time has been something around memory, and I think like, you know, we're still obviously figuring out what that means, as is everyone kind of in the space. But one of the experiments that we did and one of the proof of concepts that we did was technically what it was is, uh, you would basically create threads. You'd push messages to those threads. In the background, we process the data in a few ways. One, we like put it into some semantic store. That's the semantic memory. And then two, we do some like extraction and reasoning over the memories to extract, and we let the user define this, but like extract key facts or anything that's of interest to the user. Those aren't exactly trajectories. They're maybe more closer to the, to the procedural memory. Is that how you'd think about it or classify it, or? **Shunyu Yao** [21:22] Is it like about like knowledge about the word, or is it more like how to do something? **Harrison** [21:27] It's reflections basically, so in Generative Worlds. **Shunyu Yao** [21:31] Generative Agents. **Harrison** [21:31] Generative- **Shunyu Yao** [21:32] The Smallville. **Harrison** [21:33] Yeah, the Smallville one. So the way that they had their memory there was they had the sequence of events, and that's kind of like the raw events that happened. But then every n events, they'd like run some synthesis over those events to, for the, for the LLM to insert its own memory basically, and it's, it, it's that type of memory. I don't know how that would be classified, but- **Shunyu Yao** [21:50] Uh, I, I think of that more as a semantic memory. But to be fair, I think it's just one way to think of that. But whether it's semantic memory or procedure memory or whatever memory, that's like, like a abstraction layer. But in terms of implementation, you can choose whatever implementation for whatever memory, so they're totally kind of orthogonal. I think it's more of a good way to think of the things because, like, from the history of cognitive science and, you know, cognitive architecture and how people study even neuroscience, right? That's the way people think of how human brain organizes memory. And I think it's more useful as a way to think of things, but it's not like for semantic memory you have to do this kind of like way to retrieve or fine-tune, and for procedure memory you have to do that. Like, I think those are totally orthogonal kind of dimensions. **Harrison** [22:35] How much background do you have in kind of like cognitive sciences, and how much do you model some of your thoughts on- **Shunyu Yao** [22:41] Ooh. That's a great question actually. And, uh, I think one of the undergrad kind of influence for my like follow-up research is I was doing like a internship at MIT's Computational Cognitive Science Lab with like, you know, Josh Tenenbaum, and he's like a very famous cognitive scientist, and I think a lot of, a lot of his ideas still influence me today. Like, uh, think of things in like computational terms and getting interested in language and a lot of stuff. Yeah. Or even like developer psychology kind of stuff I think still influencing me today. Yeah. **Swyx** [23:14] As a developer that tried out LangMem, the way I view it is just it's a materialized view of a stream of logs. And if anything, that's just useful for context compression. I don't have to use the full context to run it over everything, but also it's kind of debuggable. Like if it's wrong, I can show it to the user, user can manually fix it, and I can carry on. **Harrison** [23:31] That's a really good analogy. **Swyx** [23:32] Yeah. **Harrison** [23:32] Yeah, I like that. I'm gonna steal that for sure. **Swyx** [23:34] Please, please. You know I'm bullish on memory databases. Um, I guess Tree of Thoughts? **Alessio** [23:39] Um, yeah, Tree of Thought, I mean, you had a, you had- **Shunyu Yao** [23:41] I, I feel like I'm relieving the defense again in like a podcast format. **Alessio** [23:45] Yeah, no, I mean, it was a... You had a banger. Well, this is the one where you're already successful, and we just like, you know, highlight the, the glory. It was really good. You mentioned that since thinking is kinda like taking an action, you can use like action searching algorithms to think of thinking, so just like you would use tree search to like find the next thing. And the idea behind Tree of Thought is that you generate all these possible outcomes and then find the best tree to get to the end. Maybe back to the latency question, you can't really do that if you have to respond in real time. So what are maybe some of the most helpful use cases for things like this? Where have you seen people adopt it where the high latency is actually worth the, the wait? ### SWE-Bench **Shunyu Yao** [24:22] For things that you don't care about latency, obviously, for example, if you're trying to do math, right? If you're just trying to come up with a proof. But I feel like one type of task is more about searching for a solution, right? You can try a hundred times, but if you find one solution, that's good. Like for example, if you're finding a math proof or if you're finding a good code to solve a problem or whatever. And I think another type of task is more like reacting, right? For example, if you're doing customer service, you're like a web agent booking a ticket for like a end user, right? Those are more like kind of reactive kind of tasks, right? You have to... Or more real-time tasks, right? You have to do things fast. They might be easy, but you have to do it reliably, and you care more about like can you solve 99% of the time out of 100. But for the, the type of search type of tasks, then you care more about, you know, can I find one solution out of 100? So it's kind of symmetric and different. **Swyx** [25:12] Do you have any data or like intuition from your user base, like what's the split of these type of use cases? Like how many people are doing more reactive things and how many people are experimenting with like kind of deep long search? **Harrison** [25:24] I would say like ReAct's probably like the most popular. I think there's aspects of reflection that get used. Tree of Thought probably like the least so. There's a great tweet from Jason Wei, I think your now colleague, and he was talking about like prompting strategies and how he thinks about them. And I think like the four things that he kind of had was like one, how easy is it to m- implement? How much compute does it take? How many tasks does it kind of like solve, and how much does it improve on those tasks? And, and I'd add a fifth, which is like how likely is it to be relevant with when the next generation of models kind of come out? And I think if you look at kind of like those axes and then you look at like, uh, you know, ReAct, Reflection, Tree of Thought, it tracks that the ones that score better are used more. Like ReAct is pretty easy to implement. Tree of Thought's pretty hard to implement. Um, right, like the amount of compute, yeah, a lot more for Tree of Thought. The tasks and how much it improves, I, I don't have amazing visibility there, but I think like if we're comparing ReAct versus Tree of Thought, ReAct just dominates on the first two axes so much that... My question around that was gonna be like how do you think about like these prompting strategies, cognitive architectures, whatever you wanna call them, when you're, when you're thinking of them, what are the axes that you're judging them on in, in your head when you're thinking whether it's a good one or, or a less good one or? **Shunyu Yao** [26:39] Right. Right. I think there is a difference between like a prompting method versus like a research in the sense that like for research, you don't really even care about does it actually work on practical task or does it help whatever. I think it's more about like the idea or the, the, the principle, right? Like what is the direction that you're like unblocking and whatever. And I think for the, for like a actual prompting method to solve like a concrete problem, I would say like simplicity is very important because the simpler it is, the less decision you have to make about it, and it's easier to design, it's easier to propagate, and it's easier to, to do stuff. So always try to be as simple as possible. And, uh, I think latency obviously is im- important, like if you can do things fast and you don't want to do things slow. And I think in terms of the actual prompting method you use for a particular problem, um, I think we should all be in the minimalist kind of camp, right? You should try the minimum thing and see if it works, and if it doesn't work and there's absolute reason to add something, then you add something, right? Like there's absolute reason that you need some tool, then you should add the tool thing. If there's absolute reason to add reflection or whatever, you should add that. Otherwise, if chain of thought can already solve something, then you don't even need to use any of that. **Harrison** [27:57] Yeah, or if just better prompting can solve it, like, you know, you could add a reflection step or you could make your instructions a little bit clearer and it's a lot easier to do that. **Shunyu Yao** [28:05] I think another interesting thing is like I personally have never done those kind of like weird tricks. I, I think all the prompts that I write are kind of like just talking to a human, right? It's like, I don't know, like, like I never say something like, "Your grandma is dying and you have to solve it to..." I mean, those are cool, but I feel like we should all try to solve things in like a very intuitive way, like just like talking to your coworker and that, that should work 99% of the time. That's my personal take, yeah. **Swyx** [28:30] The problem with how language models, at least in the sort of GBT-3 era, was that they're, they over-optimized to some sets of tokens in sequence. So like what reading the Kojima et al paper that was listing step by step, like he tried a bunch of them and they had wildly different results. It should not be the case, but it is the case, and hopefully we're getting better there. **Shunyu Yao** [28:51] Yeah. I think it's also like a timing thing in the sense that if you think about this whole line of language model, right? Like at the time, it was just like a text generator. We don't have any idea how it's going to be used, right? And obviously at the time you will find all kinds of weird issues because it's not trained to do any of that, right? But then I think we have this loop where once we realize chain of thought is important or agent is important or tool using is important, what we see is today's language models are heavily optimized towards those things. So I think in some sense they become more reliable and robust over those use cases and, uh, you don't need to do as much prompt engineering tricks anymore- **Swyx** [29:30] Yeah **Shunyu Yao** [29:30] ... to, to, to solve those things. I feel like in some sense, I feel like prompt engineering even is like a slightly negative word at the time because in reference to all those kind of weird tricks that you have to apply. But I think we don't have to do that anymore. Like given today's progress, you should just be able to talk to like a coworker and if you're clear and concrete and, you know, being reasonable, then it should resolve those things for you. Yeah. **Swyx** [29:51] Yeah. The way I put this is, uh, you should not be a prompt engineer because it is the goal of the big labs to put you out of a job. **Shunyu Yao** [29:58] You should just be a good communicator. Like if you're a good communicator to human- **Swyx** [30:01] Yeah **Shunyu Yao** [30:01] ... you should be a good communicator to language model. **Harrison** [30:02] And I think that's the key though, 'cause oftentimes people aren't good communicators to these language models. **Swyx** [30:07] Yes. **Harrison** [30:07] And that is a very important skill, and that's still messing around with the prompt, and so it depends what you're talking about when you're saying prompt engineer. **Shunyu Yao** [30:14] But do you think it's like very correlated with like are they like a good communicator to humans? You know, it's like- **Harrison** [30:20] It may be, but I also think I would say on average, people are probably worse at communicating with language models than to humans- **Shunyu Yao** [30:25] Oh, that's for sure. That's for sure **Harrison** [30:26] ... right now at least, 'cause I think we're still figuring out how to do it. You kind of expect it to be magical and there's probably some correlation, but I'd say there's also just like people are worse at it right now than talking to humans. **Shunyu Yao** [30:36] Yeah. We should, we should, uh, make it like a You know, like a elementary school class or whatever- **Guest** [30:41] Mm-hmm **Shunyu Yao** [30:41] ... like how to talk to language models. I don't know. **Guest** [30:43] Uh, yeah, that's very pro that. Yeah. **Shunyu Yao** [30:44] Yeah. **Guest** [30:44] Before we leave the topic of, uh, trees and searching. Not specific about Q* but there's a lot of questions about MCTS and this combination of tree search and language models, and I just had to get in a question there about how seriously should people take this? **Shunyu Yao** [31:00] Again, I think it depends on the tasks, right? So MCTS was magical for Go, but it's probably not as magical for robotics, right? So I think right now the problem is not even that we don't have good methodologies. It's more about we don't have good tasks. It's also very interesting, right? Because if you look at my citation, it's, like, obviously the most cited are ReAct, Reflection and Tree of Thoughts are methodologies, but I think, like, equally important if, if not more important line of my work is, like, benchmarks and environments, right? Like WebShop or SWE-Bench or whatever. And I think in general, what people do in academia that I think is not good is they choose a very simple task like AlphaWord and then re- they apply overly complex methods and to show they improve two percent. Uh. Um, I think, like, you should probably match, you know, the level of complexity of your task and your metho- method, right? I feel like where tasks are kind of far behind the, the, the method in some sense, right? Because we have some good test time approaches like whatever ReAct or, or Reflection and Tree of Thoughts or, like, there are many, many more complicated test time methods afterwards. But on the benchmark side, we have made a lot of good progress this year, last year, but I think we still need more progress toward that, like better coding benchmark, better web agent benchmark, better agent benchmark, not even for web or, or code. I think in general we need to catch up with, with tasks. **Guest** [32:28] What are the biggest reasons in your mind why, why it lags behind? **Shunyu Yao** [32:32] I think incentive is one big reason. Like, if you see, you know, all the method paper are cited like 100 times more than the, the task paper. And also making a good benchmark is actually quite hard, and it's almost like a different set of skills in some sense, right? I feel like if you want to build a good benchmark, you need to be like a good kind of product manager kind of mindset, right? You need to think about why people should use your benchmark, why it's challenging, why it's useful. If you think about, like, a PhD going into, like, a school, right? The prior skill that expected to have is more about, you know, can they code this method and can they just run experiments and can solve that. I think building a benchmark is not the typical prior skill that we have. But I think things are getting better. I think more and more people are starting to bu-build benchmarks, and people are seeing that as, like, a way to get more impact in some sense, right? Because, like, if you have a really good benchmark, a lot of people are gonna use it. But if you have a super complicated test time method, like it's very hard for people to use it. **Guest** [33:36] Are evaluation metrics also part of the reason? Like, for some of these tasks that we might wanna ask these agents or language models to do, is it hard to evaluate them, and so it's hard to get an automated benchmark? Obviously, with SWE-Bench, you can, and with coding it's, it's easier, but... **Shunyu Yao** [33:51] I think that's part of the, like, the skill set thing that I mentioned because I feel like it's like a product manager because there are many dimensions and you need to strike a balance and it's really hard, right? If you want to make sense very easy to auto-gradable, like automatically gradable, like easy to grade or easy to evaluate, then you might lose some of the realness or practicality. Or like it might be practical, but it might not be as scalable, right? For example, if you think about text game, human have pre-annotated all the rewards and all the language are real. So it's pretty good on auto-gradable dimension and the practical dimension if you think about, you know, practical, like actual English being practical, but it's not scalable, right? Like, it takes like a year for, like, experts to, to, to build that game. So it's not really that scalable. And I think part of the reason that SWE-Bench is so popular now is it kind of hits the balance between those three dimensions, right? Easy to evaluate and being actually practical and being scalable. Like, if I were to criticize upon some of my prior work, I think WebShop, like, is my initial attempt to get into benchmark world, and I'm trying to do a hard, good job striking the balance, but obviously we make it auto-gradable and it's really scalable. But then I think the practicality is not as high as actually just using GitHub Issues, right? Because you're just creating those, like, synthetic tasks. **Guest** [35:14] Are there other areas besides coding that jump to mind as being really good for being auto-gradable? **Shunyu Yao** [35:20] Maybe mathematics. **Guest** [35:21] Oh, classic. **Shunyu Yao** [35:23] Yeah. **Guest** [35:23] Do you have thoughts on, uh, AlphaProof, the, the new DeepMind paper? **Shunyu Yao** [35:28] I think it's pretty cool. Um. **Guest** [35:30] Yeah. **Shunyu Yao** [35:32] I think it's more of a, you know, it's more of like a confidence boost or like, uh, sometimes, you know, the work is not even about, you know, the technical details or the methodology that it chooses or the, the concrete results. I think it's more about a signal, right? **Guest** [35:48] Yeah, existence proof, like- **Shunyu Yao** [35:49] Yeah, yeah. It's like- **Guest** [35:50] Can be done **Shunyu Yao** [35:51] ... this direction is exciting and it kind of encourages people to work more towards that direction. I think it's more like a boost of confidence, I would say. Yeah. **Guest** [36:00] So we're gonna focus more on agents now and y- you know, um, we were, have a special, all of us have a special interest in coding agents. I would consider Devin to be the sort of biggest launch of the year as far as AI startups go, and, uh, you guys in, in the Princeton group worked on SWE-Agent alongside of SWE-Bench. Tell us the story about SWE-Agent. **Shunyu Yao** [36:20] Sure. So I think it's kind of like a trilogy. It's actually a series of three works now. ### CoALA **Guest** [36:26] Okay. **Shunyu Yao** [36:26] So the actually, the first work is called Intercode. **Guest** [36:30] Don't remember that. **Shunyu Yao** [36:30] But it's not as, it's not as famous, I know. And the second work is called SWE-Bench, and the third work is called SWE-Agent. And I'm just really confused why nobody's working on coding You know- ... it's like a year ago, but I mean, not everybody's working on coding obviously, but a year ago, like literally nobody was working on coding and I was really confused. And the people that were working on coding are, you know, trying to solve human eval in like a sick to sick way. There's no agent, there's no chain of thought, there's no anything. They're just, you know, fine-tuning the model and improve some points and whatever. Like, I was really confused because obviously coding is the best application for agents because it's all gradable. It's super important. You can make everything like API or, or code action, right? So I was confused, and I collaborated with some of the students in Princeton, and we have this work called InterCode, and the idea is, first, if you care about coding, then you should solve coding in an interactive way. Meaning more like a Jupyter Notebook kind of way than just writing a program and seeing if it fails or succeeds and, and stuff, right? You should solve it in an interactive way. That's because that's exactly how humans solve it, right? If I tell you to, you know, write a program like next token, next token, next token and stop and never do any edits, and you cannot really use any terminal or whatever tool, it doesn't make sense, right? And that's the way people are solving coding at the time. Basically, like sampling a program from a language model without chain of thought, without c- tool call, without refactoring, without anything. So first point is we should solve code, coding in a very interactive way, and that's a very general principle that applies for various coding benchmarks. But also, I think you can make a lot of the agent task kind of like interactive coding. If you have Python and you can call any package, then, then you can literally also browse internet or, or do whatever you want, like control a robot or whatever. So that seems to be a very general pattern. But obviously, I think a bottleneck is at the time we're still doing, you know, very simple tasks like human eval or whatever coding benchmark people proposed. Like, they were super hard 2021, like 20%, but they're like 95% already in 2023. So obviously the next step is we need better benchmark. And, uh, Carlos and John, which are the first authors of SWE-Bench, I think they come up with this great idea that we should just script GitHub and solve what-whatever human engineers are solving. And I think it's actually pretty easy to come up with this idea, and I think in the first week they already made a lot of progress, like they script the GitHub and they make all the same, but then there's a lot of painful info work and whatever, you know. I think the idea is super easy, but the engineering is super hard, and I feel like that's a very typical signal of a good work in the AI era now. So- **Swyx** [39:18] I, I think also, I think the filtering was challenging because if you look at open source PRs, like a lot of them are just like, you know, fixing typos. **Shunyu Yao** [39:26] I, I think it's challenging, and to be honest, we didn't do a perfect job at the time. So if you look at the recent- **Swyx** [39:31] Yes **Shunyu Yao** [39:31] ... blog post with OpenAI- **Swyx** [39:33] Yeah **Shunyu Yao** [39:33] ... like, uh, we improved the, the filtering so that, you know- **Swyx** [39:36] Yeah **Shunyu Yao** [39:36] ... it's more solvable. **Swyx** [39:37] I think OpenAI was just like, "Look, this is a thing now. We have to fix this." Like, like- **Shunyu Yao** [39:43] Yeah **Swyx** [39:43] ... these students just like, you know- **Shunyu Yao** [39:44] It's, uh- **Swyx** [39:44] ... rushed it. **Shunyu Yao** [39:45] It's a, it's a, it's a good convergence of interest for me, right? So- **Guest 2** [39:48] Yeah, was that, was that tied to you joining OpenAI or like was that just unrelated? **Shunyu Yao** [39:53] Yeah, it's a co- coincidence for me, but it's a, it's a good coincidence. **Swyx** [39:56] There is a history of anytime a big lab adopts a benchmark, they fix it because, you know, otherwise it w- **Shunyu Yao** [40:01] Yeah **Swyx** [40:01] ... it's a broken benchmark. **Shunyu Yao** [40:02] Yeah. **Swyx** [40:03] Yeah. **Shunyu Yao** [40:03] So naturally, once we propose SWE-Bench, the next step is to solve it, right? But I think the typical way you solve something now is you collect some training samples, or you design some complicated agent method, and then you try to solve it, right? Either super complicated prompt or you build a better model with more training data. But I think at the time we realized that even before those things, there's a fundamental problem with, with the interface or the tool that you're supposed to use, right? Because that's like a ignored problem in some sense, right? Like what your tool is or how that matters for your task. So what we found concretely is that if you just use the text terminal off the shelf as a tool for those agents, there's a lot of problems, right? For example, if you edit something, there's no feedback, so you don't know whether your edit is good or not, and that makes the agent very confused and makes a lot of mistakes. And there are a lot of like small problems you will say. And well, you can try to do prompt engineering and improve that, but it turns out to be actually very hard. And we realized that the interface design is actually a very omitted kind of part of agent design. So we did this SWE-agent work, and the key idea is just even before you talk about, you know, what the agent is, you should talk about what the environment is, and you should make sure that the environment is actually friendly to whatever agent you're trying to apply, right? And that's the same idea for humans, right? Like, if I give you... Like text terminal is good for some tasks, like Git pull or, or whatever, right? But it's not good if you wanna look at, you know, browser and whatever, right? So also like, you know, browser is a good tool for some tasks, but it's not a good tool for other task. We need to talk about how to design the interface in some sense where we should treat agents as our customers, right? It's like when we treat humans as a customer, we design human-computer interfaces, right? We design those beautiful desktops or browser or whatever so that it's very intuitive and easy for human to use. And this whole great subject of HCI is all about that. I think now the research idea of SWE-agent is just we should treat agents as our customers- ... and we should do like, you know- **Swyx** [42:16] AICI. **Shunyu Yao** [42:17] Yeah, ACI. **Swyx** [42:18] Exactly. **Guest 2** [42:19] So what are the tools that, uh, that SWE-agent should have or a coding agent in general should have? **Shunyu Yao** [42:24] It's, uh, for SWE-agent, it's like a modified tex-text terminal which kind of adapts to a lot of the patterns of language models to make it easier for language model to use. For example, now for edit, instead of having no feedback, it will actually have a feedback of, you know, actually here you, you introduced like a syntax error, and you should probably want to fix that, and there's end i- ended error there. ### ACI **Shunyu Yao** [42:45] And that makes it super easy for the model to actually do that. And there's other small things like how exactly you write arguments, right? Like, do you want to write like a multi-line edit or do you want to write a single line edit? I think it's more interesting to think about the way of the development process of an ACI rather than the actual ACI for like a concrete application because I think the general paradigm is very similar to HCI and psychology. Right. Basically, for how people develop HCIs, they do behavior experiments on humans, right? I do every test, right, like, uh, which interface is actually better, and I do those behavior experiments, kind of like psychology experiments to humans, and I change things. And I think what's really interesting for me for this SWE-Agent paper is we can probably do the same thing for agents, right? We can do every test for those agents and do behavior tests. And through the process, we not only invent better interfaces for those agents, that's the practical value, but we also better understand hu- agents. Just like when we do those A/B tests, we do those HCI, we better understand humans. Doing those ACI experiments, we actually better understand agents, and that's pretty cool. **Harrison** [43:51] Besides kind of like that A/B testing, what are other kind of like processes that people can use to think about this in a, in a good way? **Shunyu Yao** [43:58] That's a great question, and I think SWE-Agent is like an initial work, and what we do is the, the kind of the naive approach, right? You just try some interface, and you see what, what's going wrong, and then you try to fix that. You do this kind of iterative fixing. But I think what's really interesting is there going to be a lot of future directions that's very promising if we can apply some of the HCI principles more systematically into the interface design. I think that would be a very cool interdisciplinary research opportunity. **Harrison** [44:26] You talked a lot about kind of like agent computer interfaces and interactions. What about like human to agent kind of like UX patterns? Like if... Yeah, curious for any thoughts there that you might have. **Shunyu Yao** [44:39] That's a great question, and in some sense, I feel like prompt engineering is about human to agent- **Swyx** [44:46] To agent, yeah **Shunyu Yao** [44:46] ... interface. But I think there can be a lot of interesting research done about... So prompting is about how humans can better communicate with the agent, but I think there could be interesting research on how agents can better communicate with humans, right? When to ask questions, how to ask questions, like what's the frequency of asking questions. And I think those kind of stuff could be very cool research. **Harrison** [45:08] Yeah, I think some of the most interesting stuff that I saw here was also related to coding with, with Devin from Cognition, and they had the three or four different panels where you had like the chat- **Shunyu Yao** [45:16] Yeah **Harrison** [45:16] ... the browser, the terminal, and I, I guess the code editor as well. **Swyx** [45:19] Yeah, there's more now. **Harrison** [45:20] There's more. Okay. I'm n-not up to date. **Shunyu Yao** [45:22] Yeah, I think they, I think they also do a good job on ACI. **Harrison** [45:24] They did. Yeah. **Swyx** [45:25] Yeah. The, the, I think that's the main learning I have from Devin. They, they cracked that. They actually, there was no foundational planning breakthrough. The planner is like actually pretty simple, but ACI- **Shunyu Yao** [45:34] Right **Swyx** [45:34] ... uh, that they broke through on. **Shunyu Yao** [45:35] I think making the tool good and reliable is probably like ninety percent of the whole agent. Once the tool is actually good, then the agent design can be much, much simpler. On the other hand, if the tool is bad, then no matter how much you put into the agent design planning or research or whatever, it's still gonna be trash. **Harrison** [45:53] Yeah, I'd argue the same- Same with like context and instructions. Like, yeah, go hand in hand. **Swyx** [46:00] On the tool, how do you think about the tension of like, for both of you, I mean, you're building a library, so even more for you, the tension between making now a language or a library that is like easy for the agent to grasp and write versus one that is easy for like the human to grasp and write? Because, you know, the trend is like more and more code gets written by the agent, so why wouldn't you optimize the framework to be as easy as possible for the model versus for the person? **Shunyu Yao** [46:25] I think it's possible to design interface that's both friendly to humans and agents, but what, what do you think? **Harrison** [46:29] We haven't thought about that from the perspective... Like, we're not trying to design LangChain or LangGraph to be friendly. But I mean, I, I think it- **Swyx** [46:36] Whoa, whoa, whoa. What? **Harrison** [46:37] To, to be, to be friendly for- **Swyx** [46:38] Be careful **Harrison** [46:38] ... to be friendly for agents to write. But I, I mean, I think we see this with like I saw some paper that used TypeScript notation instead of JSON notation for tool calling- **Swyx** [46:49] Yes **Harrison** [46:49] ... and it got a lot better performance. So it's definitely a thing. I d- I haven't really heard of anyone designing like a syntax or a language explicitly for agents, but there's clearly syntaxes that are better. **Shunyu Yao** [46:59] I think function calling is a good example where it's like a good interface for both human programmers and for agents, right? Like for developers, it's actually a very friendly interface because it's very concrete, and you don't have to do prompt engineering anymore. You can be very systematic. And for models, it's also pretty good, right? Like it can use all the existing coding kind of... So I think we need more of those kind of designs. **Swyx** [47:22] I will mostly agree and on a slightly disagree in terms of this, which is like whether designing for humans also overlaps the designing for AI. So Malte Ubl, who's the CTO of Vercel, who is creating basically JavaScript's, uh, you know, competitor to LangChain, they're observing that basically, like if the API is easy to understand for humans, it's actually much easier to understand for LLMs, for example, because they're not overloaded functions. They don't behave differently under different contexts. They, they do one thing, they, and they always work the same way. It's easy for humans, it's easy for, for LLMs, and like that makes a lot of sense. And, and obviously adding types is, is another one. Like, like type annotations only help give extra context, which is really great. So that's the agreement. And then a disagreement is that I've actually, when I use structured output to do my chain of thought, I have found that I change my field names to hint to the, the LLM of what the field is supposed to do. So instead of saying topics, I'll say candidate topics, and that gives me a better result because the LLM was like, "Ah, this is just a draft thing I can use for chain of thought." And instead of like summaries, I'll say topic summaries to link the previous field to the, the current field. So like s- little stuff like that, I find myself optimizing for the LLM where I w- I as a human would never do that. **Shunyu Yao** [48:32] Interesting. It's kind of like the way you optimize the prompt is might be different for humans and for- **Swyx** [48:38] Yes **Shunyu Yao** [48:38] ... machines. You can have a common ground that's both clear for humans and agents, but to improve the human performance versus improving the agent performance, they might move to different directions. **Swyx** [48:49] Might move different directions. There's a lot more use of metadata as well, like descriptions, comments, code comments, yeah, annotations and stuff, annotations and stuff like that, yeah. **Harrison** [48:57] I would argue that's just you communicating to the agent- ... what it should do, and, and maybe you need to communicate a little bit more than to humans 'cause models aren't quite good enough yet. But like, uh, I don't think that's crazy. I don't think that's like- **Swyx** [49:09] It's not crazy. I will bring this in because it just happened to me yesterday. I was at the Cursor office. They held their first like user meetup, and they were... I was te- telling them about Um, the LLMO concept and why basically every interface, every tool is being redesigned for AIs to use rather than humans, and they're like, "Why?" Like, "Can't we just use Bing and Google for LLM search? Why must I use Exa or the... What's the other one that, that you guys work with? Uh," **Harrison** [49:32] Uh, Tavily. **Swyx** [49:33] Tavily. Web search API dedicated for LLMs. So- **Shunyu Yao** [49:36] What's the difference of them- **Swyx** [49:36] Exactly **Shunyu Yao** [49:37] ... to, to Bing API for that? **Swyx** [49:38] Exactly. **Harrison** [49:39] There weren't great APIs for search. Like, the best one, like, the, the one that we used initially in LangChain was SERP API- **Swyx** [49:46] Yes **Harrison** [49:46] ... which is, like, maybe illegal, I'm not sure. And, like, you know, it, uh, and now there are, like, venture-backed companies doing- **Swyx** [49:54] And then shout out to DuckDuckGo, which, which is free. **Harrison** [49:56] Yes, yes. **Swyx** [49:56] Yeah. **Harrison** [49:56] Um, yeah. I do think there are some differences, though. I think you want-- Like, I think generally these APIs try to return small amounts of text information, clear legible field. It's not a massive JSON blob, and I think that matters. I think, like, when you talk about designing tools, it's, it's not only the-- It's, it's the interface in the entirety, not only the inputs but also the outputs that really matter. And so I think they try to make the outputs- **Shunyu Yao** [50:18] They're doing ACI. **Harrison** [50:20] Yeah, yeah, absolutely. **Shunyu Yao** [50:21] Really. **Harrison** [50:21] Absolutely. **Swyx** [50:21] So, like, there, there's a whole set of industries that are just being redone for A-ACI. It's weird. Uh, and so my, my simple answer to, to them was, like, the error messages. When you give error messages, they should be basically prompts for the LLM to take and then self-correct. Then your error messages get more verbose actually than you normally would with a human. **Shunyu Yao** [50:40] Right. **Swyx** [50:40] Stuff like that. Like, little... Honestly, it's not that big. Again, like, is this worth a, a venture-backed industry? Alessio can tell us. Uh, This-- But, like, I, I, I think Code Interpreter, I think, is a new thing. **Shunyu Yao** [50:52] I hope so. We invested in it, so it'd be, so... I think that's a very interesting point. You're trying to optimize to the extreme, then obviously they're gonna be different. ### UX Design **Harrison** [50:59] Yes. **Shunyu Yao** [50:59] For example, the arrow- **Swyx** [51:00] 'Cause we take it very seriously. **Shunyu Yao** [51:01] Right. **Swyx** [51:02] Yeah. **Shunyu Yao** [51:02] The arrow for, for, like, language model, the longer the better. But for human, that will make him very nervous and very tired, right? But, but I guess the, the point is more like maybe we should try to find a co-optimized common ground as much as possible. And then if we have divergence, then we should try to diverge. But it's more philosophical now. But I think, like, part of it is, like, how you use it. So- **Swyx** [51:22] Right **Shunyu Yao** [51:23] ... Google invented the PageRank because ideally you only click on one link, you know? Like, the top three should have the answer. But, like, with models it's like, well, you can get 20. So- **Swyx** [51:31] Sure **Shunyu Yao** [51:31] ... those searches are more like semantic grouping in a way. **Swyx** [51:34] Yeah. **Shunyu Yao** [51:34] It's like for this query, I'll return you, like, 20, 30 things that are kinda good, you know? **Swyx** [51:39] Yes. **Shunyu Yao** [51:39] So it's less about ranking, and it's more about grouping. Another fundamental thing about ACI is the difference between human and machine's kind of memory limit, right? So I think what's really interesting about this concept of ACI versus HCI is interfaces that's optimized for them. You can kind of understand some of the fundamental characteristics, differences of humans and machines, right? Why, you know, if you look at find or whatever terminal command, you know, you can only look at one thing at a time or... That's because we have a very small working memory. You can only deal with one thing at a time. You can only look at one paragraph of text at the same time. So the interface for us is by design, you know, a small piece of information, but more temporal steps. But for machines, that's, that should be the opposite, right? You should just give them 100 different results, and they should just decide in context what's the most relevant stuff and trade off the context for temporal steps. That's actually also better for language models because, like, the cost is smaller or whatever. So it's interesting to connect those interfaces to the fundamental kind of differences of those. **Harrison** [52:44] When you said earlier, you know, we should try to design these to maybe be similar as possible and diverge if we need to, I actually don't have a problem with them diverging now- **Shunyu Yao** [52:52] Mm-hmm **Harrison** [52:52] ... and seeing venture-backed startups emerging now 'cause we are different from machines code AI, and it's just so early on. Like, they may still look kind of similar, and there may still be small differences, but it's still just so early, and I think we'll only discover more ways that they differ, and so I'm totally fine with them kind of, like, diverging early and optimizing for the different tasks. **Shunyu Yao** [53:12] I agree. I think, I think it's more like, you know, we should obviously try to optimize human interface just for humans. We're already doing that for 50 years. Uh, we should optimize agent interface just for agents, but we might also try to co-optimize both and see how far we can get. There's enough people to try all three directions. Yeah. **Swyx** [53:31] There's a thesis I sometimes push, which, uh, which is the sour lesson as opposed to the bitter lesson, which we always inspired by human development, but actually AI develops its own path. **Shunyu Yao** [53:40] Right. We need to understand better, you know, what are the fundamental differences between those- **Swyx** [53:45] Yeah **Shunyu Yao** [53:45] ... creatures. **Swyx** [53:46] It's funny when f-fairly early on this pod you were like, "How much grounding do you have in cognitive development and human, human brain stuff?" And I'm like, "Maybe that doesn't matter." And actually, so, like, I, I w- in my original agent's blog post, I had a picture of the human brain, and now it's m- looks a lot more like a CPU. Canonical picture of the LLMO is a C- is kind of like a CPU with all the input and output going into it. And I think that, yeah, that's probably- **Shunyu Yao** [54:10] Yeah, I, I- **Swyx** [54:10] ... the more scalable system **Shunyu Yao** [54:11] ... I, I think the problem with, like, a lot of, like, cognitive scientists, like, is that, like- **Swyx** [54:15] They think by analogy, right? **Shunyu Yao** [54:16] They, they think, you know, the only way to solve intelligence is through the human way. **Swyx** [54:20] Right. **Shunyu Yao** [54:21] And therefore, they, like, have a lot of critics for whatever things that are not cognitive or, or human. But I think a more useful way to use this knowledge is to think of that as just a reference point. I don't think we should copy exactly what's going on with human all the way, but I think it's good to have a reference point because this is a working example how intelligence work. **Swyx** [54:40] Yeah. **Shunyu Yao** [54:41] And if you know all the knowledge and you compare them, I think that actually establishes more interesting insights as opposed to just copy that or not copying that or opposing that. I think comparing is the way to go. **Swyx** [54:53] I feel like this is an unanswerable question, but I'll just put it out there anyway. If we can answer this, I think it'll be worth a lot, which is can we separate intelligence from knowledge? **Shunyu Yao** [55:01] That's a very deep question actually. And, uh, to have a little history background, I think, uh, I think that's really the key thesis at the beginning of AI, if you think about Newell and Simon and all those symbolic AI people, right? Basically, they are trying to create intelligence by writing down all the knowledge For example, they write a, like, a checker- **Swyx** [55:23] WordNet **Shunyu Yao** [55:23] ... yeah, or write a checker program. Basically, how you would solve the checker, you write down all the knowledge and then implement that. And I think the whole thesis of symbolic AI is we should just be able to write down all the knowledge, and that just creates intelligence, but that kind of fails. And I think really, I think a great, like, quote from Hinton is, "I think there are two approaches to intelligence. One approach is let's deal with reasoning or thinking or knowledge, whatever you call that, and then let's worry about learning later. The other approach is let's deal with learning first, and then let's worry about, you know, whatever knowledge or reasoning or thinking later." And it turns out right now at least, like, the second approach works, and the first approach doesn't work. And I think there might be something deep about it. Does that, does that answer your question? **Swyx** [56:09] Uh, partially. I, I think, um, Apple Intelligence might change that. **Shunyu Yao** [56:12] Can you explain? **Swyx** [56:13] If, if this year is the year of multi- multimodal models, next year is, like, on-device year, and Apple Intelligence basically has hot-swappable capabilities, right? Like, they have, like, 50 LoRAs that they swap onto a base model that does different tasks, and that's this first instance that we have of, of the separation of, of, um, intelligence and knowledge, and I, I think that's, that's a really interesting approach. Obviously, it's not exactly knowledge. It's just more styles. **Shunyu Yao** [56:36] Yeah, or more about con- context. **Swyx** [56:38] Yeah, it's more about context. **Shunyu Yao** [56:39] It's like, it's like you, you can have the same model deployed to 10 million phones with 10 million context NC if- **Swyx** [56:44] For on-device deployment, I think it's just super important. Like, if you can boil out... Like, I actually r- have most of my problems with AI news when the model thinks it knows more than it knows because it combines knowledge of intelligence. I, I want it to have zero knowledge whatsoever, and it only has the ability to parse the things I tell it. **Shunyu Yao** [57:01] I kind of get what you mean. I, I feel like it's more like memorization versus kind of just generalization in some sense, right? **Swyx** [57:07] Yeah, raw ability to understand things. **Shunyu Yao** [57:08] You don't want it to know, like, facts, like, you know, who is the president of United States. They should be able to just call internet and use a tool to- **Swyx** [57:15] Yes **Shunyu Yao** [57:16] ... to solve it. **Swyx** [57:16] Right? Because otherwise it's not gonna call the in- the, the, the tool if it thinks it knows. **Shunyu Yao** [57:19] I kind of get what you mean. I, I, I think it is, uh- **Swyx** [57:22] That's why it's valuable. **Shunyu Yao** [57:23] Okay. So if that's the case, I guess my point is I don't think it's possible to fully separate them- **Swyx** [57:29] Yeah **Shunyu Yao** [57:29] ... because, like, those kind of intelligence kind of emerges. Like, even for humans, you can't just operate in a intelligent mode without knowledge, right? Throughout the years you learn how to do things and what things are, and it's very hard to separate those things, I would say. Yeah. **Swyx** [57:46] But what if we could? As a meta strategy, I'm trying to keep as a stack ranked list of, like, what are the 10 most valuable questions in AI. **Shunyu Yao** [57:55] You can, you can think of, you can think of knowledge as a ca- cache of intelligence in some sense. **Swyx** [58:00] Uh, ooh. **Shunyu Yao** [58:01] Like, like if you, you, if you have... Right? If you have, like, wikihow.com- **Swyx** [58:06] Yeah **Shunyu Yao** [58:06] ... saying, like, you should tie a shoelace using the following steps, you can think of that te- piece of text as, like, a cache to intelligence. **Swyx** [58:13] Right. **Alessio** [58:13] I guess that's kinda like reflection anyway, right? **Shunyu Yao** [58:16] Right. **Alessio** [58:16] It's like you're storing these things as memory, and then you put them back. So without the knowledge, you wouldn't have the intelligence- **Shunyu Yao** [58:22] Right **Alessio** [58:22] ... to do it better. **Shunyu Yao** [58:23] Right. **Alessio** [58:24] I had a couple things. So we had, uh, Thomas Shalom from Meta, uh, to talk about Llama 3.1. They mentioned- **Swyx** [58:29] Then he started talking about Llama 4. **Alessio** [58:31] Yeah. He was- **Swyx** [58:31] I was like, "Whoa, okay." **Alessio** [58:33] Hey. And he said it's gonna be, like, really focused on agents. Um, I know you talked before about, you know, it's next token prediction enough to get to, like, problem-solving. If you... Say you got the perfect environment, they got the terminal, they got everything, and if you were to now move down to the model level and say, "I need to make a model that is better for, like, agentic workflow," where would you start? ### Data & Models **Shunyu Yao** [58:54] I think it's data. I think it's data because, uh, like, changing architecture now is too hard, and we don't have a good, better alternative solution now. I think it's mostly about data, and agent data is obviously hard because people just write down the final result on the internet. They don't write down how they, like, step-by-step how they do the thing on the internet, right? So naturally, it's easier for models to learn chain of thought than tool call or whatever agent self-reflection or search, right? Like, even if you do a search, you won't write down all the search processes on the internet. You would just write down the final result. And, uh, I think it's a great thing that Llama 4 is gonna be more towards agents. That means, I mean, that should mean a lot for a lot of people. Yeah. **Harrison** [59:36] In terms of data, you think the right data looks like trajectories basically of, of a ReAct agent or of- **Shunyu Yao** [59:44] Yeah. I mean, I have a paper called FireAct. Do you still remember? **Harrison** [59:47] No. **Shunyu Yao** [59:48] Okay. **Swyx** [59:49] Tell us. **Shunyu Yao** [59:49] Okay. That's one of the not famous paper, I guess. **Swyx** [59:52] It's not even on your website. **Alessio** [59:54] How are we supposed to find it? **Harrison** [59:56] It's on, it's on Google Scholar. I've got it pulled up. **Alessio** [59:58] Okay. **Shunyu Yao** [59:59] It's, it's not... It's, it's, it's been rejected for, for, like, a couple times now. **Alessio** [1:00:03] Oh, no. **Swyx** [1:00:03] Yeah, but now it's on Leina Space- **Alessio** [1:00:05] Yeah, yeah, yeah **Swyx** [1:00:05] ... so everybody will find it. **Shunyu Yao** [1:00:06] Anyway, I think the idea is very simple. Like, you can try a lot of different agent methods, right? ReAct, chain of thought, reflection, whatever, and the idea is very simple. You just have very diverse data, like tasks, and you try very diverse agent methods, and you filter all the correct solutions, and you train a model on all of that. And then the benefit is that it should somehow learn, you know, how to use simpler methods for simpler tasks and harder methods for harder tasks. I guess the problem is we don't have diverse high-quality tasks. That's the bottleneck for... **Harrison** [1:00:35] So it's gonna be trained on all code? **Shunyu Yao** [1:00:37] Yeah, let's hope we have more better benchmarks, yeah. **Alessio** [1:00:39] In school, that kinda pissed me off a little bit when you're doing, like, a homework or, like, exercises for, like, calculus. Like, they give you the problem, then they give you the solution. **Shunyu Yao** [1:00:48] Right. **Alessio** [1:00:48] But there's no way without the professor or the TA to get, like, the steps- **Shunyu Yao** [1:00:51] Right **Alessio** [1:00:52] ... to actually how you got there. **Shunyu Yao** [1:00:53] Right. **Alessio** [1:00:53] And so I feel like because of how schools are structured, we never broke this thing down. But I feel like if you went to every university and it's like write down step-by-step the solution to every single problem in the set and make it available online, that's a start to make this data set better. **Shunyu Yao** [1:01:07] I think it's also because, you know, it's... It might be hard for you to write down your chain of thought even when you're solving the same because- Part of that is conscious in language, but maybe even part of that is not in language. And okay, so a funny side story. So when I wrote down the ReAct thing, I was talking to my Google manager, right? Like, "You know what we should do? We should just hire, you know, as many people as possible and let them use Google and write down exactly what they think, what, what they search on the internet, and we train model on that." But I think it's non-nontrivial to, to write down your thoughts, like, if you're not trained to do that. If I tell you, like, "Okay, write down what you're thinking right now," it's actually not as trivial- **Guest** [1:01:48] Mm-hmm. Yeah **Shunyu Yao** [1:01:48] ... a task as you might imagine. **Guest** [1:01:49] It, it m-might be more of a diffusion process than a - **Shunyu Yao** [1:01:52] Also, I think- **Guest** [1:01:52] ... autoregressive process. **Alessio** [1:01:53] But I think the problem is starting with the experts, you know, because the- **Shunyu Yao** [1:01:56] Right **Alessio** [1:01:56] ... there's so much, like, muscle memory- **Shunyu Yao** [1:01:58] Right **Alessio** [1:01:58] ... in what you do once you've done it for so long. **Shunyu Yao** [1:02:00] Right. Right. **Alessio** [1:02:00] That's why we need to, like, get everybody to do it, and then you can see, like, how they are- **Guest** [1:02:05] Separate, separate knowledge and intelligence. **Alessio** [1:02:06] Exactly. **Shunyu Yao** [1:02:07] The, the, the simplest way to achieve AGI is literally just, just record re- the ReAct trajectory of every human being and just put them together, you know? **Guest** [1:02:15] Yeah. **Shunyu Yao** [1:02:15] Like, what, what do you have thought about? **Guest** [1:02:17] Yeah. **Shunyu Yao** [1:02:17] What do you have done, let's say, on the computer, right? Imagine, like, thought experiment. Like, you, you write down literally everything you think about and everything you do on the computer, and you record them, and you train on all the successful trajectories by some metric of success. I think that should just lead us to AGI. **Guest** [1:02:33] My, my first work of fiction in, like, ten years was exploring that idea of if you-- what if you recorded everything and uploaded yourself. I'm pretty science-based, like, you know, but probably the most, like, spiritual woo-woo thing about me is I don't think that would lead to consciousness or AGI just because, like, there's something in... There's, like... There's a soul. **Alessio** [1:02:50] Mm-hmm. **Guest** [1:02:50] You know? Like that is the, uh, unspeakable quality of- **Shunyu Yao** [1:02:53] That's if it emerges through scale, right? **Guest** [1:02:57] We can simulate that, for sure. What do you think about the role of few-shot prompting for some of these, like, agent trajectories? That was a big part of the original ReAct paper, I think, and as we talk about showing your, your work and how you think, like- **Shunyu Yao** [1:03:10] I feel like it's becoming less used than zero-shot prompting. What's your observation? **Guest** [1:03:16] I'm pretty bullish on it, to be honest, for a few reasons. Like, one, I think it can maybe help for more complex things, but then also, two, like, it's a form of prompting, and prompting is just communicating with the model what you want it to do. And sometimes it's easier to just show the model what you want it to do than write out detailed kind of like instructions. **Shunyu Yao** [1:03:32] I think the practical reason it has become less used is because the agent kind of scaffold become more complex or the task you're trying to solve is becoming more complex. It's harder to annotate a few-shot examples, right? Like in the chain-of-thought era, you just write down three lines of things. It's very easy to write down few-shot or whatever. But I feel like annotation difficulty has become harder. **Guest** [1:03:54] I think also one of the reasons that I'm bullish on it is 'cause I think it's a really good way to achieve kind of like personalization. Like, if you can collect this through feedback automatically, you can then use that in the system at a user level or something like that. Again, the issue with that is- **Shunyu Yao** [1:04:07] I think it's- **Guest** [1:04:07] ... for complex things, that doesn't really work because you're- **Shunyu Yao** [1:04:08] It's probably more useful as, like, a automatic, you know, prompt, right? If you have- **Guest** [1:04:13] Yeah **Shunyu Yao** [1:04:13] ... some way to retrieve examples and put it in, like, automatic pipeline to prompt. But I feel like if you're a human, you're manually writing now, I feel like peop- more people will try to use zero-shot. **Guest** [1:04:23] Yeah, but if you're doing a consumer product, you're probably not gonna ask- **Shunyu Yao** [1:04:26] Right **Guest** [1:04:26] ... end user-facing people to write a prompt- **Shunyu Yao** [1:04:29] I agree. I agree **Guest** [1:04:29] ... or something like that. **Shunyu Yao** [1:04:29] Yeah. **Guest** [1:04:30] But I think the thing that you brought up is also really relevant here, where you can collect u- feedback from a user, but it's usually at the top level. And so then if you have three or four or five or however many LLM calls down below, how do you disperse that feedback to those? And I don't have an answer for that. **Alessio** [1:04:46] There's another super popular paper that you author called CoALA, Cognitive Architectures for Language Agents. **Shunyu Yao** [1:04:52] I'm not sure if it's super popular. **Alessio** [1:04:53] Well, I, I, I think, uh, I hear- **Shunyu Yao** [1:04:54] Compared to- **Guest** [1:04:55] People speak highly of it here- **Alessio** [1:04:56] Yeah **Guest** [1:04:56] ... within my circles. **Shunyu Yao** [1:04:57] Right. **Guest** [1:04:57] Uh, so shout out to Charles Fry who told me about it. I think that was our most popular webinar we did- **Shunyu Yao** [1:05:01] It is the most popular webinar **Guest** [1:05:02] ... on LinkedIn. Yeah. **Shunyu Yao** [1:05:03] I think Harrison promoted the paper a lot. Th-thanks to him. **Guest** [1:05:07] I'll read what, what you wrote in here, and then we-- you can just kind of go take it wherever. Uh, CoALA organizes agents along three key dimensions, their information storage divided into working and long-term memories, their action space divided into internal and external actions, and their decision-making procedure, which is structured as an interactive loop with planning and execution. By the way, I think your communication is very clear, so kudos on, on how you do these things. Take us through the sort of three components or... And you also have, like, this, this development diagram, which I think is really cool. I think it's figure one in your paper for, for people reading along. Normally people have input, LLM output, then they develop into, all right, language agents that takes an action into environment and has observations, and then they, they go into this, the CoALA architecture. **Shunyu Yao** [1:05:47] Shout out to my, uh, co first author, Tad, who made figure one. **Guest** [1:05:51] Yeah. **Shunyu Yao** [1:05:52] It's like, you know, figure is really good. You don't even need color. You just- **Guest** [1:05:55] Exactly. **Shunyu Yao** [1:05:56] Black and white. One of the motivation of CoALA is we're seeing those agents become really complicated. I think my personal philosophy is try to make things as simple as possible, but obviously this field has become more complex as a whole, and it's very hard to understand what's going on. And I think CoALA provides a very good way to, uh, understand things in terms of those three dimensions. And I think they are pretty first principle because I think this idea of memory is pretty first principle if you think about where memory, where information is stored. And you can even think of the ways of neural network as some kind of long-term memory because that's also part of the information is stored. I think a very, like, first principle way of thinking of agents is pretty much just a neural network plus the code to call and use the neural network. Obviously, also maybe plus some vector store or whatever other memory modules, right? And thinking through that, then you immediately realize is that the kind of the long-term memory or the, the persistent information is first the neural network and second the, the code associated with agent that calls neural network and maybe also some other vector stores. But then there's obviously another kind of storage of information that's shorter horizon, right, which is the context window or whatever episode that people are using. Like, you're trying to solve this task, the information happens there, but once this task is solved, then the information is gone, right? So I think it's very systematic and first principle to think about where your information is and thinking organizing them Through categories and time horizon, right? So once you have those information stores, then obviously for agent, the next thing is what kind of action can you do? And that leads to the concept of action space, right? And I think one of the fundamental difference between language agents and the previous agents is that, uh, for traditional agents, if you think about Atari or video game, they only have like a predefined action space by the environment. They only have external actions, right? Because they don't have complicated memory or information kind of devices to do internal thinking. I think the contribution of ReAct is just to point out that we can also have internal actions called thinking. And obviously, if you have long-term memory, then you also have retrieval or writing or whatever. And then third, once you have those actions, which action should you do? That's the problem of decision-making. And, uh, and the three parts should just fully describe a agent. **Harrison** [1:08:18] Awesome. **Swyx** [1:08:18] We solved it. We have defined agents. **Harrison** [1:08:19] Yeah, it's done. **Swyx** [1:08:20] Does that- there's anything that you normally say about agents not fit in that framework? 'Cause you also get asked, uh, this question a lot. **Harrison** [1:08:28] Um, I think it's very aligned. **Swyx** [1:08:30] Yeah. **Harrison** [1:08:30] Um, if we think about a lot of the stuff we do, I'm just thinking out loud now, but a lot of the stuff we do on agents now is through LangGraph. LangGraph we would view as kind of like the code part of- **Swyx** [1:08:40] Yeah **Harrison** [1:08:40] ... what defines some of- **Shunyu Yao** [1:08:41] Yeah **Harrison** [1:08:41] ... these things and how they- **Shunyu Yao** [1:08:42] Also defines part of the decision-making. **Harrison** [1:08:44] Yeah. **Swyx** [1:08:44] Decision procedure, that's what I was thinking, actually, yeah. **Harrison** [1:08:46] Yeah. And actually, one analogy that I like there is like some of the code and, and part of LangGraph, and I'm actually curious what you think about this, but like, sometimes I say that like the LLMs aren't great at planning yet, so we can help them plan by telling them how to plan in code, 'cause that's very explicit, and that's a good way of communicating how they should plan and, and, and stuff like that. **Swyx** [1:09:04] Yeah. That- that's a Devin playbook as well. **Shunyu Yao** [1:09:06] What do you mean by that? Like, give them like a DFS algorithm or... **Harrison** [1:09:09] No, something like much simpler, like you could tell agent in a prompt like, "Hey, every time you do this, you need to also do this and make sure to check this." Or you could just put those as explicit checks in kind of like the, the decision-making procedure- **Shunyu Yao** [1:09:20] Right, right, right, right **Harrison** [1:09:20] ... or something like that. And the more complex it gets, I think the more we see people encoding that in code. And, and another way that I say this is like, all of life really is communication, right? And so you can do that through prompts or you can do that through code, and code's great at communicating things. It really is. **Shunyu Yao** [1:09:35] Is this the most philosophical episode we've ever had? **Swyx** [1:09:38] This is the deepest one I've ever had. Okay, this is great. **Shunyu Yao** [1:09:39] That's good. That's good. That's good. **Swyx** [1:09:40] Yeah. Well, we talk 'cause we talking about agents, you know? **Shunyu Yao** [1:09:42] Yeah. **Harrison** [1:09:42] I think the biggest thing that we're thinking a lot about is just the memory component, and we touched on it a little bit earlier in the, in the episode, but I think it's still like very unsolved. I think like clearly semantic memory, episodic memory are types of memory, I think, but like where the boundaries are, is there, are there other types, how to think about that. I think that to me is maybe one of the bigger unsolved things in terms of agents is just memory. Like what, what does memory even mean for agents? **Swyx** [1:10:07] That's another top high-value question. Is it a knowledge graph? **Harrison** [1:10:11] Uh, it- **Shunyu Yao** [1:10:13] I think that's one type of memory. **Harrison** [1:10:15] Yeah. If you're using a knowledge graph as a hammer to hit a nail, it's, it's, it's not that. But I think, I think practically what we see is it's still so application specific what relevant memory is, and that also makes it really tough to answer generically like what is memory. So like it could be a knowledge graph. It could also be like, I don't know, a list of instructions that you just keep in a list. **Swyx** [1:10:36] Yep. **Shunyu Yao** [1:10:37] A meta point is I feel sometimes we underestimate some aspects where humans and agents are actually similar, and we overestimate sometimes the differences. You know, I feel like, I mean, one point I think that's shared by agents and humans is like we all have very different types of memories, right? Some people use Google Doc, some people use Notion, and some people use paper and pen. Like you can argue those are different types of long-term memories for people, right? And each person develops its own way to maintain their long-term memory and diary or whatever. It's a very kind of individual kind of thing. And, uh, I feel like for agents probably there's no like single best solution. But what we can do is we can create as many good tools as possible, like Google Doc or Notion equivalent of agent memory. And we should just give the choice to the agent, like what do you want to use? And through learning, they should be able to come up with their own way to use the long-term memory. **Harrison** [1:11:30] Or give the choice to the developer who's building the agents, 'cause I think it also, the, it might, it depends on the task. **Swyx** [1:11:36] Yeah. **Harrison** [1:11:36] Like I use- **Swyx** [1:11:37] I think we want to control that one. **Harrison** [1:11:39] Right now, I would agree with that for sure, because I think you need that level of control. I use Linear for planning for code. I don't use that for my grocery list, right? Like depending on what I'm trying to do, I have different types of long-form memory. **Swyx** [1:11:50] Maybe if you tried, you would have a gorgeous kitchen. **Shunyu Yao** [1:11:52] Do you think our, like tool-making kind of progress is good or not good enough in terms of, you know, we have all sorts of different memory stores or retrieval methods or whatever? **Harrison** [1:12:04] On the memory front in particular, I don't think it's very good. I think there's a lot to still be done there. **Shunyu Yao** [1:12:08] What, what do you think are lacking? **Swyx** [1:12:09] Yeah, like you have a memory service. Like what's missing? **Harrison** [1:12:12] The memory service we launched I don't think really found product market fit. I think like, um, I mean, I think there's a bunch of different types of memory. I'll probably write a blog. I, I mean, I have a blog that I've published at some po-point on this. But I think like right off the bat, there's like procedural memory, which is like how you do things, and I think this is basically episodic memory, like trajectories of correct things. But there's also then I think a very different type is like personalization. Like I like Italian food. **Shunyu Yao** [1:12:36] It's kind of a semantic memory. **Swyx** [1:12:37] That's also maybe like a system prompt. **Harrison** [1:12:39] Yeah, exactly. Exactly. **Shunyu Yao** [1:12:41] Yeah. **Harrison** [1:12:41] It could be a sema-- It depends if it's semantic over like raw events or over reflections over events. **Shunyu Yao** [1:12:46] Right. Again, semantic procedure, whatever, is just like a categorization. What really matters is the implementation, right? **Harrison** [1:12:51] And so one of the things that we'll probably have released by the time this podcast comes out is right now in LangGraph, LangGraph is very stateful. You define a state for your graph, and basically a run of an agent operates on a thread. It's very similar to threads in OpenAI's Assistant API. But you can define the state however you want. You can define whatever keys, whatever values you want. Right now, they're all persistent for a single thread. We're gonna add the ability to persist that between threads. So then if you basically want to scope a memory to a user ID or to an assistant or to an organization, then you can do that. And practically what that means is you can write to that channel whatever you want, and then that can be read in other threads. We're not making any kind of like claims around what the shape of memory is, right? You can write kind of like what you want there. I still think it's like so early on, and we see people want, needing a lot of control over that. And so I think this is our current best thought. This is what we're doing around memory at the moment, is basically extending state to beyond a thread level. **Shunyu Yao** [1:13:47] I feel like there's a trade-off between, you know, complexity and control, right? For example, like Notion is more complex than Google Docs, but if you use it well, then it give you more capability, right? And it's like different tool might suit different applications or scenarios or whatever. **Harrison** [1:14:02] Yeah. **Shunyu Yao** [1:14:02] We should make more- Good tools, I guess. **Swyx** [1:14:04] My quick take is when I started writing about the AI engineer, this was kind of vaguely in my head, but, like, this is basically the job. Everything outside the LLM is, is the AI engineer that the researcher's not gonna do. **Alessio** [1:14:15] Wi- this basically maps to LLM- **Swyx** [1:14:18] Yeah **Alessio** [1:14:18] ... LLMOS. **Swyx** [1:14:19] Uh, I would, I would add in the code interpreter, the browser, and the other stuff, but yeah, this is mostly it. **Shunyu Yao** [1:14:25] Yeah, those are the external- **Alessio** [1:14:26] I mean, those are the tools, yeah. **Swyx** [1:14:27] Yeah, yeah. **Shunyu Yao** [1:14:27] Those are the external environment which is- **Swyx** [1:14:29] Yep **Shunyu Yao** [1:14:29] ... small box at the bottom. **Swyx** [1:14:30] Yeah, yeah. So then having this, like, reasonable level of confidence that, like, I know what things are, then I wanna break it. I wanna be like, "Okay, like, what's coming up that's gonna blindside me completely?" And it, it really is maybe, like, omnimodel, where, like, everything in, everything out, and, like, does that change anything? Like, if you scale up models, like, 100 times more, does, does that change anything? **Shunyu Yao** [1:14:50] That's actually a great, great question. I think that's actually the- ... last paragraph of the paper that's talking about this. I also got asked this question when I was interviewing with OpenAI. They were like- **Swyx** [1:15:01] Oh **Shunyu Yao** [1:15:01] ... "What if..." **Alessio** [1:15:02] Please tell us how to pass OpenAI interviews. **Shunyu Yao** [1:15:05] Like, is, is any of this still true if, you know- **Swyx** [1:15:08] If you 100X everything, yeah **Shunyu Yao** [1:15:10] ... if, if we make the model much better? My longer answer to this, you should just refer to the last paragraph of the paper, which is, like, a more prepared, longer answer. I think the shorter answer is understanding is always better. It's like a way of understanding things. Like, the solid experiment that I write at the end of the paper is imagine you have GPT-10, which is really good. Like, it doesn't even need chain of thought, right? Just input, output, just seq2seq, right? It doesn't even need to do browsing or whatever. Or maybe it still needs some tools, but let's say, like, it's really powerful. Like, then I think even in that point, I think something like CoALA is still useful if we wanna do some neuroscience on GPT-10. It's like kind of doing human kind of neuroscience, right? Which module actually correlates to which- **Swyx** [1:15:52] You want it to be inspectable. **Shunyu Yao** [1:15:53] Yeah, like you want to inspect what is the episodic memory, what is the decision-making module, what is the... It's kind of like dissecting the human brain, right? And you need some kind of prior kind of framework to help you do this kind of discovery. **Alessio** [1:16:05] Cool. Just one thing I wanna highlight from your work, we don't have to go into it, is, uh, TauBench, um- ### TauBench **Shunyu Yao** [1:16:11] Oh, yeah **Alessio** [1:16:11] ... which, uh- **Shunyu Yao** [1:16:12] We should definitely cover this. **Alessio** [1:16:13] Yeah, I'm a, I'm a big fan of simulative AI. We had a, a summer of simulative AI- **Swyx** [1:16:17] Yeah, another term we're trying to coin. Hasn't stuck, but I'm, I- I'm gonna keep at it **Alessio** [1:16:20] ... uh- **Shunyu Yao** [1:16:20] I'm, I'm, I'm really glad you covered my zero citation work. I'm really happy. **Alessio** [1:16:23] No, uh- **Swyx** [1:16:24] Zero citation work. **Alessio** [1:16:25] Well, now it's one. Now it's one. First citation. **Shunyu Yao** [1:16:27] Oh, one citation? **Alessio** [1:16:28] First... It's me. **Shunyu Yao** [1:16:29] Oh, really? **Alessio** [1:16:29] It's me right now. **Shunyu Yao** [1:16:30] Okay. **Alessio** [1:16:30] We just cited it here, so that counts. **Swyx** [1:16:32] Yeah, yeah, yeah. It's like one citation. **Shunyu Yao** [1:16:32] Does this show on Google Scholar? **Alessio** [1:16:33] We'll write, we'll write a paper about this episode. **Swyx** [1:16:35] Um, uh, one citation. **Shunyu Yao** [1:16:36] Oh. **Swyx** [1:16:36] One citation. **Shunyu Yao** [1:16:37] Oh. **Alessio** [1:16:37] Let's go. **Shunyu Yao** [1:16:38] Last time I checked, it's still zero. It's nothing. **Alessio** [1:16:41] It's lit. It's awesome. **Shunyu Yao** [1:16:42] Okay. **Alessio** [1:16:42] This one was funny because you have agents interact with, like, LM simulated person. **Shunyu Yao** [1:16:47] Right, right. **Alessio** [1:16:47] So it's, like, actually just another agent. **Shunyu Yao** [1:16:49] Right. **Alessio** [1:16:50] Right? **Shunyu Yao** [1:16:50] Right. **Alessio** [1:16:50] So it's, like, agent simulating with other agents. This has always been my thing with startups doing agents. I'm like, "One day there's gonna be training grounds for companies to train agents that they hire." Actually, Singapore is the first country to build a cyber range for cyber attack training, and I think you'll see more of that. So what was the inspiration there? Most of these models are bad at it, which is great. You know, we have some room for... I, I think the best model is 4o at, like, 48% average- **Shunyu Yao** [1:17:18] Right, right **Alessio** [1:17:18] ... so there's a lot of room to go. **Shunyu Yao** [1:17:19] Right. **Alessio** [1:17:19] Yeah, any fun stories from there, directions that you hope that people take? **Shunyu Yao** [1:17:23] Yeah. First, I think shout-out to Sierra, which is this, uh, very good startup which, uh, was founded by Brett Heller and, uh, Clay Bever. And Sierra is a startup doing conversational AI, so what they do is they, uh, they build agents for businesses. Like suppose you have a business and you have a customer service, we want to automate that part. And, uh, then it becomes very interesting because it's very different from coding a web agent or whatever people are doing because it's more about how can you do simple things reliably. It's not about, you know, can you sample 100 times and you find one good mass prove or kill solution. It's more about you chat with 100 different users on very simple things. Can it be robust to solve, like, 99% of the time, right? And then we find there's no really good benchmark around this, so that's one thing. I guess another thing is obviously this kind of customer service kind of domain. Previously there are some benchmarks, but they all have their limitations, and I think you want the task to be kind of hard, and you want user simulation to be real. We don't have that until LLM, so data sets from 10 years ago, like either just have trajectories conversating with humans, or they have very fake kind of simulators. I think right now is a good opportunity to... If you really just care about this task of customer service, then it's a good opportunity because now you have LLMs to simulate humans. But I think a more general motivation is we don't have enough agent benchmarks that target this kind of robustness, reliability kind of standpoint. It's more about, you know, code or web, so this is a very good addition to the landscape. **Alessio** [1:18:58] If you have a model that can simulate the persona, like the user, the right way, shouldn't the model also be able to accomplish the task, right? If he has the knowledge of, like, what the person will want, then it means- **Shunyu Yao** [1:19:09] This, this is a great question. I think it really stems from, like, asymmetry of information, right? Because if you think about the customer service agent, it has information that you cannot, you cannot access, right? Like the, the APIs it could call or, you know, the policies of internal company policy, whatever. And that I think very interesting for TauBench is, like, it's kind of okay for the user to be kind of stupid. So you can imagine, like, there are failure cases, right? But I think in our case, as long as the user specify the need very clearly, then it's up to the agent to figure out, for example, what is the second cheapest flight from this to that under that constraint, very complicated reasoning involved. Like, we shouldn't require users to be able to solve those things. They should just be able to clearly express their need. But then if the task failed, then it's up to the agent. That makes the evaluation much easier. **Alessio** [1:19:59] Awesome. Anything else? **Shunyu Yao** [1:20:01] I have one last question for Harrison, actually. **Alessio** [1:20:03] Nice. **Swyx** [1:20:03] Oh, no, no, that's not this podcast. You can't do that here. **Shunyu Yao** [1:20:07] I mean, there are a lot of questions around AI right now, but I feel like perhaps the biggest question is application Because if we have great application, we have super app, whatever, that keeps the whole thing going, right? ### Future **Harrison** [1:20:18] Yeah. **Shunyu Yao** [1:20:18] Obviously, we have problem with infra, with chip, with- ... with transformer, with whatever, S4, a lot of stuff. But I, I do think the biggest question is application. And I'm curious, like, from your perspective, like, is there any things that are actually already kind of working but people don't know enough? Or, like, is there any, like, promising application that you're seeing so far? **Harrison** [1:20:38] Okay. So I think one big area where there's clearly been success is in customer support. Um, both companies doing that as a service, but also larger enterprises doing that and building that functionality in- inside. **Shunyu Yao** [1:20:50] Right. **Harrison** [1:20:51] There's a bunch of people doing coding stuff. We've already talked about that. I think that's a little bit... I wouldn't say that's a success yet, but there's a lot of excitement and stuff there. One thing that I've seen more of recently, um, I guess the general category would be, like, research style agents. Specific things recently would be, uh... Like, I've seen a few, like, AI SDR companies pop up, where they basically do some data enrichment. They, they get a company name, they go out, find, like, you know, funding, uh- **Shunyu Yao** [1:21:18] What is SDR? **Swyx** [1:21:20] Sales development rep. It's an entry-level job title in B2B SaaS. **Harrison** [1:21:24] Yeah. So, so- **Swyx** [1:21:26] I don't know why I know this. **Harrison** [1:21:26] You were very quick on that. Yeah. **Swyx** [1:21:28] The PhD mind cannot comprehend. **Harrison** [1:21:31] And, and so I'd classify that under the general area of kind of like research-y style agents. I think, like, legal falls in this as well. **Shunyu Yao** [1:21:39] Hmm. **Harrison** [1:21:39] I think legal is, yeah, they're a, a pretty good domain for this. Um- **Shunyu Yao** [1:21:44] I wonder how good Harvey is doing, but- **Harrison** [1:21:46] Yeah. **Swyx** [1:21:47] There was some debate, but, uh, they raised a lot of money, so who knows? **Harrison** [1:21:50] I'd say those are... Those, those are a few of the categories that jump to mind. **Shunyu Yao** [1:21:54] Like entry type kind of research. **Harrison** [1:21:56] On the topic of applications, though, the thing that I think is most interesting in this space right now is probably all the UXs around these apps and the different things besides chat that might come out. I think two that I'm really interested in, one, for the idea of this AI SDR. I've seen a bunch of them do it in kind of like a spreadsheet-style view, where you have like, you know, 10 different companies or hundreds of different companies and five different attributes you want to run up and then each cell is an agent. **Shunyu Yao** [1:22:21] And I guess the good, the good thing about this is, like, you can already use the first couple rows of spreadsheet as a few shot example or whatever. **Harrison** [1:22:27] There's so many good things about it. Yeah- **Shunyu Yao** [1:22:28] Right **Harrison** [1:22:28] ... you can, you can test it out on a few. It's a great way for humans to run things in batch, which I don't... It's a great interface for that. **Shunyu Yao** [1:22:34] Right, right, right. It's still kind of elusive to do this kind of like PhD kind of research, but I think those kind of entry type research where it's more repetitive and- **Harrison** [1:22:41] Exactly **Shunyu Yao** [1:22:42] ... it should be automated. **Harrison** [1:22:43] And then the other UX which I'm really, really interested is, is kind of like when you have agents running in the background, how can they... Like ambient style agents, how can they- **Swyx** [1:22:50] Proactive **Harrison** [1:22:50] ... reach out to you? **Swyx** [1:22:52] Yep. **Harrison** [1:22:52] So I think, like, as an example of this, I have an email assistant, um, that runs in the background. It triages all my emails, and it tries to respond to them, and then when it needs my input, like, "Hey, do you want to do this podcast?" It reaches out to me. It sends me a message to actually- **Shunyu Yao** [1:23:04] Oh, you actually... Oh, you, you have it and it is live? **Harrison** [1:23:06] Y- yeah. Yeah, yeah, yeah. **Swyx** [1:23:06] Thank you, agent. **Harrison** [1:23:07] I use it for all my emails. **Swyx** [1:23:07] Thank you, agent for- **Harrison** [1:23:08] Well, we did Twitter. I don't have it hooked up for email yet, but- **Shunyu Yao** [1:23:10] Did you rewrite it with LangChain? **Harrison** [1:23:11] Yeah, of... LangGraph was... I will open source it at some point. **Shunyu Yao** [1:23:14] LangGraph or LangChain? **Swyx** [1:23:15] Yeah, yeah. I want- **Harrison** [1:23:16] Uh, both. **Swyx** [1:23:17] Yeah. **Harrison** [1:23:17] Both. So at this point, LangGraph for the orchestration, LangChain for the integrations with the different models. **Shunyu Yao** [1:23:22] Hmm. **Harrison** [1:23:23] Um- **Shunyu Yao** [1:23:23] I'm curious how the low-code kind of direction is going right now. Are people- **Swyx** [1:23:27] Ah. We talked about this. **Shunyu Yao** [1:23:28] Oh, sorry. **Harrison** [1:23:29] Yeah. **Swyx** [1:23:29] It's not low-code. **Harrison** [1:23:30] LangGraph is not low-code. So- **Shunyu Yao** [1:23:32] You can cut this all out. **Swyx** [1:23:33] No. **Shunyu Yao** [1:23:33] Cut this all out. **Swyx** [1:23:34] No, no, no, no. People, people will tune in just for this. **Shunyu Yao** [1:23:36] Right, right, right. **Harrison** [1:23:36] Well, it actually, it actually has to do with UXs as well. So, like, probably stems back to this idea of, I think, like, what it means to build with AI is changing. Like, I still really, really strongly believe that developers will be a core kind of, like, part of this, largely because we see, like, you need a lot of control over these agents to get them to work reliably, but there's also very clearly components that you don't need to be a developer for. Prompting is kind of like the most obvious one. With LangGraph, one of the things that we added recently was, like, a LangGraph Studio. So it's, uh, we called it kind of like an IDE for agents. You point it to your code file where you have your graph defining code. It spins up a representation of the graph. You can interact with it there. You can test it out. Uh, we've hooked it up to kind of like a persistence layer, so you can do time travel stuff, which I think is another really cool UX that I first saw in Devon and was... Yeah. **Swyx** [1:24:22] Devon's time travel is good. **Harrison** [1:24:24] The UX for Devon in general, I think- **Swyx** [1:24:25] Yeah **Harrison** [1:24:25] ... you said it, but that was the novel. That was the best part. But to the n- low-code/no-code part, the way that I think about it is you probably want to have your cognitive architecture defined in code. Um- **Shunyu Yao** [1:24:37] The decision-making procedure, right? **Harrison** [1:24:38] Yes. But then there's parts within that that are prompts or maybe configuration options, like something to do with RAG or something like that. We've seen that be a popular configuration option. **Shunyu Yao** [1:24:48] So is it useful for programmers more, or it's for, like, people who cannot program? Or I guess if you not cannot program, it's still very complicated for them. **Harrison** [1:24:55] It's useful for both. I think, like, we see it being useful for developers right now, but then we also see... Like, there's often teams building this, right? It's not one person, and so I think there's this handoff where the engineer might define the cognitive architecture. They might do some initial prompt engineering. **Shunyu Yao** [1:25:08] It's easier to communicate to the product manager or- **Harrison** [1:25:10] It's easier to show them what's going on, and it's easier to let them c- control it, and, and maybe they're doing the prompting. And so, yeah, I think what the TLDR is, like, what it means to build is changing and also, like, UXs. UX in general is interesting, whether it's for how to build these agents or for how to use them as end consumers, and there might also be overlap as well. And it's so early on, and no one knows anything, but I think UX is one of the most exciting spaces to be innovating in right now. **Shunyu Yao** [1:25:35] That's too ACI. Yeah. **Harrison** [1:25:36] Yeah. **Swyx** [1:25:37] Okay. Uh, yeah. Well, uh, that's another theme that we cover on the pod. Uh, we st- we had the first AI UX meetup, and we're trying to get that going. It's, it's not a job. It's just people just tinkering. **Shunyu Yao** [1:25:47] Well, thank you guys so much- **Swyx** [1:25:49] Yeah. For indulging us **Shunyu Yao** [1:25:50] ... for indulging us. **Harrison** [1:25:50] Thank you. **Swyx** [1:25:50] Yeah. Harrison, you're amazing as a co-host. Uh, we'd love to have you back. Like, it's definitely- **Shunyu Yao** [1:25:54] You're hired. **Harrison** [1:25:55] I, I just try to- **Swyx** [1:25:55] Yeah **Harrison** [1:25:55] ... I listen to you guys for inspiration and stuff. **Swyx** [1:25:58] It's actually really scary to have you as a listener because I, I don't want to misrepresent... Like, I talk about 100 companies, right? And- ... God forbid I get one of them wrong and, you know. Yeah. **Harrison** [1:26:07] I'm sure all of them listen as well. **Swyx** [1:26:08] Yeah. **Harrison** [1:26:08] Not to, not to add pressure. **Swyx** [1:26:10] So and- **Shunyu Yao** [1:26:12] Okay **Swyx** [1:26:12] ... Shunyu, yeah, thank you so much. Uh, it was, it was a pleasure to have you on. And, um, you had one of the most impactful PhDs in this sort of AI wave. So I don't know how you do it, but I'm excited to see what you do at OpenAI. **Shunyu Yao** [1:26:22] Thank you. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.