# ⚡️Traversal: Causal ML and Reinforcement Learning Latent Space · 2025-10-05 Anish and Raaz, co-founders of Traversal, explain how their agentic architecture combines causal machine learning and LLMs to turn incident troubleshooting from a massive search problem into an intelligent, context-driven process. Both have PhDs in causal ML and RL from MIT and Berkeley, and they applied that expertise to root cause analysis in complex enterprise systems with petabytes of fragmented data across logs, metrics, traces, and Slack. Their system dynamically combines statistical tests (for time-series) with semantic understanding (via LLMs) to winnow billions of signals to root cause candidates in under two minutes. They note that reasoning models like o3 are critical for complex incidents, while Claude is better for tool calling and unsticking. Traversal uses a mix of infrastructure size and investigation count for pricing, and sees self-healing as a continuum: 10–20% of issues can be autonomously resolved now, with 30–40% requiring senior engineer sanity-check within 6–12 months. They are hiring in New York and emphasize that building proprietary evals is core IP, making public benchmarks a tension. ## Questions this episode answers ### How does Traversal use causal machine learning for incident root cause analysis? Anish explains that when an incident occurs, many correlated spikes appear, making it hard to distinguish root cause from spurious correlations. Traversal applies causal ML and reinforcement learning to filter out 'fake needles' by statistically testing causal relationships, intelligently narrowing the search space across petabytes of data rather than relying on simple correlation. [6:08](https://addtry.com/7dc7bcfe-23c6-4c0a-a080-b0117509e4c1?t=368000) ### What is the scale of data involved in Traversal's enterprise incident troubleshooting? Raaz details that during an investigation, Traversal analyzes 'a few hundreds of millions of logs, tens of thousands of time series, a few hundred pull request deployments, and thousands of dashboards,' totaling millions of tokens. The system searches through petabytes of data fragmented across tools like Datadog and Elastic, aiming to deliver an answer within two minutes. [16:07](https://addtry.com/7dc7bcfe-23c6-4c0a-a080-b0117509e4c1?t=967000) ### How does Traversal charge for its AI incident resolution product? Anish notes that pricing is currently based on infrastructure size (e.g., number of hosts) and investigation volume. For limited self-healing cases, they sell outcomes directly. Full outcome-based pricing will mature as their technology improves, but today customers are more comfortable with infrastructure-based models due to the complexity of incidents. [37:21](https://addtry.com/7dc7bcfe-23c6-4c0a-a080-b0117509e4c1?t=2241000) ## Key moments - **[0:00] Introductions** - [1:08] Traversal was founded in January 2024 with no idea; the co-founders met daily at WeWork to brainstorm. - [2:08] Citadel Securities engineer Ahmed brought the incident response problem that became Traversal's focus. - [3:39] Raaz jokes about causality: 'When I say it, then I'm allowed to say it because I have a degree, but you gotta follow it.' - **[5:03] The Idea Maze** - [5:03] Anish admits: 'I was on the plane trying to understand what a trace is. I'd never heard the word SRE before.' - [5:42] Incident response is a 30–40-year-old problem without a great solution, making it a massive market for AI. - [6:08] When a latency spike occurs, a thousand other spikes happen simultaneously; causal ML distinguishes root causes from spurious correlations. - [6:44] The troubleshooting haystack includes logs, metrics, traces, code, configs, and Slack messages across petabytes of enterprise data. - [7:30] Anish predicts AI-powered coding assistants will increase software incidents as engineers ship code they don't understand. - [8:15] Bogumil from Sequoia asked Anish: 'I've heard this pitch 20 times before. Why on earth should we go with you?' - **[9:33] Demo** - [9:52] At Fortune 50 companies, incident troubleshooting spans 50–100 teams and must sift petabytes of data to deliver an answer in under two minutes. - [11:27] Traversal's demo starts with two inputs: an approximate time of the incident and a brief description of the problem. - [14:22] Traversal autonomously builds investigation context by querying live logs, metrics, and traces, hopping from service to service. - [15:46] A Traversal investigation processes hundreds of millions of logs, tens of thousands of time series, and thousands of dashboards, totaling millions of tokens. - **[17:00] Semantics + Stats** - [17:09] Traversal's AI agents winnow petabytes of telemetry by dynamically deciding which statistical tests to run on time-series data, since LLMs struggle with raw numbers. - [18:48] Raaz describes Traversal's core framework as 'semantics meets statistics,' intertwining LLM semantic understanding with statistical time-series analysis. - **[20:26] Self-Healing** - [20:50] Traversal starts with read-only access to build trust, then enables self-healing by matching root causes to pre-approved automation scripts. - **[22:12] Business Model** - [22:44] Anish positions Traversal as the 'Switzerland' of observability, selling investigation outcomes rather than data storage, and remaining agnostic to data sources. - [24:41] Traversal targets complex incidents where no runbook exists, not simple alert automation, and its customers validate the product at scale. - [27:51] The 'hero engineer' problem: for complex incidents, 50–100 engineers gather on a call, hoping one hero finds the root cause aha moment. - **[29:44] Model Picks** - [30:01] Traversal finds reasoning models like OpenAI o3 critical because they must argue across many symptoms in a complex architecture. - [31:11] Raaz says GPT-5's verdict is undecided: 'It starts thinking when it shouldn't,' and the team will decide tomorrow after systematic testing. - [32:23] Raaz contends that deterministic workflows are insufficient because real enterprise systems lack perfect instrumentation, requiring world knowledge and reasoning iterations. - [33:35] Traversal is shifting agentic parts to Anthropic because Claude models are better at tool calling and 'unstucking' from incorrect paths. - [34:47] Traversal moved from vibe-based model selection to detailed eval pipelines, and invested heavily in benchmarking as core IP. - **[35:31] Evals & Benchmarks** - [35:31] Anish declares that for AI agent companies, rigorous evaluation is core IP; Traversal hired four European PhDs to build it. - [38:01] Self-healing is active for 10–20% of incidents; the next tier requiring senior sanity checks will be automated in 6–12 months. - **[38:32] Healing Timeline** - [40:14] Raaz predicts full AI re-architecting of code bases based on production issues is at least a couple of years away. - **[41:03] Wrap-Up** - [42:56] Anish says chaos engineering in staging environments doesn't convince customers; only historical production incidents build trust for AI troubleshooting. ## Speakers - **Alessio** (host) - **Swyx** (host) - **Anish** (guest) - **Raaz Dwivedi** (guest) ## Topics Evals, Agent Infrastructure, Enterprise ## Mentioned Anthropic (company), OpenAI (company), Traversal (company), Claude (product), Claude Code (product), Cursor (product), Datadog (product), Dynatrace (product), Elastic (product), Foro (product), GPT-5 (product), Gemini (product), Grafana (product), Sentry (product), ServiceNow (product), Splunk (product), ThousandEyes (product), o3 (product) ## Transcript ### Introductions **Raaz Dwivedi** [0:04] Hey everyone, welcome to the Latent Space Podcast. This is Alessio, founder of Kernel Labs, and I'm joined by Swyx, founder of Small AI. **Swyx** [0:10] Hello, hello. Uh, we're delighted to be in the remote studio today with Anish and Raaz of Traversal. Welcome. **Anish** [0:16] Thank you for having us. **Raaz Dwivedi** [0:17] Yeah, thank you. **Swyx** [0:19] Yeah, uh, so Anish, uh, I, I've met you guys before. You, uh, you had a... You were kind enough to, like, show me a demo of Traversal before your launch, and then you spoke at AIE. Maybe you can kick us off with, like, explaining your backgrounds and how you got into starting Traversal. **Anish** [0:34] Yeah, absolutely. I'll start and then I'll, I'll let Raaz add to it. So my background in the world, I guess like, like, uh, you, I'm from Singapore. Uh, been in the US now for 15 years. Most of my time has been in, in technical institutions. Um, I did my PhD at, at MIT in Computer Science about 10 years ago. I just fell in love with AI and ML and wanted to understand it deeply. And most of my research at MIT was in a field called, uh, causal machine learning, so the idea that, you know, correlation isn't causation, so how do you get these AI systems to pick up cause-and-effect relationships from data? And also reinforcement learning, which is, to me, fundamentally about how do you search large spaces effectively. Um, and I loved it, and my goal was to kind of be in AI research for the rest of my life, and I thought the best version of that is to be a professor and was lucky enough to, to get into Columbia's faculty, so I'm a professor there of operations research and, and computer science. But I think with all the developments happening in LLMs and, and AI agents, you know, as I was making the transition from PhD life to faculty life, it just felt there's so much huge explosion of activity of... and of research activity in industry, especially in, like, startups with research in their DNA taking on, like, a very technical problem. It felt like a good time to stretch myself in that way and, and so that was kind of my philosophical reason for, for starting Traversal, and obviously Raaz and Raj and Ahmed, who are my other co-founders, they all moved to New York at the same time as me. I've known Raaz and Raj for a long time from MIT, and so we just, you know, all saw the puck-- where the puck was going with AI, so started the company together in, in Jan of 2024 without really an idea, honestly. We just had an id- We just basically said... We, we used to come to Wework every day at 9:00 a.m. and be like, "Let's think till something comes to us." And, uh, but we were looking for something pretty specific, I think something at the intersection of causal machine learning, reinforcement learning, and how it intersected with AI agents. This is before it was cool, like in Jan 2024. Not that many people are talking about it, but we felt like this is, has to be important because that's how you act in the world. Um, and then our fourth co-founder, Ahmed, who was from Citadel Securities before, he was the one who really got us on this problem, and I think he had dealt, as you can imagine, uptime is of utmost important at a, at a place like Citadel, and so troubleshooting and incident response is something they think about a lot. As we looked into it, it kind of felt like a perfect problem for us to take on, and we can get into why, but it felt perfectly at this border of causal ML, reinforcement learning and LLMs and AI agents. It fit really nicely. So I can talk about why exactly, but I'll, I'll let Raaz introduce himself. **Raaz Dwivedi** [2:59] Um, hey, I'm Raaz. My background is, I mean, I, I grew up in India. My background is also in AI research. I came to US actually exactly 10 years ago on August 14th, 2015, and for my PhD at Berkeley, I started in foundations of machine learning more broadly, but then I made my way towards causal machine learning through, actually through a startup, uh, in observability for wireless systems. They're trying to do root cause analysis for, uh, when Wi-Fi goes down, is it a, is it a device problem? Is it a LAN problem? So that kind of, uh, problem, it, during an internship, provoked my interest into this field, and I moved in that direction through the rest of my PhD. So we say correlation isn't causation, and I joke, well, when I say it, then I'm allowed to say it because I have a degree, but you gotta follow it. And then I, I went to MIT for my postdoc, and I met Anish there, and we had very similar research interests. We vibed together and started dreaming of, you know, uh, how do we convert theorems into companies? But that, uh, uh, that really took off when I also got a job in New York a-as a faculty in Cornell Tech, and I was moving, um, again, similar department, operational research and computer science, and Anish and I were like, "Okay, we will be in New York. We have dreamt enough. Maybe it's time to do something about it." And we were like, "What do we know?" We said, "We know AI. We know causal ML. What we don't have is a team, but slowly it glued together, and we don't have a problem. Slowly we figured it out." And here we are. I think, uh, it's a very exciting space of, uh, you know, a very rich problem, no matter how you look at it, the pain point, the data richness from an AI's perspective, the product experience from a user experience. So it's a v- It, it kind of feels like a, a blessing in disguise that we landed on this problem, although this is not the most natural problem that would have come to, you know, either of us during our research time. So yeah. **Anish** [4:51] Yeah. Tell us more about that. I, I think this was one of the first almost obvious ideas when AI came out. I think I've seen dozens of AI SRE companies. It was almost kind of like a meme at some point. ### The Idea Maze **Anish** [5:03] But then obviously you guys figured something out. So I'm curious how much of it is your unique approach versus timing with the model performance versus, you know, the market being ready. Like, maybe talk people through how you went through the idea maze to, to get there. **Raaz Dwivedi** [5:19] Yeah. So funnily enough, I remember when we were flying to San Francisco to pitch, I was on the plane trying to understand what a trace is. I'd never heard the word SRE before. So we really came to this problem from, like, first principles, having no understanding of any of this. And so I think for the idea maze, we went through a few different ideas of what, how causal machine learning and, and LLMs fit together, like what A/B testing for LLMs looks like, what dynamic prompt tuning for LLMs look like. Then we thought about what is something like Amplitude look like, like product analytics for LLMs. So all in this general space of observability, but, like, hovering around it. And I think it was really when we saw this problem from the world of, like, core engineering that it kind of clicked, I think specifically because the market is really big, I think, and it's a long-standing problem, that it's not like you have to make a bet that LLMs will get bigger or, like, A/B testing will be important for these systems. This is, like, a long-standing- **Anish** [6:08] 30, 40-year-old problem without really a great solution so far. So I think that's really what got us interested, and then I think in terms of the why we felt with a unique edge because I think at the heart of it, if you, if you, you know, just trim it down to the core of it, it's basically if some spike that happens, like latency shoots up or something like that, and you're trying to figure out why, the problem is there's a thousand other spikes happening at the same time. And so, and you're tr- and you're trying to figure out is a given spike a symptom of the issue, a spurious correlation 'cause something else is wrong, or truly the root cause, right? And so it's about, like, finding this needle in a haystack with many fake needles everywhere, and that's exactly what Causal ML and RL do. It's about searching for this haystack and figuring out how to filter these fake needles. And so it fit aesthetically and also in terms of, like, our technical expertise really well. And I think we looked at the haystack as, like, logs and metrics and traces and code and config files and, you know, Slack messages. So, and that's exactly what these LLMs are good at understanding. And you know, it's at any large enterprise you're dealing with like petabytes of data. It's like a massive search problem, and so you cannot just put all of it into context of an LLM and hope something great happens. You have to search the data sequentially and adaptively, right? And that's what these agentic systems are fundamentally about. And you're, like, query autonomously many different systems, whether it's Elastic, Grafana, ServiceNow, Datadog, Elastic. And so in terms of, like, the generational wave of technology and what we thought we'd be good at, it fit really nicely. And I think that maybe the third thing is that w- everything we'd seen at that point in the world of, like, AI Swe, I guess, like all of these coding, AI-powered coding assistants, like, that was just starting to pick up. And so it's pretty clear to us that, you know, and we were starting to use it ourselves, and sometimes we didn't understand what the code was doing, but, you know, we shipped it. And, and so it's like, well, if this is, clearly this is gonna happen a lot more. And so if, and if it breaks, no one's gonna have any clue as to how it broke or why it broke. And so rather than... So we're gonna spend all of our time firefighting and being on call and doing QA work, which I think would suck, right? If that's what engineering discipline became. And so we felt like something like Traversal, which is thinking about software maintenance, not software development, has to come about to, like, fit with that wave. And so that was in, in the sense of market timing, I guess. Yeah. So that, that was the all the confluence of factors that came, that made it work for us. And I mean, obviously, the other thing is that, um, we got funded, so that helped. Uh, right. Right, just, and I think the reason that happened was that, you know, like our, our first, our first VC backer was Sequoia, and I think they had met, I think like, I think like 20 VCs, 20, uh, AI SRE companies before this, which unbeknownst to us, and that was the first question. Uh, like Bogumil, who's a board member from there, asked us, like, "I've heard this pitch 20 times before. Why on earth should we go with you?" And so we, we kinda explained, you know, like how we were thinking about it, and I think it was, I think through that conversation for a few days, it was clear to both of us that this is, like, really an AI problem, not an observability problem at the heart of it. And so then a team like us who doesn't know anything about observability but knows a decent amount of AI are the right kind of people to solve it. I'd say that's honestly what last year was about, was just being a research org trying to figure out how to make this work. **Alessio** [9:17] It would be great if we can do maybe a demo in a little bit, so maybe queue that up. But can you talk through the, um... Yeah, just talk about the inputs and then kinda like the runtime of Traversal, so to speak, what it does, and then kinda the output of it. I think today what I do, for example, is like I have a Sentry MCP server that gets my issues. I put it in cursor, write a fix, figure out if it works, ship it, hope it fixed it. What, what's kinda like the future of that with, with Traversal? ### Demo **Alessio** [9:49] Like, what does it look like to be a user? **Anish** [9:52] Yeah. So I think a, a couple of things. One is I think that works really well, that kind of general flow you have when you have... When it's either the problem is really localized, and so the error log you're getting from, say, Sentry, is very high signal, and it connects to a small code base nicely. Now, imagine you are a Fortune 50 company. Most of the people who are dealing with these kinda software operations, it's incredibly distributed, right? And building the fragmentation across 50, 100 teams, you cannot do that, essentially. There's just no way. For, for a variety of reasons, because you have to, like, be querying over petabytes of data, and it cannot... It's not like a simple MCP high-intent query. Like, you have to be searching the space in a much more, uh, intelligent way, and you have to build a shared context across all of these different systems while limiting, while respecting rate limits, because all of, any sort of serious observability system will put, you know, rate limits on you. And so how do you... And typically people want an answer in two minutes. Like, that's what we've learned from just, like, working with customers. Two minutes is kind of the sweet spot. And so how do you search through petabytes of data in two minutes with many, many fragmented pieces of those systems, right? And so I think that in some ways is just as much an infrastructure problem as an AI problem. And I think if I think about the last six months, we've been almost more of an infrastructure company than an AI company, like just figuring out how to search these things effectively, the right way to cache this data, uh, so you can do it real-- you can search it in real time. So I think that's where the complexity is. I think for like very, very localized things, you know, it's, it's okay, but when you're dealing with like production grade scale, it, it gets hard quickly. **Alessio** [11:19] Yeah, let's jump into a demo. I think that's, you know, most people that listen also run into the same problem, so I think they'll, they'll understand it better. **Anish** [11:28] Um, let me share my screen. There are two fundamental inputs. One is an approximate guess of when the incident began or it was first noticed, so that was, in this case, evening on April 22nd, and then a brief description of what happened. And you can see it's like s- something was not working. People couldn't select some region while they were trying to create buckets. You can add more context and use images to kickstart an investigation as well. **Alessio** [11:56] And so when people are creating this, are they being notified of the issue somewhere else, and then they come to you? Or are you automatically prefilling this so that they can start their investigation? **Anish** [12:09] Typically, I'd say it auto-triggers off an incident channel or an alert channel, and so that's typically the flow that happens. Sometimes people will come to our UI for, well, two reasons. One is if they want to do a deep dive into an investigation, because on Slack you just get a small piece of information, basically like the leads. If you want to do a deep dive as to why we believe that is the answer, giving all of the associate evidence, doing follow-ups, all that kind of stuff can happen on our UI. The reason people would come to this page is a few reasons. One is sometimes there may not be an incident, but they just kind of want to get a sense of what's going on with their system. So sometimes on, you know, let's say people will go to a, a, like Grafana or Datadog or something, even when something isn't wrong, to just get like a health check of like, "Is everything okay? Is everything cool?" And so people kind of do that, too. It's like, "Oh, this, you know, this part of my systems feels kind of slow. Can you just figure out what's going on?" For this, like this general health check type thing. So I'd say that's a typical use case for coming in the UI and beginning a manual investigation. **Alessio** [13:11] And then, so the next step is kind of like context creation from an engineering perspective. How do you guys think about building the context? And then maybe if you can talk a bit about long context and whether or not that's something useful to you or, uh, you know. Eh, I think there's always this question of like, if you could put all the data in, would it be better? Like I think here you have definitely more data than can fit in the context window. So I'm curious like where the sweet spot is. **Raaz Dwivedi** [13:36] So context, so I think there are a few ways people do add context. You know, which teams, which services, which alerts they are looking at, which dashboards they are looking at. So I think those nuggets are definitely helpful in making the investigation faster. One of the things that we have shied away from is asking too much from the user, because then what is the value add of Traversal, right? That if you have to find all that context and give it to the AI to go do the last mile delivery. So the onus is on us, so let me just run it and show you what is happening. So when, when the user asks this question, now Traversal is going and figuring out what all contexts should be pulled in for this particular issue that the user is describing. And in the background, it's leveraging who is asking the question, at what time the question is being asked, and its own understanding of the knowledge base or the system architecture in the background. Based on that, it's making queries to, you know, the, of course, the live data, the logs, the metrics, the traces, and is building that context and is doing this sequential hopping as it's building this context to hop from service to service, as you can see, from index to index, and finding these three services are impacted. Okay, but they are also connected like this, so let me go hop there. So this kind of context building is being done by AI, and that's why it takes a couple of minutes. But the user can always warm start when they come to the UI or on Slack by sharing that, "Hey, begin the investigation. Here is the three things I already know." And then it can accelerate the findings of the AI. But, um, yeah, so that's how we are thinking about context. Um- **Anish** [15:20] And I'll add that, um, a lot of times the context is implicitly getting built by things like what is growing in the Slack channel and like the LLM kind of parsing all the relevant nuggets that are getting formed in that war room by itself, and it's using that to guide the search as well implicitly through the process. But we found that the less you... Especially like a highly stressful time, like an incident, the less you can ask of a, uh, you know, an on-call engineer, the, the better in some sense. **Raaz Dwivedi** [15:46] Yeah. But to the, to... I think your question of how much context is being used by AI to come to the answer, you can see that, you know, it has looked at a few hundreds of millions of logs, tens of thousands of time series, a few hundred pull request deployments, and thousands of dashboards. If you just take the token count off of this much data, it's easily a few million tokens. But then how do you do the processing of it so that the-- whenever it hits an LLM or a reasoning or an orchestrator agent, the context is maintained? So that goes into, you know, the tool orchestration or the tool building and memory management that we do on our end, so that at any given point, none of the LLMs or agents are overwhelmed. So that's part of our Agentic architecture, that raw number of context, raw context length is a few million tokens, which cannot fit in a single LLM, but we still try to manage the right pieces to get to the answer quickly. **Alessio** [16:52] Yeah. So right now, if I were one of the VCs that have seen all the pitches, I would be like, "Well, aren't you just getting all this data and then asking an AI what to do?" And I think maybe here is where you start putting some of the Causal ML work background that you guys have. ### Semantics + Stats **Alessio** [17:09] So can, can you maybe now show the forks? So there's kind of like the way that a lot of companies does it, which is like, hey, you, you build the context. Now you ask the LLM to figure out why this was happening. What are like some of the things that you're doing differently, um, that you can talk about? **Anish** [17:25] So I'll note a few things, uh, I'll let Raaz add. So I guess, uh, there's... Building the context itself is where a huge amount of the, the, the pain is, right? Because you're dealing with, let's say, petabytes of data, like let's say a billion possible symptoms, and you have to get it down to a small number of symptoms. And that entire-- So like it's this like massive data processing pipeline where you're reducing-- We're getting more and more relevant context over time, and there's LLMs throughout that process. And each of those LLMs are-- There, there's basically AI agents throughout that process helping to, to, um, winnow down the, the relevant amount of information. And the way it's winnowing down that information is by running these kinds of statistical tests that we have built in a proprietary way. Because most of the data you're looking at is like time series data- And these LLMs are really bad at, at processing time series data, right? And that's really where, like, good statistics comes in. So those are basically, like, statistical tests or the toolkit that the AI agent has access to, and like almost like a data scientist, it's deciding, you know, uh, dynamically what is the next best collection of statistical test to run to filter down the search space, and also use at the same time the semantic information associated with the logs, the metrics, the traces, and so on and so forth. It's like a, it's like a continuous, um, cutting down of the data, and the way the data is being cut down is by an AI agent system dynamically deciding what statistical test to run at that point in time. And Raaz, you wanna add to it? **Raaz Dwivedi** [18:48] Yeah. Uh, yeah, I think the question is you build all this... As Anish said, you have to build this context in an intelligent fashion. You can use LLM for the semantic side of it and statistics for the time series aspects of it. So what we have built is like, you know, semantics meets statistics kind of framework. Uh, let me show you a brief diagram that helps you navigate this huge amount of data. So if you look at this, at a, at a high level, it is a reasoning agent that is trying to sift through your observability stack, and you have so many different kinds of data, and then process them, and then go through, find all the signals in that data. So at any given time, I showed you there are ten billion time series. Somebody comes and tells me, "I am having these particular issues." Which of those time series should I look at? It, it's unclear, and that reasoning has to be built into the agent. The agent is not good at looking at time series data, so that's, that is what statistics needs to take care of. But statistics doesn't understand what is the relationship between latency and memory usage and disk utilization. So that is the LLM part. So how do we intertwine these various tools to sift through the data and then come up with, you know, sifting through so much data to this one line root cause candidate that is also grounded in factual symptoms that are being observed in the data is where, you know, I would say it, it's not just a simple LLM call or a simple MCP React agent that can do this easily. **Alessio** [20:26] Great. So now people come here, and then what's the, what's the next step? Are people moving off to then fix the issue? So I know in the screenshot you showed before, uh, you suggested rolling back one of the commits. ### Self-Healing **Alessio** [20:39] Are you guys mostly focused on fastest path to just stopping the bleeding, so to speak? Or are you also helping companies think about, you know, long term, this is how you can fix this? **Anish** [20:50] Yeah, it's a, it's a good question. I'd say typically when you start with a company, we only ask for read-only access to your, to your system, because that's what security is typically comfortable with. And people also really hate it if you tell them that you're gonna install, like, not an AI agent, but, like, an agent on their infrastructure host, right? Because that's even more of a computational overhead. So what people are generally comfortable with is read-only API access. So... And you have to kind of build trust there. Once you work with a company for a while, then obviously the next question that comes up is, "Can you kind of self-heal the system for me?" And there's a few different answers to that. One is simple things like pull back this, revert this commit or something like that, if it's, if it's very localized that way. And the other thing is that a lot of these companies, especially large companies, have, have these automation scripts to heal their system, and there are, like, thousands of them. And the problem is they don't know which one to run, like restart this pod, autoscale this particular part of your infrastructure, and so on and so forth. And so they don't know which one to run, and the thing is, if you find the root cause correctly, then these LLMs are actually pretty good semantically understanding what is the... what each of these different healing scripts are doing, and it can then kind of connect it or match it to the right one to run. And, and that's how we've been at least threading the needle of self-healing, because people don't want to give you full write access where you can just do whatever to their system. So it has to be like a whitelisted set of commands, and typically that takes the form of preexisting scripts that, that they have found to be helpful. ### Business Model **Alessio** [22:12] That makes sense. I have another maybe slightly related question, which is the business model of these things. I think we talked about it on the podcast a little bit, but you had... You know, like I said, I use the Sentry MCP, right? But Sentry has a Sentry SEER product, but I don't pay for it because I use the MCP for free. How... Do, do the data sources, like, are they your friends? Like, are they happy that you're building this? You know, like, will they... Uh, does, you know, does Elastic want to build something here? Like, what are, like, the competitive dynamics in the market? **Anish** [22:44] Yeah, it's a good question. I think at least what we have seen is that as you start hitting, you know, even midsize and, and large organizations, it's incredibly fragmented. Like, they're using like seven or eight different tools. They're using... I mean, like one company we work with using Datadog, ThousandEyes, Splunk, Dynatrace, Datadog, ServiceNow. And I think, you know, and if you want to troubleshoot, you have to kind of traverse through all of these different systems, pun intended, right? And so that's the... And that's really the pain, right? Is, is the fragmentation of the system and fragmentation of teams. Now, the problem is that all of these different companies, the- typically the way they work is, is they price based on the amount of data they're storing, right? And so, you know, they have very little incentive for, for company A to provide you any insight on data being stored on, on company, observability company B, right? Because that's just not... It doesn't fit with their pricing model as it stands. And so as a r- But the problem is to troubleshoot well, you have to go talk to both of them in an iterative way almost. And that's why I think there's a chance for a company, a new inc- a new company such as ourselves to, um, to have value, because we're not trying to sell you data storage. We're trying to sell you an outcome, which is like the investigation itself, and we're reasonably indifferent as to where the data is coming from and agnostic, right? And kind of the Switzerland of, we don't really care where the data is. So I'd say that's been at least our perspective on why a company such as ourselves has the right to win, even though, you know, there are these incredible incumbents in the space of observability. **Alessio** [24:08] Is this the pitch that you give investors? How, how did you go into the ... Into you pitching the same thing and you're like, "Oh, we'll just be the Switzerland. We'll fix it. It's all good." What, what, what do you think... I, I guess, like, you know, if you do some self-reflection, it's like, what do you think is really unique about the approach you took? **Raaz Dwivedi** [24:25] That really resonated with people, especially when you don't really have a background that is like in this space, right? Like I, I think like, you know, in a black box, you would imagine, okay, the founder of Traversal is probably somebody who ran DevOps or something in one of these big clouds or something like that. **Anish** [24:41] Yeah, I think it was two things. As, as, as Shawn said, like fundamentally, this is a space of show, not tell. And I think, um, we just had good references of, of customers actually using the product at scale and dealing with complex incidents. The c-- I think the, the, the long-term, uh, technical edge will be when you deal with complex incidents, and I'd say most of the companies that have been in the space, they haven't really put a product out that works and people are using it, um, all the time. And also, their use case has typically been like smaller, easy runbook automation for alerts versus like actually dealing with incidents where no one has any answer. And so I think the biggest thing was just showing and having, you know, being able to point to customers that are actually using it. And, and so I think that's probably the, the number one thing. And then I think the second part of it is explaining why this is such a hard AI problem, where it's not just like, "Oh, let's throw a wrap... chat, ChatGPT wrapper around your telemetry and, and something magical happens," and why that's not possible. And so I think explaining that really carefully, and we can talk about what those key AI challenges are. So I think, I think it was a mix of those two things, which is the, you know, actually having it work, and I don't think we've seen any other c-company in our space having something actually work at production. And then second, a v- a very clear distillation of, of the key AI problems, which makes it an AI problem, not an observability problem. **Swyx** [26:01] Yeah. I, I mean, and, and to be clear, I think, um, when Alessio was asking like, you know, the, the, the, the, the, the pitch, it was just, you know, you have very high-profile investors. You know, I got connected to you by Bucky, who was at Kleiner at the time, and, uh, now he's at Lightspeed. And obviously, uh, it looks like you have Brian as well, who I know from Temporal, uh, f- uh, and previously at Index Ventures. So basically like very, very top tier. You know, I know a little bit about your customer list, which you've talked about at the, the Doors Fair as well. And so like, yeah, I mean, it's, it's just a really, uh, fascinating go-to-market that you've done. And then obviously that also, um, there, there's a, there's a lot of like the chicken and egg, right? Like you have to build the software in, in design partnership with all these impressive names, and everyone's trying to do it, and you've somehow figured out how. So that's, that's all we're trying to figure out. **Anish** [26:49] And I mean, I can, I can talk through what some of the, the hard AI problems are here. I think one is, you know, if you think about AI agent company, like AI agent in some sense is a continuum, not a binary, right? It's like you, you can be more agentic, at least in my opinion. And that's fundamentally about how much of the workflow is deterministic versus non-deterministic. Like at the simplest form, you have, you know, a fully predetermined workflow where in each of those steps there's, there's some LLM being called, you know, in one of those steps or two of those steps, and people call it an agent as well, right? But that's really like writing traditional software with, and, you know, a machine learning routine in between. And then you have kind of the full extreme, which is the full control flow is decided by an AI agent, right? And that's really hard to do. And I'd say in our world of troubleshooting, you think about it, especially with complex incidents, there's no... We, we have 50, 100 engineers on a call trying to figure out what happened. And, and the reason that's happening is because there is no playbook, and so we're just hoping that we throw more bodies at it, somehow, you know, some hero engineer will figure out what happened, aha, and then the shared context will somehow crystallize, right? And that's kind of a crazy way of doing things, but that is the best we've done so far because of how complex the problem is. And so I think it's one of those situations where it is one of the most agentic problems, in my opinion, out there, troubleshooting like incidents. And that's why I think it's such a hard problem, uh, to take on and why it's a, it's a good AI problem because you cannot like rely on a runbook. And you have to kind of generate a lot of synthetic data. You have to think very carefully about statistics because these LLMs are really bad about thinking about time series data. So the, these confluence of these few factors makes it, I think, a problem that's like prime for, you know, a team such as ourself that comes out of left field, not from central casting for DevOps. Was well suited to solve it. **Raaz Dwivedi** [28:33] And if, if I may add, I think one other thing is, I think that there is runbooks, right? People talk about is it just a runbook automation product? That's what happens, you know, for regular alerts, and that's what we call AI automation rather than AI intelligence. And then there is incidents where you're completely in the dark. Some people have some ideas, but then you need a lot of collaboration, coordination, knowledge sharing. So I think for us when we began, the questions were we don't understand observability as much, we don't understand these workflows as much. But that also forced us to build in a way that would generalize. Like we couldn't cut corners because ironically we didn't know what the corners are. So how do you cut them? So I think, so then when what we built runs and it actually finds value, we know that it is not hard coded because we didn't know what to hard code. So that's why like there's... Of course, accuracy is, is a long-term battle in this space because the incidents are getting more and more complex. The scale is getting huge and huge. But when our AI run finds an answer, we know it's a good signal. Like it is not by fluke because there is no way we would have guessed it otherwise. So I think there was also some aspect of that in the excitement with the results that they would not have just automated runbooks. ### Model Picks **Swyx** [29:44] Yeah. Awesome. Um, one, one double click on, um, just like the agentic control flow and, um- **Raaz Dwivedi** [29:50] Mm-hmm **Swyx** [29:51] ... and reasoning models. Any commentary on just like what the best models have been so far for you? Do you use any open source models in your, in your stack? Is that useful or are you just frontier models all the way? Uh, and then like obviously if you, if you like GPT-5, let's, we can talk about GPT-5 as well. **Raaz Dwivedi** [30:06] So I think we, we keep trying all the models out there. A few things to note, I think one of the reasons we, we have tried fine-tuning and that does help in certain use cases, but for some of the enterprise customers we cannot bring our own model because of all the AI regulations. The industry is still trying to figure out how conservative or liberal they have to be with the AI usage. So we built an architecture where the AI is rip and replace as long as you Tell me which endpoints to hit, we'll be able to point it. So we have typically found that reasoning models are critical for our use case. And actually, it is the use case where reasoning models are a necessity. You are trying to argue across so many symptoms and such a complex architecture of what the root cause is, and a simple, you know, Foro and these, the flagship models would not work. So reasoning works. Now, the next question is between Anthropic, you know, the three usual suspects, Anthropic, Gemini, and OpenAI. We have typically found historically that OpenAI was better, the o3 in general, but it was also because it was the most easy to use for us to build on top of, so we kept optimizing for it. So there is also this implicit advantage of if you have a well-supported infrastructure, people can move fast. Lately, we have seen Gemini start doing pretty well as well. But again, it's like the infra support is slightly lagging, so then we tend to default to OpenAI for many use cases. GPT-5 versus not, I think the verdict is still not clear. Like GPT-5 starts thinking when it shouldn't think. So , so that's like, that has been our feeling, but I would say we are still undecisive about it, and we are running some final tests, uh, and tomorrow we'll make a decision should we switch to in some places where GPT-5 makes sense. Like we use a collection in many places, but like for the heavy lifting, we, I will decide tomorrow. So far we have not seen overwhelming positive signal yet. **Swyx** [31:57] Okay. I mean, th- yeah, it's interesting. First of all, I think that, that, that eval on GPT-5, I think OpenAI will be very interested to hear that. They always wanna know. I mean, they're gonna fix bugs in and around it, right? Like I think this is, uh, one of the early days still. I think like for, for you guys actually, like, you know, I think there's one of these like more controversial views out there that actually you don't really need reasoning models, especially if you have a pretty deterministic workflow that's like, you know, you're, if you're finding out a lot and you're, you're sort of narrowing it down. You know, it's a pretty constrained domain. You don't have to reason from first principles a lot of the time. You can just kind of like literally spell out the steps to, to, to debug. So there's an argument to not use workflows, but, uh, not use reasoning models, but it sounds like you're pretty o3, uh, centric. **Raaz Dwivedi** [32:39] Yeah, and I think the reason is very simple. No amount of workflows will suffice for a big enterprise. Like you have to link together some of the missing pieces, some of the poorly instrumented data, and that requires world knowledge and a few iteration with the world knowledge. So I think the way we think about o3 versus Foro, like just for reasoning versus not, is do I take my first thought seriously or do I rethink about it a few times, right? So if you compose Foro ten times, maybe you can beat, but then you're-- aren't you essentially reinventing o3? So I think that's how we think about it, is that no amount of workflows will ever be sufficient because of the missing pieces in system context and lack of good instrumentation in any real enterprise scale system. **Swyx** [33:25] And then, and then coming-- Yeah. Oh, s-sorry. **Alessio** [33:27] I was just gonna say one thing that sometimes people bring up is how the Claude models are much better at tool calling. **Raaz Dwivedi** [33:32] Yes. **Alessio** [33:32] Um, has that been your experience too? **Raaz Dwivedi** [33:35] Yeah. So I think for, uh, some of the agentic part of the stack, we are shifting towards Anthropic because they're agentic and their tool calling, especially the unstucking part, you know, when you go down a wrong path and you, you build context that forces you to keep going down the wrong path. I think unstucking there is signal that Anthropic and Claude are, are, are better. So I would say it's real news from our perspective as well. And i-- like we have seen it even like when you compare, for example, Claude Code versus Cursor. Like we are trying to leverage signals from every use case. So internally in the company, we use Cursor and Claude Code, and you see that there is actually some real news in Claude Code as well. So I agree. Like- **Alessio** [34:18] Yeah **Raaz Dwivedi** [34:18] ... it's, uh, slightly better right now. **Alessio** [34:20] Do, do you have some sort of like formal eval internally that you run or is it mostly vibes and then some tests? **Raaz Dwivedi** [34:29] So, uh, I mean, when we began, it was vibes. Uh, but, uh, now we do have, you know, proper benchmarks and very detailed, detailed eval pipeline that informs our decision. And that's why I was saying that, you know, if you were to ask me, uh, for five months ago, is GPT o3 better than o1? I was like, yeah, based on my vibes and a couple of incidents, but for GPT-5, I said the verdict is still pending because we are being more systematic about it. Uh, and I think for our problems, if you think about incident RCA, there's a lot of structure in the problem that it's not only about you go from a symptom to root cause, did you get it right? Yes, that's the gold standard. But how do you leverage, how do you break down this problem and see are you making partial progress in that direction is important to, you know, help both the eval and the calibration of your architecture and potential fine-tuning with synthetic data. I would say investment into eval at an earlier stage is highly recommended for that reason. ### Evals & Benchmarks **Swyx** [35:31] Yeah. I would add that in some ways I find, you know, the best AI companies will always have to be the edge of what the models can do, right? I think you, you always wanna be threading the line. If, if everything works all the time, then you're not really pushing the limit and you're not innovating, right? So I think you always kinda wanna be pushing. And so if you're always pushing, then sometimes your system will work, sometimes it won't work. And so, and so then evaluation, I would say it's become sometimes a big bottleneck. You're spending the most engineering hours just evaluating. And so the-- I think the best companies will be investing a lot of time in really good eval, and I think that's core IP. I tell that to the company all the time, is like one of the core pieces of IP of this company is how well we do eval. And so I think, I think that would be true of, of companies in our space and more broadly, um, as, as AI agent companies. Yeah. And now we, now that we've seen it, we have, we've just hired like probably four people , uh, from the, the world of, of PhDs in Europe. So, you know, Europe's people love benchmarking, right? And eval, so, so, so that's the type of person who we've hired a bunch of now to, to help build that out eval suite. **Raaz Dwivedi** [36:34] And, and basically they were like, "What will I do?" And I was like, "Create the benchmark and beat the benchmark." They're like, "Oh, that sounds exciting." **Alessio** [36:42] But it's funny. Um, on the more business side, you said, um, O3 is obviously very expensive, right? Um, and you mentioned earlier, uh, about outcomes. How do you think about that? We had different... You know, we had Brett Taylor on the podcast talk about outcome kinda like based, resolution based. Dharmesh Shah from HubSpot also talked about different kinda like, you know, results versus work. How do you guys charge for the product? I'm sure a lot of it in the early stages is like, "Hey, look, we'll figure it out," you know. But where do you see kind of like the market going? Like, do customers respond more to like base fixed fee that is kinda like tied to amount of work? Do they like the outcome-based thing for things like this? **Anish** [37:21] Yeah. It's, I'd say it's, uh, certainly we're in the phase of we're, we're learning and the market is learning as well. I'd say it's a function of two things. One is the size of the search space, which is like how much, what is the fundamental like universe that we have to search through? And the best proxy of that is fundamentally the, the, the complexity of your, of your infrastructure, like number of hosts, number of containers, whatever, that kind of thing. And then the second one is the number of investigations you're running. Though I'll say that it's different in customer support versus our world, because there you could actually sell a full outcome, like in a pretty binary sense. Like was it done, was it not done? In our world, I'd say the maturity of, of the technology is such where you can basically give really good leads, right? Um, and so it's not a full outcome where the system is healed every time. And so then I think people have a bit more of an aversion to paying for a purely in terms of outcome based, because it's not there yet. So I think as the, as the, as our technology matures, as our market matures, then I think we'll move to more of a full outcome-based pricing. But right now it's a mix of the size of your infrastructure and then the number of investigations you're running, maybe leaning more towards size of infrastructure than, than investigations. But for some of the settings where we are actually now able to do like full end-to-end self-healing, then, you know, we are trying to sell the full outcome itself. ### Healing Timeline **Alessio** [38:32] Um, how far away do you think we are, uh, as we kind of wrap for the self-healing part? Do you think that's like one year, two years, three years? Or maybe like what are like, what percentage of tasks? Because there's obviously like different difficulties, you know, um, for like the easier ones, medium ones, hard ones. What are like the timelines in your guys' mind? **Anish** [38:53] Raaz, you wanna take this? **Raaz Dwivedi** [38:54] Yeah, sure. So I would say I think, as Anish said earlier, agents is a continuum. I think self-healing is also a continuum, right? If you ask people who work in infrastructure, they are like, "Kubernetes auto-scaling, that's a kind of a self-healing." So, uh, so that analogy aside, our traversal is already doing self-healing for a universe of issues where it's reliable to do a good root cause and then do self-healing. So I would say those kind of issues are like, say they occur 10 to 20% of the time, where end to end you are free and you can... It's like think of if an L1 and L2 engineer can just do it, that is kind of where AI is today. Now, for then there is this level of 30 to 40% of the incidents or issues where you need to involve, um, you know, a senior engineer for sanity checking. I think that healing will appear in, I don't know, six months to a year. That will be comfortably... The technology will be there. Its implementation or the form it takes will continue to be a collaboration with the enterprise. Now, if you go to the, the question that you had asked earlier in the podcast of like, how are you thinking about the long-term, you know, re-orchestration or re-architecting of the code base? I think we are a tad bit away from that kind of a full agentic system that, you know, just reorganizes the whole code base. I would say, well, AI always keeps surprising you, but at least a couple of years away, if not more, where you can then rely on creating the code base. And, and I would say that there is a difference between, you know, code-based debugging and re-orchestrating the code, and then it interacting with an external system. That is like, you can build a lot of unit tests and make your code perfect. That's what many, uh, good companies already do. But when rubber hits the road, then you start seeing the road bumps that you had never accounted for. So what I'm talking about a couple of years away is the issues where you have to see the real production system play out and run into issues that you cannot forecast just by unit testing. I think the things that can be easily taken care of by unit testing would be taken care of in the next year or two, a lot, the bulk of those things. But the more severe ones, I think there is still time. ### Wrap-Up **Alessio** [41:03] Okay. I think that's a good overview of w- where we are. Uh, just to end off, any, uh, things you wanna get on a soapbox about? Any predictions? Um, any rants? **Anish** [41:14] I think in terms of rants, I don't have rants, but I'd say in, in the space of us and, and many other spaces, you know, I think more showing, less telling. I feel there's a lot of people saying the same thing, and so I think that, you know... **Alessio** [41:26] Okay. I think that would be the title of the episode. **Anish** [41:28] Yeah. So I think, I think that would be good in general. And I think maybe... I don't know what-- I think the one interesting problem is that how do you create benchmarks, right? Because for us, the benchmarks we create are such core IP for us because that's how we decide how we're doing. And then there's a world in which you wanna go make that public, but if you make it public, that's you also giving away so much of like what is core to you. And so I think that is like a, a stalemate we find ourselves in, in our world, and I'm sure exists in other worlds as well that are adjacent, whether security or coding. That's, that's something to think about, I guess, is, is how do we collectively move everyone at the same time, retain core IP, which, which I said a bit part of it is benchmarking. So I think- **Alessio** [42:04] I- I'm actually curious about that. When, uh, your customers are evaluating it, how do you run that? Like we have a company, Dropzone, that is a AI SOC agent, so they do something similar to you in the security space still. One idea that I've heard is like, "Hey, you can create these kinda like simulation environments." Uh, we have a company called Veris AI that does this for reinforcement learning. But there's also this application of like you can use the same environment to see how an agent that I would buy from you will behave in it instead of using it as a data generator. Is this something that like it's been on your mind? Like is there anybody doing interesting work there? Just like how do I speed run the POC? Because it's also best for you, right? Like if you can put the Traversal agent in a simulation And just parallelize all these different tests. That's much better than doing a POC for like a month or two months, but it feels like that's not really happening yet. **Anish** [42:56] Yeah, I think the problem is, you know, sometimes we've, we've worked with customers where they say, "Okay, let's, let's, let's, you know, basically do chaos engineering in a staging environment, and we'll see how you do, and then based on that, we'll decide." And then almost invariably, I promise you, you, you will, we'll crush it there, and then they'll be like, "No, no, no." Actually, they're like, "Do you really know?" We have to go in production. And you're like, "What the hell?" Right? So, so I think that's kind of the issue. And then if you're trying to build a high fidelity enough simulator of your production environment, then it's just a production environment. So then I think that's the problem, I guess, is that we run into. And so the best thing I think we've found so far is if they give you like 10 historical incidents or something like that, that from production, and assuming that their data retention of your production telemetry is there, that's probably the best way we've found to, to speed run it versus like a complex environment and, and creating synthetic bugs there. Somehow that doesn't have that... You cannot get a big contract done that way, is what we've found. **Raaz Dwivedi** [43:50] I'll quickly add, I think in the spirit of, you know, the previous question and this question is like everybody... The reason nobody gets convinced on staging environment is that everybody thinks that their environment is unique, right? Like it's the most complex thing on Earth. So they're like, "Can your AI actually figure out the complexities of my system?" Although there are a lot of similarities across environments, but the belief that mine is unique. So I think unless you hit production and show value there, conviction cannot be built in a demo or a staging environment. **Alessio** [44:20] Cool, guys. Thanks for joining us. This was great. Um, any role that you are hiring for or any other call to action for people? **Anish** [44:28] Yeah. I think we are certainly hiring aggressively. We're all, uh, in person in, in New York City, so we're building like a, at least for now, a, a very much like a New York company, and, uh, we're proud of that. So people who either live in New York or wanna move to New York, uh, and wanna work at a cool company in the intersection of AI infrastructure, we are, we're hiring on all fronts. So, so please do reach out to us. **Alessio** [44:51] Cool. Thanks, guys. **Raaz Dwivedi** [44:52] Welcome. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.