# DeepWiki: The GitHub Encyclopedia Latent Space · 2025-05-21 Silas Alberti from Cognition (Devin) introduces DeepWiki, a free no-login product that indexes over 30,000 open-source repos and generates AI-powered wikis and deep research answers. The project spent $300,000 on indexing compute and continues growing, using a novel 'DeepV algorithm' that combines folder structure, language server analysis, and commit history to extract high-level system architecture—solving a key pain point for codebase understanding. Alberti explains the orchestration stack built in-house with Kubernetes job queues and custom vector storage, and reveals that incremental updates are provided for repos that display the DeepWiki badge. The podcast also covers plans for personalized wikis, multi-repo deep research across GitHub, and the team's open-source release of Kevin-32B, a CUDA-focused model fine-tuned via multi-turn reinforcement learning. ## Questions this episode answers ### How does DeepWiki extract high-level architecture from a codebase? Silas Alberti describes that DeepWiki extracts high-level systems beyond folder structure by using graphs from the language server, commit history, and contributor patterns. They work with a competitive programmer, Gennady, on graph algorithms. It doesn't run the code, relying on offline signals. This structured understanding is then used to create the wiki's architecture sections, unlike typical RAG systems that operate on local file context. Silas calls it "the key alpha" of DeepWiki. [7:00](https://addtry.com/0539a04b-1c2c-4927-a504-d81cf1d1df29?t=420000) ### How does DeepWiki keep its wikis updated for free, and what's the badge strategy? Silas says that due to compute costs, incremental updates for all repos would be infeasible. They noticed over a thousand projects had added a DeepWiki badge to their READMEs. To avoid letting those wikis go stale, they now detect the badge presence and automatically keep those repos updated. This serves as a win-win: projects displaying the badge get ongoing updates, driving more exposure for DeepWiki. It's a growth hack that rewards adoption. [14:02](https://addtry.com/0539a04b-1c2c-4927-a504-d81cf1d1df29?t=842000) ### What is Cognition's Kevin-32B model and how was it trained? Silas shares that Cognition released Kevin-32B as their first open-source model, a fine-tune of QwQ. The project, led by three research interns, explores multi-turn reinforcement learning on a CUDA kernel conversion task where the model converts Python to CUDA and gets reward based on output correctness and speed. The multi-turn training made the model better at refining itself over multiple tries, producing more aggressive optimizations compared to a single-turn version that was too cautious to avoid non-compiling code. [28:21](https://addtry.com/0539a04b-1c2c-4927-a504-d81cf1d1df29?t=1701000) ## Key moments - **[0:00] Intro** - [1:00] DeepWiki started as a free side project for open-source Q&A and grew beyond expectations, says Silas Alberti. - **[1:06] DeepWiki Origins** - [2:30] DeepWiki saw its second-highest usage day after launch, with growing adoption in Asia, notes Silas Alberti. - **[2:50] Launch & Security** - [3:15] DeepWiki's indexing compute spend may soon exceed $1 million, Silas Alberti estimates. - [4:04] DeepWiki initially indexed 30,000 repos chosen by recency-weighted stars, prioritizing newer projects over older high-star count. - **[7:00] Code Understanding** - [7:00] Extracting high-level system architecture is the key capability that sets DeepWiki apart from other code RAG tools, says Silas Alberti. - [8:20] Silas Alberti reveals that DeepWiki uses language server graphs and commit history, not just folder layout, to infer code structure. - [10:00] Cognition built its own vector database for DeepWiki instead of using an existing one, says Silas Alberti. - **[10:19] Indexing & Integration** - [13:00] DeepWiki auto-refreshes wikis for any repo that embeds the DeepWiki badge, creating a viral growth loop. - **[13:07] Keeping Updated** - **[15:31] Research & Eval** - [17:00] DeepWiki relies on a small set of manually curated, high-quality evals to keep answers grounded and reduce hallucinations. - **[19:23] Live Demo** - **[21:24] Doc Improvements** - [23:00] Silas Alberti: 'Repo setup is actually one of the hardest parts about development.' - **[24:50] Cross-Repo Research** - [27:00] Silas Alberti proposes expanding DeepWiki to cross-GitHub deep research, enabling comparisons of implementations across all open-source repos. - **[28:21] Kevin-32B Model** - [29:00] Cognition released Kevin-32B, an open-source model trained with multi-turn RL to convert Python to optimized CUDA kernels. - **[31:48] Outro** ## Speakers - **Swyx** (host) - **Alessio** (guest) - **Silas Alberti** (guest) ## Topics Developer Tools, Compute, Language Models ## Mentioned Cognition (company), GitHub (company), OpenAI (company), Sourcegraph (company), DeepWiki (product), Devin (product), Kevin-32B (product), VS Code (product) ## Transcript ### Intro **Alessio** [0:03] Hey everyone, welcome back to the Lit In Space Podcast with another lightning episode. This is Alessio, partner and CTO at Decibel, and I'm joined by my co-host Swyx, founder of smol.ai. **Swyx** [0:14] Hello, hello, and today we have Silas Alberti from Cognition, also known as Devin. Welcome. **Silas Alberti** [0:21] Super cool to be on here. I'm glad. **Swyx** [0:24] Uh, Silas, I, I've run into you a, a couple times. You're just generally in sort of the Bay Area. I think a lot of people have this misconception of Cognition as being an East Coast team, obviously because of the founding mythology of the, the team. But you guys have a significant SF presence now, right? **Silas Alberti** [0:40] We're actually mainly based here in SF. I mean, yeah, the founding mythology is, I think we were a nomadic company in the beginning, so we had hacker houses in different places. I think the first one was actually in Burlingame, then it was in New York, then back on the, on the West Coast, and then back in New York. And it just happened that, like, at the original launch, uh, we were based in New York, and that was, like, when we got a lot of attention, and we had, like, this team photo in New York. But yeah, we've been permanently in the Bay since April of last year. ### DeepWiki Origins **Swyx** [1:07] Yeah. Awesome. And, uh, you know, I think a lot of people know about Cognition and Devin, and we can dive into that as well. You know, both... It sounds like both Alessio and I are, are casual users of Devin. Uh, but the thing that caused this episode now or, you know, the reason I, I, I asked for this episode was DeepWiki, which apparently you drove. So maybe you wanna tell people about what DeepWiki is and some of the origin story. **Silas Alberti** [1:30] Yeah, I mean, DeepWiki, um, first of all, I was kind of excited to see the reception. I mean, it, it was, um, supposed to be this, um, like, open source focused, like, free side project almost, but it, um, became bigger than expected. So the idea is, um, I think we've, like, always wanted to have, like, a Q&A tool that you can use on any open source code base. So we kind of wanted to build, like, deep research but for GitHub. That was the original initial framing. And I think it has, like, two components. It has the wiki itself, which is basically like AI-generated com- code documentation for any code base on GitHub, and then there's, like, the, like, Deep Research agent that leverages both this, like, wiki, but also, like, any code file on the code base and can answer questions for you. **Swyx** [2:13] One thing I was thinking was, like, it, it... Cognition already had, like, a wiki feature, right? Like, uh, you c- you could make internal wikis of your repos. So, so you just kind of exposed it, or what was the sequence there? **Silas Alberti** [2:25] Yeah, I think obviously it was, um, a thing that we developed and kept improving over time. But then, yeah, I think the, um, the particular thing that we had to make happen for DeepWiki was, like, scaling up the infra so we could support, like, tens of thousands of open source repos at the same time. And, um, I think also just, like, making it be this, like, no signup required product that you can just use really quickly, just swap out the repo URL, um- ### Launch & Security **Alessio** [2:52] Yeah. And, uh, do you have any updated numbers of the announcement tweet that you did that's, like, two weeks ago, three weeks ago? You had 30,000 repos, four billion lines of code, uh, $300,000 in compute spend just to do the indexing. Uh, where is the, where is the project now? Are you... Is it growing a lot every week, or, like, does it kind of reach an equilibrium at some point? **Silas Alberti** [3:14] Yeah, I think it's actually, um, been kind of exciting to see that, like, a couple of days ago we... Like, obviously there was, like, a launch spike and then, like, um, there was, like, some stabilized usage, but now it's actually, like, kind of like in a growth phase again. And I think, um, like, I need to check the latest numbers, but I think we had our, like, second highest day after the launch recently, which is kind of awesome. Um, I think there's also, like, a lot of usage in Asia, which has been really exciting to see. We actually recently shipped, like, d- wikis in, like, other languages for that. Um, I think in terms of compute spend, so we, um, think we should be now at, like, um, the high, like, tens of thousands of repos. So I guess you can, like, do the rough math, like, we might be approaching quite significant numbers of compute spend here. I mean, um, maybe we'll, like, break the million soon. Yeah. **Alessio** [4:05] And then what- **Silas Alberti** [4:06] Yeah **Alessio** [4:06] ... how, how do you think about, um, repo selection? So when you started, how did you decide what were kind of like the founding repos that you sp... Because I know a user can submit them today, uh, but when you started, did you just look at the most popular, or was there, uh, is it based on language, you know, anything there? **Silas Alberti** [4:25] Yeah, so initially we just thought, um, we wouldn't even allow people to index repos on their own, 'cause, um, I think that was actually, like, kind of like a pretty s- difficult infra lift to, like, kick off these indexing jobs, um, and, like, create the task queues for that and so on. So we were just like, "Oh, like, maybe this is, like, kind of like the hall of fame of the best open source repos, and they'll just get this, like, free wiki." And we actually thought very carefully about, like, which repos to pick, so it was some combination of, like, the top starred repos, but some weighting for, like, recency. For example, like, a repo that is just, like, two months old, um, and has, like, a couple hundred stars, it's probably, like, more important than a repo that's, like, eight years old but has, like, 2,000 stars. I think, um, yeah, so it's kind of like the recency-weighted top 30K repos. And a lot of our usage is on, actually on these, like, top 30K repos. So I would say, like, like, 80%, 90% of users just, like, check up on the popular repos. **Swyx** [5:18] And, and, like, this is an ambitious project. Like, it... I think any time anyone launches any AI-powered thing with no sign-in, that can be pretty popular right off the bat, obviously, 'cause it's immediately useful. But then there's a, a security question involved of, like, um, you know, were you worried, or did you, you know, did you, did you put in, like, I don't know, fingerprinting or, like... What's, like, the standard security stack that people should know about when, like, launching something like this? Like, I want to launch something like this, but I, I don't want to expose inference to the internet. **Silas Alberti** [5:46] Yeah, no, I think it's something we were definitely worried about. I think we have, like, some protections, like some, um, like rate limit heuristics to make sure we're not getting DDosed. Fortunately, we have, like, really good security guys, and I kind of, like, just, like, trusted their judgment on, like- **Swyx** [6:01] Okay **Silas Alberti** [6:01] ... what is the right level of protection here. I actually think we erred on the side of just, like- Being pragmatic here, though. I think there is obviously, like, always a risk, but I think we just wanted to ship something. And I, I, I just, like, really don't-- didn't wanna have a sign-in, 'cause that's just, like, so annoying. **Swyx** [6:13] Yeah, it is. It is. Perhaps we could also dive into some of the design choices inside of one of these repos. Is that, is that instructive? Um- **Silas Alberti** [6:21] I'm gonna- **Swyx** [6:21] Could also do this. Unless, Alessio, you have oth-other questions before I go into this. **Alessio** [6:25] No, that's good. I just indexed smol-podcaster, so once that's done, we can look at it. **Swyx** [6:31] Okay. Well, okay, I'll do VS Code first because that is the, uh, $12 billion GitHub project that everyone can fork. So if I, if I kick off VS Code- **Silas Alberti** [6:41] That's a hard one. **Swyx** [6:41] Yeah, it's a really hard one. Yeah, I, I actually know this code base 'cause I, um, I've done, I've done a VS Code extension a little bit. So mess with Monaco, mess with the extension system, uh, and actually also had to write the, uh... some stuff for the terminal. What are the design choices here? Like, how do you get, uh, a wiki out of a code base? ### Code Understanding **Silas Alberti** [7:00] I think the hardest thing, honestly, is the high level structure in the systems, right? Like, this is, I think, the key alpha that, like, DeepWiki has that I haven't seen anywhere else yet, 'cause, um, I mean, there's, like, a lot of, like, RAG systems that, like, people have built where, like, you just retrieve a bunch of files and, like, obviously we can, like... LLMs can read these files and start to understand, but this is, like, a very local context. So the thing that we wanted to figure out for DeepWiki, and this is also something that, like, we wanted for Devin, and this is what inspired this project in the first place, how do you take a code base and then extract the high level systems? And that's kind of like the structure on the left side, right? Um, you... Like, in a... Like, this is what an engineer would have in their head that, like, you, you're annoyed about and, like, the agents don't understand. Uh, and we kinda, like, wanted to solve that problem. **Swyx** [7:48] So you always have an architecture overview, you always have courses, core services, and then obviously these are all custom. And, and then, and then you also visualize it, which I, I find quite useful. Are there any useful heuristics that you, that you want to share? Like, sometimes it's not obvious when, when a project this big covers so many modules, which is top level and which is a child of the top level. No, but they just put it all in the same folder. Or do you, do you just go by folder structure? **Silas Alberti** [8:20] Yeah, I think folder structure, um, is definitely, like... The way I view it is honestly there's, like, these different graphs you can, like, map onto a code base, right? There's, um, the simplest one is the folder structure graph, but unfortunately it's, like, actually a pretty bad one because people sometimes just, you know, like, put all the components in this folder and all the servers in this folder. **Swyx** [8:39] Yeah. And the- **Silas Alberti** [8:39] And it doesn't really- **Alessio** [8:40] LSPs everywhere, jumping everywhere, yeah. **Silas Alberti** [8:42] Like, things we found helpful, there's, like, the language server graph. There's also the commit history. That's actually a really interesting one because what did people contribute to? Con-contributors usually own certain systems and also, like, how did things develop over time? We found that to be, like, a very interesting signal. **Swyx** [9:02] Right. Yeah. **Silas Alberti** [9:02] One of the people who contributed to this project was, uh, Gennady. I don't know if you know about Gennady. Gennady is, um, like, the number one ranked competitive programmer, has, like, six IOI gold medals and is, like, really, really good. Happens to be really, really good at graph algorithms, so it was, um, certainly a pleasure to have him, like, figure out, like, cool graph algorithms. **Swyx** [9:21] Yeah. It's super awesome. So the, the inputs are, let's say, file system, commits, I think, and then, uh, LSP as well. Anything around, like, dynamic linking, which is, I think, the, the hard part, right? By dylin- dynamic linking, I mean, like, the only thing that is shared between systems is, like, a string, and it's, like, sort of dynamically invoked at runtime. Right? Like that's the hardest- **Silas Alberti** [9:39] Yeah. No, yeah, we don't, uh, we don't run the code base. Um. **Swyx** [9:43] Yeah. **Silas Alberti** [9:44] This is something that, um, I guess Devin could do if, if it actually, like, has it as a... Like, that's the cool thing about the Devin agent. It has, like, an actual VM. It can run the code base. But for the indexing right now, we just use kinda, like, offline signals, yeah. **Swyx** [9:58] Awesome. Um, Alessio, you wanna share your run and... So Alessio has a code base that he's-- that is public that we also like to share on the podcast because it's like a, you know, like a meta podcast thing about podcasts. **Alessio** [10:11] Let me see if it's done indexing. **Swyx** [10:13] So- **Alessio** [10:13] We were just talking about this. **Swyx** [10:14] Oh, yeah, okay. Okay, I can, I can, I can buy time, Gen. I was also curious- **Alessio** [10:18] Yeah, it's still... It, it has two to ten minutes. ### Indexing & Integration **Swyx** [10:21] Is it done? **Alessio** [10:21] So it should be, it should be done soon. **Swyx** [10:23] It's fine. Okay. What is the indexing... S- indexing processes, do you have an orchestrator that you like? So I, you know, I, I'm just, like, the, the default workflow person. I used to work at Temporal. There's other solutions like Restate or, um, Airflow, you know, what-whatever. Like, do you know, do you know what the orchestration system is? **Silas Alberti** [10:43] Yeah. We, we have, like- **Swyx** [10:44] You wrote your own 'cause you're so good. **Silas Alberti** [10:46] We have this culture of, like, building everything in-house. Um, it's also kinda funny, like, we, um... There's, like, a RAG component to this, and I've been actually... I, I personally advocated, okay, let's just, like, use a vector database and make our lives easy. But then there was this, uh, really strong infra guy, Andrew, who was like, "We don't need a vector database. We can build our own vector database." **Swyx** [11:05] This is Andrew Gao? **Silas Alberti** [11:06] Uh, this is Andrew He. **Swyx** [11:08] Okay. **Silas Alberti** [11:08] Also a famous competitive programmer. I think he's top 10 in the world as well. Um, but also happens to be, um, like, an infra god. Yeah. Um, and so, um, he's been actually, like, owning all the infra for this. Um, yeah, I think we just, like, schedule indexing jobs in Kubernetes, have, like, a system that, like, manages it. You know, there's a queue obviously because it, it's... Like, especially during the launch, there was, like, a lot of demand at once. Sometimes it's, like, random spikes. I don't know how they happen. There were, like, 2,000 indexing jobs in the queue suddenly. There was, like, the other day. Um, I think right now the queue is not that long, so I think it should be fine. **Swyx** [11:46] Amazing. **Alessio** [11:47] How do you think about consumption of DeepWiki for obviously people, but then for LLMs? I think the good thing that you have is there's, like, the relevant source files under each service that kind of links back to it. Is there an easy way, say, if I'm using that repo in Cursor to, like... or, you know, obviously Devin, to use that kinda, like, deep link? To pull in the right documentation from DeepWiki, because I saw the URLs, they're not-- they're kinda like, you know, uh, they have a header number and then a subindex, so there should be an easy way to get one file into an LLM to say these are, like, the pages and these are, like, the links to the relevant source files, and then have that being put into the agent context. Is that kind of the idea of how you use it internally at Devin? **Silas Alberti** [12:30] Yeah. Yeah, I think, uh, if you wanna use it with, like, other services, happy obviously, uh, for people to do this, um, you can, like, put the links in, and they can just, like, crawl the website. There's also a guy who just built, like, an MCP server for DeepWiki, which I thought was pretty cool. Yeah, I think if you use Devin, I mean, Devin just, like, has kind of Wiki deeply integrated into z- into its brain, to the extent that, like, even, um, like the underlying structure that is, like, not quite exposed, uh, to the front end, um, of how it, like, views the systems and, uh, which files belong to which pieces. So, um, I guess, like, this is part of what, uh, makes Devin hopefully smarter than other agents. ### Keeping Updated **Alessio** [13:07] Yeah. And then I also saw you have auto-refresh thing on the right that is not enabled. It's the idea that basically you're re-updating it based on PRs that get merged. Like, how, how do you kinda see, yeah, keeping... Th-these are always, like, great projects to, like, start right, but now it's like next month- **Silas Alberti** [13:23] This is the growth hack **Alessio** [13:24] ... it's gonna be more out of date, and then later it's gonna be more up to date. What, what's the process for keeping them on in sync? **Silas Alberti** [13:31] Yeah, dude, this is, like, a thing we were, um, debating a lot, right? Because it is pretty expensive to do these indexing jobs, right? Um, so first of all, we made it a little bit cheaper to, like, do these incremental updates. If you, like, pay for Devin, for example, like, we actually update it on literally every commit incrementally. I guess Wiki might have some, like, latency, but, like, some of the other indexes, like, get always updated. For the free product, it would just not be feasible to, like, do, like, close to, like, a hundred thousand repos, like, every day. So we found, like, okay, there was more than a thousand projects already had the, like, DeepWiki badge to every... Which I thought was, like, super awesome to see. Um, like, way more than I expected. And I felt kinda bad that we would just let these wikis go stale. So we just decided, okay, how about we just detect whether you have the Wiki badge, and then we'll keep it updated for you? Kinda like a nice win-win situation. **Swyx** [14:30] Yeah. Um, I mean, i-to me, this is the growth hack, right? Like, also, if you, you get featured on the VS Code README, like, a lot of people will find the DeepWiki as well. As you n- as you know, I think, I have made my own extension such that every, um, uh, every repo has a DeepWiki page that I can- **Silas Alberti** [14:50] Is that an extension that I can use? **Swyx** [14:53] Uh, yeah, it's like s- but, like, it's not a well-named extension 'cause I repurposed an existing extension to, to do that. So it's called the Netlify Browser Extension. I think it's under my GitHub or something. And I just- **Silas Alberti** [15:08] Okay, Neti- **Swyx** [15:08] ... I just added a DeepWiki button. 'Cause it, it has the Netlify button- **Silas Alberti** [15:12] Okay **Swyx** [15:12] ... and I just added DeepWiki in there. So yeah, here it is. It's on Chrome. There we go. People were like: "Hey, this, this looks like the, you linked the wrong thing." And I'm like: "No, it's the right thing. I just didn't update any-" "... anything about this. I just added it to my own extension." But yeah, I, I think it's, I think it's useful. ### Research & Eval **Swyx** [15:31] Basically, it's just an always up-to-date README that is, uh, that is deeply researched. How would you compare actually to, like, this as, like, a deep research thing, right? Like, 'cause, like, uh, OpenAI just added deep research for GitHub repos, which is kind of encroaching in on DeepWiki a little bit. **Silas Alberti** [15:49] Yeah, no, definitely. I mean, yeah, I think this is, um, kinda like a thing I would expect a lot, a lot of people to ship at some point as well. I think it's part of the reason why we want to ship it fast, 'cause we've, like, had this sitting around for a while, but, like, uh, mostly paywall behind Devin, and now it's, like, available to everyone on open source. Um, I mean, I think I feel pretty confident that, like, our deep research is like, especially given this, like, high level systems thing, like the DeepV algorithm gives, just, like, gives you better answers. I think we can actually try it out if you want, um, because there's, like, this question asking thing at the bottom. And we could, like, try to ask a question to VS Code code base. **Swyx** [16:22] Sure. What should I ask? **Silas Alberti** [16:25] Well, um, let's see. Let's ask how is... Like, how do I Let's, let's, let's ask something concrete about the code base. For example, how are inline suggestions implemented? **Swyx** [16:39] Okay. And do I want the deep research button or not? Uh, what does this do? **Silas Alberti** [16:43] So deep research just makes it, like, take more time and whether you have a better answer. Maybe if you want to, like, not wait a long time, you can just do the normal one. So, um, it's less smart. So usually I prefer- **Swyx** [16:52] Yeah **Silas Alberti** [16:52] ... deep research, but it's, um, gets the point across. **Swyx** [16:56] Okay, so here there's a little bit of code rag. Uh, actually I did wanna ask, are you-- is, have you found any code embedding model that's good? Do you guys have your own? What are you doing for code rag? **Silas Alberti** [17:07] I guess I can't, probably can't share the exact thing that powers our act. **Swyx** [17:10] Okay. Sure. Obviously, there's... It's just the, yeah, there's just a lot of images, so I had to ask. Okay, how inline suggestions are implemented in VS Code? Unfortunately, I don't know the right answer, so it's hard to, hard to justify or it's hard to evaluate. Evals are hard. How do you eval DeepWiki? **Silas Alberti** [17:30] Well, I think, yeah, we have our own set of evals, and so yeah, I think it's just, like, kinda, like, work that you have to do to create evals and make them good. Um, I think usually, um, like small numbers of very high quality evals are the way to go, and, like, we curate them manually and make sure they're really good. Um, I think the good thing is, like, this is, like, very grounded, you know, like I think the hallucinations are usually not that bad. Like, usually you just, like, notice that it just, like, didn't find the thing that you asked about. But in this case, it seems like it just found, like, a thing that's called inline completions controller, like that's probably the thing that we were asking about. And that's, like, super helpful. Like, actually, like the, the reason why I wanted to watch DeepWiki was 'cause I was actually- Have the desire to use this all the time. You know, like, there's like some open source project you work with, you just like wanna find where is this implemented, and then, like I just like I find it really quickly. **Swyx** [18:26] Okay. I, I launched the deep version anyways 'cause it was, it was like one click and it was like, "Go deeper." I'm like, "Okay." I don't really know it'll do anything. I think that's one of the questions about the various forms of deep research that are out there now. It's like, I don't really know what I get apart from some preset amount of deeper, but it's, it's unclear the quality improvement, if there even is a quality improvement, right? Maybe this is the complete answer, and there's no better answer to get. **Silas Alberti** [18:53] Yeah, I feel like for this question, definitely the fast answer seems to get you most of what you want. There's like sometimes things where like the deeper version... Like, especially if you want it to like find like initial thing and then like explore like the code like around it, and like actually like jump to definitions, like traverse the code base, that's where like the deeper becomes very helpful. The deep version usually takes like a couple minutes though. **Swyx** [19:21] Yeah. Yeah, yeah. Awes- awesome. Um, all right. I got the smol-podcaster results. Let's see. ### Live Demo **Silas Alberti** [19:29] Awesome. **Swyx** [19:29] Share screen. All right. I have-- I'm not, I actually have a private fork that I'm building, so this is a little, um, outdated, but it was great. So I liked that. Let's do system architecture. Hello architecture. This is pretty good. Replicate OpenAI Anthropic, and then this is how we store files, interfaces. Um, yeah. There's like transcribe, how we process transcript, do the show notes, chapters, pick clips for YouTube. Um, use the transcript because they use a different subtitle format. It's very cool. I mean, this is like- **Silas Alberti** [20:09] Also it's not like, you know, so often I just want exactly this for a code base, you know. **Swyx** [20:13] Right. **Silas Alberti** [20:14] If I... When I was onboarding onto Devin, you're like- **Swyx** [20:16] Yeah **Silas Alberti** [20:16] ... **Swyx** [20:18] Yeah. And there, there's like some funky stuff like how, uh, how Hue thinks in Celery. Yeah. Like I, I think this part, like when some... if somebody wants to contribute, it's like pretty easy to go through the Python, but then figuring out how everything gets kind of scheduled, uh, because it takes so long to do the transcript and, and all of that. Yeah. It's good. Also, like the... There's this other thing to sync, like audio and video get edited differently because you cannot really cut all the silence in video because otherwise it looks... It's like jumping. So here it's like, yeah, it's like Levenshtein distance for like segments and, yeah. This is better than the read me of the project, for sure. Let me see. Where is it? Uh. Yeah. I mean, I think there's a world in which we don't need readmes anymore because we actually have- **Silas Alberti** [21:12] Right **Swyx** [21:12] ... the world where the code documents itself. **Silas Alberti** [21:15] Yeah, totally. I should put the, I should put the bun. That's what I need. **Swyx** [21:21] Button in there. **Silas Alberti** [21:22] Get the badge. Get the badge in the thing. One, one thing we've been wondering also is how do we make this like actually the place that you-- people just feel comfortable using as their documentation? ### Doc Improvements **Swyx** [21:32] I see. **Silas Alberti** [21:33] Yeah. **Swyx** [21:33] Oh, you actually want that. Yeah. **Silas Alberti** [21:35] I mean, like we've been thinking about like how do we make it editable, um, while balancing that like you kind of like also want it to like refresh automatically and like new commits as they come in. Um- **Swyx** [21:45] Hmm. **Silas Alberti** [21:46] So you probably don't want the thing where you just like directly edit the texts. **Swyx** [21:50] I mean, probably just, you know, like a clear demarcation between human written and AI written sections that you can place in somewhere and you can pr- you can remember that and, you know, I think you guys are pretty good at diffing or like generating diffs. **Silas Alberti** [22:04] Yeah. That's probably a form with a dealership soon. **Swyx** [22:08] Yeah. **Silas Alberti** [22:09] Interesting. **Swyx** [22:09] I think you need... You should add, um, code execution, and then people can just test. You know? I, I think that's something maybe like, uh, 18 months ago, two years ago, there's like a, a few YC companies that try to do like interactive docs where like you could actually run the code in the docs to like see what would happen. **Silas Alberti** [22:26] Oh, I see what you mean. Yeah. **Swyx** [22:27] A- and I think that also like for, for the other line it helps for like verification, right? It's like if it doesn't run, you can actually figure out why and fix it. Um- **Silas Alberti** [22:37] Yeah **Swyx** [22:37] ... I think that's how you get... You add to it. **Silas Alberti** [22:41] You know, it, Alessio has a code sandbox to sell you if you need. **Swyx** [22:44] Well, yeah. But that's, uh... But everything, everything, everything will need to run code, so. **Silas Alberti** [22:50] Everything needs to run code. **Swyx** [22:51] I, I think like there's some projects where that is helpful and some where they're not, and so oftentimes you need to specially prepare the repo or like, you know... Yeah. There, there's all these... The world is so not standardized. Like the closest- **Silas Alberti** [23:03] Right **Swyx** [23:03] ... we've gotten to is Devcontainers, basically, and Devcontainer adoption is like nothing. Like no- nobody... Like I'm still scared of Devcontainers. **Silas Alberti** [23:10] It's a, it's a huge problem that we face with Devin actually because, um, like one of the things that makes Devin still very unique among like all the coding tools is that it's, it's like this like in the cloud agent that you can like parallelize and spawn like 10 instances of. So Devin actually needs like the full dev environment to be set up, and I think you guys probably both went through this when you onboarded with Devin. It's like probably like the highest friction onboarding point that we still need to like make- **Swyx** [23:37] Yeah **Silas Alberti** [23:38] ... as easy as possible. **Swyx** [23:39] Oh, y- I mean, you guys have come a long way. I, I mean, I first tried you out like early last year, right? You guys have come a long way on the onboarding, but still like there is... Yeah. I mean, you know, you want, you want the magic thing of like, just connect my GitHub, and then you can scan everything, and then like, you know, the PRs just work. Like no, like I have to go in and manually set up every single repo so that you have a snapshot that, uh, you can start from, right? **Silas Alberti** [24:01] Yeah. **Swyx** [24:01] But I, I think, you know, that if I'm doing serious work, that is a cost I will pay because I expect it to pay off down the line when you can actually do PRs for me, so. **Silas Alberti** [24:11] Yeah. We, we, we've, we've been trying to get like Devin smart enough to just set up the repo for yourself. But it turns out that like repo setup is actually one of the hardest parts about development. **Swyx** [24:20] Yes. Yeah. **Silas Alberti** [24:21] This is also like when you onboard a new company, like actually getting us like- It's all these, like, different VPNs and keys and, like, all things you need to figure out. And, like, it's obviously not documented at all. **Swyx** [24:30] Yeah, and infrastructure, right? Like, like Redis and, like, you know, Postgres and, like, whatever your local networking thing is, if you, if you use that kind of... Like, if you have, like, a microservices system, good luck. Like, I don't know how you can do that. It's interesting. Okay, cool. Um, any other... Like, any other sort of directions or questions about DeepWiki that you're exploring that maybe you can just put out there and, you know, let, let people tell you what they want? ### Cross-Repo Research **Silas Alberti** [24:51] Well, uh, yeah, I guess the biggest one is just, like, okay, how do we not only add in but also, like, personalize DeepWiki? Um, I'm very curious, like, people have said, "Okay, I kind of like have some opinions on, like, how to structure. Can I influence it? Can I..." Maybe, um, maybe the ideal response to, like, a deep research question is actually not just, like, an answer, but, like, its own, like, small wiki page that also has, like, diagrams and everything. I do actually think it can generate diagrams if it wants in a, one of the responses. And, uh, and actually the last one that I'm pretty, uh, pretty excited about, uh, that I'm sure- curious to hear you guys' opinion on is basically, like, what if you had deep research, but not just on a single code base, but on, like, all of GitHub, you know? So, like, let's say you're like, "Oh, I want to implement, like, a markdown inputs." Can you find across, like, all the open source repos, like, what is a great way to do this? **Swyx** [25:42] Yeah. I, I think if you can figure out what's still being maintained... I have this problem with Rails just because the Rails has such a long history. There's, like, so many gems that do the same thing, but some of them are kinda outdated, some are not. But from a search perspective, it's hard to figure that out until you get in. Because a lot of them have, like, a lot of stars, but, like, the last commit is, like, seven years ago or, like, whatever. Or maybe they're not compatible with, like, the latest Rails or whatnot. So I think that would be cool. I'm guessing that the issue you're gonna run into is, like, forks. It's like people just fork something and they make some maybe one, two small changes, and then it's, like, a different project but the same. I'm curious how you're gonna end up solving that. Yeah. **Silas Alberti** [26:21] Yeah, I think that's definitely a thing, but I guess there, there you can use some heuristics just on, like, number of stars that, like, the project has or, like, how relevant it is. Yeah, I think the cool thing that the Wiki has that you wouldn't have otherwise is, like, if you just do, like, pure, like, RAG on, like, such a big base of, uh, code files, it'll just be, like, pretty bad at a certain point. You know, on a single code base, sure, I can see it, but on, like, tens of thousands of code bases, uh, it's tougher. But I think actually the wiki pages are kinda like a nice abstraction on top of the code base, um, that seems to be, like, a very nice pool to do search on. So I'm, I'm pretty interested in that. **Swyx** [26:57] Yeah, single code base is good. Uh, I would also refer you to Sourcegraph, um, because they, they have also indexed all public GitHub repos, and they provide search over all of them. So as your MVP, use that and see what you would pull in in context, because you kinda have to rebuild Sourcegraph or partner with them. **Silas Alberti** [27:15] I'm a big fan of Sourcegraph, by the way. I love it. **Swyx** [27:17] Oh, okay. Okay. Yeah. And, uh, so, like, they already have the infra for this, and I've used it, like, once or twice because you... Because everyone can do token search very well, but if you want to do semantic search on code, it's a different animal that, uh, is not solved. **Silas Alberti** [27:34] Yeah, the thing that I find useful, and the thing that I also imagine, like, the deep research across GitHub to look like is basically you just, like, have this, like, random function, and you wanna see how it is being used, for example, right? And, like, Sourcegraph sometimes can do this, where you just, like, search for this particular symbol across all of GitHub and, like, see, like, nice example implementations. But I think there's actually, like, so much more you can do if you have, like, more, like, semantic and deep research style. Like, like, what are the ways that people have implemented, you know, inline suggestions in their VS Code forks? **Swyx** [28:04] Yeah. Yeah. Yeah. Okay, cool. Well, there's lots more R&D, but I think this is a huge step. Any call to action? I, I'm definitely, uh, I, I'm inviting you guys to, to speak at the conference next month. I think Scott is, is at, at least doing a talk. I'm not sure if we- **Silas Alberti** [28:18] Yes **Swyx** [28:19] ... are doing something yet. Any other call to action? ### Kevin-32B Model **Silas Alberti** [28:21] Well, please, um, try out DeepWiki. Send me any feature requests. Link it on your README if you want it to be auto-refreshed. And, um, also check out, we launched another thing recently. I wanna give it a shout-out to the three new research interns on our team, Kevin-32B, our first open source model release. So, um, that is also worth checking out. I think it's actually a really sick project, and, um, I was pretty, pretty excited with the results there. **Swyx** [28:47] Yeah, it's like a fine-tune QWQ, right? Do you, do you guys in- intend... Is this, like, part... It looks like the Notion page is no longer shared. Where is it? Oh, here it is. Okay. Um- **Silas Alberti** [28:57] Yeah, you can pull up the blog. I think it's up at the bottom now. **Swyx** [29:02] Is this, is this an area, like... So Devin does not expose a model selector, right? Like, there's just, like, whatever Devin's in-house model is, we have no idea. They used to use GPT-4, and then you've, you've come up frequently on, on, uh, OpenAI's stuff. So I, I presume that a large part of it is OpenAI. But it's, looks like you're also exploring open models. **Silas Alberti** [29:27] Yeah, this was obviously, um, like, an internal kind of, like, research project. And I think the thing we wanted to explore here is can we, um, get multi-turn reinforcement learning working on, like, a nice environment that is, like, good code execution feedback. CUDA kernels are just so cool because you can just, like, compare the Python implementation with the CUDA implementation, so the task is to convert from Python to CUDA. **Swyx** [29:50] Mm. **Silas Alberti** [29:50] And then you can just check are the outputs the same, and then, like, how much faster it is. And it's, like, a really nice reward signal. And, um, I think the thing that was particularly interesting here is that, like, the multi-turn reinforcement learning actually did something. Um, basically, as you trained the model, it not only became, like, better in the first try, but better at refining itself. **Swyx** [30:09] Yeah. And, like, sometimes, like, I always see that you can actually cut the steps once you, like, have a reasonably good signal on, like, the, the path for solving these things. **Silas Alberti** [30:19] Yeah. Yeah, I think one, one cool thing also, basically, like, the single turn model that was just trained on, like, getting the best result after one turn, it would basically be a little bit, like, too careful, 'cause it- **Swyx** [30:29] Mm **Silas Alberti** [30:30] ... could risk writing, like, non-compiling code. **Swyx** [30:32] Yeah. **Silas Alberti** [30:32] Whereas, like, the multi-turn model would maybe do, like, more aggressive optimization, but then, like, there's, like, one compiler or one bug, and it has, like, three more turns to fix it. So it was kinda nice. **Swyx** [30:43] Yeah. Fascinating. Yeah. It was interesting, like, to s- to compare this, especially this diagram, with the RFT launch from OpenAI recently, um, because this is exactly how they're writing their verifiers. I mean, obviously this is, like, the standard R- RL way, but it would be useful to see how widespread this becomes. Like, I... This seems like a very powerful thing to do, but also, like, a very advanced thing, only if you have verifiable rewards, which seems to be coding and math, basically. **Silas Alberti** [31:16] Yeah, I think that's almost how I view, like, the future of application layer companies. 'Cause, I mean, yeah, you see, like, the different, the labs are also now creating these, like, RL platforms, and you can soon, like, customize models with RL on your personal, like, on your, like, company's use cases. So a lot of what, like, application layer companies have to figure out is, like, how to encode their product and their, like, customer needs into RL grading and RL feedback. **Swyx** [31:40] Yeah. Awesome. Cool. Um, that's it for me. Thank you so much for your time. Thank you for DeepWiki. I think it's a huge service to the community. **Silas Alberti** [31:48] Yeah. Thank you for having me on. This was great, and, um, talk to you all soon. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.