RAG is a hack - with Jerry Liu of LlamaIndex
Oct 12, 2023 · 1:13:25
In this episode of Latent Space, Jerry Liu of LlamaIndex argues that while RAG is fundamentally a hack, it remains the most practical approach for grounding LLMs with external data, and his open-source framework now serves 600,000 monthly downloads. Liu recounts how LlamaIndex originated from a hackathon at Robust Intelligence, evolving from an experimental tree-index to a modular toolkit spanning data loading, retrieval, synthesis, and agent loops. He explains the trade-offs between RAG and fine-tuning, noting that context window limits force algorithmic rather than learned optimizations, but RAG offers transparency and access control that fine-tuning cannot. Liu details the company’s $8.5M Greylock raise, the launch of LlamaHub for community-contributed data loaders, and the open-source SEC Insights app as a production-grade template. Looking ahead, he emphasizes the need for better retrieval benchmarks and predicts that future personalization will move beyond vector stores into model-internal memory architectures.
