Ramp

The Future of AI Infra: from Kubernetes to Agent Sandboxes — Akshat Bubna, Modal CTO

Jul 8, 2026 · 59:10

Modal CTO Akshat Bubna explains how the cloud platform evolved from a serverless runtime to an AI cloud built for elastic inference, agent sandboxes, and post-training workloads. He argues that Kubernetes was never designed for bursty GPU-heavy AI workloads, so Modal built a decorator-based infrastructure that co-locates compute requirements with code. The company added GPUs a year before ChatGPT and now powers inference for custom models at Suno, Runway, and robotics firms, using GPU snapshotting to slash cold starts. Modal's new Auto Endpoints incorporate open-source DeFlash speculative decoding for frontier-level performance. For reinforcement learning rollouts, Modal provides up to 100,000 sandboxes simultaneously. The platform spans 17 cloud providers, features private IPv6 networking via eBPF, and supports serverless multi-node training with RDMA. Bubna also reveals Modal's shift from developer experience to agent experience, noting that agents benefit from the same minimalist SDK and observability tools that humans use.

Devin’s 80% Moment: Background Agents, 7x PRs, & End of Hand-Held Coding — Walden Yan & Cole Murray

May 28, 2026 · 1:09:33

Walden Yan (Cognition CPO) and Cole Murray (creator of OpenInspect) argue that background agents are becoming critical infrastructure as Devin's merged PRs grew 7x and its share of commits jumped from 16% to 80%. They explain why architecture matters, particularly the decision to separate the agent's brain from the machine (harness out of the box) for security and permissions, and why full VMs beat Docker for running real applications. Testing, they stress, is a harder problem than computer use—requiring orchestration of services, feature flags, and multi-model coordination. Memory remains unsolved, with Devin using auto-generated 'Knowledge' and exploring file-system-like approaches. Both caution against uncontrolled vibe coding, which regresses codebases to the worst engineer's style, and advocate for hybrid frontier/sub-frontier systems to balance cost and capability.