SF Compute

Why AI Labs With Unlimited GPUs Still Fail — Anjney Midha, AMP

Jun 18, 2026 · 1:00:37

Anjney Midha, CEO of AMP, argues that AI labs with unlimited GPUs still fail due to misaligned culture and infrastructure waste, proposing a compute grid modeled on independent system operators to pool demand and supply. At Google, 95% node utilization was considered an outage, yet most clusters today don't reach that, with waste compounding at scale. AMP’s grid, starting at scheduling, aims to make FLOPs flow like megawatts, having secured 1.3 gigawatts of demand. Midha explains Anthropic cracked coding because 'luck favors the prepared mind'—their four years of paranoia and scarcity created a culture that OpenAI’s abundance couldn't replicate. He also shares a 14-year mission in end-of-life prediction, arguing AI can reduce the 30% of Medicare/Medicaid spend on end-of-life care. He warns that too much capital too early makes labs fragile because without hardship they fail to define their P0.

⚡️ Beyond Transformers with Power Retention

Sep 23, 2025 · 32:59

Jacob Buckman, CEO of Manifest AI, discusses their solution to AI's expensive computational bottleneck: the ever-growing KV cache for long context inference. Power Retention replaces attention in transformers with a fixed-size memory, achieving 10X training speedup at 64K tokens and 100X inference speedup over FlashAttention. Manifest releases Vidrial, a just-in-time CUDA kernel framework that finds optimal configurations, and Metamorphosis, a process to convert existing transformers like StarCoder-3B into Power Retention models in just two hours of mid-training. Buckman demonstrates PowerCoder, a 3B code model that matches baseline loss after 10K steps and handles contexts up to 32K and beyond. The open-source release includes kernels and tools, aiming to build community trust and encourage adoption by inference providers. He also highlights the need for genuinely long-context datasets beyond internet text, such as human trajectories.

SF Compute: Commoditizing Compute

Apr 11, 2025 · 1:12:02

Evan Conrad, co-founder of SF Compute, argues that GPUs behave like a real estate business, not a traditional cloud, because price-sensitive customers value every incremental GPU and will switch for a 10% margin. CoreWeave succeeded by selling locked-in long-term contracts to low-credit-risk customers like Microsoft and OpenAI, ignoring short-term demand. He predicts hyperscalers and providers like Together and DigitalOcean will lose money on GPU clusters because software margins cannot match the hardware costs. SF Compute started as an AI lab forced to sublease its cluster monthly to avoid bankruptcy, then evolved into a market where anyone can buy H100s by the hour via dynamic pricing—often below $1/hour for short bursts. Utilization stays near 100% as prices adjust. Future plans include cash-settled futures to reduce financial risk across the industry, while the brand deliberately stays anti-hype and calm.