Harbor

[State of AI Papers 2025] Fixing Research with Social Signals, OCR & Implementation — Team AlphaXiv

Dec 31, 2025 · 34:45

AlphaXiv founders Raj and Rayhahn describe building the go-to platform for navigating AI research by obsessing over UI and getting authors like the Laura team to comment directly on papers, then evolving from comments to feeds to benchmarks to AI assistants. They detail how they beat Hugging Face Papers by enabling direct paragraph-level commenting and early traction with high-signal authors, and how they use social signals (views, comments, Twitter) to filter the archive firehose. They discuss the broken academic review process—20% AI-generated reviews at ICLR, quality collapsing under exponential submission growth—and argue papers are becoming less important than implementations, Docker containers, and real-world usability. Their vision: making it trivial to spin up any trending paper's code directly in browser, focusing on the top 0.1% of implementable papers that applied researchers actually care about.

Terminal-Bench 2.0: the most impt coding agent benchmark of 2025 gets a v2! Launch + Q&A w/ founders

Nov 8, 2025 · 35:01

Terminal-Bench co-creators Mike Merrill, Alex Shaw, and collaborators Andy Konwinski, Ludwig Schmidt, and Aatash launch Terminal-Bench 2.0, an 89-task coding benchmark with rigorous human and LM-assisted verification to test agent performance on real-world tasks like designing DNA primers and installing Windows 3.11. They reflect on Terminal-Bench 1's problems—easy tasks and reproducibility—and detail spending 300 hours manually auditing each task. They introduce Harbor, an open-source package for running sandboxed agent rollouts, standardizing task format and integrating with agents like Codex and Claude Code. Aatash shows using Harbor to generate thousands of training traces for DataComp Agent, running 1,000 parallel sandboxes. Future plans include expanding to scientific computing.