OpenAI's Habitat storage platform: 70M req/s lessons and a 2-engineer Rust rewrite

Part one of OpenAI's engineering series on Habitat, the storage platform behind ChatGPT (1B+ weekly users, 500+ PB stored, 22M requests per second at the edge, 70M+ per second across all services, 100k+ requests per second during storage failure recovery). The writeup covers the migration from a shared Python library into a globally distributed service: asyncio tuning to bound scheduling delay under load; fixing an aiohttp LIFO-connection-pool metastable failure by moving to FIFO; a custom HTTP/2 fan-in front end on Envoy; an internal NoSQL layer with a TAO-style graph API; and Rockset-based CDC for online/offline migrations with a disaster-recovery escape hatch. The kicker: in Q2 2026, Habitat v3 — a complete rewrite in Rust — was built by 2 engineers in months using Codex and GPT-5.5, replacing a Python service second only to Kubernetes in core share, at ~6x CPU and ~15x memory efficiency, now serving ~95% of production traffic.

The architecture lessons (metastable failures in connection pools, asyncio tail latency, HTTP/2 fan-in economics) apply to any high-scale Python-to-Rust migration. But the headline data point is organizational: a 2-engineer, agent-assisted full rewrite of a top-2-by-cores OpenAI service at 6x/15x efficiency — currently the strongest public evidence for AI-assisted rewrites of business-critical infrastructure, and a benchmark for what 'AI-native engineering team' now means in practice.