On 2026-11-05 GitHub published a deep dive into a new storage backend it calls Project Rubicon, and the framing is the thing to notice first. This is not a performance tweak sold to humans typing git push. It is a rebuild justified almost entirely by machines: GitHub says it designed the system so that many clients — coding agents above all — can write to a single repository at the same time. If you build web apps with AI and have started pointing agents at your repos, this is the server-side half of the story you have been living on the client side.
What actually happened
GitHub's busiest repositories used to see, in its own words, "at most dozens of writes in a single second," and that was treated as a bursty peak. Agents change the shape of that load from a spike to a floor. Every push an agent makes fans out: a continuous integration run starts, a deployment may fire, and other agents fetch the new state. So reads scale with writes, and both climb together. To put a number on the trend, GitHub reports that one busy repository averaged 44M requests per day in March 2026 and 181M requests per day by May 2026 — roughly 4x in two months.
The old architecture, called Spokes, stored full copies of a repository across several machines and needed a quorum of those replicas to acknowledge every write. That is a sensible design when writes are rare and precious. It falls apart when writes become constant and concurrent, because every writer is forced to coordinate with every replica. GitHub set itself a blunt goal for the replacement: "at least two orders of magnitude more scale."
How Rubicon works
The design rests on two principles GitHub states plainly: "minimize coordination, and decouple storage from compute." Both follow from one observation about Git — most of what a repository holds never changes.
Git objects — the commits, trees, and blobs that make up your history — are immutable and content-addressed. Once written, a given object is the same forever. Rubicon puts all of that immutable data into Azure Blob Storage: the packs, their indexes, read accelerators, and even cached fetch responses. Blob storage is cheap, durable, and effectively unlimited, and because objects never change, many readers and writers can touch them without ever stepping on each other. No coordination is needed for the bulk of the data.
What does change is tiny: the references. A branch or a tag is just a pointer that says "main is now this commit." That pointer is the only part two writers can genuinely conflict over, so it is the only part that needs agreement. Rubicon isolates it in a dedicated, replicated log called Ref-CAS, which holds the references and a manifest of packs and stores only pointers — never object data. Each commit gets a sequence number, and the repository's version is simply the sequence number of its latest commit. A reader that already holds the current version is told nothing changed after a single comparison, so an unchanged repository is almost free to serve.
With that split, compute becomes disposable. Rubicon's serving nodes are "virtual replicas" whose local disks are nothing more than rebuildable caches. A write uploads its pack to blob storage, then commits one Ref-CAS transaction. A read checks the version, pulls whatever packs it is missing into its cache, and serves from there. Maintenance moves out of the way too: background compaction nodes lease shards of a repository's packs and merge them into larger packs, so the housekeeping that used to run on the machines answering your requests now runs somewhere else entirely.
The payoff GitHub reports from internal testing: 7,200 pushes per second to a single repository, and 5,000 concurrent agents writing to one repo, with latency staying flat. For contrast, the old approach bottlenecked references at a single blob capped in the mid-hundreds per second.
Why this matters if you build with AI
The practical lesson is that the wall you hit running parallel agents was probably never your laptop. Worktrees isolate files, and you can spin up as many agent processes as your machine tolerates, but they all push to the same remote, and that remote's single-writer coordination was the real ceiling. Rubicon is GitHub lifting that ceiling, and it does it without asking anything of you: git push, git fetch, and git branch behave exactly as before, so no client changes are required.
There is also a reusable pattern here worth stealing even if you never run an agent fleet. Separate the large, immutable, content-addressed data from the small mutable pointer that actually needs consensus. Push the first into cheap object storage where concurrency is free, and pay the cost of coordination only on the second. That shape — objects in a blob store, a thin replicated log for the pointer — shows up any time you need many writers and a single source of truth, and GitHub just published a production-scale worked example of it.
What is not known yet
The honest caveats are large. The 7,200-pushes and 5,000-agent figures are GitHub's own internal tests, not independent benchmarks, and the post gives no exact latency numbers — only a "flat latency curve." Rubicon is in limited private preview, with a broader rollout promised vaguely as "early next year," and it is a GitHub.com backend: there is no self-hosted or Enterprise Server package announced, so a team on its own infrastructure gets none of this yet. GitHub also lists a long tail of future work — fetching single objects instead of whole packs, an atomic multi-file commit API aimed squarely at agents, partial and global replicas — which is another way of saying the agent-native parts are not finished. And the whole justification rests on a forecast that agent load will keep climbing the way the March-to-May curve did.
My take
This is Amar, writing in the first person. What I like about Rubicon is that it names the real constraint instead of hiding it behind a bigger box. For months the advice for running parallel agents has been about the client — git worktrees, separate ports, a second database — and all of that is true, but it dodges the fact that two agents still fight over one branch pointer on the server. GitHub's answer is the correct decomposition: the pointer is the only thing that needs consensus, so make everything else refuse to coordinate.
I am also a little wary. "Built for 5,000 agents" is a benchmark, and a repository with 5,000 agents writing to it is, to me, usually a smell — a sign the work was never cut into pieces that could live in separate repos or separate branches with human-sized review. Rubicon removes a ceiling I would rather most teams never touch. The borrowable idea is the better prize: content-addressed data in a blob store, a thin log for the one mutable pointer. That I will reach for in my own systems long before I ever push an agent fleet hard enough to feel the old limit.
