# Designing a URL shortener for its worst day > Resilience is a design input, not a deep dive at the end. The first in a series: the smallest system everyone knows, designed around the question of what is allowed to die. By Matthew D. Webb · 2025-12-29 · 16 min read Canonical: https://mdwebb.io/articles/system-design/designing-a-url-shortener-for-its-worst-day/ AI involvement: AI did the heavy drafting and wrote the diagram engine that animates the figures; I supplied the brief, the argument, and the veto. --- Most system design write-ups are written for interviews, and it shows. The shape is always the same: functional requirements, an API, a row of boxes, a cache in front of a database, and somewhere near the end a short, polite section about what might go wrong, positioned like the fire exits on an aircraft safety card. Nobody involved believes the aircraft will catch fire. The section exists to be gestured at. I want to argue for the opposite order. Failure is not a chapter you append once the design works; it is the material you design with. Every box you draw and every arrow between two boxes is also, whether you notice or not, a decision about blast radius: what breaks together, what degrades instead of dying, which failures a user can see and which stay a private matter between you and the pager. I have written before about why [resilience is mostly organisational](/articles/the-anatomy-of-a-resilient-system/); this series is about the part that is genuinely architectural. The whiteboard is where resilience gets decided. The incident channel is just where you find out what you decided. So the format for these pieces is the one I actually use. Dispatch the parts everyone agrees on quickly, because consensus is not where design lives. Then do the real work: which single request must survive, which components are allowed to die, in what order, and what each of those choices costs. And because it is nearly 2026, one more pass the older write-ups never needed: what changes when a growing share of the traffic, and of the operations around the system, has a model in the loop. First specimen: the URL shortener. Chosen precisely because it is boring. The baseline holds no surprises for anyone, which makes it the cleanest possible place to show the method before the systems get harder. One more thing before we start: this series shows its working. The diagrams are live drawings, not exhibits: drag the boxes about if you disagree with the layout, and hit tidy when you regret it. The napkins are pinned up where the arithmetic actually happened, and the implementation notes fold out wherever the code earns its place on the sheet. A design you can only look at is a poster. ## The parts everyone agrees on Take a long URL, hand back a short code. Follow the short code, get redirected. Custom aliases if the name is free, optional expiry, and an expired link should answer 410 rather than 404, because "this existed and was withdrawn" is a different fact from "this never existed". The numbers matter more than the features. Call it a billion stored links and a hundred million daily users, with redirects outnumbering creates by something like a thousand to one. A billion rows at a few hundred bytes each is half a terabyte, an amount of data that fits on the kind of SSD you can buy in an airport. There is no big data problem here. There is a read latency problem, and a small database standing behind it. The API is two verbs: a POST that creates a mapping, a GET on the code that answers with a 302 and the long URL in the Location header. The 302 deserves a sentence, because it is the first genuinely load-bearing choice. A 301 invites every browser and intermediary to cache the redirect permanently, which is faster, and also surrenders the hop through your server. Keep the hop. It is your control point: the place you can expire a link, retarget it, count it, or kill it when it turns out to be hostile. Giving that up to save one round trip is the kind of optimisation you regret at the worst possible moment.
Implementation notes · the read path, end to end ```sql -- One table carries the whole product. The primary key IS the index that -- serves every redirect; nothing else here is load-bearing. create table links ( code text primary key, -- base62, 7 chars long_url text not null, created_at timestamptz not null default now(), expires_at timestamptz, -- null = immortal disabled boolean not null default false -- the kill switch the 302 buys ); ``` ```ts // The handler the whole system exists for. Cache first, store only on a // miss (through the coalescer; see the cache funeral), and the 302 keeps // the hop, and with it the kill switch, ours. export async function redirect(code: string): Promise { const cached = await cache.get(code); const link = cached ?? (await coalesced(code, () => db.links.find(code))); if (!link || link.disabled) return new Response(null, { status: 404 }); if (link.expiresAt && link.expiresAt < now()) { return new Response(null, { status: 410 }); // withdrawn, not missing } if (!cached && link) await cache.set(code, link, ttlWithJitter()); return Response.redirect(link.longUrl, 302); } ```
Code generation is the one genuinely contested choice. Hashing the URL invites collisions you must then detect and retry around; random codes place the same bet with a different generator. A global counter rendered in base62 has no collisions at all: seven characters gives three and a half trillion codes, and the only awkwardness is that a counter is a point of coordination. So blunt the coordination: each write instance leases a block of a thousand values from a Redis counter with one atomic increment and mints locally until the block runs dry. Lose an instance, lose its block, and nobody cares. The requirement was uniqueness, not continuity, and a unique constraint in the database stands behind even that.
Implementation notes · minting codes without a meeting ```ts // The counter, blunted. One INCRBY buys a thousand local mints; uniqueness // survives a crash because a lost range is simply never handed out again. const BLOCK = 1000; let next = 0; let ceiling = 0; async function nextCode(): Promise { if (next === ceiling) { ceiling = await redis.incrby("counter", BLOCK); // atomic, once per block next = ceiling - BLOCK; } return base62(next++); } const ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"; function base62(n: number): string { let out = ""; do { out = ALPHABET[n % 62] + out; n = Math.floor(n / 62); } while (n > 0); return out.padStart(7, "0"); } ```
That is the consensus design, and I have deliberately not lingered. The interesting thing in the diagram is not any box; it is that there are two lanes behind the gateway, because that split is where the actual designing starts. ## Find the request that must survive The first question I ask of any system, before storage engines and before API shapes, is: which single request is this system for? Not which requests it serves. Which one it exists to serve, the one that must keep working when everything else is negotiable. Here the answer is brutally clear. The redirect is the product. A create that fails is a mild annoyance to one person holding a long URL; a redirect that fails is a broken link in front of an audience, multiplied by every place that link was ever pasted. The read lane is the product. Everything else is staff. Once you have that answer, availability stops being a single virtuous number and becomes a budget you allocate deliberately. "Four nines" sounds like a value statement; as arithmetic it is fifty-two minutes of failure a year, and that budget is spent almost entirely by the path users can see. So I refuse to give both lanes the same contract. The redirect path gets the four nines and the engineering spend that implies. The write path gets three nines and an honest error message, because nobody has ever abandoned a URL shortener over a create that failed for ninety seconds at 3am. Pretending the two halves deserve equal treatment does not make the system fairer; it makes both halves more expensive and neither more reliable. This is also where the bulkheads get placed, and the placement only works if it happens now. The read lane must not know the counter exists. The write lane must not be able to poison the cache. Isolation drawn on the whiteboard costs a line; isolation retrofitted during an incident costs a quarter. ## Design the funerals before the features Here is the same system again, worth keeping in view for this part. I think about this as rehearsing the funerals: every component gets one, on paper, before it gets to production.
**The cache dies first**, because it always does, and because it is the failure people design for least despite depending on it most. On a system this read-heavy the cache is not an optimisation; it is the thing actually serving your traffic, with the database as its understudy. Kill it and a thousand reads per write become a thousand reads per write aimed at Postgres, most of them for the same few hot keys. The instinct is to call this a capacity problem and buy a bigger database. It is a coordination problem: ten thousand in-flight requests all asking the same question. Request coalescing, one lookup in flight per key with everyone else waiting on that answer, turns the herd into a queue. Serving stale entries while the refill happens is unusually guilt-free here, since a mapping essentially never changes after it is minted. I have written about [why caching goes wrong](/articles/caching-is-easy-until-it-isnt/) in general; this system is the rare case where the cache's dishonesty is nearly harmless and its absence is fatal. The trap is recovery: a cold cache pointed at full traffic is the herd again with better branding. Warm it gradually, stagger the TTLs so a million entries stop expiring in the same second, and only then trust it with the crowd.
Implementation notes · one flight per key ```ts // The whole defence is a Map. One lookup in flight per key; everyone who // arrives while it is airborne shares the same promise. Losing the map on // restart costs one brief stampede, which is the cold-start story anyway. const inFlight = new Map>(); export function coalesced( code: string, load: () => Promise, ): Promise { const existing = inFlight.get(code); if (existing) return existing; const flight = load().finally(() => inFlight.delete(code)); inFlight.set(code, flight); return flight; } ```
**The counter dies second**, and the correct amount of drama is none. This is the bulkhead paying out: the read lane cannot be touched by a failure it has no path to. Write instances holding leased blocks keep minting until the blocks run out; the unlucky remainder fail fast with a clean error rather than hanging on a dead connection, because a fast honest failure is recoverable and a slow ambiguous one poisons everything upstream of it. When the counter returns it resumes from the last replicated value, some blocks are gone forever, and the sequence has holes. It was never a promise of continuity. If predictable codes bother the security review, encrypt the counter with a small block cipher before encoding and the sequence stops being legible; the resilience story does not change. **The database dies last**, the funeral everyone rehearses for and the one this design attends most calmly. Writes are straightforwardly down, or degraded to a queue if you can stomach the hard question a queue implies: do you hand back a short code before its row exists? That is a promise to serve a redirect you cannot yet serve, and I would rather fail the create than lie about it. The read path is the half worth watching: with a warm cache and stale-while-down semantics, redirects for the hot working set keep flowing while a replica is promoted. The system's most-used feature can outlive its own database for a while. Not forever, and the misses get 503s, but "degraded" and "down" are different words in the incident channel for a reason. The failover itself should be boring, automated and rehearsed. A restore you have never practised is not a recovery plan, it is a rumour. Notice what made all three funerals quiet: nothing heroic, no clever middleware, just decisions about isolation and degradation order that were available for free at design time. The system fails in layers because it was built in layers. That is the whole thesis, and it cost three sentences on a whiteboard. ## Share nothing you would have to reconcile Multi-region is where designs like this usually grow their first committee: if codes must be globally unique and two regions both mint them, who arbitrates? My rule is that the best coordination protocol is the one you delete. Give each region a disjoint slice of the counter space and global uniqueness becomes a property of arithmetic rather than of agreement. No consensus round, no fencing tokens, no split-brain arbitration. There is nothing to fight over because nothing is shared.
Reads are served wherever the user is; mappings replicate asynchronously between regions. Asynchronous means lag, and lag means a code minted in Leeds might 404 in Ohio for a heartbeat. I will take that trade every time on this workload, because a fresh link takes longer to paste into a group chat than it takes to replicate. When a region dies, geo DNS drains its traffic to the survivor: latency rises, availability does not, and the returning region catches up from the replication stream with nothing to reconcile. Failover becomes an exercise in routing rather than negotiation, which is exactly the kind of exercise you want at 3am. The multi-region design that survives is the one that arranged, in advance, to have nothing to argue about. ## Same discipline, newer weather None of the physics above changes because a model showed up. Redirects are still reads, counters still lease blocks, and the read lane still deserves its unequal contract. What changes is the temperament of the traffic and the toolkit of the operators, and I find both changes reward exactly the decisions already made. Start with the clients. A growing share of creates now arrives from agents: pipelines shortening links in bulk, assistants minting URLs mid-conversation, tools wired in over MCP-shaped integrations. Agent traffic is polite in aggregate and pathological in the particular: it arrives in bursts, retries with terrible enthusiasm, and will happily replay a create it already succeeded at because a timeout ate the response. So the write API grows idempotency keys, the discipline payments systems learned the hard way: same key, same code, no duplicate rows. Rate limits move from IP address to credential, and the 429 carries a Retry-After header that well-built agents actually honour, which turns backpressure into a conversation instead of a fight. This is not exotic. It is the write lane's three-nines contract being taken at its word by clients that read contracts literally.
Implementation notes · creates an agent can safely retry ```ts // Same key, same answer. A replayed timeout gets the code it already // earned rather than a duplicate row, which makes retrying safe, which is // the only kind of create an agent should be offered. export async function createLink(req: CreateRequest): Promise { const key = req.idempotencyKey; if (key) { const seen = await db.idempotency.find(key); if (seen) return seen.link; // a replay, not a new wish } const link = await mint(req); // lease-backed nextCode() + one insert if (key) await db.idempotency.put(key, link.code, DAY_TTL); return link; } ```
Abuse is the sharper edge. A shortener is redirection as a service, and redirection as a service becomes phishing infrastructure the moment you stop paying attention. The modern answer involves a classifier, often a large model, scoring destinations for malice. My rule here is absolute: the model stays off the synchronous path. A model endpoint is the least available dependency you will ever own, and welding your create flow to it trades your uptime for its; putting it in the redirect flow would be unforgivable. Score asynchronously, after the create, and let the 302 you refused to give up earlier do its real job: the hop through your server means a link found guilty can be flipped to a warning page or a 410 seconds after the verdict, for every copy of that link everywhere. And then the operators. This is where I think the AI story is genuinely good, provided one boundary holds. Models are excellent at the reading half of incident response: spotting the anomaly no static threshold catches, correlating a p99 wobble with the deploy that caused it, drafting the timeline while the humans debug, keeping the runbook honest against what the last incident actually required. The funerals rehearsed above are precisely the well-instrumented territory where that help compresses minutes into seconds. What the model does not get is the pager. An agent that can restart your cache is an agent that can restart your cache in a loop, and anything with write access to production inherits the blast radius conversation from the rest of this article. The judgement stays with a person for the same reason the counter got a bulkhead: capability was never the constraint. Containment is. ## What travels to the next system Strip away the shortener and what remains is the method, and it is portable. Name the request that must survive, and let it be unpopular that the others matter less. Price availability as a budget, not a virtue. Put bulkheads where the whiteboard makes them free. Rehearse every component's funeral before production schedules one for you. Share nothing across regions you would ever have to reconcile. And keep models, however capable, off the path your users are standing on. The consensus design is opening theory: worth knowing cold, and not where the game is decided. The game is decided in the positions after the book runs out, and production always plays you out of book. The next entries in this series take the same method into systems where the answers stop being comfortable: where the data actually is big, where writes are the product, and where serving stale is not a mercy but a lawsuit. The shortener was the method on training wheels. It will not stay that easy. ## Further reading - [Hello Interview: Design Bit.ly](https://www.hellointerview.com/learn/system-design/problem-breakdowns/bitly): the interview-facing treatment of this system, done properly. I used it to sanity-check the baseline arithmetic, then spent this article on everything it files under the closing deep dives. - [Timeouts, retries, and backoff with jitter (Amazon Builders' Library)](https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/): everything your retrying clients, human and agent alike, should have read before they met your API. - [Sharding & IDs at Instagram](https://instagram-engineering.com/sharding-ids-at-instagram-1cf5a71e5a5c): unique IDs at scale without a coordination service, from people who clearly also disliked meetings. - [How Complex Systems Fail (Richard Cook)](https://how.complexsystems.fail/): eighteen reasons the tidy single-component funerals above will, in real life, arrive three at a time.