← back to the garden

pi-storm · part 04 of 09

Where Does a WebSocket Coordinator Live?

planted · June 18, 2026
#pistorm#websockets#cloud-run#serverless#gcp#architecture

Three parts in, pi-storm has not needed a server. The Rust kernel runs in the browser. The worker pool that pushed it to 10.7 billion darts per second runs in the browser. The whole thing deploys as static files. That was deliberate. My default for early builds is serverless-first, and a static page is the most serverless thing there is: no instances, no patching, no bill for idle time.

Then the project hit the sentence that ends that streak: two browsers, on two different machines, need to agree on one number.

Somebody has to hand out work units, collect results, and keep the running total. That somebody cannot be a browser tab, because tabs close. It has to be a coordinator. And a coordinator is a server, which means the serverless-first thesis is about to meet its first real test.

What the coordinator actually is

Before deciding where it lives, it is worth being honest about how small it is.

The coordinator hands out work units shaped like {jobId, unitId, seed, count}, receives results shaped like {inside, total}, and keeps the tally. Its entire state is a few in-memory maps: connected nodes, outstanding units, aggregated totals. Mine is Node with Fastify and a WebSocket server, and the core logic is a few hundred lines.

WebSocket is not decoration here. The nodes are browsers, so the coordinator has to push work to them rather than wait to be polled, and it needs to know the instant a node disconnects so it can reclaim that node's units. A long-lived, bidirectional connection is the natural shape. That one requirement, long-lived, drives everything that follows.

The obvious serverless answer, and where it goes wrong

Serverless-first says: reach for the thing that scales to zero. So the first sketch was the textbook one, a WebSocket API in front of functions.

This exists and works, but look at what it does to the design. Functions are short-lived by definition, so the platform holds the socket for you and invokes a function per message. Your connection state can no longer live in memory, because there is no durable memory. Every map I listed above moves into an external store. Assigning a unit becomes a read-modify-write against a database. A node disconnecting becomes an event you hope arrives. Cold starts sit between a browser saying "done" and the tally updating.

I would be taking a 300-line in-memory object and smearing it across a database, a queue of function invocations, and a pile of retry semantics. That is a distributed system, built to serve a project that does not yet have ten users. Serverless-first is supposed to reduce the amount of system I operate, and here it was increasing it.

The second sketch was Cloud Run, which genuinely supports long-lived WebSocket connections. Deploy the container, done. Except Cloud Run's whole value is autoscaling, and autoscaling is exactly the failure mode.

Split brain, the disqualifier

Picture the default configuration under load: Cloud Run adds a second instance. The load balancer spreads incoming WebSocket connections across both. Now eight browsers are connected to instance A and eight to instance B.

Each instance has its own in-memory maps. Each one believes it owns the job. Each one hands out units from its own view of the queue and keeps its own tally. There is no shared truth, so there are two totals, both wrong. Nodes on A can never verify results from nodes on B, which quietly destroys the redundancy scheme that later parts depend on. That is split brain, and with a stateful in-memory coordinator it is not a tail risk, it is the guaranteed outcome of the second instance starting.

The standard fix is to externalize the state: put the maps in Redis, make every instance stateless, let the platform scale freely. That is the correct answer for a system that needs to scale. So I did the honest sizing exercise instead of the reflexive one.

The coordinator does no compute. The browsers do the compute. The coordinator's job is bookkeeping for, at most, dozens of connected nodes, and a single Node process does that without noticing. Scaling out solves a load problem I do not have, and creates a consistency problem I would then need Redis and careful locking to solve. I would be paying complexity to enable a capability I cannot use.

(For completeness: Cloudflare Durable Objects are the platonic answer to this exact shape, a single-threaded stateful object with a stable address. If my infrastructure lived on Cloudflare I would have reached for one. It lives on GCP, and adding a second cloud to avoid one small server is its own kind of overengineering.)

The decision: one instance, on purpose

So the coordinator runs on Cloud Run with autoscaling turned into a constant:

"autoscaling.knative.dev/minScale" = "1"
"autoscaling.knative.dev/maxScale" = "1"

min = max = 1. One instance, always on, state in memory, exactly as the code was written.

Both halves of that matter. max = 1 is the split-brain guard: the platform is not allowed to create a second source of truth. min = 1 admits that a WebSocket coordinator cannot scale to zero, because scaling to zero drops every connection and evaporates the state. That second half is a real dent in the serverless-first thesis. I am paying for an idle instance overnight, which is precisely the thing the thesis exists to avoid.

The remaining objection is the single point of failure, and here the answer comes from the grid's own design rather than from infrastructure. The protocol is built so that losing a participant is routine: a node that disappears mid-work just gets its units reassigned. A coordinator restart is the same event wearing a different hat. Nodes reconnect, unfinished units are reissued, the computation continues. The next part is entirely about watching that healing happen, so I will not spoil the demo, but it is why "the one instance might restart" is an acceptable cost instead of a disqualifier.

The deploy fought back anyway

For the record, shipping this to pi.inoltro.tech involved almost no WebSocket problems and a full week of GCP ones. The coordinator sits behind an external HTTPS load balancer with Cloud Armor and a Google-managed wildcard cert, DNS on Cloudflare in DNS-only mode. Along the way: Terraform wants Application Default Credentials, which gcloud auth login does not give you. Five credentialed gcloud accounts on my machine meant every command needed an explicit --account and --project, and docker push silently uses whichever account the credential helper thinks is active. A Domain Restricted Sharing org policy blocked the allUsers invoker binding that a public Cloud Run service needs, and since the external load balancer does not authenticate to Cloud Run, you cannot avoid allUsers; it took a project-level policy override, with the load balancer and Cloud Armor as the actual gate. None of this is a tutorial, it is just the honest texture of "one tiny server" on a real org's cloud.

Where the thesis lands

Serverless-first survives part four, but it comes out amended. The default is still: reach for the thing that scales to zero, and let the browser carry everything it can. The amendment is: when a component is genuinely stateful, small, and singular, one deliberate server beats a distributed workaround. The failure mode is not choosing a server. The failure mode is refusing to, and building a database-backed imitation of an in-memory map to preserve an aesthetic.

min = max = 1 is that judgment written into config, where the next person can read it.

Update (July 2026): this Cloud Run instance later wedged in production under real WebSocket load, and the coordinator moved to a plain VM. The single-instance judgment held; the serverless platform under it turned out to be the wrong fit for long-lived sockets at volume. That story, and what it does to the thesis above, is When the pinned instance wedged.

Previously: one browser, every core. Next: two machines, one pi, and pulling the plug mid-computation.