By the last part, the grid could survive nodes dying. This part is about nodes lying, which is a much worse problem, because a dead node costs you throughput and a lying node costs you correctness.
Here is the situation honestly. A pi-storm node is a browser tab. I do not control the machine, the browser, or the person behind it. Anyone can open dev tools and rewrite the worker. Anyone can skip the browser entirely and speak the WebSocket protocol from a script, accept work units, compute nothing, and return whatever JSON they like. The coordinator receives { inside: 157061, total: 200000 } and has no idea whether a CPU produced it or a random number generator did.
So the design question for this phase was: how do you accept math from strangers without redoing all of it yourself?
Why the obvious defenses do not work
The first instinct is a plausibility check. For a Monte Carlo pi unit, inside / total should be near pi/4, so reject anything far off. That catches a lazy liar who returns inside: 0. It does nothing against a competent one who returns a value near pi/4 that no honest machine would ever produce. A statistical filter cannot tell a plausible lie from an honest result, because a plausible lie is designed to look like an honest result.
The second instinct is recomputation: the coordinator recomputes every unit and compares. That works, and it also deletes the entire point of the project, because now the trusted server is doing all the compute and the grid is decoration.
The way out was purchased two parts ago, in the kernel, and at the time it looked like an implementation nicety. The kernel uses a seeded PCG32 PRNG, no OS randomness anywhere in the compute path, and WASM's IEEE-754 float math is deterministic. So a work unit is a pure function: same (seed, count) in, byte-identical {inside, total} out, on any honest machine, any OS, any CPU.
That determinism converts verification from recomputation into comparison. I do not need a trusted party to check a result. I need a second stranger.
R=2: two strangers must agree to the byte
Every real unit is assigned to R=2 distinct nodes. The coordinator never verifies math. It encodes each submission canonically (for pi, inside then total as little-endian u32, eight bytes of hex) and compares the encodings:
// Group opinions by canonical byte encoding.
const groups = new Map<string, string[]>();
for (const [nodeId, output] of u.submissions) {
const key = this.workload.encode(output);
const g = groups.get(key);
if (g) g.push(nodeId);
else groups.set(key, [nodeId]);
}
if (groups.size === 1) {
// All R distinct nodes byte-agree -> verified.
const [winners] = groups.values();
this.verifyUnit(u, winners, nowMs);
return;
}
If both encodings match, the unit is verified and aggregated exactly once. For a liar to slip a wrong value into the aggregate, it would have to byte-match another node's wrong value, which means either collusion or guessing an exact 64-bit encoding.
Note what "distinct" is doing there. The coordinator tracks every node ever involved with a unit and never assigns it the same unit twice, so a node can never be its own second opinion.
If the two encodings differ, someone is wrong, and with two opinions you cannot tell who. The unit flips to mismatch and gets assigned to a third distinct node. Two of three byte-agree, the majority wins, the dissenter is penalized. There is one degenerate case: all three differ. That is impossible for honest nodes, so at least two parties lied, but indistinguishably. Nobody is blamed, every submission is discarded, and the unit requeues as a fresh R=2 attempt on other nodes.
Canaries: questions I already know the answer to
Byte-equality catches liars probabilistically, one collision at a time. Canaries catch them with proof.
Every Nth minted unit is a known-answer unit: the coordinator precomputes its true output with a trusted TypeScript reference kernel (golden-vector-pinned against the Rust/WASM one). The critical property is that a canary is indistinguishable on the wire. It is minted from the same monotonic seed sequence as real units, carries no marker, and verification metadata never leaves the coordinator. A node cannot tell whether it is answering a real unit, a canary, or a tiebreak, so it cannot behave honestly only when watched.
Canaries get redundancy 1, because the reference output is the second opinion. A pass earns trust, and doubles as a live WASM-versus-TS byte-equality determinism check on whatever CPU that node happens to have. A failure is not suspicion. It is proof of lying, and the node is ejected immediately.
Trust, quarantine, ejection
Each node carries a trust record, scored as (1 + good) / (1 + good + 4 * bad), where canary failures count four times over. The score scales the node's assignment pipeline, so a suspect gets less work before it gets none. Below 0.5 the node is quarantined: it receives only freshly minted canaries until it either fails one (ejected) or passes enough to recover. Real units are never risked on suspects, and an honest node that hit one fluke can climb back out.
Ejection purges the node's unverified submissions and reopens those slots. Its already-verified contributions stay, and that is safe by construction: each one was byte-equal with an independent node, so it stands regardless of what the ejected node did later.
One honest caveat: identity is per-connection. An ejected client can reconnect as a fresh node with reset trust. Raising that bar is future work, and I would rather say so than pretend the door is locked.
The test that mattered: a subtle liar, canaries off
The adversarial harness injects a wire-protocol liar into a grid of three honest headless browsers. The scenario I actually cared about disables canaries entirely and makes the liar competent:
// inside ~ count*pi/4, shifted by ~5.4 sigma plus seed jitter:
// passes any integrity guard or statistical sniff (pi ~ 3.16),
// can never accidentally byte-match an honest output.
function subtleLie(unit) {
const count = unit.params.count;
const jitter = Number(BigInt(unit.params.seed) % 37n);
const inside = Math.min(count, Math.floor((count * Math.PI) / 4) + 997 + jitter);
return { inside, total: count };
}
No guard rejects this. No plausibility filter flags it. The only detection path left is R=2 disagreement, escalation, and majority. And that path alone was enough: every lie disagreed with an honest partner, every tiebreak went 2 against 1, three tiebreak losses hit the bad-result threshold, and the node was ejected. The harness asserts the liar's tombstone shows zero accepted submissions, poisoned units were purged and re-verified, the no-corruption invariant held, and the global pi stayed within tolerance.
That run is why I trust the design. Canaries are the fast path, but the system does not depend on them.
The bill: 2.5x to 2.9x
None of this is free, and the coordinator measures exactly what it costs. The verification tax is compute spent across every recorded submission, winners, losers, canaries, and later-purged lies, divided by verified aggregated compute. R=2 puts a hard floor of 2.0 on it. In practice I measured 2.5x to 2.9x, the high end under active tiebreak churn from the liar.
So the honest summary is: to trust one unit of math from strangers, I pay for roughly two and a half to three. That number is not a footnote. It sits at the center of whether any of this makes economic sense, and it comes back with force in the final scorecard.
What I find satisfying is that the entire verification engine, byte grouping and tiebreaks and canaries and trust, never once looks inside an output. It compares opaque encodings. Which means it should work for any deterministic workload, not just pi. That is the next part.
Previous: Distributed and healing. Next: It was never about pi.