Data poisoning and blast radius: how one bad write propagates through shared memory
Data poisoning is a single wrong or malicious write that downstream agents treat as ground truth. Because shared memory exists to be read and built on, that one fact fans out: every agent that reads it derives new conclusions from it, those conclusions get written and read in turn, and the corruption amplifies across derived state. The blast radius is everything that transitively depended on the poisoned fact. You contain it not by hoping writes are clean but by structural controls — provenance on every fact, confidence earned through corroboration before a claim is promoted, and corrections appended rather than silently overwritten, so the bad fact and everything it touched stay traceable and reversible.
Poisoning: a bad write the system treats as truth
Data poisoning is the moment a wrong value enters shared memory and is read by something else as ground truth. The write can be accidental — a misparsed date, a scrape of a stale page, a confident hallucination — or it can be adversarial, a claim planted specifically so other agents will act on it. The mechanism that does the damage is the same either way, and it is the mechanism that makes shared memory useful in the first place: agents read what is already written so they don’t have to re-derive it. A memory worth reading is, by the same property, a memory where a poisoned fact spreads at the speed of trust.
This is the garbage-in, garbage-out law with a multiplier. In a single computation, garbage-in produces one garbage-out and stops. In shared memory the garbage-out is persisted, becomes the garbage-in of the next agent, and the chain continues. The bad value gets laundered: by the third hop nobody is reading the original suspect claim, they’re reading well-formed conclusions that happen to rest on it.
Blast radius and fan-out
The blast radius of a poisoned fact is everything that transitively depended on it. It is set by fan-out: how many readers consume the fact, and how many writers each of those readers becomes. One fact read by five agents, each producing three derived facts, has already touched fifteen downstream claims after a single hop — and those fifteen have their own readers.
poisoned fact
├─ agent A reads → writes A1, A2 (depends on poison)
│ └─ agent C reads A1 → writes C1 (depends transitively)
└─ agent B reads → writes B1 (depends on poison)
This is write amplification in the institutional-memory sense: a single bad input causes many bad writes, because derived state copies the error forward rather than re-validating it. The corruption is cascading — poisoned input yields poisoned conclusions, which are themselves inputs to the next layer. Each layer makes the error cheaper to believe and more expensive to remove, because removing it means re-examining everyone who leaned on it. A claim that was trivial to fix on day one is load-bearing by day three. (The ripple effect of one bad fact is the plain-language version of this same cascade.)
Containment is structural, not vigilant
You cannot contain poisoning by asking every agent to be careful, because the failure happens through careful behavior — each downstream agent is correctly trusting what is written. Containment has to be a property of how writes are recorded and promoted.
Provenance on every fact. Each fact carries where it came from and, where relevant, what it was derived from. That turns “what is the blast radius?” from an unanswerable question into a graph walk: start at the poisoned fact and follow the edges forward to every conclusion that read it. Without provenance the radius is effectively unbounded — you can’t distinguish a claim that leaned on the bad input from one that didn’t. This is why HiveMind keeps facts attributed and traceable rather than anonymous and absolute.
Corroboration before promotion. A lone claim should not get the standing of a settled one. In HiveMind, trust is layered: confidence is earned from distinct, independent agents agreeing, and no agent certifies its own truth. A poisoned fact that only one source asserts stays low-confidence, so the cascade stalls before it becomes the ground everyone stands on. Requiring independent agreement before a claim is treated as fact is the quorum idea — see the quorum, Byzantine faults, and confidence deep-dive for why self-certification and single-source emphasis don’t count.
Confidence weighting at read time. Readers don’t have to treat every stored value identically. A claim’s confidence — how much corroboration it has earned — travels with it, so an agent can weight a thinly-supported fact accordingly instead of building load-bearing decisions on it.
Correct by appending, not overwriting
The instinct on finding a poisoned fact is to delete it. That is the worst option, because a silent overwrite destroys exactly the evidence you need: the fact that a poisoning occurred, when, and what was built on it before the fix. HiveMind is append-mostly — corrections are new facts, not destructive edits — so the bad fact and its retraction both stay in the record. The blast radius stays computable after the correction, and the change is reversible if the correction itself turns out wrong. (See the append-only log deep-dive for the underlying mechanics.)
Because every machine holds a full local copy synced peer-to-peer, this record lives on your own devices — there is no central store to quietly rewrite history in. The goal is not a memory that never ingests a bad fact. It is a memory where, when one slips in, you can see how far it reached and pull it back out without unwinding the world.
Frequently asked
What's the difference between data poisoning and an ordinary bug?
A bug produces a wrong result inside one computation; when the process ends, the wrong result usually ends with it. Data poisoning persists the wrong value into shared state where other agents read it as ground truth and build on it. The harm is not the single bad value — it's that the value becomes an input to other writes, so the error is laundered into apparent fact and amplifies. Poisoning can be accidental (a misparse, a stale scrape) or adversarial (a deliberately planted claim), but the propagation mechanics are identical.
How do you measure or bound the blast radius after a bad write is found?
You need provenance: each fact records where it came from and what it was derived from. With that dependency graph you can walk forward from the poisoned fact to every conclusion that read it, directly or transitively — that set is the blast radius. Without provenance the radius is unbounded in practice, because you cannot tell which downstream claims leaned on the bad input and which were independent.
Why correct by appending instead of overwriting the bad fact?
A silent overwrite erases the evidence. You lose the fact that a poisoning happened, you lose the timeline, and you lose the ability to find what was built on the wrong value before the fix. Appending a correction keeps both the bad fact and its retraction in the record, so the blast radius stays computable and the change is reversible if the correction itself turns out to be wrong.
Related
Take yourself out of the loop.
Let your agents do the lifting while you keep the judgment.
Get the Playbook