TTL, staleness, and cache invalidation for agent knowledge

There are only two hard things

Phil Karlton’s line — “there are only two hard things in computer science: cache invalidation and naming things” — is a joke that has aged into a design principle. A cache is a stored answer that was correct when it was computed. The hard part is never storing it; it is knowing when the stored answer stopped being the right one. Agent memory is exactly this problem at scale: a corpus of facts that were each true at the moment they were recorded, and a reader that must decide which ones are still safe to act on now.

The naïve model treats a fact as present or absent — if it’s in the store, use it. That model has no concept of when, so it serves a crashed service’s “is running” record with the same confidence as a recorded decision. The fix is to make freshness a first-class property of every fact, alongside its content and its source.

TTL and expiry

The simplest freshness primitive is a time-to-live (TTL): a duration attached to a fact that says “this may be relied on for at most this long.” When the TTL elapses, the fact expires — it is not deleted, but it is no longer authoritative. Expiry is a deadline, and its great virtue is that it requires no cooperation from anyone: even if the world changes silently and no update is ever filed, the clock still moves and the fact eventually becomes suspect on its own.

The cost of pure expiry is that it is blunt. A fact can expire while still perfectly true, and a fixed TTL is a guess about a fact’s volatility. That is why a single global TTL is the wrong design — see knowledge has a shelf life for why durable facts and volatile facts need completely different windows.

Push vs pull invalidation

There are two ways to keep a cached fact honest, and they trade reliability against latency.

• Push invalidation — the source notifies every replica the instant a fact changes. This gives the tightest freshness: a reader is told to drop or refresh the entry immediately. But it only works if the source knows who holds the fact, is willing to notify them, and no notification is ever lost. Miss one push and a replica silently believes a stale value forever.
• Pull revalidation — the reader checks freshness itself, on access. No cooperation from the source is required beyond answering “is this still current?” The cost is a round-trip at read time and a window of staleness between checks.

Agent knowledge is gathered from a world that rarely pushes: services go down, people change their minds, files change with no event emitted. So agent memory leans on pull, with push treated as an optimization layered on top where a source happens to be cooperative — never as a foundation you can assume.

Bounded staleness

Pull-based freshness needs a contract, or readers can’t reason about what they’re holding. That contract is bounded staleness: a guarantee that any fact you read is no older than some bound Δ relative to the truth — and crucially, that the system knows the bound and can tell you when a fact has exceeded it. This is a consistency model in its own right, weaker than strong consistency but far more honest than “whatever’s in the cache.” Instead of pretending a fact is current, the store says: this fact is at most Δ stale; past Δ, treat it as unverified. Pairing each fact with the logical time it was last confirmed makes the bound checkable — the companion deep-dive on causal consistency and vector clocks covers the clocks that make “how old, relative to what” answerable.

Soft-state and revalidate-on-read

A fact under a freshness bound is soft-state: it is true only as long as it keeps being refreshed, and it decays toward “doubtful” if it is not. Soft-state must be actively renewed; the absence of a renewal is itself information. The renewal mechanism is revalidation — and the key insight is that validation is cheaper than re-fetching. Revalidate-on-read means: when a reader touches a fact past its freshness window, the system doesn’t blindly re-fetch its content; it asks the source whether the fact is still current. If yes, it refreshes the fact’s timestamp — a confirmation that the value is still true as of now — without rewriting the value. That confirmation is real evidence, not redundant noise; it is what advances the fact’s currency.

How HiveMind applies it

HiveMind keeps an append-mostly record: facts are added, and forgetting is deliberate soft decay rather than silent deletion. That model fits freshness naturally — expiry never erases a fact, it marks the fact suspect and surfaces it for revalidation, and a re-confirmation appends a fresh observation rather than overwriting history. Because each machine holds a full local copy synced peer-to-peer, freshness is evaluated locally: a reader on any device checks a fact’s window against its own clock without a round-trip to a central server, and your data never leaves your devices to be revalidated. Expiry there is a flag, not a delete — closely related to how removal is modeled as another fact rather than destruction, covered in the tombstones and compaction deep-dive.

The result is a corpus where every fact carries not just what it claims but when it should be doubted. Present-or-absent is a one-bit model of knowledge; freshness is the dimension that keeps a shared memory from quietly serving the past as if it were the present.

Frequently asked

Related