Table of Contents

Catalog #

The capstone of 24 canonical blocks. This file is not an index bolted on — it is the OWNER of the meta-layer: the four patterns that recur across the corpus and were, until now, defined nowhere. It is also the design procedure: the topological order in which the blocks’ questions become meaningful.

The corpus in one sentence:

guarantees are manufactured at the root (quorum) and weaken,
deliberately and with named prices, as they flow up through
leases, logs, caches, and projections toward the user.

Part I — The Meta-Patterns (owned here) #

1. The Three Enemies #

Every one of the 23 technical bottlenecks (*) is a local projection of three primitives. The enemies are the real design space; the stars are their per-block costumes.

ENEMY 1 — STALENESS: the gap between a recorded claim and the moving world.

commit point*            queue        claim of effect vs the effect
view-vs-reality*         scheduler    claim of capacity vs the fleet
modeling ignorance*      state_machine claim of outcome vs the unknowable
revocation freshness*    policy       claim of authority vs the grant's death
binding coordinate*      checkpoint   claim of position vs captured state
invalidation delivery*   cache        claim of freshness vs every copy
paused holder*           lease_fence  claim of ownership vs one's own expiry (reflexive)
uncommitted suffix*      log          claim of permanence vs the provisional tail (premature)
promotion decision*      replication  claim of durability vs the async window
proving death*           GC           claim of unreachability vs unseen readers
                                      (+ IRREVERSIBILITY amplifier: the only
                                      unretryable mistake in the catalog)
herding on stale signal* routing      claim of load vs correlated deciders
completeness bet*        materialized claim of "seen everything" vs late events
recycled name*           identity     claim of reference vs a reassigned name
hot invariant*           transaction  (partial: serialization is the bill of
                                      one shared claim)
semantic gap* (~^o)      versioning   claim of shape vs drifted meaning —
                                      near-^o: meaning is not machine-checkable

ENEMY 2 — COLLAPSE: the system eating itself under load.

goodput*                 backpressure capacity spent on dead work; metastable
the peak*                capacity     provisioned always, needed briefly;
                                      the statistical bet and its correlated loss
(herding* is dual-cited: stale signal + manufactured correlation)

ENEMY 3 — SILENT OMISSION: the lie without a clock.

silent incompleteness*   index        the answer that omitted, invisibly
torn visibility*         snapshot     the answer that was NEVER true
observer inside*         telemetry    the witness who dies with the patient —
                                      omission CORRELATED with severity
(recycled name* is kin: the lookup that succeeds and lies)

And the catalog’s only true ^o:

Termination^o            quorum       FLP: no deterministic recipe exists.
                                      everything practical is a bet on
                                      partial synchrony. safety* is one
                                      recipe (Intersection); liveness has none.

The honest contract of the whole corpus: you will never be wrong (Intersection*), you will probably eventually decide (Termination^o), and everything in between is a named, priced weakening.

2. The Coordinate (the unowned primitive, now owned) #

One primitive, five faces across five files:

a coordinate = a position in a version order

produced   checkpoint_replay: state bound to its history position (binding*)
consumed   snapshot: the visibility coordinate, one per read (one door)
laddered   log: appended → durable → committed → applied (four named positions)
fenced     lease_fencing + identity: the epoch that rides a name
stamped    versioning: the version that travels with the artifact

The consumer offset is the canonical three-facet instance: position (log), progress claim (checkpoint), redelivery consequence (queue). Design rule: every coordinate names its version order and its arbiter — and the only coordinate that never lies is the commit index, minted at the root (quorum).

3. The Meta-Ladder (nine instances of one gradient) #

Every ladder is the same axis: guarantee strength vs cost, with HOPE at the bottom rung and PROOF at the top.

boundary      enforcement:    convention → protocol → kernel → crypto → physical
cache         freshness:      TTL → invalidation → validator → lease → version-pin
GC            death proof:    decree → visibility → convergence → reachability
index         answer contract: approximate → candidate → exact
quorum        read path:      follower → R-quorum → lease-read → ReadIndex → log
lease_fencing validity:       wall-clock → heartbeat → quorum / issuer
retry         dedupe:         algebra → CAS → sequence+epoch → record → txn coupling
transaction   isolation:      RU → RC → SI → serializable (write skew in SI's gap)
identity      source:         location → assigned → content-derived

The recurring design question: what does hope buy you at the bottom rung, and what forces the climb? The recurring dissolution: the top rungs (content-addressing, quorum, algebra) don’t guard the enemy — they make it meaningless. Four dissolutions on record: content- addressing kills tearing, conflict, recycling, and memoization-key drift; quorum kills the promotion trade and the paused holder; CRDT algebra kills ordering conflicts and hot-invariant contention; level-triggering kills lost-event fragility.

4. One Door (atomic publication, four descriptions unified) #

GC:           publish new metadata atomically; delete last
snapshot:     a coherent view is entered through exactly one door
materialized: cutover step 5 — one generation pointer flips once
log:          the high watermark — the fence between immutable body
              and provisional tail

One mechanism: state and its coordinate become visible together or not at all, through a single root. Never assemble reality from two separately-fetched parts. Its violation is Enemy 3’s torn visibility.

5. The Gap/Overlap Dial (eight sites) #

Every ownership or generation handoff forces one choice — you get gap or overlap, never neither:

1. Kafka rebalance         eager (gap) vs cooperative-sticky (overlap, fenced)
2. volume detach/attach    lease_fencing's handoff phase
3. CDC snapshot→stream     Debezium's seam
4. reindex/rebuild catch-up materialized's cutover step 3
5. quorum reconfiguration  joint consensus = overlap, fenced
6. replication bootstrap   snapshot + delta seam
7. resharding              keys unowned vs two owners
8. expand/contract         dual-write = chosen overlap, paid in divergence checks

Overlap is made safe by idempotency + fencing; gap is paid in availability. Choose out loud.

Part II — The Tier Lattice #

TIER 0   identity                    the naming substrate: every key,
                                     coordinate, and token is minted here
TIER R   quorum (the root)           the arbiter: manufactures the only
                                     staleness-immune facts (commit index,
                                     terms) and exports them upward
TIER 1   COMPONENTS (nouns)          queue, scheduler, boundary, policy,
                                     cache, log, replication, shard(=the
                                     Splitting operator itself)
TIER 2   PROTOCOLS (verbs)           state_machine (substrate formalism),
                                     checkpoint_replay (recovery),
                                     backpressure (dynamics),
                                     garbage_collection (maintenance),
                                     lease_fencing (ownership keystone),
                                     retry_idempotency (duplication),
                                     snapshot (visibility),
                                     versioning (evolution — the time axis)
TIER F   FACES (arrow-dense)         index, materialized, routing —
                                     read-side and traffic-side composites
TIER E   EVALUATION (the crown)      capacity (money), telemetry (evidence) —
                                     the only tiers that close feedback
                                     loops back into the lattice

Symmetric pairs (the lattice’s structure):

produce / consume       checkpoint_replay ↔ snapshot   (the coordinate)
manufacture / consume   transaction ↔ retry_idempotency (atomicity domains)
gather ownership / gather duplication   lease_fencing ↔ retry (the reunions)
remove / rewrite        GC's two parts
statics / dynamics      queue+scheduler ↔ backpressure
space / time            everything ↔ versioning
adjudicate / hold       GC's pin registry ↔ snapshot's pins

Part III — The Topological Design Order (the procedure) #

Ordered by dependency of meaningfulness: each tier’s questions are senseless before the previous tier’s answers. A DAG with back-edges only from Tier E — which is what “evaluation layer” structurally means.

STEP 0  NAME            identity: entities, sources (digest vs mint vs
                        location), dual layers, scopes, composite keys
STEP 1  INVARIANTS      state_machine: states, transitions, ignorance
                        states; transaction axis 1: name every invariant
                        BEFORE any machinery — and try to shrink it in
                        the schema
STEP 2  TRUTH           log: what order, the four ladder positions, the
                        ack's rung; quorum: what is agreed, which quorums
                        intersect, one history or intersecting views
STEP 3  OWNERSHIP       lease_fencing: who may act, anchored to what,
                        fenced where (every mutation target), gap-or-
                        overlap on every handoff
STEP 4  GEOMETRY        shard: the key so expensive invariants land
                        inside slices; replication: topology, ack window,
                        convergence, placement; scheduler: what allocates
                        the scarce thing
STEP 5  WORK & FLOW     queue: the waiting discipline; retry_idempotency:
                        identity + rung + atomicity domain per effect;
                        backpressure: walk the chain for unbounded
                        buffers, propagate deadlines
STEP 6  READ SIDE       snapshot: the one door; cache: the freshness
                        rung + full key; index: the answer contract;
                        materialized: time semantics + watermark policy
STEP 7  MEMORY          checkpoint_replay: the coordinate, atomically,
                        stamped with the epoch; GC: pins registered,
                        proof of death chosen, publish-then-delete
STEP 8  CROSSINGS       boundary: concern × mechanism × strength matrix;
                        policy: where authority lives, staleness named;
                        routing: granularity, key authenticity,
                        decorrelated feedback
STEP 9  TIME            versioning: stamps in-band, rollback direction
                        tested, envelope + order, expand/contract with
                        gated contract
STEP 10 EVALUATE        capacity: walk the ledger, size the bet, decide
                        who suffers; telemetry: decision-first signals,
                        SLI from the user's chair, what silence means
        (back-edges: telemetry → backpressure's controllers;
         capacity → scheduler's allocations — the closed loops)

Interview compression of the whole procedure:

name it → invariant it → agree on it → own it → split and place it →
work it safely → derive its views → remember and forget it →
guard its crossings → evolve it → price it.

Part IV — Star Registry (block → cleave → star → flagship → one-liner) #

Block	Structural cleave	Star	Flagship recipe	One-liner
identity	source ladder	recycled name*	dual layer (name+UID)	the most dangerous lookup is the one that succeeds
state_machine	transition authority	modeling ignorance*	Unknown/Suspect states + timeout exits	give ignorance a state, a timer, and an exit
transaction	invariant boundary	hot invariant*	split / commute / escrow / single-writer	contention is the invariant’s bill
log	record semantics	uncommitted suffix*	high watermark	the log lies about its own tail
quorum	consensus vs overlap	Intersection* / Termination^o	shared persistent honest witness	never wrong; probably, eventually, decided
lease_fencing	validity substrate	paused holder*	downstream fencing	the old owner will not stop itself
shard	key scheme	skew*	recipes outside the partitioner	the partitioner counts keys; the bill counts work
replication	write topology	promotion decision*	fence + honest RPO/RTO in daylight	the incident will not wait for the meeting
scheduler	decision question	view-vs-reality*	optimistic bind → lease → fence → reconcile	every decision is provisional
queue	retention semantics	commit point*	idempotency / outbox	ack ≠ effect
retry_idempotency	operation identity	atomicity domain*	marker inside the domain	exactly-once is a boundary, not a system
backpressure	limited quantity	goodput*	deadline propagation	never serve work whose caller gave up
snapshot	visibility coordinate	torn visibility*	one door	a torn view was never true
cache	freshness ladder	invalidation delivery*	ladder-climb + zookie	staleness is manufactured here on purpose
index	answer contract	silent incompleteness*	one-sided-by-construction (bloom)	how would anyone know what was omitted?
materialized	fill trigger / family	completeness bet*	watermarks + declared late policy	late events are the terrain, not the failure
checkpoint_replay	what is authoritative	binding coordinate*	atomic manifest + epoch stamp	state means nothing without its position
garbage_collection	proof vs equivalence	proving death* (+irreversible)	tombstone = deletion’s own 2PC	the only mistake you cannot take back
boundary	concern vs mechanism	(matrix, not star)	strength per cell	a namespace is not a wall
policy	where authority lives	revocation freshness*	zookie: staleness as a parameter	the further authority travels, the harder the recall
routing	decision granularity	herding*	power of two choices	a fleet agreeing on stale news is a stampede
capacity	the ledger	the peak*	who-suffers, chosen in advance	the P2A ratio is the economics
telemetry	the decision served	observer inside*	deadman switch	the dead don’t page
versioning	what is versioned	semantic gap* (~^o)	tombstone the identifier	the wire parses; only discipline understands

Part V — Closing Notes #

Boundary.md carries a matrix instead of a star — correctly: its content is the concern×mechanism×strength grid, and its “bottleneck” (the shared kernel, in the k8s compilation) is an instance, not a primitive.

The catalog’s own VSA, one level up: lease_fencing was the corpus’s strategic bottleneck (essential to nine consumers, owned nowhere) until written; this file resolves the remaining four unowned meta-patterns. After it, the corpus is closed: every mechanism has one definition, every seam two signatures, every star one owner, every enemy a roster.

What this file cannot do: retrieval speed and transfer. Those are the next system — cards from Part IV’s rows, reverse drills from the enemy roster (symptom → star → recipe), and mock walks of Part III against real prompts. The knowledge is done; the performance is practice.