A successful write is a local fact. A system with more than one node has a second question to answer: did the other copies get there, too—and what happens if they didn’t?
That’s why I keep the put path and the repair path separate in StreamHive. A one-shot put waits for a matching acknowledgement and retries within a finite budget. That can handle a lost acknowledgement or a short interruption. It cannot promise that a peer which was offline for hours will catch up.
Two loops, two guarantees
StreamHive’s peer inventory exchange is the slower loop. It runs at startup and periodically, compares paged inventories, and asks for keys the local store is missing. Repair responses are bounded; a large exchange can continue later instead of growing into unbounded work.
That distinction changes what a healthy signal means. “The write was accepted” describes one operation. “The peers are converging” describes a continuing process across time, restarts, network limits, and storage state. I want the system to expose enough counters and status to tell those stories apart.
Integrity before repair
There is little value in repairing a replica with bytes that do not match the key they claim to represent. For content-addressed blobs, StreamHive checks key and content at apply and repair-source boundaries. A damaged source is skipped rather than sent as a valid repair.
The hash does not solve every storage problem. It gives one precise check: these bytes match this content-derived key. It does not tell a node how to resolve conflicting application-level records or discover every peer in a changing network.
Repair is normal work
Distributed storage is often introduced as “copy this object to another machine.” That is the postcard version. The operational version is a loop: accept, acknowledge, compare, repair, and report when the loop is limited or incomplete.
A copy gets the bytes somewhere once. A repair path gives the system a way to notice when reality drifted—and a bounded way to move it closer again.