Flipping a brain to log-canonical
In this section
A brain that has been migrated to the segment log still runs that log as a mirror: the generation tree (the files under entities/) is the canonical, every write lands there first, and the log is written after it. The native writer (brainy serve taking writes itself) commits only into a brain whose log IS the canonical, so a mirror brain refuses every native write by name until it is flipped.
The flip is one-way. It writes a mark (_system/canonical-mode.json) recording when, under which decision and by whom; from then on the brain opens log-canonical on every engine that knows the mark, its tree's bookkeeping is frozen, and asking for the mirror back is refused by name.
npx brainy flip-log-canonical /srv/brains/acme --dry-run
npx brainy flip-log-canonical /srv/brains/acme --decision flip-acme --authorized-by opsBASHWhat it guarantees
Nothing a reader could see is lost. Before the mark, the tree and the log are compared row by row, by id, at
_revandupdatedAt. The mark is written only when every row agrees. A mark over a disagreement would make every row the log lacks vanish from every keyed read.The tree's version wins, because the tree was the canonical. Every row the tree holds and the log does not (tree-only), and every row the tree holds at a newer version (drifted, tree newer), is republished to the log first: one ordinary commit per 1,000 rows, with an empty change, through the same commit path a write takes. The log receives exactly the record a flush would have written, read back from the tree. Each commit spends one generation stamped
origin: system:reconcile, so the history says which generations were reconciliation and which were writes.No writer can land between the proof and the mark. The flip closes the brain, takes the brain's writer lease itself, compares again and writes the mark under that lease.
A store already flipped is left alone. A second run reports the existing mark and does nothing.
What blocks it, by name
Refusal | When | What to do |
|---|---|---|
| The brain was never migrated to the segment log (no canonical pointer), or its pointer serves the tree. | Run the migration ceremony first, then flip. |
| Another process holds the brain: a lease on another host, or a live process on this one. A dry run refuses too. | Stop the process that serves the brain. Its own close drains the writes it was still holding. |
| The log holds a live row the tree does not have (log-only), or a newer version of a row than the tree has (drifted, log newer). | A ruling, per brain. See below. |
| The tree and the log still disagree after the reconciliation. No mark is written. | Report it: the reconciliation's own commits did not land. |
Why the log-ahead classes are refused rather than reconciled. On a mirror the log is written from the tree, so the brain's own commits never put the log ahead. Two things do. A delete whose log tombstone was still held in memory when the process died leaves the log holding the row the tree removed. A copy taken while the brain was running can capture the log after the tree. In both cases the tree-canonical answer is to publish the tree's state over the log's (a tombstone, or the older version). That choice changes what the log says happened, so the flip does not make it for you.
The dry run
--dry-run takes no lease, opens nothing and writes nothing. It reads the lease record (and refuses if the brain is held), compares the tree and the log as they stand, and prints the classes, what an apply would republish, and the refusal an apply would meet.
Its comparison is unsettled. A brain whose last writer crashed with writes still held in memory shows those rows as tree-only until an open's recovery runs. The apply opens the brain (recovery included) and flushes before it compares.
The receipt
--json prints the whole receipt; without it the same facts are printed as text.
Field | What it says |
|---|---|
|
|
| The lease record as found: |
| The first comparison: per family, rows on each side and every class counted, the first 20 ids of each named, the comparison's wall and peak memory |
| The entity and relation rows republished |
| The first and last generation the reconciliation spent, and how many |
| The comparison the mark was written on |
| The mark: |
| Open and flush, compare, reconcile, close, compare again, mark, total — milliseconds, each measured |
Exit codes: 0 flipped, already marked, or a completed dry run; 1 refused (the class name and its sentence on stderr, and under refused in the JSON); 2 usage.
What it costs
One open and one clean close of the brain, two full comparisons, and one commit per 1,000 reconciled rows. A comparison walks every row of both sides once and holds one family's ids in memory at a time.
MEASURED on a copy of a production brain (1.37 million generations, about 95 GB allocated), in a shared build-box lane:
Step | Wall | What it found or did |
|---|---|---|
Dry run (no open) | 24–38 s | entities: tree 94,208 / log 94,190: 18 tree-only, 2 drifted (tree newer), 0 log-only · relations: tree 192,125 / log 192,113: 12 tree-only, 0 drifted, 0 log-only · peak resident 3.0 GB |
Open and flush | 1.4–1.7 s | The open's recovery replayed writes the copy's fact log held above its tree (21 VFS files, their 21 folder links, and newer versions of 6 rows) into the tree, and none of them reached the log. This is why the apply compares only after the open |
First comparison | 24.5–32.6 s | 39 tree-only and 8 drifted entities, 33 tree-only relations, nothing log-only · 192,146 ids held at peak, peak resident 5.3 GB |
Reconciliation | 0.42 s | 47 entities and 33 relations republished in 2 commits (2 generations) |
Close | 0.5 s | A clean close |
Second comparison | 22.3–30.6 s | Every row agrees |
Whole flip | 48–66 s | The 1-minute load average was 3–23 during these runs (other jobs shared the machine) |
After the flip, the same 51 reads (get, a find on a user field, related from, to and node, neighbors, a VFS folder listing and file read) on a fixed sample gave identical answers. The native write set opened the flipped copy in 1.1–1.4 s on its first claim, including a graph cold load of 84 SSTables in 95–121 ms. Later claims in the same process reused the loaded graph (0.0 ms).
Rolling back: what an older engine does with a flipped brain
The mark has existed since 12.0.0, and every engine since then reads it. An older engine opening a flipped brain serves it log-canonical too. It does not reopen it as a mirror, and it refuses every door that would write the frozen tree (an enabled: false open, a state replacement, a history compaction, a rollback of the migration).
Relationships the native writer lands are the one difference. This release's related({ from }) reads relationships through the log and returns them (measured on the flipped copy: a relationship written over the wire was returned by related({ from }) and related({ to }) after a reopen). A 13.1.x engine reads related({ from }) from the tree, where the native writer writes nothing, so it does not return those relationships. related({ to }) and get return them in both.
There is no command that un-flips a brain. The exact way back is to restore a snapshot taken before the flip (backup and restore). Every write since that snapshot is lost, so take one immediately before flipping.