Upgrading from 7.x to 8.0

Opening a 7.x on-disk store with Brainy 8.0 runs a one-time, in-place layout migration the first time the store is opened. It is automatic (autoMigrate defaults to true), it runs once, and it stamps a marker so every later open is a no-op.

Almost everything about the upgrade is transparent: your entities, relationships, metadata, and indexes migrate and rebuild without any action on your part. This guide covers the one case that needs attention — Virtual Filesystem (VFS) content — and how 8.0 recovers it for you.

TL;DR

  • Just upgrade to @soulcraft/brainy@8.0.12 (or later) and open the store. If a previous upgrade left VFS content stranded, 8.0.12 heals it on open, with no operator action.

  • Want to force or script it? Call await brain.vfs.adoptOrphanedBlobs().

  • The recovery is non-destructive and idempotent — it copies, never moves, and running it twice is a no-op.

What the migration does

The 7.x on-disk layout stored entities under a per-branch path (branches/<head>/entities/...). 8.0 uses a flat layout (entities/...). On first open, Brainy:

  1. Collapses branches/<head>/entities/* into the flat entities/* layout.

  2. Rebuilds the derived indexes (vector, graph, metadata) and count rollups from the canonical entities.

  3. Stamps _system/migration-layout.json so re-opening is a no-op.

  4. Takes an automatic pre-upgrade backup of the directory before it starts, and removes it once the upgrade is verified complete (see The safety net below).

The log line to expect (once per store):

[brainy] Migrating a 7.x branch layout (branches/main) to the 8.0 flat layout
  in place — 13175 entity files. This runs once; back up the directory first if
  you need a rollback (8.0 does not keep the old layout).

The one thing that needs recovery: VFS content

If your application uses the Virtual Filesystem (VFS) to store file content (for example, a CMS that keeps page documents at paths like /pages/homepage/page.json), that content is held as content blobs.

7.x kept those blobs in the branch system's copy-on-write area (_cow/). 8.0 removed the branch/copy-on-write system, and its content blobs live in the content-addressed store (_cas/). The layout migration moves entities — it does not move the VFS content blobs. Left alone, a 7.x store's VFS blobs would remain in _cow/, and a read of one would throw:

VFS: Cannot read blob for /pages/homepage/page.json:
  Blob metadata not found: 6b87cfb71ad2f04602a5c157214dc42000...

8.0.12 recovers them automatically

Brainy 8.0.12 adds an on-open recovery pass that runs right after the layout migration. It scans _cow/, and for every content blob whose blob: + blob-meta: pair is not already in _cas/, it copies both across into the 8.0 store. It:

  • Heals a fresh 7→8 upgrade (where _cow/ still holds the blobs), and

  • Heals a store already upgraded by an earlier 8.0.x that stranded them — the recovery is gated on the presence of _cow/ and its own marker, not on the layout-migration marker, so an already-migrated store is still healed.

  • Is a cheap no-op on a native-8.0 or fresh store (there is no _cow/), and on a store already healed (a marker records completion so later opens skip the scan).

So the operator action for a stranded store is simply: upgrade to 8.0.12 and open it.

import { Brainy } from '@soulcraft/brainy'

// Opening the store is all that is required — recovery runs during init().
const brain = new Brainy({ storage: { type: 'filesystem', path: '/data/my-store' } })
await brain.init()

// The previously-failing read now succeeds.
const page = await brain.vfs.readFile('/pages/homepage/page.json')

On a store that needed recovery you will see:

[brainy] Recovered 337 VFS content blob(s) stranded by a 7→8 upgrade
  (adopted _cow/ → _cas/ in place).

Forcing recovery explicitly

If you would rather run the recovery deliberately (for example, in an upgrade script that asserts a clean result before flipping traffic), call it directly:

const result = await brain.vfs.adoptOrphanedBlobs()
// → { cowBlobs, adopted, alreadyPresent, incomplete }
console.log(`adopted ${result.adopted}, already present ${result.alreadyPresent}`)
if (result.incomplete > 0) {
  // One or more _cow/ blobs are missing their bytes or metadata — investigate
  // _cow/ before discarding your own backup. See "The safety net" below.
}

adoptOrphanedBlobs() self-initializes the brain, so it is safe to call immediately after construction. It returns:

Field

Meaning

cowBlobs

Distinct content-blob hashes found in _cow/.

adopted

Newly copied into _cas/ on this call.

alreadyPresent

Already in _cas/ (a prior open or run adopted them).

incomplete

A _cow/ blob missing its bytes or its metadata — skipped rather than half-adopted.

Verifying recovery

After opening under 8.0.12, confirm a previously-failing path reads:

const content = await brain.vfs.readFile('/pages/homepage/page.json')
console.log(content.toString().slice(0, 80))

If you scripted it with adoptOrphanedBlobs(), a clean result is incomplete === 0 and adopted + alreadyPresent === cowBlobs.

The safety net: pre-upgrade backup

Brainy takes an automatic pre-upgrade backup and removes it once the upgrade is verified complete. In 8.0.12 that verification includes VFS content: if the blob recovery reports incomplete > 0, the completion marker is not stamped (the next open retries) and the pre-upgrade backup is retained so you still have a rollback while blobs remain unaccounted for. You will see:

[brainy] VFS blob recovery adopted N blob(s) but M orphaned _cow/ blob(s) are
  missing their bytes or metadata and were left in place. The pre-upgrade backup
  is being retained; inspect _cow/ before discarding it.

The recovery never deletes anything from _cow/, so the original blobs stay in place for inspection or a manual rollback regardless.

Who is affected

Any 7.x store that used the VFS to store file content (so it has a _cow/ area) is a candidate for stranded blobs on a 7→8 upgrade. Stores that never used the VFS have no _cow/ content blobs and are unaffected.

Because the recovery is gated on the presence of _cow/, it is self-selecting and safe to roll out everywhere:

  • On a store with stranded blobs → it adopts them.

  • On a native-8.0, fresh, or non-VFS store → it is a no-op on the existence check.

There is no configuration to set and nothing to opt into. Upgrading to 8.0.12 and opening each store is sufficient.

Rollback

The recovery is copy-only, so no rollback of the recovery itself is ever needed. If you need to roll back the whole 7→8 upgrade, restore the directory from your pre-upgrade backup (retained automatically while recovery is incomplete, or your own snapshot) and pin @soulcraft/brainy@7.x. 8.0 does not keep the old branch layout in place, so a directory-level restore is the rollback path.