Brainy

Docs Brainy

The Brainy API Contract

Name: Soulcraft
Address: US

In this section

Brainy ships as two engines. Open Brainy (@soulcraftlabs/brainy, MIT) is the reference implementation. Brainy (@soulcraft/brainy) is the native engine. Until now they were coupled as a pair: a native release was pinned to one reference-engine version, and the fleet gated on that tuple. A pair is not a contract — it says these two builds were tested together, never this is what a Brainy is.

This page is the contract. It states what every conforming implementation must do, in terms no implementation owns. The version number below is what the fleet gates on.

contract version: 1

Read it from code instead of from this page:

import engine from '@soulcraft/brainy'
engine.contractVersion()          // 1 — the engine plugin's door

import { Brainy } from '@soulcraft/brainy'
new Brainy(config).contractVersion()   // 1 — the API layer's doorTS

and from packaging:

// @soulcraft/brainy's package.json
{ "brainyContract": 1 }JSONC

The machine-readable form of everything below is docs/api-contract.json, emitted from src/contract/contract.ts by scripts/contract/emit-manifest.mjs. That TypeScript module — not this page, and not either engine's .d.ts — is the source of truth; this page is its prose, and the JSON is its wire form.

§0 What "conforming" means

An implementation conforms to contract 1 when:

Every REQUIRED door in §2–§10 exists with the stated shape.
Every stated semantic holds — the ordering laws, the merge law, the cascade law, the visibility law, the refusal law.
Every error class in §11 is exported under its stated name and thrown under its stated condition.
The operator law (§12), the field-addressing law (§13) and the health law (§14) hold exactly, including their documented divergences.
It answers contractVersion() === 1.

An OPTIONAL door may be absent. Optional is not "second class" — it is the honest marker for a door the convergence program has not yet committed to owning in both engines. Promoting an optional door to required is a MAJOR change (§1).

What conformance does NOT promise

Not identical ids. relate() mints a relation id per call. Two engines seeded with identical inputs agree on (from, to, type, subtype), never on the relation's id. Compare edges by that tuple.
Not identical tie-breaks in vector top-k. Two rows at exactly equal distance may be ordered either way. What IS promised: the result COUNT is the same, and every returned id is a member of the tied set — never a different row.
Not identical performance. The contract is about answers.

§1 The compatibility rule

Change	Version move
A new OPTIONAL door	minor — additive
An operator moving from refused to served	minor — additive
A new error class	minor — additive
A door removed	major
A door's answers narrowed	major
An ordering or merge law changed	major
An OPTIONAL door promoted to REQUIRED	major
An operator moving from served to refused	major

contractVersion() answers the MAJOR number — that is the number a fleet service gates on, because a minor move can never break it. Additive changes within a major are visible in the emitted manifest's door list, which src/contract/contract.test.ts diffs against a fresh emit; a contract change that forgets to re-emit reds the gate.

§2 Lifecycle

Door	Shape	Req
`init`	`init(overrides?: Partial<BrainyConfig>): Promise<void>`	required
`close`	`close(): Promise<void>`	required
`flush`	`flush(): Promise<void>`	required
`clear`	`clear(): Promise<void>`	required
`use`	`use(plugin: BrainyPlugin): this`	required
`getActivePlugins`	`getActivePlugins(): string[]`	required
`isInitialized`	`get isInitialized(): boolean`	required
`ready`	`get ready(): Promise<void>`	required
`newId`	`newId(): string`	required

init() is idempotent and polymorphic. Calling it twice is a no-op, and every door that needs an open brain reaches it lazily through the same override chain — so a subclass that gates on init() gates the lazy path too.

close() releases every hold on the storage root: the writer lock, open file descriptors, and mmap regions. A conforming implementation can be reopened on the same root by a fresh process immediately after close() resolves.

flush() is the durability boundary. Once it resolves, every accepted write is visible to a fresh reader of the same root. Writes are durable at their own cadence too; flush() is how a caller demands the boundary now.

§3 Write

Door	Shape	Req
`add`	`add(params: AddParams<T>): Promise<string>`	required
`update`	`update(params: UpdateParams<T>): Promise<void>`	required
`remove`	`remove(id: string): Promise<void>`	required
`addMany`	`addMany(params: AddManyParams<T>): Promise<BatchResult<string>>`	required
`updateMany`	`updateMany(params: UpdateManyParams<T>): Promise<BatchResult<string>>`	required
`removeMany`	`removeMany(params: RemoveManyParams): Promise<BatchResult<string>>`	required

The id law. A caller-supplied id is honored exactly. An omitted one is minted by the engine.

The merge law. update() MERGES: keys present in the supplied metadata replace their stored values, keys absent from it survive untouched, and _rev advances by exactly one. There is no replace-mode update.

The cascade law. remove(id) deletes the entity AND every relation incident to it. After it resolves, related({ node: id }) answers [] — not a dangling edge, not a relation pointing at a missing endpoint. Removing an id that does not exist is a no-op, never an error.

The projection law. When add() resolves, every derived projection the engine serves reflects the write — with ONE exception: the semantic (vector) projection when the vector is embedded asynchronously. waitForIndexed() (§10) is the door for that case; the alternative — a read that silently answers from a projection that has not caught up — is banned.

§4 Read

Door	Shape	Req
`get`	`get(id: string, options?: GetOptions): Promise<Entity<T> \\| null>`	required
`batchGet`	`batchGet(ids: string[], options?: GetOptions): Promise<Map<string, Entity<T>>>`	required
`find`	`find(query: string \\| FindParams<T>): Promise<Result<T>[]>`	required
`similar`	`similar(params: SimilarParams<T>): Promise<Result<T>[]>`	required
`counts`	`get counts(): { entities(); relationships(); byType(); bySubtype(); topTypes(); … }`	required
`explain`	`explain(params: FindParams<T>): Promise<{ query; fieldPlan; warnings }>`	required
`pagination`	`get pagination(): { find(); count(); meta() }`	optional
`streaming`	`get streaming(): { entities(); search(); relationships(); pipeline(); process() }`	optional

The absence law. get() answers null for a row that does not exist — it does not throw, and it never answers a partially-populated entity. batchGet() OMITS missing ids from its map rather than mapping them to null.

The three-state law for find() (this is the load-bearing one):

A query that cannot be answered THROWS. If you get [], the query resolved and honestly matched nothing. If you get results, they are ordered by what you asked for. There is no third state.

An unresolvable field name, an unserviceable operator, or an option the engine cannot honor is a typed refusal (§11, §12, §13) — never an empty array and never a silent fallback to insertion order. Accepted-and-ignored options are banned under the same law.

The ordering law.

Entities MISSING the orderBy field (or holding null) sort last in both directions — never dropped from the result.
Ties break by id ascending, in both directions. Ordering is fully deterministic.
With no orderBy and no vector, result order is unspecified. Compare result SETS across implementations, not sequences.

The visibility law. Every entity carries one of three tiers:

Tier	Default `find()`	Reached by
`public`	visible	always
`internal`	hidden	`{ includeInternal: true }`
`system`	hidden	`{ includeSystem: true }`

system is the engine's own tier. A consumer cannot create a system-tier row through AddParams.visibility; the VFS root (00000000-0000-0000-0000-000000000000) is the well-known member every conforming brain holds.

counts.entities() and counts.relationships() are O(1) ledger reads, not scans, and the by-dimension doors read the same ledger. A count that would require a store walk to be exact is a health question (§14), not a count.

§5 Graph

Door	Shape	Req
`relate`	`relate(params: RelateParams<T>): Promise<string>`	required
`unrelate`	`unrelate(id: string): Promise<void>`	required
`updateRelation`	`updateRelation(params: UpdateRelationParams<T>): Promise<void>`	required
`related`	`related(paramsOrId?: string \\| RelatedParams): Promise<Relation<T>[]>`	required
`relateMany`	`relateMany(params: RelateManyParams<T>): Promise<string[]>`	required
`graph`	`get graph(): GraphApi<T>`	optional
`auditGraph`	`auditGraph(options?): Promise<GraphAuditReport>`	optional

Traversal directionality. related({ node }) is UNDIRECTED — every edge touching the node, in either direction. related({ from }) and related({ to }) are directed. type and subtype narrow the edge set. A node with no edges answers [].

Relation identity. relate() mints the relation id; a caller cannot supply one. Cross-implementation comparison is by (from, to, type, subtype) (see §0).

§6 Transact

Door	Shape	Req
`transact`	`transact(ops: TxOperation<T>[], options?: TransactOptions): Promise<Db<T>>`	required
`now`	`now(): Db<T>`	required
`generation`	`generation(): number`	required
`asOf`	`asOf(target, options?): Promise<Db<T>>`	optional
`diff`	`diff(a, b): Promise<DiffResult>`	optional
`history`	`history(id, options?): Promise<EntityHistory>`	optional

The batch law. transact() applies its operations in order and atomically: either every operation commits or none does. Later operations in one batch OBSERVE earlier ones — [{op:'add', id, …}, {op:'update', id, …}] commits one row carrying the merged result, and the returned Db's receipt names both operation ids.

The generation law. generation() is monotonic and advances on every committed write. now() returns an immutable Db value pinned to the current generation; a Db never changes underneath its holder.

CAS. A write carrying an expected _rev that no longer matches the stored revision throws RevisionConflictError — it never overwrites. A transaction committed against a generation that has since advanced incompatibly throws GenerationConflictError. A pinned generation that has been compacted away throws GenerationCompactedError. All three are refusals, never silent last-write-wins.

§7 Aggregate

Door	Shape	Req
`defineAggregate`	`defineAggregate(def: AggregateDefinition): void`	required
`queryAggregate`	`queryAggregate(name, params?): Promise<AggregateResult[]>`	required
`removeAggregate`	`removeAggregate(name: string): void`	required
`trackField`	`trackField(name, options?): void`	optional

An aggregate is incrementally maintained, not computed at query time: a write that changes a member row updates the aggregate's state on the write path. queryAggregate() reads that state.

The where inside an AggregateSource obeys the SAME operator law as find() (§12) — identical vocabulary, identical verdicts, identical refusals. An operator that refuses in find() refuses here too.

§8 Embed

Door	Shape	Req
`embed`	`embed(data: any): Promise<Vector>`	required
`embedBatch`	`embedBatch(texts: string[], options?): Promise<Vector[]>`	required
`similarity`	`similarity(textA: string, textB: string): Promise<number>`	optional

embed() produces exactly the vector add() would store for the same data. embedBatch() returns one vector per input, in input order.

The contract does NOT fix the embedding model or the vector dimensionality — those are per-brain configuration, and two brains with different embedders are both conforming. What the contract fixes is that a brain's embedder is consistent with itself: the vector embed(x) returns is the vector a subsequent add({ data: x }) stores.

§9 VFS

Door	Shape	Req
`vfs`	`get vfs(): VirtualFileSystem`	required

The virtual filesystem is stored in the same brain as system-tier entities, rooted at the well-known id 00000000-0000-0000-0000-000000000000. Its rows are therefore reachable through find({ includeSystem: true }) on any conforming implementation — which is exactly why the root id is part of the contract rather than an implementation detail.

§10 Health and repair

Door	Shape	Req
`health`	`health(): Promise<{ overall; checks }>`	required
`getIndexStatus`	`getIndexStatus(): Promise<{ initialized; projections; … }>`	required
`repairIndex`	`repairIndex(options?): Promise<RepairReport>`	required
`diagnostics`	`diagnostics(): DiagnosticsResult`	required
`requireProviders`	`requireProviders(keys: string[]): void`	required
`storageAdapter`	`get storageAdapter(): BaseStorage`	required
`warm`	`warm(): Promise<WarmReport>`	optional
`maintenanceDebt`	`maintenanceDebt(): Promise<MaintenanceDebtReport>`	optional
`waitForIndexed`	`waitForIndexed(path?, opts?): Promise<void>`	optional
`stats`	`stats(): Promise<BrainyStats>`	optional

The semantics are ADR-008's, restated here as contract rather than as one engine's design (docs/ADR-008-index-health-and-repair.md):

Health is accounting, not sampling (D1). Every projection maintains an exact coverage ledger on its write path and reports health as a subtraction. A sampled probe may be a diagnostic; it may never be a verdict.
Verdicts are graded and named (D2). health().checks names every condition; isReady()-style readiness is a VIEW of the same report. There are no unnamed latches.
Rebuild is online and generational (D3). New structures are built beside the serving ones and swapped atomically. Reads never go dark for a rebuild.
Heal is incremental, and never read-triggered (D4). The common cure is re-posting the named missing items — O(missing), not O(store). A read path may refuse loudly; it may never start a store walk. repairIndex() is the one explicit operator door.
A throwing probe is heal: 'none' (D5), with the error in detail. Flakiness can never buy a rebuild.

The serving law (the D2/D4 amendment of 2026-08-26, co-signed by both engines): heal grades one question only — could an answer be WRONG? — never how expensive is the fix? Serving is withheld ONLY by these named, rebuild-graded invariants:

index-initialized · durable-state-present · manifest-residency ·
replay-clean · strand-latch

strand-latch withholds serving only for the lost-mapper, torn-manifest, replay-unclean and archive-orphan classes. Everything else — a saturated vector repair, a coverage shortfall, a short generation during an online rebuild — is repair-graded: healthy: false, the row red with the number, and serving: true. An implementation that withholds serving for any other reason is non-conforming; that exact defect took two fleet services down in one day before the amendment.

requireProviders(keys) THROWS when any named key fell back to the engine default. It is the structural gate the native package's loud-refusal law is built on — diagnostics() reports, requireProviders() enforces.

§11 Error classes

Every name below is an EXPORTED class of a conforming implementation's public surface — a caller cannot instanceof-narrow a refusal it cannot import. BrainyError is the base every engine refusal extends.

Class	Thrown when
`BrainyError`	base class for every engine refusal
`EntityNotFoundError`	a door requiring an existing entity gets an id that does not exist (not `get()`, which answers `null`)
`RelationNotFoundError`	a door requiring an existing relation gets an id that does not exist
`UnresolvableFieldError`	a field name resolves to neither user metadata nor one of the ten `system.*` scalars
`InvalidFieldAddressError`	a field address is not addressable at all (an unknown `system.*`, or a plumbing name)
`UnsupportedFindOptionError`	a `find()` option the engine cannot honor is supplied
`RevisionConflictError`	a CAS write loses the race
`GenerationConflictError`	a transaction is committed against an incompatibly-advanced generation
`GenerationCompactedError`	a pinned generation has been compacted away
`SpeculativeOverlayError`	a speculative overlay could not be reconciled with the committed log
`StoreInconsistentError`	canonical storage and the committed log disagree irreconcilably
`PendingFlushDurabilityError`	durability was requested for writes a pending flush could not make durable
`CanonicalEnumerationUnavailableError`	a maintenance walk cannot enumerate canonical rows
`VectorIndexNotReadyError`	the vector projection cannot be trusted to answer
`MetadataIndexNotReadyError`	the metadata projection cannot be trusted to answer
`GraphIndexNotReadyError`	the graph projection cannot be trusted to answer
`DerivedArtifactMissingError`	a required derived artifact is absent and cannot be rebuilt in place
`ProtectedArtifactError`	a write would damage an engine-protected artifact
`MigrationInProgressError`	a door is called while an on-disk migration holds the brain
`WaitForIndexedTimeoutError`	`waitForIndexed()` reached its deadline
`TornRecordError`	a canonical record on disk is partially written

§12 The operator law

Full accounting: docs/filter-operator-conformance.md. The contract's binding statement is the jointly frozen set and its three verdicts — served, served beyond the baseline, refused by name. There is no fourth verdict, and a quiet wrong answer is never one of them.

Served (both engines answer identically) — 18 tokens:

eq · equals · ne · notEquals · in · oneOf · gt · greaterThan ·
gte · greaterThanOrEqual · lt · lessThan · lte · lessThanOrEqual ·
between · contains · exists · missing

Combinators (clause-level, served by both) — 3 tokens:

allOf · anyOf · not

Served beyond the baseline — the native engine answers these three posting-set expressions exactly (hasAll = intersection, noneOf = universe minus union, excludes = negation of contains); the reference engine's index path answers all three with [] today. The native engine is the correct side, and this is a RULED divergence, pinned on both sides:

hasAll · noneOf · excludes

Refused by name — the native engine THROWS, naming the operator and the field; the reference engine's find() resolves with []:

startsWith · endsWith · length · matches

The refusal is a proof, not a shortfall: the metadata index stores normalized posting keys (lowercased, trimmed, one posting per array element, canonical numbers), which provably erases the distinctions those four operators read. matches carries a second, independent reason — JavaScript RegExp and Rust's regex are different dialects. The durable cure is to make the predicate's answer an indexed value at write time and filter on it with a served operator.

The vocabulary is closed. An operator token outside these 25 value operators is not "unsupported" — it is a typo, and a conforming implementation refuses it by name rather than matching nothing.

Known divergence — negation with case or padding. The native engine's metadata index normalizes a posted string (lowercase + trim); the reference engine's does not. Positive operators are blind to this (the index answers a superset, find() narrows it exactly), but a NEGATION (ne / noneOf / excludes) on a string field whose only difference from the operand is letter case or surrounding whitespace drops rows the reference engine keeps. Ruled, documented, pinned on both sides.

§13 The field-addressing law

Full text: docs/field-addressing.md. One rule, no exceptions:

A bare field name always means user metadata. Engine scalars are reached explicitly as system.<field>.

Origin — never name — decides the storage key. An engine scalar can only land on a system.* key; a user metadata field can only land on a bare key. The two never merge, which is what makes the rule enforceable at the storage layer rather than by resolution priority. A user field named level, createdAt or type is ordinary user data and always works.

The ten entity scalars (system.<name>):

id · type · subtype · createdAt · updatedAt ·
confidence · weight · visibility · service · createdBy

The relation scalars (system.<name>):

verb · sourceId · targetId · subtype · createdAt · updatedAt ·
confidence · weight · visibility · service · createdBy

Invisible plumbing — never resolvable, never indexable, never orderable, not bare and not through system.:

vector · connections · level · data · _rev

Reserved namespace. A user metadata field whose name would flatten into system.* (a field literally named system.foo, or an object field named system) is REFUSED at write time. A scalar field named exactly system is ordinary user data.

The refusal names both candidates. system.<anything else> does not exist and refuses. An unresolvable bare name refuses with a message naming both the missing user field and the system.* correction.

§14 What the contract does not cover

Deliberately outside contract 1, so it cannot be mistaken for a promise:

On-disk layout. Canonical storage layout is docs/canonical-layout-spec.md's subject, versioned separately. Contract 1 is an API contract; two conforming implementations may hold different bytes. The canonical seam (below) is the bridge until Stage 2 moves canonical storage into the native core.
Neural import, MCP, integrations, OData. Shipped by the reference engine and re-exported by the native package, but not contract-mandated at version 1.
Performance. docs/performance-budget.md carries the CI-enforced latency budget for the native engine. It is a product commitment, not a contract term.

§15 The canonical seam

Until Stage 2 lands the segment-log canonical inside the native core, the native package reaches canonical storage through the reference engine's FileSystemStorage. That dependency is ONE named module — src/api/canonicalSeam.ts — which enumerates the exact adapter methods the engine calls and asserts every one is present at open, refusing by name if the reference engine drops one.

The seam is named so that it can be RETIRED, not so that it can grow. Its surface is frozen at contract 1; adding a method to it is a deliberate, reviewed change, and Stage 2 deletes the module.

§16 How this page stays true

src/contract/contract.ts is the source of truth; this page is its prose.
scripts/contract/emit-manifest.mjs emits docs/api-contract.json from it.
src/contract/contract.test.ts fails red when the committed manifest and a fresh emit disagree, AND when any door named here is absent from the shipped reference-engine surface, AND when the operator or field-addressing vocabularies drift from the shared catalogs.
src/conformance/oracle.test.ts runs one workload against three subjects — the reference engine, the native provider stack, and the one-install package constructed the public way — and asserts the same answers.
scripts/check-contract.mjs (npm run check:contract) verifies both engines answer contract 1 before a release.

Finding: four alias spellings in the operator page are not in the shipped vocabulary

docs/filter-operator-conformance.md's table lists is, isNot, greaterEqual and lessEqual as served aliases. Open Brainy 10.4.3's shipped VALUE_OPERATORS set declares 25 tokens and none of those four is among them — the engine throws BrainyError('INVALID_QUERY') on an unrecognized operator, so those four spellings refuse rather than serve. This contract publishes the SHIPPED vocabulary (read from the engine at gate time by operatorCatalog.loadBrainyOperatorTokens()), not the four extra spellings. Filed for the operator page to correct; recorded here so the difference is never discovered as a surprise.

Previous ← Brainy Performance Next ADR-002 — DiskANN as brainy's billion-scale index option →