brainy serve
In this section
brainy serve is the engine answering on an address. It is a Rust binary that links the engine's core as a library — not a JavaScript server in front of the native addon, and not a wrapper around a Node process. The same code the .node addon exposes is called directly, in one process, with no napi crossing on any request path.
Two ways to run
A library mount. A Rust host process embeds the crate and mounts the doors on a listener it owns. The host keeps its own runtime, its own address, and its own shutdown.
A thin CLI. brainy serve, standalone, for a host that just wants the engine to answer on an address.
brainy serve # binds 127.0.0.1:8300
brainy serve --listen 10.0.0.5:8300,h2 # one listener, HTTP/2
brainy serve --listen 10.0.0.5:8300,h2 \
--listen 0.0.0.0:8443,h3 # LAN h2 + an edge h3 listenerThe starter. With no arguments at all, the binary opens ./brain, serves it on a port the operating system says is free — loopback, never a wildcard — and prints the snippet an MCP client is configured with. Nothing is asked and nothing is configured. It does not CREATE a store: a store's identity, manifest and counts ledger are written by the engine's own open, and a directory merely shaped like one is the empty success this engine will not produce, so an absent store is a refusal naming the one command that makes one.
brainy # the starter
brainy mcp # the stdio MCP server over ./brainThe door surface is a law
Every public engine verb reaches every transport, and no transport keeps its own list. The route table, the SSE endpoints, the WebSocket method set, the MCP tool list and the gRPC service are all generated from one source: api-contract.json, the emitted manifest of the versioned API contract whose prose is api-contract.md.
A door is never quietly absent. Every door either answers or refuses by name:
NotYetNativeError— the door's implementation is moving into the engine's core and is not there yet. The refusal carries the door's name and the release it arrives in. Never a no-op, never an empty success.InProcessOnlyError— the door hands back a live in-process object (a plugin, a storage adapter). A wire can carry what such an object reports; it cannot carry the object. The refusal names what to call instead.
Both appear in the door's MCP tool description as well, so an agent reads the refusal before it writes the call.
Every door's response, answer or refusal alike, also carries the server-side measured wall of that call — Server-Timing: brainy;dur=<ms> on HTTP (and _meta.ms merged into a JSON body), brainy-ms in a gRPC response's metadata, and meta.ms on a WebSocket answer or an SSE stream's closing caught-up frame — one clock, taken once in the dispatcher, never a second timer per door or per transport.
brainy host names, on every engine refusal, whether waiting is worth it. Every BrainyError a door throws carries its own retryable boolean (api-contract.md §11); brainy host's dispatcher asserts it into the wire envelope explicitly for every transport it serves — error.retryable in the HTTP and WebSocket-session JSON body, and in MCP's structuredContent.error.retryable — so a caller reads the same fact wherever it is listening. HTTP additionally answers a Retry-After header (whole seconds) exactly when retryable reads true: the refusal's own estimate when it carries one (the write-band ceiling's cadence, the metadata index's integrity check, estimated from the bytes it still owes), otherwise a fixed 2 seconds (src/host/refusals.ts, retryAfterSecondsFor). It is a cadence to back off by, never a promise the underlying job finishes by then; the refusal's own message names the real wait (whenServing({ families: [...] })) or the operator cure (repairIndex(...)). Neither the header nor the field changes any status code a consumer already maps — GraphIndexNotReadyError, MetadataIndexNotReadyError and VectorIndexNotReadyError still answer 503 whichever way retryable reads.
The contract door
GET /contract.json serves a generated envelope — {service, version, builtAt, contractVersion, layers} — describing every door on every transport, every refusal class, and the server's own doors. It follows the fleet's contract-door standard: generated from the table that enforces it, stamped with the build that serves it, and pinned by the server's own suite against a committed copy.
Consumers vendor it at build time, commit the copy, and diff it in CI. A difference is a red build, never silent drift.
GET /agent and GET /llms.txt sit beside it. They are onboarding prose for an arriving agent — how to get a key, where the contract door is, how to connect over MCP. They are not the contract, and they never substitute for it.
Names and defaults
Contract service name |
|
Contract envelope |
|
Onboarding |
|
Engine doors |
|
SSE change feed |
|
WebSocket |
|
MCP streamable HTTP |
|
MCP stdio |
|
MCP instructions |
|
MCP resources | the store's |
The starter |
|
gRPC |
|
Default bind |
|
Listener flag |
|
h3 certificate |
|
Brains |
|
Keys file |
|
Service key | the admin doors (health, diagnostics, repair, snapshot, warm, stats) |
First start, no keys file | mints one key and prints it once |
Issuers file |
|
Minted key |
|
Client resolution order. The fixed LAN name brainy.example.internal (a hosts line, DNS once there is more than one), then the public https://brainy.soulcraft.com, then BRAINY_URL last.
Transports
The order these landed in, and why.
1 · HTTP/1.1 and HTTP/2, from day one. Both on the same listener. TLS is optional and the host terminates it at the edge; the server speaks cleartext h2 on a private address, which is what a LAN wants and what removes a certificate from the fleet's critical path.
2 · MessagePack, negotiated per request. JSON is the default. Accept: application/msgpack returns MessagePack and Content-Type: application/msgpack accepts it — the same documents, one serde model, two encodings. Vectors and bulk pages are the reason: a 1,024-element vector of doubles is 9,227 bytes of MessagePack against 16,256 of JSON — a 1.76x saving (MEASURED, the codec's own a_vector_is_smaller_in_msgpack pin), because JSON spells every float in decimal and MessagePack does not. A page of a thousand rows carries that difference a thousand times, and the segment log already stores its records as MessagePack, so wire and store share a codec.
The saving is smaller than the arithmetic on a raw f32 vector suggests, and the reason is stated rather than rounded away: the document model holds numbers as f64, so a vector element costs nine bytes on the wire where the engine stores four. Serving vectors as f32 roughly halves that again; it lands with the doors that return vectors, where the width is part of the answer's shape.
Every door round-trips identically in both encodings, and a pin holds that. A body whose bytes do not decode cleanly — including a JSON body sent to a MessagePack door, which would otherwise read as the integer 123 and discard the rest — is refused by name rather than half-read.
3 · gRPC, generated from the contract. --listen host:port,grpc serves it. The .proto is generated at build time from the contract — one rpc per door, named by the door, with the service implementation generated beside it — so there is no list of methods anyone can forget to update, and a pin holds the gRPC surface equal to the HTTP surface door for door. docs/brainy.proto is the committed artifact a consumer runs protoc over to get a typed stub in Python, Go, Java or C#.
A door is typed by its name, not by per-door fields. The contract carries TypeScript signatures, not field schemas, and deriving protobuf fields from a signature string would be a guess a generated client's compiler would happily bless — so one request and one answer message carry the brain, the id and the door's document as the same JSON everything else speaks. Typed messages arrive when the contract grows real parameter schemas.
A door's refusal is carried in the answer, never as a gRPC status. A status is a transport verdict and a refusal is an engine answer; a caller that must tell "the network failed" from "the write path is not native yet" cannot if the second is folded into the first. Only the transport's own failures are statuses — no credential, and a body the server cannot read.
Every rpc is unary for now: the change feed's held tail is served as server-sent events over HTTP/1.1 and HTTP/2, and no server-streaming rpc carries it yet.
4 · HTTP/3 over QUIC. --listen host:port,h3, with --tls-cert and --tls-key. The same doors, the same encodings, the same pipeline — HTTP/1.1, HTTP/2 and HTTP/3 differ in how they frame a request and not at all in what a request is, so no transport carries its own copy of the policy.
The certificate is the host's. QUIC has no cleartext mode: TLS 1.3 is part of the protocol, not a layer over it. This server mints nothing — one it generated itself would be one no client could verify and every client would be taught to skip verifying, which is worse than no TLS because it looks like TLS. Naming an h3 listener without both files refuses at startup, and deliberately does not fall back to h2 on that address: a host that asked for h3 and silently got h2 would advertise h3 to its clients and be wrong. Every certificate problem is named separately — missing, unreadable, a PEM with no CERTIFICATE block, a PEM with no PRIVATE KEY block, a key belonging to a different certificate — because "TLS failed" is the least useful thing a server can say at startup.
ALPN offers exactly h3; offering more would let a client negotiate a protocol the listener does not serve. Alt-Svc is advertised from the h1/h2 answers only when an h3 listener actually exists, since advertising an alternative that is not there teaches a client to spend a round trip failing. Shutdown closes the endpoint and waits for idle rather than cutting streams, which to a client would look exactly like a network failure.
Protocols are per-listener, and that is the only knob. A host enables h3 only where it helps — a public edge serving browsers and mobile — while the LAN runs h2 with MessagePack. One server, one door table, a listener list the host names.
MCP
Every door is a tool. MCP is a transport here, not a curated selection, so a client that speaks only MCP reaches exactly the engine an HTTP client reaches — and the two land on the same dispatcher, so neither can serve a method the other does not.
tools/list is the generated table, whole. A tool's input schema is generated from its binding: the path parameters it names are required strings, and a door whose contract signature declares parameters also takes body. It does not claim to know the body's fields — the contract carries signatures, not schemas, and naming fields from a type name would be a guess wearing a specification's clothes. The signature travels verbatim in the description instead.
A door's refusal is a result, not a protocol error. A write that is not native yet, or a key that does not reach a brain, comes back as tool content the model reads, with isError: true and the machine-readable class in structuredContent. A protocol error is invisible to a model, and a refusal an agent cannot read is a refusal it will retry forever. Protocol errors are kept for what they are: a malformed request, an unknown method, a missing argument.
Resources are records with addresses: a brain's census, an entity, an entity's history. Every template names the door that reads it, and reading one calls that door through the same path everything else uses. A resource whose read has no implementation is not offered — which is why there is no relation resource: the contract has no door that reads one relation by id, and related is a query, not an address.
Over stdio the process runs AS a principal: BRAINY_KEY names the key, read from the environment and never from an argument, because an argument is visible to every process on the machine. Nothing but JSON-RPC goes to stdout.
The server teaches its own use. Each tool description carries the contract's summary and signature AND one generated line about when to reach for that door; initialize.instructions carries the store's own ceremony from /_brainy/mcp/instructions.md, or a compact default measured against a 700-token budget; and the store's /skills/ documents are served as resources at brainy://skills/<path>, so improving a skill is a write rather than a release. The protocol revision is 2025-06-18, one revision and not a range, refused by name at the MCP-Protocol-Version header. The whole model — connect snippets, the title and description rules, what refuses today — is MCP.
Measured
Conditions. Job serve-measure2, exclusive lane on a 32-core box, load average 2.7 at start, release build, loopback, a sparse-preserving copy of the the venue store (7.1 GB allocated, 158 GB apparent) — never the archive. 600 requests per configuration (640 at c=64). Percentiles are nearest-rank. Every figure is served latency from a caller's socket: the whole cost of an answer including framing, not the engine's time in isolation.
door | transport | enc | c=1 p50 | c=8 p50 | c=64 p50 | c=64 p95 | best answers/s | bytes/answer |
|---|---|---|---|---|---|---|---|---|
| h1 | json | 0.068 ms | 0.087 ms | 0.536 ms | 1.042 ms | 96,629 | 156 |
| h2 | json | 0.028 ms | 0.070 ms | 0.418 ms | 0.535 ms | 145,693 | 29 |
| h2 | msgpack | 0.027 ms | 0.066 ms | 0.410 ms | 0.551 ms | 147,478 | 19 |
| h1 | json | 0.079 ms | 0.131 ms | 0.596 ms | 0.843 ms | 95,645 | 687 |
| h2 | json | 0.040 ms | 0.103 ms | 0.854 ms | 1.207 ms | 75,190 | 559 |
| websocket | json | 0.022 ms | 0.204 ms | 1.616 ms | 1.929 ms | 44,548 | 608 |
paged read | h1 | json | 2.713 ms | 2.765 ms | 9.287 ms | 13.736 ms | 6,375 | 48,273 |
paged read | h2 | json | 2.372 ms | 3.019 ms | 9.853 ms | 14.823 ms | 6,219 | 48,143 |
paged read | h2 | msgpack | 2.355 ms | 3.003 ms | 9.476 ms | 13.720 ms | 6,227 | 39,661 |
The gRPC row is withheld pending re-measurement: the run that produced this table still carried a transport defect on that path (tonic's tcp_nodelay applies only to a listener tonic itself binds, never to an incoming it is handed), and a number taken with a known defect in it is not a measurement. The fix is in and pinned; the row returns with the first sweep taken on the exclusive lane after it, and not before — a figure measured beside other work is not a receipt.
What MessagePack actually buys, plainly: bytes, not latency. A get answer is 19 bytes against 29, and counts 551 against 559 — and at those sizes there is no latency gain to speak of (0.027 ms against 0.028 ms at c=1). The saving is real only where the payload is: the 48 KB paged answer drops to 39.7 KB, about 18%, with p50 within noise of JSON. Choose it for bandwidth on large or bulk answers, not for speed on small ones.
Two defects this measurement found, both invisible to a green unit suite and both now fixed and pinned:
Every h2 and gRPC answer carried a 40 ms delayed-ACK stall —
TCP_NODELAYwas unset on accepted sockets. HTTP/1.1 escaped it because its writes coalesced past the interaction, which is exactly why latency has to be measured per transport rather than inferred from one.The WebSocket answered nothing — 600 of 640 requests lost. The HTTP/1.1 upgrade was never performed, so every handshake succeeded and every frame after it went nowhere.
Sovereignty
It binds to the address the host names.
127.0.0.1:8300when nothing is named, and never0.0.0.0by default.It never contacts any host. Not for weights, not for licence validation, not for telemetry.
Keys are files the host places, mode
0600. The engine mints nothing — except a first start with no keys file, which mints one key and prints it once.A minted
bk1token is verified, never minted, by this server: the issuer — Soulcraft Identity, or a self-hoster's own minting service, itself run throughbrainy-serve issuer keygen/token mintas the reference implementation — signs it; this server only checks the signature against a key in the issuers file, another file the host places at mode0600. See Minted keys.There is no UI, no login federation, and no generative model in this binary. The language seam is configured to the host's provider.
The language provider is resolved from the process's environment at startup, in the same precedence a bare new Brainy() uses on the Node side — BRAINY_LANGUAGE_PROVIDER, then exactly one recognized vendor key, then a local model server on a well-known port, then none. A misconfiguration (two vendor keys with no explicit choice, or an endpoint the enterprise outbound gate refuses) stops the server before it binds anything. See docs/language-providers.md for the full matrix, every environment variable, and every refusal.
Keys and principals
A key names a principal — a person, a business, or a service — and the brains that principal may reach. The service key is separate: it reaches the engine mechanics (health, diagnostics, repair, snapshot, warm, stats) on every brain and reaches no principal's rows.
Every request carries a principal or the service key. Refusals are typed and named.
The bare service key is never the author of a write. A raw write (add, update, remove, the batch and relation doors, transact), a file write (any vfs write op) or a memory write presented with it refuses ServiceKeyCannotWriteError (400), the host's refusal by the host's name: it answers engine mechanics on every brain and speaks for nobody, so a row or a file it wrote would carry an author that is not a person. It still calls every read and every mechanic. The host lets a service that names a caller (X-Brainy-Caller) write as that person; this server does not read that header yet, so a service key that names one is refused here as well, and the refusal says so.
Minted keys
A keys-file line is one durable secret every holder of it shares — right for a service acting as itself, wrong for an arriving AGENT: anima, an outside provider's own agent, a third party, each of whom should carry their OWN credential, scoped to one brain and to read or read/write, revoked by expiry rather than by editing a file everyone else's key also lives in. A minted key is that credential: a short-lived, signed claim an ISSUER hands the agent, which this server VERIFIES rather than looks up. The engine still mints no bearer credential of its own for anyone but the stranger at first start — an issuer is always an external identity.
The format
bk1.<base64url(payload JSON)>.<base64url(ed25519 signature over the payload bytes)>TEXTPresented the same way a static key is: Authorization: Bearer bk1.eyJ…. The payload:
Field | Meaning |
|---|---|
| The issuer id — a line in the issuers file. |
| Optional. Which of that issuer's keys signed this, for rotation. |
| The agent or user id this token names. Opaque to this server. |
| The one store this token opens. One path segment. |
|
|
| Issued-at, Unix seconds. |
| Expiry, Unix seconds. Enforced with a 60-second clock-skew allowance. |
| Optional. Checked against the host-placed revoked list when present; otherwise carried, never tracked for replay by this server. |
| Optional. An array of non-empty strings — the roles this bearer holds. Absent means no claim at all, |
The signature covers the exact bytes of the decoded payload segment — never a re-serialised or canonicalised form — so a verifier hashes what it was handed and nothing else.
The issuers file
~/.config/soulcraft/brainy-serve-issuers (or $BRAINY_SERVE_ISSUERS), mode 0600, placed by the host — refused before it is read if the group or the world can read it, the same law the keys file already holds. One line per key:
# <issuer-id> ed25519:<base64 public key> [kid]
soulcraft-identity ed25519:MCowBQYDK2VwAyEA… 2026-09
self-hosted-issuer ed25519:MCowBQYDK2VwAyEA…TEXTThe optional third field names the key for rotation: a token whose kid matches a line is checked only against that line; a token with no kid is checked against every key its issuer has, so a fleet can add a new key before retiring the old one instead of every held token failing at the moment of rotation.
A missing issuers file is not a refusal to start — it means this server trusts no issuer, so every bk1 token is answered BRAINY_KEY_UNKNOWN_ISSUER, the same "not placed yet" story an empty keys file already tells. The file is re-read when its bytes change, checked at most once every 5 seconds — never per request, never a filesystem watcher (SIGHUP, inotify): a rotated kid is honoured within that window, with no restart. A file that exists but fails to parse refuses this server's START the first time; once running, a later edit that fails to parse is logged and the last good table is kept rather than dropped.
A per-issuer entitlement ceiling is planned but not yet enforced. A self-hosted door that mints its own tokens (brainy host --sign-in, see docs/host.md) is meant to be limited to entitlement absent or "expired" — never "suite", which only Soulcraft Identity's own issuer id may carry. Neither verifier (this one or the TypeScript host's) checks that limit today; every issuer in this file is trusted for either value until it does.
The owner scope
The owner triple — ["read","write","owner"], in any order — is the OWNER's own credential: minted by a human sign-in only, never by an agent's key. It implies the full read/write grant and additionally may call the three doors that RULE on guardrails rather than merely mutate a brain's own rows: guardrails.ratify, guardrails.unlock and guardrails.freeze (activating a proposed rule, lifting a lock, freezing every tier-1 rule). Every other guardrail door — guardrails.check, guardrails.declare, guardrails.propose, guardrails.retire, guardrails.audit — is unaffected: an agent may tighten its own leash, propose anything else, and read what the rules stopped, from any credential.
Calling one of the three ruling doors with anything short of the owner triple — a plain ["read","write"] token, a ["read"] token, or a keys-file static key (whose scope is unconditionally None, since the keys file carries no owner marker of its own) — refuses GuardrailOwnerScopeRequiredError (403), named by the door and checked before that door's own disposition, the same "authorisation before disposition" order every other refusal in this section already keeps. There is no narrower "owner, read-only" shape: a ruling either carries the full triple or the array is not this scope.
The revoked list
A minted token is a bearer credential good until its own exp — this server contacts nothing outside itself to learn anything sooner (see Sovereignty above). To revoke one early — a leaked token, an offboarded agent — the host places revoked.json beside the issuers file (--revoked <path>, or the issuers directory's brainy-serve-revoked by default), copied VERBATIM from auth's own public feed by the deployment; this server never fetches it itself:
{ "revoked": [ { "jti": "3f9a…", "exp": 1799999999 } ] }JSONexp is the revoked token's own expiry, carried so a stale entry can be dropped once the token would have expired on its own terms anyway — it is not a deadline on the revocation. A token whose jti is listed, and still inside that entry's own exp, refuses BRAINY_KEY_REVOKED (401) at the same ladder position BRAINY_KEY_BAD_SIGNATURE holds: revocation is asked only of a payload already proven the issuer's own.
Like the issuers file, this one is re-read when its bytes change, checked at most once every 5 seconds, no restart. A missing file means no jti this server is shown is ever revoked — a legitimate, documented state, not a refusal. A present file that fails to parse refuses this server's START the first time; on a later reload it is logged and the last good list is kept.
The revocation's latency of record, end to end, is the sum of three numbers, none of them this server's alone: the feed's own max-age (however long auth is willing to serve a stale answer), the host's own pull cadence (however often the deployment copies the feed into revoked.json), and up to 5 seconds for this server's own check window. A self-hosted deployment that runs no such pipeline places its own revoked.json by hand, or none at all — an absent file is a legitimate choice, not a missing feature.
The refusals
Eight classes, none of them a contract door's own — the same family UnauthenticatedError, ForbiddenError and the transport's other local refusals already belong to, documented here because this is where a caller learns to expect them rather than in the generated engine contract, which knows nothing about bearer credentials:
Class | Status | When |
|---|---|---|
| 401 | The bearer does not parse as |
| 401 | The token's |
| 401 | No key this issuer is trusted for verifies the signature. |
| 401 | The token's |
| 401 | Past |
| 403 | The door mutates the store and the token's scope is |
| 403 | The door addresses a brain other than the one the token names. |
| 403 | The door rules on guardrails ( |
A missing or unrecognised bearer of ANY shape (no header, a non-bearer scheme, an unknown static key) still answers the existing generic UnauthenticatedError — deliberately: a static key is a secret an attacker can probe near-misses against, and a specific refusal there would be a hint. A bk1 token is signed, not secret, so there is nothing to be gained by hiding which of the checks it failed and real operator value in being told which.
Every transport answers the same way: HTTP/1.1, h2 and h3 in the JSON or MessagePack body (error.name); gRPC as UNAUTHENTICATED for the five identity-time classes — BRAINY_KEY_MALFORMED, BRAINY_KEY_UNKNOWN_ISSUER, BRAINY_KEY_BAD_SIGNATURE, BRAINY_KEY_REVOKED, BRAINY_KEY_EXPIRED — the message text names which, since gRPC's status is coarser than HTTP's, and as the ANSWER's own refusal — the same shape ForbiddenError already uses — for BRAINY_KEY_SCOPE, BRAINY_KEY_BRAIN and GuardrailOwnerScopeRequiredError, which are known-identity refusals, not credential failures; MCP and the WebSocket session both ride the same HTTP-shaped body their other refusals do. Authorization: Bearer bk1.… works identically over MCP, including stdio via $BRAINY_KEY.
The operator commands
Reference implementations — a self-hoster's own issuer, or a test rig; Identity mints its own keypairs and tokens the same way, never by shelling out to this binary:
# Generate a keypair. Prints the issuers-file line; writes the PRIVATE key
# ONLY to --out, mode 0600 — never to stdout.
brainy-serve issuer keygen --issuer soulcraft-identity --out ./issuer.key
# Mint a token. Reads the issuer id (and kid, if any) from the file above.
brainy-serve token mint \
--issuer-key ./issuer.key \
--brain wnw \
--sub agent:anima \
--scope read,write \
--ttl 3600BASHA worked example
$ brainy-serve issuer keygen --issuer example --out /tmp/example.key
[brainy serve] private key written to /tmp/example.key at mode 0600 — never printed.
[brainy serve] append this line to the issuers file:
example ed25519:8f2c1a9e4b7d0f3c6a5e8b1d4c7f0a3e6b9c2d5f8a1e4b7c0d3f6a9e2b5c8d1f
$ echo 'example ed25519:8f2c1a9e4b7d0f3c6a5e8b1d4c7f0a3e6b9c2d5f8a1e4b7c0d3f6a9e2b5c8d1f' \
>> ~/.config/soulcraft/brainy-serve-issuers
$ chmod 600 ~/.config/soulcraft/brainy-serve-issuers
$ brainy-serve token mint --issuer-key /tmp/example.key --brain wnw \
--sub agent:anima --scope read --ttl 600
bk1.eyJpc3MiOiJleGFtcGxlIiwic3ViIjoiYWdlbnQ6YW5pbWEiLCJicmFpbiI6IndudyIsInNjb3BlIjpbInJlYWQiXSwiaWF0IjoxNzU3MjAwMDAwLCJleHAiOjE3NTcyMDA2MDB9.MEUCIQ…
$ curl -H 'Authorization: Bearer bk1.eyJ…' http://127.0.0.1:8300/v1/brains/wnw/countsBASH(Keys are throwaway; the payload above is illustrative, not a real signature.) A running brainy-serve serve picks up the new issuers-file line on its own — within 5 seconds of the file's bytes changing, no restart needed.
The writer model, and the pool
One writer per store, and the lock decides. The writer of a store is whoever opens it read-write; the store's lease admits exactly one; every other attach is read-only. That is one law for the whole engine, not a server policy: the lease is locks/_writer.lock, its clean-close record is locks/_writer.close, and the claim, the staleness verdict and the fence are native/src/brain/writer.rs — the same module Brainy.init() reaches across a napi boundary in a Node process and this server links directly as a library. Two implementations of one lock is how a store ends up with two live holders, which is a class of incident this engine has already paid for once.
One holder per brain inside this process. The server is the holder, and every door goes through the pool. There is no second way to open a store here: a reader opened beside the pool would be a second holder of a single-writer store.
The writer lease, and which doors move it
POST /v1/brains/{brain}/open — the init door — is the read-write open, and the only door that takes a lease. It is service-key only. It answers with a writer block that says which role this server got:
It won the lease.
role: "writer", the lease's own pid, hostname,startedAtand heartbeat, andclaimPath— how it came by it:fresh(no lease was there),leftover-after-clean-close(bookkeeping an orderly shutdown could not remove — nothing to recover),stale-after-crash(the previous writer left no close record, so this open owes recovery and says so),forced(an operator's{"force": true}, never an inference).Another live process holds it.
role: "reader", the holder named by pid and host, andrefused: "BRAINY_WRITER_LOCKED"— the engine's own code. The store is still served for reads. This server never queues behind a holder, never polls for one to leave, and never downgrades silently: the refusal is in the answer, by name, and a second warning says the open went read-only.
A read never claims. The pool opens a store for whichever door needs it first, and almost every door is a read; a pool that claimed on admission would make this server the writer of every store it was asked to read from.
POST /v1/brains/{brain}/close gives the lease back and records the orderly close, then drops the pool's handle — the canonical reader, its descriptors and its mmap regions. A brain another door is still answering from stays alive until that door finishes. The recorded close is what lets the NEXT open know the store was closed cleanly as a fact rather than infer it from a pid, so an orderly stop owes it: the server also releases every lease it holds on shutdown, after the listeners drain, and narrates each one.
The lease outlives a residency, deliberately. Eviction is a memory decision; the writer role is not. A store this server is the writer of stays a store this server is the writer of across an eviction and a readmission — otherwise the arbiter would be handing the writer role to whatever process claimed next.
A claim after a crashed in-process writer answers at once and replays behind the doors. When the previous writer was the in-process engine and did not close cleanly, the projections (metadata, column, graph, vector) stand behind the log's head with nothing to attest them, and bringing them level is work that grows with the store (MEASURED, a 4.1 GB venue tenant: 395 to 431 seconds as the claim's own wait). A window of at most 200 ids is still replayed inside the claim. A larger one is not: init answers as soon as the lease and the log are held, the replay runs on its own thread, and until it has finished
reads of the brain are served from the attach;
every write is refused with
ProjectionReplayRunningError(HTTP 503,retryable,retryAfterMs, and the projection,doneandtotalids andelapsedMs), andinitandhealthsay how far the replay is (aprojections-replaying-behind-the-doorswarning and awriter-replaycheck). A replay that itself refused answersProjectionReplayRefusedError(HTTP 500, not retryable): close the brain and claim it again, and the claim plans the replay from the projections' marks anew.
Closing the brain, or stopping the server, stops the replay at a slice boundary and leaves the marks where they were, so the next claim owes the same replay and nothing is counted twice.
Rows the crashed writer acknowledged and never mirrored into the log are counted and named. The in-process engine acknowledges a write when its entity tree and write-ahead logs hold it, and mirrors its commits into the segment log at a flush or a close; a kill in between leaves rows in the tree that this server, which serves from the log alone, does not see. When the lease records an unclean close, a census of the tree against the log runs behind the doors, and init (asked again once it has finished) and health answer rows-acknowledged-by-the-previous-writer-are-not-in-the-log with stranded: { entities, relationships, firstEntities, firstRelationships, treeEntities, treeRelationships, censusMs }. Nothing is lost - the rows are in the tree - and today's cure is to open the store once with the in-process engine and close it cleanly, which mirrors them into the log, and then claim it again.
What a lease buys, and what it does not, on this build
This build answers flush on the wire, eleven ENGINE write doors (add, the create path; update, the overwrite; remove; their batch forms addMany, updateMany and removeMany; the relationship doors relate, unrelate, updateRelation and relateMany; and transact) and one record-profile write door (memory.turn) — the doors listed below still refuse by name with the release they arrive in, because the core's executing write path for them is still moving in. add commits a row through the log and the four projections, and the row reads back — through this server's own get, which reads a log-canonical store's log, and through a cold in-process reopen of the same store — pinned end to end on the real binary against an in-process add of the same payload (src/native/serveAddWire.e2e.test.ts). An add on an id the store already holds is an OVERWRITE and refuses NotYetNativeError by name, naming the id and pointing at update.
update reads the row as it stands from the log by its id, decides the merge in the same core the engine calls in process, and commits it as a removal of the view the row was indexed under followed by the new one — so the row keeps its dense int and every structure keyed by it. It answers what the host answers ({ value: null, brain }), refuses a stale ifRev with RevisionConflictError (409) and an unknown id with EntityNotFoundError (404), and writes nothing on either. What it does not serve yet refuses NotYetNativeError by name before any effect: new data with no explicit vector (re-embedding needs the embedder), deferEmbedding, retention, an empty or all-zero vector, and a row whose embedding is still pending.
remove reads the row as it stands from the log and rebuilds the view its indexes retract from that STORED record — never from anything the caller sent — then commits one tombstone. It answers { value: null, brain }; an id the store does not hold is an answer too, and nothing is written for it. A row that is an endpoint of live relationships leaves with them: every relationship it is an endpoint of is removed in the same commit, as the engine's own remove() deletes them.
The batch forms run each row through the single door's own path and commit once per CHUNK (chunkSize, else 500, the filesystem store's own batch size): each chunk is one durable transaction — one generation — that lands whole or not at all. That is the engine's own boundary for removeMany; for addMany and updateMany it is a difference from the engine BY DESIGN — in process each of their rows is its own generation — so a change-feed reader sees one commit per chunk from this server. An updateMany naming one id twice in a chunk cuts the chunk before the repeat, and an addMany naming one id twice refuses the second as the overwrite it would be in sequence. Each answers the engine's BatchResult — { successful, failed: [{ item, error }], total, duration } — and a refused row has no effect. With continueOnError: true a refused row is listed in failed and the rest carry on; without it the batch STOPS at the first refused row and the call refuses with that row's own error, carrying landed (every row that committed, and stays committed), stoppedAt and notLanded, so a caller always knows exactly which rows landed. For removeMany the chunk holding the refused row does not commit. Rows run in the request's order: the engine's parallel mode, under which continueOnError: false does not stop an addMany or an updateMany, is not copied. removeMany by { type, where } refuses NotYetNativeError by name; by ids an id the store does not hold counts as removed and writes nothing, as in process.
Every native write applies the engine's write rules before any effect: the brain-wide subtype rule (on by default in the engine, and the only setting this server has) refuses a row with no subtype as ValidationRefusedError, VFS rows excepted; a row whose (type, subtype) a profile the store declares governs is checked against it and refuses RecordProfileError with the rule and the field. addMany checks every item before its first row, so a batch that breaks a rule refuses whole — with itemIndex and landed: [] — and nothing lands. Rules a process registers only in its own memory (requireSubtype(type, …), trackField) bind that process alone and are not applied here.
memory.turn decides through the SAME core the engine calls in process and lands through the SAME committer add's own arm does (docs/profile-writes-in-rust.md §9). Each write door is flipped only once its own read-back and wire pin are proven.
A write needs two things, and a refusal names which is missing: this server must hold the store's WRITER LEASE (init takes it) — without it a served write door refuses NotTheWriterError (409) — and the store's canonical must be the segment log with its one-way log-canonical mark — otherwise it refuses CanonicalNotWritableError (409). A tree-canonical store's records are written by the engine's own TypeScript storage adapter, and a migrated store still running as a mirror owes that adapter its tree; no native committer changes either. Both refusals carry the store's canonicalMode and the sentence that says what it means for writes.
So a store whose lease this server holds can be written through the doors above and through nothing else until it is closed. That is not left for an operator to discover from a failed write:
init's answer carries awriter-holds-this-storewarning naming what this build can write, whether this brain's write set is open, and the release the rest arrive in — for both families, ENGINE and record-profile, each its own field (writeDoors,profileWriteDoors).The per-brain
healthdoor carries it as awrite-doorscheck (the ENGINE doors) and aprofile-write-doorscheck (memory.turnand its kin), each naming its own served and refusing doors —passonly when a served door in that family exists AND this brain's write set is open,warnotherwise — and the store's writer as awriter-leasecheck —passwhen this server holds it and the fence still holds,warnwhen another live process does (the ordinary fleet shape, not a fault),failwhen the lease exists and cannot be parsed (this server will not judge a lease it cannot read) or when this server's own fence has been lost.GET /v1/healthreportspool.writersandpool.writerBrains— how many of this population's stores this process is the writer of, and which — and awriteDoorsblock naming the served and the refusing ENGINE doors, read from the same binding table the per-brain check reads. Two lists is how a door comes to answer while a health check says it refuses.
Brains are named at startup (--brain <name>=<path>, repeatable) and none is opened until a door needs it, so a server holding a thousand names starts in milliseconds and pays for the stores it actually serves.
The budget is derived, never configured. The control-group limit when the process runs under one, otherwise total memory; the kernel's availability figure; and this process's own resident set, which is not the population's. A fifth of the reachable limit is held back, floored, because an arbiter that budgets to the last byte evicts on a schedule set by the kernel's out-of-memory killer. A box that publishes neither a control-group limit nor a memory total is floored and says so. There is one environment override and reading it is narrated every time, naming the number it replaced.
Eviction is a clean close; readmission is an attach. The whole handle goes and everything it held is released; the next door on that brain reopens it and waits — it is never refused. A brain is closed when it has been quiet for sixteen of its own measured access intervals (floored at thirty seconds so it is never closed between two phases of one request, capped at thirty minutes), or when the population is over budget, largest measured footprint first.
A brain with a door in flight is never evicted — excluded, not ranked last. A brain answering a request is not idle whatever it costs.
GET /v1/health reports the arbiter: what it may hold, what it holds, how many brains it knows, and whether the budget was floored or overridden.
Read-only attach
The file doors — vfs.readFile, vfs.readdir, vfs.stat, vfs.exists, vfs.getxattr, vfs.listxattr, and through them the MCP skills and the store's own instructions — read through a read-only attach. It is worth saying exactly what that is, because it is not a read-only mode flag:
Nothing is claimed and nothing is written. No writer lock is taken by the attach itself — the lease is
init's business and nothing else's, so a server that was never asked to open a store read-write never touches its lock directory. No directory is created, no log is grown or rotated, no archive is swept, no torn artifact is quarantined, no census is written, and no background job starts. Dropping the handle releases every map; there is nothing to close, because nothing was claimed.It judges servability by the OWNING open's law, not by its own. A reader with a second opinion about whether a store may be served is a second truth.
It refuses; it never repairs. Every write the owning open makes is correct because that process owns the store. A reader that quarantined a torn SSTable would be deleting evidence from a store another process is writing, and one that created a missing id mapper would turn a lost mapper into "this store is empty". So a store that cannot be served exactly as it stands refuses by name, carrying the reason: a writer handoff in flight, a torn SSTable, a record profile from a newer writer, an unreadable generation stamp, a metadata projection nothing attested.
The unflushed tail is part of the read. Mapping only the sealed SSTables would answer "no such file" for everything written since the last flush.
One generation per attach
An attach is frozen at the generation it resolved, for the whole life of the brain's residency. A file written after it is invisible to it — not stale for a while, invisible.
Re-attach is how a reader moves forward, and it happens exactly where the pool's lifecycle already happens: a brain that is evicted and readmitted attaches again, at whatever generation is serving then, and a restart does the same. There is no refresh timer and there will not be one. A reader that re-resolved its generation under a caller would answer two requests of one session from two states of the store, and nothing in either answer would say so.
The attach itself is paid by the FIRST file door on a residency, not by admission: a read by id needs the canonical reader and nothing else, so opening a brain does not map a metadata read set that only the path doors read through.
The counts door follows the writer's commits
_system/counts.json is written by the TypeScript engine alone, and it stamps the file with ledgerGeneration: the store generation its numbers are exact through. A writer that commits to the log — brainy-serve or brainy host — does not move that file, so on a log-canonical store the file is behind the log by exactly the commits made since the stamp. The counts door does not answer the file as it stands. It brings the numbers to the log's head first:
It walks the commits above the stamp. Every row those commits touched is read twice from the log — as it stood at the stamp and as it stands now — and the first image is forgotten and the second counted, by the engine's own counting rules: a hidden tier (
internal,system) counts in the all-tier totals only; a type counts when the tier does; an entity counts as vectored when its vector has landed. A number never wraps below zero.The cost is those commits' rows, plus opening the log's views once per walk. The rows come from the log's order index with no record decoded, and each costs two point reads. Opening a view scans its family's active (unsealed) segment, and that part scales with the segment, not with the commits. MEASURED on a copy of a 17,880-entity store (a 12.8 MB active commit segment of 106,355 records and a 42 MB active entities segment), on a shared build lane, not a receipt: the first answer of a handle, which opens the commit partition, takes about 66 ms; an answer that walks 3 to 6 new rows takes about 135 ms; an answer that finds nothing new takes 0.7 to 1.7 ms, the cost of the file read it replaces. A handle pays the walk once for a given stamp and head, and when the head moves it walks only the new commits. A stamp that sits at the flip (below every commit boundary) reads each touched row's version chain instead, once: the ledger a writer persists afterwards starts every later walk at a known place in the log.
A writer persists the reconciled ledger beside the stamp. After a flush, at most once every 30 seconds while it runs, and once more at the close when a flush left it behind (a persist opens the log's views under the write-set lock, which costs the size of the log's active segments, not the commits: MEASURED 420 to 610 ms on a copy of a 104 GB brain, so one per flush held the lock for half of every second), under its own key (
nativeLedger) insidecounts.json, tagged with the stamp it was derived from. It writes nothing when no commit landed since the last persist, and it never rewrites a key the engine owns: the engine reconciles its per-type statistics from that very stamp when it opens, so a stamp moved past the commits would make it skip them. When the engine rewrites the file the key goes with it, and the tag can no longer match.It never answers a stale file as if it were current. A ledger that cannot be brought to the head comes back with
allCountsSuspect: trueand the reason inallCountsSuspectReason: no stamp (a session that did not close cleanly), a stamp below the flip, a stamp ahead of the log, a record that will not count, a total that would go below zero.
The document gains three fields, all additive: ledgerGeneration (the store generation the numbers are exact through, null without a stamp), allCountsSuspectReason (null when nothing is in doubt) and reconciled (null when the file answered as it stood; otherwise fromGeneration, throughGeneration, rows, forgotten, noted, startedFrom — stamp or native-ledger — and placement — log-window or version-chain).
What the file doors answer
readFile resolves the path through the store's own path posting — never a directory walk — reads the content-addressed blob, and verifies the bytes against their own address before returning them. A blob whose bytes do not hash to their own address is corruption and is reported as such, never served. utf-8 is the default encoding and it REFUSES rather than decode lossily, naming the offset where text stops and base64 as the cure.
readdir answers a directory's DIRECT CHILDREN, which is the engine's own question. It reads the path keys that sit under the directory's prefix as one ranged seek of the index, keeps the keys with no further / in them, and reads the rows of those alone in one batch — never the whole path space, which is what it used to do (twelve seconds on the root of a 90,000-node brain, and a different question). It reads the path and not the parent field or the graph because the parent field is not kept in step when a file or folder is moved (one real brain listed 48 of its 1,781 directories differently by parent than by the engine's containment edges), and the path is: on that brain the listing by path equalled the engine's listing for every directory. recursive: true asks for every descendant, each entry named relative to the directory listed. A file refuses ENOTDIR and a path that holds nothing refuses ENOENT.
The options are the engine's ReaddirOptions, in the request's options object: withFileTypes, withMetadata (each entry carries its entity's metadata bag; EINVAL without withFileTypes, as in process), recursive, filter (type, pattern, minSize, maxSize, modifiedAfter, modifiedBefore — an instant in epoch milliseconds, since JSON has no Date), sort (size, modified, created), order, offset and limit, applied in that order. Two shapes refuse NotYetNativeError rather than answer differently: sort: "name" (the engine's display order, which follows the process's locale and which this server does not reproduce) and a filter.pattern that is a regular expression (the engine builds one from the glob; this server matches *, ? and ., as that expression does). Entries are documents. Without a sort they come in the byte order of the names they are reported under — a directory's children by name, a recursive: true listing by the relative path it reports — which is the order the engine answers too (docs/api-contract.md, "A folder lists in name order"); that default is not the refused sort: "name", and offset and limit slice it.
One option is this server's own: withContent: true carries each text file's bytes on its entry, up to a megabyte, so a client building a resource index resolves titles and descriptions from front matter in one call rather than one call per document; a file over that cap, or one whose media type is not text, is listed with its size and its media type and says why its bytes are not there. It is off unless asked for: a listing that reads every text file in the folder is not what listing costs.
stat answers one node's recorded type, media type, size, mode, owner ids and the four instants (atime, mtime, ctime, birthtime) without opening a blob at all; it does not carry connections, the count of a node's relationships, and it names the node by its recorded path where the engine echoes the path as asked. exists answers whether a path names a node, getxattr one extended attribute (null when the node has none by that name) and listxattr the attribute names, alphabetically. readFile of anything but a file refuses EISDIR, as the engine's does.
A path resolves as the engine resolves it: runs of / collapse and a trailing / drops, the root answers from its fixed id, and any other path is a read of the path posting — under its canonical spelling, then under the legacy spelling without its leading slash. A path two rows claim is answered with the row of the smallest internal number, as the engine's resolver answers it, so the answers a file manager shows do not change when it moves to the native server; a listing names every row, and health counts such paths and names the first few. A write onto one is refused by name (PathAmbiguousError) rather than landing on whichever row the index named first.
One door, multiplexed by op — and the write half named, not hidden
vfs is ONE door: POST /v1/brains/{brain}/vfs, with an op field in the request body naming the verb (readdir when a call names none). It was GET until 12.11.0 and the change is a wire change for anyone who dials the route by hand — the path is the same, the method is not. brainy serve's own MCP tool (brainy_vfs) and its resource listing are unaffected: they call the door through its binding, not through a method.
The door's whole vocabulary is declared on both binaries. What differs is who can serve which:
|
|
|
|---|---|---|
| served, from the read-only attach | served |
|
| served |
| served natively, through the brain's write set — see "The file writes served natively" below | served |
Those are two different facts and they travel under two different names. The one read this server has no attach door for (history) refuses UnknownVfsOperationError: it is not an op of THIS door at all. A shape of a served op that this build does not do yet refuses NotYetNativeError, naming the op and the shape — today two shapes of copy, below. Neither is ever a no-op and neither is ever an empty success.
The file writes served natively
The ten write ops take the brain's write set, not the read-only attach, so the server must hold the brain's writer lease (init first) and the brain must be log-canonical. The requests are the host's: { "op": "writeFile" | "appendFile", "path", "content" | "contentBase64", "options" } answering { op, path, bytes, generation } (bytes is what the caller sent); { "op": "mkdir", "path", "options": { "recursive", "mode", "metadata" } }; { "op": "rmdir", "path", "options": { "recursive" } }; { "op": "unlink", "path" }; { "op": "setxattr", "path", "name", "value" } / { "op": "removexattr", "path", "name" }; { "op": "rename", "oldPath", "newPath" } answering { op, oldPath, newPath, generation }; { "op": "move", "src", "dest" } answering { op, src, dest, generation }; and { "op": "copy", "src", "dest", "options": { "overwrite", "deepCopy", "preserveVector" } } answering { op, src, dest, generation }.
A new file in folders that do not exist yet is one commit: the folders, the file and every containment edge land together or not at all, where the engine makes a separate commit for each. The file's bytes go into the blob store first, outside the write set's lock, and the file's text (and each new folder's path) is embedded before the answer — the engine answers first and embeds on a background worker, which this server has no counterpart for. A write that the guardrail rules refuse leaves nothing behind, not even its bytes.
A rename or a move (the engine's move calls rename) is one commit whatever it moves: the row's new path, name, folder and clock, the containment edge from the old folder to the new one, and, for a folder, the new path of every file and folder under it. The row keeps its id, its text, its vector and its bytes; nothing is embedded and no blob is touched. A death leaves the old place or the new one, never a file listed in two folders or in none. The folder's tree is read and checked before anything is written: a file or folder under it whose own stored path disagrees with the place its edges put it at refuses VfsLegacyPathShapeError (409, nothing written), because moving it would recompute that path by string arithmetic against the old one and corrupt it silently. A copy is one commit too: the new rows, their edges and the folders made on the way. A copied file is a new row that names the same bytes, embeds its text again unless preserveVector keeps the source's vector, and carries the source's attributes and a copiedFrom naming it; a copied folder is made fresh and every file and folder under it is copied by the same rule (deepCopy: false makes the empty folder, and the folders above it, only).
Where the two binaries answer differently, on purpose:
writeFileover a directory refusesEISDIR. The engine accepts it and turns the folder into a file.mkdirwithoutrecursiveunder a file refusesENOTDIR. The engine makes the folder under the file.A malformed
contentBase64refuses (InvalidVfsContentError); the host's decoder skips what it cannot read and writes what is left.A guardrail rule sees the canonical path — on both binaries.
legal/x.txt,//legal//x.txtand/legal/x.txt/all name/legal/x.txt, and a rule fencing/legalholds for each of them. (The engine used to read the path as the caller spelled it, which let the first of those slip; it reads the canonical one now.) Arename, amoveor acopyis scoped by EVERY path it touches, source first and then destination, one rule check each: a rule fencing/legalrefuses a file landing inside it as it refuses one leaving it. Each op answers to its own name (vfs_rename,vfs_move,vfs_copy): a rule naming onlyvfs_renamedoes not bind amove, as in the engine.A store with no VFS root gets one, in the same commit as the first thing written under it, shaped as the engine's open shapes it (a
vfs-rootcollection ofsystemvisibility, owned byroot, embedded from/). A request that is refused leaves no root behind.appendFilenever turns a failure to read into a replacement. The engine's append takes every failure to read the file as "no such file" and writes the appended bytes alone over it. Here a file whose bytes the store cannot give refusesEIOand stays as it is; a folder refusesEISDIR. An append builds on the version it read: when other writers replace the file under it, it reads again (up to six times) and then refusesAppendConflictError(409, nothing written) rather than write over their bytes.removexattrleaves the file's vector alone. The engine's hands the whole entity back toupdate, including the stubvector: []a metadata-only read carries, whichupdatetakes as the instruction to unvector the row: after aremovexattrthe node is no longer found by meaning. Here only the attribute changes.A removal counts a history reference for the bytes it removes, before it commits. The engine on a log-canonical store counts none, so a working-state file moving on could delete bytes an older version still named. Nothing is deleted at all on a flipped brain until its blob history counts are recounted from the log (13.3): see
docs/flip-to-log-canonical.md, "What changes afterwards".healthcarries ablob-history-countsrow atwarnandinitablob-reclaim-withheldwarning while that is so.A folder moved into its own subtree refuses
EINVAL, and so does the root, which neither renames, moves nor copies. The engine writes the cycle (the folder's path becomes a descendant of itself and its edge hangs from a child), renames the root row and every path under it, and names the copy of the root's children by string arithmetic (/backupawhere/backup/awas meant).A destination whose folder is a file refuses
ENOTDIR; the engine hangs a containment edge from the file. A destination whose folder does not exist refusesENOENTbefore anything is written, for a rename, a move and a copy alike. The engine's rename takes the file out of its old folder FIRST and then fails to find the new one, leaving a file its old folder no longer lists (it still reads, andexistsstill says yes); the engine's copy adds the row first and leaves it behind, with its path, when the folder is missing.The moved row's
parentfollows it. The engine leaves the OLD folder's id in the row's bag after a move across folders; nothing reads the field, but it names the wrong folder.A copy takes a live reference to the bytes it shares. The engine's copy takes none, so the bytes' count stays one too low. MEASURED on the released engine: a file copied and its source then removed leaves the count at 0 while the copy names the bytes, and a working-state file that takes those bytes and moves on reclaims them; a fresh engine reading the copy then refuses
Blob metadata not found. The reference is taken before the commit and given back if the copy is refused, so a death in between leaves a count one too high (a leak a recount settles) and never one too low. A source whose bytes the store does not hold whole refusesEIOrather than make a second file nobody can read.A file copied into the root is listed. The engine's single-file copy makes no containment edge when the destination's folder is the root, so the copy exists and
readdir("/")does not show it.A file copied inside a folder copy keeps its subtype, and its
parentnames its own copied folder. The engine's folder copy drops each file's subtype andparent.Two shapes of
copyrefuseNotYetNativeError, by name:overwriteover a path that already holds something (the engine writes a SECOND row at the path, so two rows answer to one), and a folder that holds anything but files and folders, a link (the engine drops the node from the copy without a word; this door will not make a copy that is quietly smaller than its source).VfsLegacyPathShapeErroris a 409. The host leaves the class out of its status table and answers 503, "try later", for a refusal no retry can change.
An overwrite, retention: 'latest-only' and the sticky-class law are the engine's: once a path is working state a plain write refuses EINVAL in the engine's words; retention: 'latest-only' continues it and retention: 'history' leaves it. A working-state file is written without embedding it. An overwrite counts one history reference for the bytes it supersedes before it commits and releases the live one only after the commit is durable, so a death between the two leaves a count too high (a leak) and never too low.
Every refusal that both binaries make is the same refusal — the class VFSError, the POSIX code and the engine's words (EEXIST, ENOENT, ENOTDIR, EINVAL naming the engine's own metadata keys). The write scope is the op's: a token minted for read is refused every write op and admitted every read op.
readdir answers the same question, in the same order, on both binaries. Each lists a directory's direct children and, for recursive: true, every descendant with names relative to the directory, and each lists them sorted by name, byte-wise on the UTF-8 of the name each entry is reported under (the order docs/api-contract.md declares; an explicit sort replaces it). (This server used to answer every node under the prefix at any depth, and the engine used to list in the order a folder's Contains edges came back; those were the two places the two parted, and both are gone.) What still differs, on purpose: this server's entries are documents ({ name, path, id, entityId, type, mimeType, size }) where the host's are names or VFSDirents — client.vfs.readdir() maps either into the shape the caller asked for; text rides an entry only under withContent; and the two refusals named above (sort: "name" and a regular-expression filter.pattern), which the host does not make.
A missing path refuses ENOENT from both binaries, under two class names. This server answers its own transport class FileNotFoundError with code: "ENOENT" and a 404; brainy host IS the engine's process, so it answers the engine's own VFSError with the same code and the same status. A file listed refuses ENOTDIR and a folder read refuses EISDIR, both as the engine's VFSError from both binaries. Match on error.code for a check that holds against either door.
The accords reads
The accords profile's five READ doors are served natively, through the same read-only attach the file doors use and under the same laws. They are reached under a brain, namespaced so they cannot collide with an engine door of the same name — read, search and resolve exist in both families:
Door | HTTP | MCP tool | Session |
|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Every one of them is BrainPrincipal: holding a key for the board is enough, and none of them is an engine mechanic. The eight accords WRITES are served too — see "The accords writes" below.
The accords writes
The accords profile's eight WRITE doors are served natively on a store this server holds the lease of and whose canonical is the log — the same two conditions every write door here meets. Each is decided by the same core the in-process engine decides by, and lands through the same committer add does, in ONE commit: the record, its parent's update, its party records and its relationships together, or nothing.
Door | HTTP | MCP tool |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
A request is read the way brainy host reads it, on every transport: accord_decide's mode (queue, vote, record) picks the door, postRound's target: 'action' is postUpdate, the hosted field names (actor, target, direction, entryKind) are accepted, and the host's input checks (a real calendar day, the id's party prefix, a kind's required fields, an escalation's target, a decision's options) refuse by the same rule and class before the door decides.
The record a door acts on is read from the log, under the lease. A vote, a round or a ruling decides against the record as this server last wrote it, rounds included — never against a reader frozen at an earlier generation.
Every record a write creates carries a vector, embedded by this server's own copy of the engine's model, as the engine embeds it in process. A box without the model refuses the write by name (
EmbedderUnavailableError) rather than landing a record no meaning-based read could find.A ruling runs from the ruling party's own record:
person:<subject>for a minted token, the keys-file name itself (person:ana@example.com) for a keys-file key. No write door creates that record; a board without it refuses the rulingEntityNotFoundError.Relationship writes need the graph projection to cover the store's head when the server claims it, as
relatedoes; otherwise a door whose plan writes a relationship refusesEngineReadErrorcarrying the projection's own class, and writes nothing.Refusals keep the core's class and status:
RoundsMovedError(carrying the rounds the caller missed),NotMyFlagError,ConsentIncompleteError,IllegalTransitionError,OverrideForbiddenError,MandateAmbiguousError,DanglingCitationError,AccordNoSuchRecordError,AccordRecordExistsErrorand the rest, each with its evidence.
A served Accords write is visible to this server's Accords READS once the brain is readmitted: the reads answer from an attach frozen at one generation per residency (see "The accords reads" above).
The in-process brain.accords.* facade forwards these same five reads. It reaches them through the SAME Stage 3 Phase A Brain facade this server runs on (native/src/brain.rs, native/src/brain_napi.rs's NativeBrain.accordsInbox() / …, wrapped by NativeBrainDoors in src/native/brain.ts) — reused rather than a second napi surface, so this door and this server answer from one implementation. The handle opens once per brain and is held at the generation it attached — a row written after the first read is invisible to that facade until the brain is closed and reopened, the same law this server's own residency pool lives under. brain.accords.inbox() (and dashboard/search/read/timeline) therefore answer for real in a TypeScript process with no socket in the loop; the eight writes still throw NotYetNativeError there too.
One read({ id }) for all nine kinds. A record id carries its own kind prefix (thread:ACC-X, decision:a-slug, round:thread:ACC-X:3), so the id names the row and there is no kind parameter to get wrong: a wrong-kind read cannot exist. A BARE protocol id is probed under every prefix in one batch — exactly one hit answers, several refuse as ambiguous carrying both ids, none refuses as not found carrying what was probed.
timeline({ id }) is one door for both families. A thread's rounds and an action's log entries are the same row kind under the same parent law, so a caller never chooses between them. The delta form (afterSeq) is one more CLAUSE, not a slice after the read, so a reader that has seen forty of nine hundred turns pays for the ones it has not seen. A timeline whose positions are not the exact contiguous range refuses as corrupt rather than answering a negotiation with a turn missing.
search is kind-agnostic, and an exact id cannot be outranked. A query containing a record's protocol id — bare or prefixed — returns that record as hit one whatever kind it is: its score is set one above the highest a word count can reach, so the property is arithmetic rather than a sort that happens to hold. The rest is the engine's own text leg over the same word postings find ranks by, inside the candidate set the caller's narrowings produced. With no query there is no relevance AND no re-sort: the search view is one find() with no order key, so the rows come back in the store's own order — the order the reference engine answers that view in, which the cross-language pin holds this one to.
inbox answers in protocol order — the mandate, the decisions awaiting the user's word, this participant's open actions, their live threads — each an ordered set of native segments, clipped at the caller's limit with the pre-truncation counts in totals so a reader knows what was cut. dashboard is the board's whole shape, and its lab lens is a predicate on a nested provenance key applied in the query rather than after the fetch.
Two halves these reads do not serve, said in the answer
The semantic leg of
search. A vector search needs the vector index and an embedding provider, and an attach maps the metadata read set and the canonical tree — neither of those. So the answer carriessemantic.served: falsewith the reason, and the text leg it did serve is named. A door that answered a hybrid query from half its legs without saying so would be the quiet wrong answer this server exists to refuse.The presence stamp. A participant's own read of their own board moves their durable last-read stamp (the profile's rule R16). That is a WRITE, and a read-only attach takes no lock and writes nothing, so
inboxanswerspresenceStamped.stamped: falsewith the release the stamp arrives in. The read half is whole.
Every answer also carries what it cost — how many segments ran, how many candidates their postings matched, how many rows were read. An accord view orders by a metadata key and an attach has no column store, so ordering a segment reads the rows it ordered: honest work, visible in the counters, and bounded by the segment rather than by the store.
The refusals these doors can hand a caller, by name
Class | HTTP | When |
|---|---|---|
| 404 | The board holds no record under that id. Carries every id that was probed. |
| 409 | A bare id names a record under more than one kind prefix. Carries both. |
| 409 | The store holds that row and its (type, subtype) pair is no kind the profile declares. |
| 409 | The board does not hold exactly one active mandate (the profile's rule S7). Carries the count. |
| 500 | A timeline's positions are not the contiguous range they must be (S5). Carries expected and found. |
| 400 | A narrowing or an ordering on a field the profile declares unindexed. Carries the field. |
| 400 | A |
| 400 | A required parameter is absent, or a cursor cannot name a position. |
| 500 | A posting, a value space, or an entity int the index named could not be read. |
| 500 | A record the index named exists and its identity record could not be read. |
| 409 | The store cannot be attached for reading as it stands — the attach's own reason, by name. The cure is an action by the store's owner, never a retry. |
MandateAmbiguousError and TimelineCorruptError are the accord PROFILE's own classes, published in the API contract with the rule each enforces; the rest are this server's and are declared in the generated contract at /contract.json.
What serves, and what refuses
Served natively today by the MODEL rather than by a store: embed, embedBatch and similarity. The engine's own all-MiniLM-L6-v2 is loaded from files on the box — the same weights the Node package maps, found beside the binary or at BRAINY_EMBED_MODEL_DIR — and nothing is fetched. A server with no model REFUSES those three doors by name and lists every directory it looked in; it never answers a zero vector, because a vector is what a row MEANS to a store and a wrong one makes every search plausible and wrong.
Served natively from the store: the canonical reader, get, batchGet, counts, explain, paged reads, aggregates, now/generation/asOf/diff/history, export, the lifecycle pair init and close — the read-write open and the lease's release — the per-brain health verdict, diagnostics, repair, snapshot, the change feed, the VFS reads (readFile, readdir, stat, exists, getxattr, listxattr), the accords profile's five reads (inbox, dashboard, search, read, timeline), and the query doors —
find's metadata legs:where(every operator the conformance table freezes),type,subtype,service,excludeVFS, the visibility law,orderBy/order, andlimit/offset. The filter is evaluated by the SAME evaluator a Node process runs, so awherecannot mean one thing here and something else in process: a range bound inside a timestamp's minute answers the rows truly inside it, a presence test on an object-valued path asks the path's indexed leaves, and a field the store declares unindexed refuses by name (UnindexedFieldError), exactly as in process.connectedwalks the graph adjacency projectionrelatedreads, by the in-process planner's own walk (each anchor on its own, breadth-first,depthhops,direction, the verb types inviaortype), and the filter is answered within the neighbours; withoutorderBythe page stands in the walk's discovery order, as in process.paginationwithcount: true:pagination.count(params), the size of the setfindserves for the same params, as{ value: <count> }— the same filter and the same visibility law, so a count never disagrees with the pages it counts.related: one hop, from the GRAPH ADJACENCY projection attached read-only — which is where an edge's endpoints actually live.{ node }is undirected,{ from }/{ to }are directed,type/subtype/servicenarrow the edge set, and the page is edge id ascending.
The change feed (GET /v1/brains/{brain}/changes, server-sent events; the onChange door on every other transport) reads the log the brain's canonical lives in: a log-canonical brain's segment log, whose frames are brainy host's frames field for field (generation, index, kind, op — update or delete — id, and on an after-image denseInt, hasVector and the entity or relation view, relation endpoint tags read at serve time), and a tree or mirror brain's fact log, whose frames are thinner (generation, op, id, denseInt, hasVector). A frame's SSE id is <generation>.<index>. The cursor reads the host's resume table (docs/host.md): a bare generation is served after, a frame id from the start of its generation, head from now, and Last-Event-ID wins over the url; a cursor beyond the brain's head, one that names no generation, and a limit that is not a positive integer refuse BadQueryParameterError (400) by name, and on a log-canonical brain a cursor beneath the log's first commit boundary refuses ChangeFeedError / FactLogRefusedError. Over HTTP/1.1 and HTTP/2 the stream is held after its caught-up frame (liveTail.held: true): each later commit is sent once it is DURABLE, never before — a frame a crash could take back would be a quiet loss — rung at the commit point by the brain's write set when this server holds it, and otherwise on a two-second belt that re-reads the log from the cursor; a heartbeat comment goes down an idle stream every 15 seconds, and a held stream never pins its brain resident. A brain closed and opened again keeps its held streams: they are told when the writer is claimed again, and the first commit after it rings them at its commit point with no wait for the belt. Over HTTP/3, and as a JSON answer on any transport, the door answers one page and ends at the head (liveTail.held: false); a resume from the cursor misses nothing.
A known refusal in the released read server, fixed in 13.2. On a store whose segment log is canonical and that has committed anything since it was migrated (every fleet store's shape), related and every door that counts a row's relationships refuse with GraphProjectionBehindError. The cause is in brainy serve, not the store: it compared the graph projection's stamp with the tree's generation file, which a log-canonical store stops writing at the migration. The store and its graph are fine. Do not rehearse brainy serve on a real store's relationships before 13.2 and read that refusal as the store's fault.
The wire shape of the three doors above, said once rather than left to be discovered from either side: get answers { entity } — a HYDRATED row (the same reserved/custom split find/related answer with, _fmt and the raw on-disk stamp never crossing the wire), or entity: null when the id is absent, which is the answer at this one door. batchGet answers { entities, absent } — entities an object keyed by id, each value hydrated the same way; absent naming every id that came back with nothing, rather than a caller having to diff its own request. readdir answers { count, entries } — never a bare array, which a caller cannot tell from a door that answered nothing at all. brainy host answers the identical three shapes at the same three doors (src/mcp/dispatch.ts's case "get"/"batchGet"/"vfs") — a client cannot tell the two binaries apart here.
Two things are said here rather than discovered. An orderBy reads one value per MATCHED row (there is no column store in a read-only attach) and therefore carries the engine's own bounded-fallback budget of 1000 matches, refusing by name above it with the cure. A stored record with no timestamps keeps none here, where the in-process door fills them with the clock. related's own page — directed, or a { node } page at any offset — used to be said here too, as a declared ordering divergence; it no longer is. The in-process door's own order is verb-int ascending, deterministically (native/src/graph_adjacency.rs's paginate_u64 sorts every verb-int read it answers, even the unpaged one), and this server reproduces that order exactly, including the { node } page's own per-side truncation at a deep offset — a client reading related cannot tell this server and the in-process door apart, at any offset, on any anchor.
related is worth a sentence about where it reads, because the obvious reading was wrong. The in-process door builds a filter over verb records (from → sourceId, to → targetId), and a stored relation record keys bare — so one hop looks like one posting read. MEASURED on a store this engine wrote: the metadata index posts an edge's verb, its subtype and its whole user bag, and posts neither sourceId nor targetId — a filter on either answers zero rows on every store. The endpoints are in the graph adjacency projection, so that is what this server maps, gated on the projection's own stamp: it answers when the stamp says the persisted trees cover the generation the store is committed at, and REFUSES by name with both numbers when they are behind. It never runs the recovery walk an in-process index runs to close that gap — that would turn one hop into a scan of every verb — and never answers the empty page a posting-driven implementation would have returned for every anchor, which reads exactly like "this node has no edges".
A ranked search is served: find with a query, either beside searchMode: "text" (the words alone) or with no searchMode at all, auto or hybrid (the words and the meaning, blended) — with any limit, offset, type, service, where, excludeVFS. The word index gathers a pool (the first 2,000 rows by match count), the meaning leg adds the 1,000 nearest rows of the vector index when the call is a blend, the two are fused by reciprocal rank (the weight of the meaning by the query's length, or hybridAlpha), the visibility rule and the filter fall the pool to the rows the caller may see, and the ranking stage orders what is left: the query is encoded by the engine's own late-interaction checkpoint (answerai-colbert-small-v1) and every candidate is scored against the store's tokens family. Both models load from files on this box (BRAINY_ENCODER_MODEL_DIR and BRAINY_EMBED_MODEL_DIR, else beside the binary or under assets/models/; nothing is fetched). The page is in the in-process engine's order, and the answer's ranking account says how many candidates were ordered, how many the family could score, and how many kept the position the fusion gave them because the family does not reach them yet.
The meaning leg reads the vector index only when it covers the store. The index is attached from the files its manifest lists (never the directory), and a store committed past the generation the index covers is served only when the log's tail above it holds no write that changes what the index holds — a metadata-only overwrite, which rewrites the row with its vector as it was, changes nothing. A query is embedded exactly as the in-process engine embeds it (BRAINY_QUERY_EMBED=fast embeds it to its own length instead: faster, and not the same bits). The server keeps the last 256 query vectors and ranking-model encodings, so a repeated or paged search costs neither model pass (BRAINY_QUERY_CACHE=0 turns that off). A store that publishes no tokens family or no vector index, an index behind the store or written by another model, a family written by another checkpoint, or a box that holds no ranking model or embedder REFUSES by name (ShapeNotServedError, saying which): the same rows in the leg's order would be a different answer, and nothing here approximates one.
similar is served: similar({ to, limit, threshold, type, where, service, excludeVFS }) answers the nearest rows to the vector of the row to names (or to a vector you bring), inside the same filters, at the narrow width the in-process engine's unranked path uses (twice the page), best first — the vector index and nothing else; no model runs. A to that names no row, or a row whose embedding has not landed yet, refuses as a bad parameter and says which.
Refusing by name because a read-only attach maps no such index, each naming what is absent and where the same query is served: a semantic leg alone — an explicit vector in a find, or a searchMode of vector/semantic; a ranked search asked of a door that cannot order by it (a count, a page) or with no words; a graph leg — searchMode: 'graph', graph traversal, a connected walk that names a subtype, and a pagination.count with connected; and the ranked and aggregated shapes — purpose, rerank: true, aggregate, fusion, near, passages and a ranked cursor. None of them is approximated: a semantic find answered from the word postings alone would rank plausibly and wrongly, and that is worse than a refusal a client can route around.
Refusing by name because this core has no read implementation of them AT ALL yet, a different gap from both of the above — not a projection a read-only attach cannot map, and not a write waiting on the commit path: neighbors (graph traversal from one entity, direction/verb-type/depth filtered), findDuplicates (near-duplicate detection over the semantic index; brainy host answers it as a bounded scan, scan candidate rows per call with a cursor, docs/api-contract.md §4), transactionLog (the reified commit log, read from the generation store), the dial's three (generationAt, historyBounds, historySamples - date-to-generation answers from the TypeScript core's in-memory time index), scanFacts (the sequential fact-log scan the TypeScript core's own heals and replays walk) and the recovery read, listRemoved, readRemoved and readRemovedContent (what was removed, a removed row's last version and a removed file's bytes - docs/removed-rows.md; natively they will read a log-canonical store's segment log, a tombstone being a record and its last version the record before it, and that handler is not written yet). brainy host's TypeScript core already serves all of them in-process; this binary's own dispatch has no arm for any of them, and no release names when one will (native/serve/src/doors.rs's CLIENT_SURFACE_READS_RELEASE).
Refusing by name because they need the writer and the TypeScript job runtime, which this read-only attach does not have: the operator ceremonies (docs/operator-ceremonies-over-the-wire.md) — today the orphan-id retraction pair and the backup pair. The backup pair is persist (POST /v1/brains/{brain}/ceremonies/persist, body { name }) and persistStatus (GET on the same path): over brainy host the disk is the one place the two deployments differ — the wire carries a snapshot NAME, and the host places it inside the folder its operator set with --snapshots-dir, never a path (docs/api-contract.md). The orphan-id pair is retractOrphanedIndexIds (POST /v1/brains/{brain}/ceremonies/orphan-ids) and orphanedIndexIdsStatus (GET on the same path, answering one run document: a run this host holds reads running, done or failed with the refusal's name; with no run held, the engine's own status decides: running with resumedByEngine: true while the engine works on a run its open resumed, done for a finished receipt, interrupted for an unfinished one nothing is working on, and none without a receipt). A ceremony walks a whole store and changes it, and a read-only attach may not. brainy host serves all four under the service key; this binary answers NotYetNativeError at any of them, with the service key or without it refused first ForbiddenError, and no release names when they arrive (native/serve/src/doors.rs's OPERATOR_CEREMONIES_RELEASE). The recount of stored content references, repairBlobReferences (POST /v1/brains/{brain}/repair-blob-references, a dry run unless apply; docs/blob-references.md), raises counts in the blob store and so needs the writer: it is brainy host's, and this binary answers it NotYetNativeError under the same release name.
Served natively, from the committer: add — the create path — update — the overwrite — remove, and their batch forms addMany, updateMany and removeMany, and the relationship doors relate, unrelate, updateRelation and relateMany, on a store this server holds the lease of and whose canonical is the log (see "What a lease buys" above for the two refusals that say which of those is missing, and what update, remove and the batch forms do not serve yet).
transact plans its operations in order, each by its single door's own planner over what the earlier ones planned, and lands them as ONE commit or not at all: a refused operation refuses the whole batch with its own class and opIndex. options.meta refuses NotYetNativeError by name (the engine records it in a transaction log the native path does not write).
clear is served, on the service key, as ONE logged generation. It tombstones every live noun and verb, guardrail, profile and accords rows among them, as the engine's own clear removes them. It keeps the store's declarations and its canonical-mode mark, and it is durable when it answers. Nothing is lost: the clear is itself a recorded generation, and asOf before it still reads every row. On a log-canonical store the engine's in-process clear() refuses with LogCanonicalClearError and points here, because it would delete the mark and leave every row in the log.
What it costs: one generation, with the write lock held for the whole clear, and the hold grows faster than the rows. MEASURED on the gate box:
about 11,000 rows took 1.0–1.3 s;
about 101,000 rows took 24.2 s, which is 0.24 ms a row against 0.09 at 10k.
Both used 28 fsyncs. Every other write waits for the whole hold, so this is an operator door for a rare moment, not a routine one.
declareSubtypeRule and declareFieldValues are served. They take the same bodies as brainy host ({ type, options: { values?, required? } } and { field, values }; the path's {id} names the type or field when the body does not), refuse bad arguments with ValidationRefusedError in the engine's own words, and write _system/write-rules.json durably. That file binds every process that writes the store, this server's next write included. Declaring a rule the store already carries writes nothing and answers persisted: false. A declaration spends no generation: it is not a row.
defineAggregate, removeAggregate and trackField are served.
The record. A definition is written into the store's aggregate-definition record, the one the engine reads, with the engine's hash. Bad definitions are refused with the engine's class and sentence, by the same check the engine applies when it registers a definition: an operator outside the vocabulary (
FilterUnreadableError), an operator no index can serve, an operand of the wrong shape, or a definition that does not parse (ValidationRefusedError). This server never saves a definition the engine would refuse.Tracking a field registers its vocabulary with this server's writer for the server's life, as the engine's
trackFieldbinds its own process. Every native noun write door checks it in the engine's words. Tracking also defines the field's standing count aggregate once.What this server does not do is maintain or query aggregates. The engine rescans each aggregate over the rows this server wrote at its first query after an open, because those rows moved the committed generation the aggregate's state was stamped at.
What a call costs and answers. No call spends a generation. A call that changes nothing writes nothing and answers
persisted: false.
Refusing by name until the core owns the write path: import and the memory profile's abstract — its rollup, its per-absorbed-row stamp and its edges are one transaction, and its decision has no native twin yet. (The accords profile's eight writes are served — "The accords writes" above — and so are all ten of the file system's — "The file writes served natively".)
Served natively, from this same committer, as of the memory profile's turn — the first record-profile write door this build executed (docs/profile-writes-in-rust.md §9; the accords profile's eight writes followed in 13.2): idempotent on (conversationId, turn), refusing TurnAlreadyRecordedError for a contradicting gist and absorbing an exact re-send, exactly as brain.memory.turn() does in process — because it decides through the identical core and lands through the identical committer add does. memory.turn flipped first: its own state read needed the identical keyed canonical read. add followed it once its own read-back was pinned: a row it lands reads back through this process's get and through a cold in-process reopen.
Refusing as in-process only: use (a plugin is code) and storageAdapter (a live object holding this process's file descriptors and its writer lock). Each names what to call instead.