Memory Projections

The design, in ten lines: what a memory projection IS, so the next one is added the same way.

1. A memory projection is a derived row, not a cache. It is durable, it is a pure function of the canonical memory rows, and it is rebuildable from them alone. Nothing else may be true of it — a projection that needs a log, a queue, or its own history is a second source of truth wearing a projection's name. (Distinct from a STORAGE projection: src/open/projectionArtifacts.ts is how the engine persists rows; this is a row the engine derives FROM rows.)

2. One fold, two callers. The incremental job cycle and the full rebuild run the SAME fold function over rows in (createdAt, id) order. That is what makes "rebuild = identical projection" structural rather than hoped for, and projections.e2e.test.ts pins it on the hash as well as the fields.

3. Every term of the fold is exact under an ordered fold — min, max, sums, a last. A term that is not (a live count, a distinct set that later rows can shrink) does not go in the row; the door computes it from a posting-list cardinality at read time, which materialises no ids.

4. The counts are historical, not live. A member counts whether or not it was later retracted, because memoryCount is how many memories were written into this conversation — monotone, and therefore exact incrementally. Under the engine's retention law a retracted row is still a row that happened — see Retire.

5. The cycle's cost is proportional to what was WRITTEN, never to what the store holds. Forward-only over a watermark; the watermark is the mark and nothing is stamped on a canonical row. The lookup of the rows to update is narrowed by conversation id in the index (in), never by paging the projection subtype — that shape is the maintenance term the cost lane exists to refuse (brainy 4.2.1).

6. A projection row is not a memory. It carries no stability, which is the fence (episodeJob.ts) that keeps a fold from reading its own output as input and deriving sessions of sessions forever. Pinned. It carries no author either, for the same reason author is not one of rule 3's exact terms: a session can span many episodes written by many different actors, and there is no min/max/sum/last of "who wrote it" that is honest about the rest. A session's own members still carry their own authors; ask memory.episodes() for them.

7. Rebuild is a DOOR, never a cadence. brain.memory.rebuildProjections(). A sweep that re-derived the whole store on a timer would be exactly the cost shape rule 5 forbids.

8. The doors report their own bound. Every answer carries rowsVisited, the real count of index rows read. A door that claims to be bounded and is not is exactly the defect projections exist to cure — context()'s thread slice, before a session projection existed to answer the question directly, walked ~2,827 rows to answer one question on a 2,669-paragraph brain.

9. A hash, and what it honestly saves. Each row carries its content hash and each digest carries the hash of the lines it named. A caller passing knownHash back is told unchanged and handed no bodies. That saves the CALLER's tokens, never the engine's reads, and the module header says so rather than implying otherwise.

10. The engine never writes prose — but it LANDS it atomically. A session row is structure: a span, two counts, a last member. Words about a conversation come from a service or from the language seam (brain.language.summarize), into a field the engine declares and does not fill. What the engine does own is the landing: memory.abstract() (src/memory/abstract.ts) writes the rollup row, the processing.abstractedIntoMemoryId on every absorbed row and the abstractedInto edges in one generation-pinned transaction, because three facts that must agree cannot be written by three calls that can half-land.

What exists today

MEMORY_PROJECTION_NAMES in src/memory/projections.ts is the closed set, and today it names exactly one: session. A name added there without a fold and a door fails to compile.

The truly virgin store — zero sessions is an answer, never a refusal. sessions()/digest() order and filter on the session profile's own fields (SESSION_FIELDS.lastAt foremost), which the profile declares lazily, on the FIRST remember() a brain ever sees. A brain that has never had one has never declared them, so querying by name would otherwise refuse UnresolvableFieldError — a true statement about the field and a false one about the question these doors exist to answer, since a brain that has never remembered anything has, honestly, zero sessions. sessions() narrows its own catch to exactly the fields its own query touches (nothing broader) and answers the empty page instead — src/memory/projectionDoors.ts's own module header, and projectionDoors.test.ts, carry the fixture. Found via session-capture-road-generic-in-engine's own session_start, the first caller in this codebase to legitimately call digest() on a brain that may never have been written to at all.

Episodes are not a projection in this module's sense, and the distinction is deliberate. An episode (Event/memory-episode, one per closed burst, maintained by the memory-episodes job and read through memory.episodes()) predates this module and is built by a different mechanism — a forward-only segmentation walk with no closed-form fold, because an episode can still be open while a session never has that state. See Housekeeping for how episodes are built. What this page's ten rules govern is the session tier above them — and the next projection this module grows, whatever it is, is held to the same ten.

The session-door WRAP write is the clean example of both, side by side. session-capture-road-generic-in-engine (12.10.0) gives the terminal session's own SessionEnd hook exactly one write: one memory.remember() row, source: 'conversation', metadata.sourceConversationId the harness session's own id. That single row feeds BOTH mechanisms from the SAME (source, sourceRef) key, on two different clocks — the incremental fold above folds it into the SESSION projection the next time memory-projections runs (so session-start's own "From your memory" section, read through memory.digest()/memory.sessions(), sees it as soon as that fold has), while the forward-only walk in memory-episodes links it into an EPISODE only once the run has gone quiet past episodeGapMs — the same row, present in one tier before the other, which is exactly the "still open" state rule 4 above says a session never has and an episode always might.