Performance Budget

This document is a contract. The latency targets below are CI-enforced regression gates, not aspirations. Code that fails a gate does not ship.

Why a budget matters

Brainy + cor is the only database that answers a single query combining:

• Metadata filter — structured where over indexed fields (LSM metadata index with Roaring bitmap evaluation)

• Graph traversal — multi-hop relationship walks with verb-type selectivity (graph adjacency LSM with verb endpoint lookups)

• Vector similarity — semantic search over the entire corpus (Adaptive DiskANN: Mode 1 for in-memory recall, hybrid for PQ-compressed billion-scale)

No pure vector database (Pinecone, Weaviate, Qdrant), pure graph database (Neo4j, Memgraph), or document database (MongoDB, Elasticsearch) does this. PostgreSQL with pgvector gets two of three with a multi-second join cost. The combined query is the differentiator and it is only differentiating if it is sub- millisecond at billion scale.

The budget locks in that property as a release criterion.

The budget

All measurements are p50 / p99 over 100 queries on a single 32- core box (workstation tier: i9-14900HX / 64 GB / NVMe Gen 4; billion-tier: bxl9000 / Ryzen 9 7950X3D / 184 GB / NVMe). Queries are drawn from a representative workload mix described in Methodology below.

Read path budget

Cross-subsystem ID resolution budget

find() returns UUID strings to JS, so every result row pays the int→UUID resolution. Batched APIs are mandatory; per-call APIs are not part of the hot path.

Write path budget

Recall floor

Recall is a correctness property, but it has a budget too: a query that returns the wrong answer in 0.5 ms is worse than one that takes 1 ms and returns the right one.

How the budget is enforced

Every gate in the table corresponds to a regression test in src/native/findPerfGates.test.ts (to be added; tracked under the Cor 3.0 release plan). The test pattern follows the existing lsmPerfGates.test.ts:

1. Build a corpus at the target scale with a representative field distribution (50 fields, mixed cardinality, ~10 verbs per noun, dim 384 vectors with clustered structure).

2. Warm the page cache with one pass over the query set.

3. Run 100 queries per shape, record p50 and p99.

4. Assert p50 ≤ budget × 1.0 and p99 ≤ budget × 1.2 (20 % headroom on p99 absorbs CI variance).

A red gate is a release blocker. The test prints the measured distribution so the regression is bisectable to a commit.

Methodology

Workload mix

The representative workload is a 70 / 20 / 10 split:

• 70 % filter-driven queries (where with 1-3 predicates, selectivity from 0.1 % to 10 %)

• 20 % vector-driven queries (vector search with optional metadata pre-filter)

• 10 % graph-driven queries (1- or 2-hop walks with verb-type selectivity)

The mix tracks the access pattern Soulcraft Memory observes in production: most queries are structured browse / filter; vector search is the discovery path; graph hops are the relationship expansion.

Hardware floor

The budget is anchored on the cor 3.0 design floor — a 32- thread Intel Core i9-14900HX with 64 GB DDR5 and NVMe Gen 4 SSD. That's roughly a $2K commodity workstation; production targets sit on top of it. The bxl9000 measurements (Ryzen 9 7950X3D, 184 GB) are the billion-tier ceiling. Numbers from cloud VMs are typically 1.3–1.8× slower for the same work because of memory bandwidth and NVMe queue depth.

What "p50" and "p99" mean here

• p50 is the median measurement over 100 query repetitions with a warm page cache. It represents the steady-state cost.

• p99 is the 99th percentile and represents the worst typical query — page-cache miss, garbage collection pause, or a background msync.

• We do not publish a p99.9 budget. p99.9 measurements are dominated by external noise (kernel scheduling, NVMe firmware latency spikes) and are not reproducible across runs.

The hot paths inside find()

A unified find({where, vector, hop, orderBy, limit, offset}) query executes against three cor subsystems in sequence. Each subsystem has a sub-budget that rolls into the totals above.

1. Filter pushdown — metadataIndex.getIdsForFilter(field, value) returns a Roaring bitmap of matching entity ints. AND / OR / NOT combinators are bitmap operations. Sub-budget: ≤ 0.3 ms for a 3-predicate AND at 100 M.

2. Vector search — NativeDiskAnnWrapper.searchWithFilter walks the Vamana graph, prefilters candidates against the metadata bitmap, scores by distance. Sub-budget: ≤ 1 ms at 1 M (Mode 1), ≤ 4 ms at 100 M (hybrid).

3. Graph hop expansion — graphIndex.getNeighbors(int_id, verb_type) returns a Roaring bitmap of neighbor ints. Sub- budget: ≤ 0.3 ms per hop at 100 M.

4. Cross-subsystem ID resolution — intsToUuidsBatch over the final result set. Sub-budget: ≤ 0.1 ms per 1 K results.

The sub-budgets sum to within the top-level budget when the query plan is correct. They blow the budget when the plan is wrong (e.g. vector search runs over the whole corpus when a 1 %-selective filter could prune it first). The query planner described in Investments below is what keeps the plan correct.

The cor 3.0 performance investments

The shadow-page rework (G4.1, G4.4, G4.2 redo per the snapshot safety contract) sets the substrate. The following investments build on it to reach the budget at billion scale:

1. Cross-subsystem query planner. Today find() runs filter → vector → graph in a fixed order. The planner uses metadataIndex.getCountForCriteria for cardinality estimates and picks the lowest-cost order per query (e.g. a 0.01 %- selective filter goes first, prunes 99.99 % of vector work). PostgreSQL does this. Pure vector DBs don't.

2. Batched cross-subsystem APIs. Every napi boundary call pays ~5 μs of overhead. Resolving 10 K UUIDs one-at-a-time costs 50 ms in napi alone, before any real work. Batched variants (intsToUuidsBatch, getNeighborsBatch, getVerbsBatchCached) amortise that to ~0.5 μs per item. The hot path uses batches exclusively.

3. Lock-free delta cache reads. The LSM delta cache (added in G4.3, applied in G4.1 + G4.4) currently sits behind a Mutex. A sharded DashMap or ArcSwap<Arc<HashMap>> MVCC pattern lifts the read-side lock entirely, lets concurrent find() calls scale linearly with cores.

4. SIMD bitmap operations. CRoaring uses portable intersect/union; modern AVX-512 popcnt and vpternlogq kernels are ~3–5× faster on bitmap-heavy filter combinations.

5. DiskANN predicate pushdown. The Vamana beam search currently scores every neighbor of every visited node. With a metadata-bitmap predicate passed into the search engine, pruned nodes are skipped during the walk, not after. Massive win for high-selectivity hybrid queries.

6. Pipelined result resolution. DiskANN currently returns all top-K candidates before resolution starts. Pipelining the resolution (intsToUuidsBatch over the first 10 candidates while the walk continues) hides the resolution latency entirely.

7. Zero-allocation hot path. find() allocates intermediate Vec<u64> and HashSet<String> per query. A bumpalo-style arena allocator (reset per query, no per-element free) removes the allocation pressure and the GC pause.

Each investment ships with the gates above as the proof point.

Forbidden patterns

The performance contract has anti-patterns the way the snapshot safety contract does:

1. Per-row napi calls on the hot path. Anything in find() that does for (id of ids) { resolve(id) } over napi is incorrect. Use a batch.

2. Lock acquisition on every read. A Mutex on the read path costs at minimum a CAS on every call. At 100 K QPS with contention, this is a bottleneck before the first real operation runs. Use ArcSwap or per-shard locking.

3. Synchronous compactions on the hot write path. Compaction is a background operation. A write that blocks for the duration of a compaction is a p99 cliff; we currently run compaction synchronously in G4.3's ShadowPageEndpointsStore (acceptable at 3.0 launch scale) but the pattern is explicitly recognised as a future fix.

4. String allocation in the inner loop. UUIDs are converted to strings for the JS boundary, but the conversion has to happen once per result row, not per intermediate set. Carry integer ids through the planner; convert at the egress.

5. Quadratic scans where indexed lookups exist. Filter evaluation that walks all entities checking each against a predicate, instead of consulting the metadata index, is the hidden cliff at scale. Every where clause is index- accelerated or it's a release blocker.

See also

• docs/snapshot-safety.md — the snapshot safety contract this performance work is co-designed with.

• docs/verification-report.md — measured baseline numbers (SIFT1M / SIFT10M / 1M synthetic, AVX-512 baseline) that the budget is anchored on.

• docs/comparison.md — comparison of brainy + cor against pure vector / pure graph / pure document databases on the triple-intelligence query.