Transaction System

Status: ✅ Production Ready

Overview

Brainy's transaction system provides atomic operations with automatic rollback on failure. All operations within a transaction either succeed completely or fail completely - there are no partial failures.

Key Benefits

• Atomicity: All operations succeed or all rollback

• Consistency: Indexes and storage remain consistent

• Automatic: Transparently used by all brain.add(), brain.update(), brain.remove(), and brain.relate() operations

• Composable: brain.transact() runs a multi-write batch through the same machinery as exactly one atomic commit

Architecture

User Code (brain.add(), brain.update(), brain.transact(), etc.)
 ↓
Transaction Manager (orchestration)
 ↓
Operations (SaveNounMetadataOperation, SaveNounOperation, etc.)
 ↓
Storage Adapter (sharding, ID-first routing)

How It Works

Every write operation in Brainy automatically uses transactions:

// Internally, this uses a transaction
const id = await brain.add({
 data: { name: 'Alice', role: 'Engineer' },
 type: NounType.Person
})

Transaction Flow:

1. Begin Transaction: TransactionManager creates new transaction

2. Add Operations: Operations added to transaction (SaveNounMetadataOperation, SaveNounOperation)

3. Execute: Each operation executes in sequence

4. Commit: All operations succeeded → changes persist

5. Rollback: Any operation failed → all changes reverted

Rollback Mechanism

Each operation implements both execute and undo:

class SaveNounMetadataOperation {
 async execute(): Promise<void> {
 // Save new metadata
 await this.storage.saveNounMetadata(this.id, this.metadata)
 }

 async undo(): Promise<void> {
 // Restore previous metadata (or delete if new entity)
 if (this.previousMetadata) {
 await this.storage.saveNounMetadata(this.id, this.previousMetadata)
 } else {
 await this.storage.deleteNounMetadata(this.id)
 }
 }
}

On failure:

• Operations rolled back in reverse order

• Previous state fully restored

• Indexes updated to reflect rollback

Compatibility with Advanced Features

Multi-Write Batches: brain.transact()

✅ The 8.0 path for atomic multi-entity writes

Single-operation methods each commit their own transaction. When several writes must succeed or fail together, use brain.transact() — a declarative batch that commits as exactly one generation, with optional whole-store compare-and-swap and durable transaction metadata:

const db = await brain.transact([
 { op: 'add', id: orderId, type: NounType.Document, subtype: 'order', data: 'Order #1042' },
 { op: 'update', id: customerId, metadata: { lastOrderAt: Date.now() }, ifRev: customer._rev },
 { op: 'relate', from: customerId, to: orderId, type: VerbType.Creates, subtype: 'purchase' }
], { meta: { author: 'order-service' } })

db.receipt.ids // resolved id per operation, in input order

How It Works:

• The batch executes through the same TransactionManager as single operations, wrapped in the generational commit protocol: before-images are staged and fsynced first, and the atomic manifest rename is the commit point — a crash anywhere before it rolls back to the exact pre-transaction bytes.

• Per-entity ifRev and whole-store ifAtGeneration provide compare-and-swap at two granularities; any conflict rejects the entire batch before anything is staged.

• The returned Db is a pinned, snapshot-isolated view of the committed state.

See the consistency model for the full guarantees (snapshot isolation, time travel, snapshots) and Snapshots & Time Travel for recipes.

Sharding

✅ Fully Compatible

Transactions work across multiple shards:

// Entities with different UUID prefixes go to different shards
const id1 = 'aaa00000-1111-4111-8111-111111111111' // Shard: aaa
const id2 = 'bbb00000-2222-4222-8222-222222222222' // Shard: bbb

await brain.add({ id: id1, data: { name: 'Entity A' }, type: NounType.Thing })
await brain.relate({ from: id1, to: id2, type: VerbType.RelatesTo })

// Transaction handles cross-shard atomicity automatically

How It Works:

• Sharding is transparent to transactions

• analyzeKey() method routes to correct shard based on UUID

• Transaction operations don't need to know about shards

• Rollback works across all shards involved

ID-First Storage

✅ Fully Compatible

Transactions work with direct ID-first paths - no type routing needed!

// Entities stored with direct ID-first paths
const personId = await brain.add({
 data: { name: 'John Doe' },
 type: NounType.Person // → entities/nouns/{shard}/{id}/metadata.json
})

const orgId = await brain.add({
 data: { name: 'Acme Corp' },
 type: NounType.Organization // → entities/nouns/{shard}/{id}/metadata.json
})

// Type changes handled atomically (type is just metadata)
await brain.update({
 id: personId,
 type: NounType.Organization, // Type change
 data: { name: 'Doe Corp' }
})

How It Works:

• Type information stored in metadata.noun field

• Storage layer uses O(1) ID-first path construction

• No type cache needed (removed in a previous version)

• Type counters adjusted on commit/rollback

• 40x faster path lookups (eliminates 42-type search)

Storage Adapter Interface

✅ Fully Compatible

Transactions go through the StorageAdapter interface, so both shipped adapters (filesystem, memory) and any custom plugin adapter inherit the same atomicity guarantees:

const brain = new Brainy({
  storage: { type: 'filesystem', path: './data' }
})

await brain.add({ data: { name: 'Entity' }, type: NounType.Thing })

How It Works:

• Transactions operate through StorageAdapter interface

• Custom adapters registered via the plugin system implement the same interface

• Atomicity guaranteed at the write-coordinator level

• Read-after-write consistency maintained inside a single Brainy process

Examples

Basic Add Operation

import { Brainy } from '@soulcraft/brainy'
import { NounType } from '@soulcraft/brainy/types'

const brain = new Brainy()
await brain.init()

// Automatically uses transaction
const id = await brain.add({
 data: { name: 'Alice', role: 'Engineer' },
 type: NounType.Person
})

// If add fails, all changes rolled back automatically

Update with Type Change

// Original entity
const id = await brain.add({
 data: { name: 'John Smith', category: 'individual' },
 type: NounType.Person
})

// Update with type change (atomic)
await brain.update({
 id,
 type: NounType.Organization, // Type change
 data: { name: 'Smith Corp', category: 'business' }
})

// If update fails, original type and data restored

Creating Relationships

const personId = await brain.add({
 data: { name: 'Alice' },
 type: NounType.Person
})

const projectId = await brain.add({
 data: { name: 'Project X' },
 type: NounType.Thing
})

// Create relationship (atomic)
await brain.relate({
 from: personId,
 to: projectId,
 type: VerbType.WorksOn
})

// If relate fails, no partial relationship created

Batch Operations

// Multiple operations, all atomic
for (let i = 0; i < 100; i++) {
 await brain.add({
 data: { name: `Entity ${i}`, index: i },
 type: NounType.Thing
 })
}

// Each add() is a separate transaction
// If any add fails, only that specific add is rolled back

Delete with Cascade

const personId = await brain.add({
 data: { name: 'Bob' },
 type: NounType.Person
})

const projectId = await brain.add({
 data: { name: 'Project Y' },
 type: NounType.Thing
})

await brain.relate({
 from: personId,
 to: projectId,
 type: VerbType.WorksOn
})

// Delete person (atomic - deletes entity + relationships)
await brain.remove(personId)

// If delete fails, both entity and relationships remain

Error Handling

Transactions automatically handle errors and rollback:

try {
 await brain.add({
 data: { name: 'Test Entity' },
 type: NounType.Thing,
 vector: [1, 2, 3] // Wrong dimension → error
 })
} catch (error) {
 // Transaction automatically rolled back
 // No partial data in storage or indexes
 console.error('Add failed:', error.message)
}

Common Error Scenarios:

• Invalid vector dimension: Automatic rollback

• Type validation failure: Automatic rollback

• Storage write failure: Automatic rollback

• Index update failure: Automatic rollback

Performance Considerations

Transaction Overhead

What a transaction costs:

• A typical single-operation write wraps 2-8 operations (metadata + data + indexes) in one transaction

• The overhead is bookkeeping (operation objects + undo state), not extra I/O on the success path

• Rollback cost is proportional to the operations already applied (each is undone in reverse order)

Optimization:

• Operations executed sequentially (not parallel) for consistency

• Rollback only happens on failure (success path is fast)

• Index updates batched within transaction

Auditing Committed Batches

Every committed brain.transact() batch is recorded in the transaction log, newest first:

await brain.transact(ops, { meta: { author: 'import-job' } })

const entries = await brain.transactionLog({ limit: 10 })
// [{ generation: 1042, timestamp: 1765432100000, meta: { author: 'import-job' } }]

Single-operation writes advance the generation counter but do not append log entries — see the consistency model for the history-granularity contract.

Best Practices

1. Let Brainy Handle Transactions

// ✅ Recommended: Use Brainy's API (transactions automatic)
await brain.add({ data, type })
await brain.update({ id, data })
await brain.remove(id)

// ❌ Avoid: Direct storage access bypasses transactions
await brain.storage.saveNoun(noun) // No transaction protection

2. Handle Errors Gracefully

// ✅ Recommended: Catch errors, transaction rolls back automatically
try {
 const id = await brain.add({ data, type })
 return id
} catch (error) {
 console.error('Add failed, rolled back:', error)
 // Decide how to handle (retry, log, alert user)
}

3. Validate Before Operations

// ✅ Recommended: Validate early to avoid unnecessary rollbacks
if (!isValidVector(vector, brain.dimension)) {
 throw new Error(`Vector must have ${brain.dimension} dimensions`)
}

await brain.add({ data, type, vector })

4. Batch Related Writes with transact()

// ✅ Recommended: writes that must land together go in one batch
await brain.transact([
 { op: 'add', id: orderId, type: NounType.Document, subtype: 'order', data: 'Order #1042' },
 { op: 'relate', from: customerId, to: orderId, type: VerbType.Creates, subtype: 'purchase' }
])

// ❌ Avoid: sequential single operations when partial application is unacceptable
const id = await brain.add({ ... })   // commits alone
await brain.relate({ ... })           // a crash here leaves the entity unlinked

5. Understand Atomicity Guarantees

What Transactions GUARANTEE:

• ✅ Atomicity within a single Brainy process

• ✅ Consistent state across all indexes

• ✅ Automatic rollback on failure

• ✅ Works with all storage adapters (filesystem, memory, custom plugin adapters)

What Transactions DON'T Provide:

• ❌ Two-phase commit across multiple Brainy instances

• ❌ Distributed locking across processes

• ❌ Cross-datacenter ACID guarantees

Design: Transactions ensure atomicity at the write-coordinator level inside one process. Cross-instance coordination, if you need it, lives in your service layer.

Testing Transactions

Unit Tests

import { describe, it, expect } from 'vitest'
import { Brainy } from '@soulcraft/brainy'

describe('Transaction Tests', () => {
 it('should rollback on failure', async () => {
 const brain = new Brainy()
 await brain.init()

 const id1 = await brain.add({ data: { name: 'Entity 1' }, type: NounType.Thing })

 try {
 await brain.add({
 data: null as any, // Invalid - will fail
 type: NounType.Thing
 })
 } catch (e) {
 // Expected failure
 }

 // First entity should still exist (rollback didn't affect it)
 const entity1 = await brain.get(id1)
 expect(entity1).toBeTruthy()
 })
})

Integration Tests

See tests/transaction/integration/ for comprehensive integration tests covering:

• Sharding integration (sharding-transactions.test.ts)

• Type-aware integration (typeaware-transactions.test.ts)

The atomicity guarantees of brain.transact() — including crash recovery through the real recovery path — are proven in tests/integration/db-mvcc.test.ts.

Troubleshooting

High Rollback Rate

Symptom: a high share of writes throw and roll back

Possible Causes:

1. Invalid vector dimensions

2. Type validation errors

3. Storage write failures (disk full, network issues)

4. Index corruption

Solutions:

• Validate data before operations

• Check storage adapter health

• Monitor disk space and network connectivity

• Review error logs for patterns

Slow Transaction Performance

Symptom: Operations take > 100ms per transaction

Possible Causes:

1. Large metadata objects

2. Remote storage latency

3. Many indexes enabled

4. Disk I/O bottleneck

Solutions:

• Optimize metadata size

• Use local caching for remote storage

• Disable unused indexes

• Use SSD storage

Architecture Details

Transaction Lifecycle

1. BEGIN
 ↓
2. ADD OPERATIONS
 - SaveNounMetadataOperation
 - SaveNounOperation
 - UpdateGraphIndexOperation
 ↓
3. EXECUTE (sequential)
 - Execute operation 1 → Success
 - Execute operation 2 → Success
 - Execute operation 3 → FAILURE
 ↓
4. ROLLBACK (reverse order)
 - Undo operation 2
 - Undo operation 1
 ↓
5. THROW ERROR

Operation Types

Storage Adapter Integration

Transactions use the StorageAdapter interface:

interface StorageAdapter {
 saveNounMetadata(id: string, metadata: NounMetadata): Promise<void>
 saveNoun(noun: Noun): Promise<void>
 deleteNounMetadata(id: string): Promise<void>
 deleteNoun(id: string): Promise<void>
 // ... other methods
}

Key Insight: Both shipped storage adapters (filesystem, memory) — and any custom plugin adapter — implement this interface. Transactions work with any storage adapter automatically.

Additional Resources

• Unit Tests: tests/transaction/Transaction.test.ts, tests/transaction/TransactionManager.test.ts

• Integration Tests: tests/transaction/integration/

• MVCC Proofs: tests/integration/db-mvcc.test.ts (atomicity, CAS, crash recovery for brain.transact())

• Consistency Model: docs/concepts/consistency-model.md

Summary

Brainy's transaction system provides production-ready atomic operations with automatic rollback. Every single-operation write is transactional out of the box, and brain.transact() extends the same guarantee to multi-write batches — one atomic commit, with compare-and-swap and durable transaction metadata.

Key Takeaways:

• ✅ Automatic: No manual transaction management needed for single operations

• ✅ Atomic: All operations succeed or all rollback — per operation and per transact() batch

• ✅ Compatible: Works with all storage adapters and features

• ✅ Coordinated: Per-entity ifRev and whole-store ifAtGeneration CAS reject conflicting batches before anything is staged

Start using transactions today - they're already built into brain.add(), brain.update(), brain.remove(), and brain.relate() — and reach for brain.transact() whenever several writes must land together.