Neural API
AI-powered analysis: clustering, similarity, outlier detection, and visualization.
Overview
The Neural API provides advanced AI-powered analysis of your knowledge graph. Discover patterns, find anomalies, cluster similar items, and generate visualization data.
import { Brainy } from '@soulcraft/brainy'
const brain = new Brainy()
await brain.init()
// Access Neural API
const neural = brain.neural
// Cluster similar items
const clusters = await neural.clusters()
// Find nearest neighbors
const neighbors = await neural.neighbors(entityId, 10)
// Detect outliers
const outliers = await neural.outliers(0.2)
Clustering
clusters()
Automatically group similar entities into semantic clusters.
neural.clusters(options?: {
algorithm?: 'kmeans' | 'hierarchical' | 'sample',
maxClusters?: number,
threshold?: number
}): Promise<Cluster[]>
| Option | Type | Default | Description |
|---|---|---|---|
algorithm |
string |
'kmeans' |
Clustering algorithm to use |
maxClusters |
number |
10 |
Maximum number of clusters |
threshold |
number |
0.8 |
Similarity threshold (0-1) |
// Basic clustering
const clusters = await neural.clusters()
for (const cluster of clusters) {
console.log(`Cluster: ${cluster.label}`)
console.log(`Members: ${cluster.members.length}`)
console.log(`Cohesion: ${cluster.cohesion}`)
}
// K-means with specific cluster count
const kmeansClusters = await neural.clusters({
algorithm: 'kmeans',
maxClusters: 5
})
// Hierarchical clustering with high similarity
const hierarchical = await neural.clusters({
algorithm: 'hierarchical',
threshold: 0.9 // Only very similar items
})
// Sample-based for large datasets
const sampled = await neural.clusters({
algorithm: 'sample' // Fast approximation
})
clusterFast()
Fast clustering for medium-sized datasets (10k-100k items).
neural.clusterFast(options?: ClusterOptions): Promise<Cluster[]>
clusterLarge()
Scalable clustering for large datasets (100k+ items).
neural.clusterLarge(options?: ClusterOptions): Promise<Cluster[]>
Similarity Analysis
similar()
Calculate semantic similarity between two entities.
neural.similar(id1: string, id2: string): Promise<number>
// Compare two entities
const score = await neural.similar(entityA, entityB)
console.log(`Similarity: ${score}`) // 0.0 to 1.0
if (score > 0.9) {
console.log('Very similar!')
} else if (score > 0.7) {
console.log('Related')
} else {
console.log('Different topics')
}
neighbors()
Find the nearest neighbors to an entity.
neural.neighbors(id: string, count?: number): Promise<Neighbor[]>
// Find 10 nearest neighbors
const neighbors = await neural.neighbors(documentId, 10)
for (const neighbor of neighbors) {
console.log(`${neighbor.id}: ${neighbor.score.toFixed(3)}`)
}
// Output:
// doc_abc: 0.954
// doc_def: 0.921
// doc_ghi: 0.897
hierarchy()
Build a semantic hierarchy from an entity.
neural.hierarchy(id: string, options?: {
depth?: number,
breadth?: number
}): Promise<HierarchyNode>
// Build hierarchy from a concept
const tree = await neural.hierarchy(conceptId, {
depth: 3,
breadth: 5
})
// tree = {
// id: 'concept_123',
// label: 'Machine Learning',
// children: [
// { id: 'concept_456', label: 'Deep Learning', children: [...] },
// { id: 'concept_789', label: 'Supervised Learning', children: [...] }
// ]
// }
Anomaly Detection
outliers()
Detect outliers - entities that don't fit well with others.
neural.outliers(threshold?: number): Promise<Outlier[]>
// Detect outliers with default threshold
const outliers = await neural.outliers()
console.log(`Found ${outliers.length} outliers`)
// Stricter threshold (more outliers)
const moreOutliers = await neural.outliers(0.3)
// Looser threshold (fewer outliers)
const fewerOutliers = await neural.outliers(0.1)
// Analyze outliers
for (const outlier of outliers) {
console.log(`ID: ${outlier.id}`)
console.log(`Score: ${outlier.outlierScore}`)
console.log(`Nearest cluster: ${outlier.nearestCluster}`)
}
Visualization
visualize()
Generate visualization data for D3.js, Cytoscape.js, or other libraries.
neural.visualize(options?: {
maxNodes?: number,
dimensions?: 2 | 3,
algorithm?: 'force' | 'tsne' | 'umap'
}): Promise<VisualizationData>
| Option | Type | Default | Description |
|---|---|---|---|
maxNodes |
number |
100 |
Maximum nodes to include |
dimensions |
2 | 3 |
2 |
Output dimensions |
algorithm |
string |
'force' |
Layout algorithm |
// Generate 2D visualization
const viz2d = await neural.visualize({
maxNodes: 100,
dimensions: 2,
algorithm: 'force'
})
// Result structure
// {
// nodes: [{ id, x, y, label, cluster }],
// edges: [{ source, target, weight }]
// }
// Generate 3D visualization with t-SNE
const viz3d = await neural.visualize({
maxNodes: 200,
dimensions: 3,
algorithm: 'tsne'
})
// Use with D3.js
const svg = d3.select('svg')
svg.selectAll('circle')
.data(viz2d.nodes)
.enter()
.append('circle')
.attr('cx', d => d.x)
.attr('cy', d => d.y)
.attr('r', 5)
Entity Extraction
extractor
Extract entities from text with intelligent caching.
// Extract entities from text
const entities = await brain.neural.extractor.extract(text, {
types: ['person', 'organization', 'technology'],
confidence: 0.7,
cache: {
enabled: true,
ttl: 7 * 24 * 60 * 60 * 1000, // 7 days
invalidateOn: 'mtime'
}
})
// Check cache performance
const stats = brain.neural.extractor.getCacheStats()
console.log(`Cache hit rate: ${(stats.hitRate * 100).toFixed(1)}%`)
Complete Example
import { Brainy, NounType } from '@soulcraft/brainy'
const brain = new Brainy()
await brain.init()
// Add some documents
const docs = [
"Machine learning is a subset of AI",
"Deep learning uses neural networks",
"Natural language processing handles text",
"Computer vision processes images",
"Reinforcement learning learns from rewards",
"This is about cooking recipes", // Outlier
]
for (const doc of docs) {
await brain.add({ data: doc, nounType: NounType.Document })
}
const neural = brain.neural
// Cluster the documents
const clusters = await neural.clusters({
algorithm: 'kmeans',
maxClusters: 3
})
console.log('Clusters:')
for (const cluster of clusters) {
console.log(` ${cluster.label}: ${cluster.members.length} members`)
}
// Find outliers
const outliers = await neural.outliers(0.2)
console.log(`\nFound ${outliers.length} outliers`)
// Generate visualization data
const vizData = await neural.visualize({
maxNodes: 50,
algorithm: 'force'
})
console.log(`\nVisualization: ${vizData.nodes.length} nodes, ${vizData.edges.length} edges`)
Use Cases
- Content organization: Automatically group similar documents
- Quality control: Detect outlier data that may be incorrect
- Recommendations: Find similar items to recommend
- Knowledge mapping: Visualize relationships in your data
- Duplicate detection: Find near-duplicate content
See Also
- find() - Semantic search
- vfs.* - Virtual Filesystem
- Triple Intelligence - Query fundamentals