Skip to main content

​
Installation

Get started with the DeepAgents retrieval package for RAG and semantic search.

​
Package Installation

Install the core package: 
npm install @deepagents/retrieval

​
Dependencies

The package includes these key dependencies:
  • fastembed - Local embedding generation
  • sqlite-vec - Vector similarity search in SQLite
  • langchain - Text splitting utilities
  • unpdf - PDF text extraction
  • rss-parser - RSS feed parsing
  • @linear/sdk - Linear API client
All dependencies are automatically installed with the package.

​
Database Setup

The retrieval system requires a SQLite database. You can use either: 
npm install better-sqlite3

import Database from 'better-sqlite3';
import { SqliteStore } from '@deepagents/retrieval';

const db = new Database('./my-vectors.db');
const store = new SqliteStore(db, 384); // 384 dimensions for BGE-Small-EN-V15

​
In-Memory Database

For development or testing: 
import Database from 'better-sqlite3';
import { SqliteStore } from '@deepagents/retrieval';

const db = new Database(':memory:');
const store = new SqliteStore(db, 384);

​
Embedding Model Setup

FastEmbed downloads models automatically on first use. Models are cached locally. 
import { fastembed } from '@deepagents/retrieval';

// Default: BGE-Small-EN-V15 (384 dimensions)
const embedder = fastembed();

// Or specify a model explicitly
const embedder = fastembed({
  model: 'BGESmallENV15',
  cacheDir: './models', // Optional: custom cache directory
});

​
Available Models

  • BGESmallENV15 (384 dimensions) - Fast, good quality
  • BGEBaseENV15 (768 dimensions) - Higher quality, slower
  • AllMiniLML6V2 (384 dimensions) - Alternative lightweight model
  • MLE5Large (1024 dimensions) - Best quality, largest

​
Basic Setup Example

Complete setup with all components: 
import Database from 'better-sqlite3';
import { fastembed, SqliteStore, ingest } from '@deepagents/retrieval';
import { local } from '@deepagents/retrieval/connectors';

// 1. Create database
const db = new Database('./vectors.db');

// 2. Initialize store (384 dimensions for BGESmallENV15)
const store = new SqliteStore(db, 384);

// 3. Create embedder
const embedder = fastembed({
  model: 'BGESmallENV15',
});

// 4. Ingest content
await ingest({
  connector: local('**/*.md'),
  store,
  embedder,
});

console.log('Ingestion complete!');

​
TypeScript Configuration

Ensure your tsconfig.json includes: 
{
  "compilerOptions": {
    "module": "ESNext",
    "moduleResolution": "bundler",
    "target": "ES2022",
    "lib": ["ES2022"],
    "esModuleInterop": true
  }
}

​
Connector Imports

Connectors are available from a separate export path: 
// Core exports
import { ingest, similaritySearch, fastembed, SqliteStore } from '@deepagents/retrieval';

// Connector exports
import { github, rss, local, pdf, linear } from '@deepagents/retrieval/connectors';

​
Verify Installation

Test your setup with a quick example: 
import Database from 'better-sqlite3';
import { fastembed, SqliteStore, ingest, similaritySearch } from '@deepagents/retrieval';
import { local } from '@deepagents/retrieval/connectors';

const db = new Database(':memory:');
const store = new SqliteStore(db, 384);
const embedder = fastembed();

// Create a test file connector
const connector = local('README.md');

// Ingest
await ingest({ connector, store, embedder });

// Search
const results = await similaritySearch('installation', {
  connector,
  store,
  embedder,
});

console.log(`Found ${results.length} results`);
console.log(results[0]);

​
Troubleshooting

​
Model Download Issues

If FastEmbed fails to download models: 
// Specify a custom cache directory
const embedder = fastembed({
  cacheDir: '/path/to/models',
});

​
SQLite Version

Ensure you’re using a recent version of better-sqlite3 that supports the sqlite-vec extension.

​
Memory Usage

For large ingestion jobs, use batching: 
// Ingestion automatically batches in groups of 40
await ingest({ connector, store, embedder });

​
Next Steps

Ingestion

Learn how to ingest documents

Connectors

Explore available connectors

Build docs developers (and LLMs) love

Get started for freeTalk to us