Skip to main content

​
Overview

The RAG (Retrieval-Augmented Generation) service is a powerful tool that helps the AI obtain the required context for generating better code suggestions. By indexing your codebase and providing relevant context to the AI, RAG significantly improves the quality and accuracy of AI responses.
The RAG service is disabled by default. You need to explicitly enable it in your configuration.

​
Prerequisites

Before enabling the RAG service, ensure you have:
1

Docker or Nix

The RAG service requires either Docker or Nix to run. For macOS users, OrbStack is recommended as a Docker alternative.
2

LLM Provider

Configure an LLM provider (OpenAI, Ollama, DashScope, or OpenRouter) for the RAG service.
3

Embedding Provider

Configure an embedding model provider for semantic search capabilities.

​
Configuration

Enable and configure the RAG service in your Avante setup: 
require('avante').setup({
  rag_service = {
    enabled = true, -- Enable the RAG service
    host_mount = os.getenv("HOME"), -- Path to mount for the service
    runner = "docker", -- "docker" or "nix"
    
    llm = {
      provider = "openai",
      endpoint = "https://api.openai.com/v1",
      api_key = "OPENAI_API_KEY", -- Environment variable name
      model = "gpt-4o-mini",
      extra = nil, -- Additional options
    },
    
    embed = {
      provider = "openai",
      endpoint = "https://api.openai.com/v1",
      api_key = "OPENAI_API_KEY",
      model = "text-embedding-3-large",
      extra = nil,
    },
    
    docker_extra_args = "", -- Extra Docker arguments
  },
})

​
Configuration Options

​
enabled
boolean
default:"false"
Enables or disables the RAG service
​
host_mount
string
default:"$HOME"
The path on the host machine that will be mounted to the container. This allows the RAG service to access your files.
You can mount:
  • Your home directory (default)
  • The project directory only
  • The root directory / for system-wide access
The mount will be read-only for security.
​
runner
string
default:"docker"
The container runtime to use. Options:
  • "docker" - Use Docker
  • "nix" - Use Nix
​
docker_extra_args
string
Additional arguments to pass to the Docker command

​
LLM Configuration

The llm block configures the language model used for RAG operations:
​
llm.provider
string
required
Model provider. Supported providers:
  • "openai"
  • "ollama"
  • "dashscope"
  • "openrouter"
​
llm.endpoint
string
required
API endpoint URL for the LLM provider
​
llm.api_key
string
required
Environment variable name containing the API key
​
llm.model
string
required
Model name to use for RAG operations
​
llm.extra
table
Additional configuration options for the LLM

​
Embedding Configuration

The embed block configures the embedding model for semantic search:
​
embed.provider
string
required
Embedding provider. Same options as llm.provider
​
embed.endpoint
string
required
API endpoint URL for the embedding provider
​
embed.api_key
string
required
Environment variable name containing the API key
​
embed.model
string
required
Embedding model name
​
embed.extra
table
Additional configuration options for the embedding model

​
Supported Providers

llm = {
  provider = "openai",
  endpoint = "https://api.openai.com/v1",
  api_key = "OPENAI_API_KEY",
  model = "gpt-4o-mini",
},
embed = {
  provider = "openai",
  endpoint = "https://api.openai.com/v1",
  api_key = "OPENAI_API_KEY",
  model = "text-embedding-3-large",
},

​
Container Management

After changing the RAG service configuration, you need to manually delete the container to ensure the new configuration is used: 
docker rm -fv avante-rag-service
The container will be recreated automatically the next time you use Avante.

​
Mount Path Considerations

Choose your host_mount path carefully based on where your projects are stored.
Path: os.getenv("HOME")Use when: All your projects are within your home directory.Pros: Secure, limits access to your user files only.Cons: Cannot access projects outside your home directory.
Path: /path/to/your/projectUse when: You want to limit RAG to a specific project.Pros: Most restrictive, best security.Cons: Need to reconfigure for different projects.
Path: /Use when: Projects are scattered across your file system.Pros: Access to all files on your system.Cons: Broader access (though still read-only).

​
Usage with @codebase

Once the RAG service is enabled, you can use the @codebase mention to leverage it: 
@codebase How does the authentication system work?
The RAG service will:
  1. Index your codebase
  2. Find relevant code snippets
  3. Provide them as context to the AI
  4. Generate a more informed response
For more information on using mentions, see the Completion Sources documentation.

​
Troubleshooting

  1. Ensure Docker/Nix is running
  2. Check that you have internet connectivity
  3. Verify API keys are set correctly
  4. Remove and recreate the container: 
    docker rm -fv avante-rag-service
  1. Consider using a faster embedding model
  2. Use gpt-4o-mini instead of larger models for LLM
  3. Reduce the scope of your host_mount
Ensure your project is within the host_mount path. If projects are outside your home directory, you may need to set host_mount = "/".

Completion Sources

Learn about @codebase and other mentions

Providers

Configure AI providers

Build docs developers (and LLMs) love

Get started for freeTalk to us