Overview RAG Chat offers several configuration options to customize its behavior, from API keys to text processing parameters and system prompts.
Environment Configuration OpenAI API Key Setup RAG Chat requires an OpenAI API key to function:
from dotenv import load_dotenv load_dotenv()
Create .env File
Create a .env file in the root directory of your project:
Add Your API Key
Add your OpenAI API key to the .env file: OPENAI_API_KEY = sk-your-api-key-here
Restart Application
Restart the Streamlit application to load the new environment variables: Never commit your .env file to version control. Add it to .gitignore to keep your API key secure.
Getting an API Key
Visit OpenAI’s platform
Sign up or log in to your account
Navigate to API Keys section
Create a new secret key
Copy the key to your .env file
You’ll need billing set up on your OpenAI account to use the API. Check OpenAI’s pricing for current rates. Vector Store Configuration Persistent Directory Documents are stored in a persistent directory:
persistant_directory = 'db'
Default Setup
Custom Location
.env Configuration
By default, the vector store is saved to a db directory in your project root. persistant_directory = 'db'
You can customize the storage location: persistant_directory = '/path/to/your/custom/db'
Or use environment variables: import os persistant_directory = os.getenv( 'VECTOR_DB_PATH' , 'db' )
Add to your .env file: VECTOR_DB_PATH = /custom/path/to/db
Vector Store Initialization def load_existing_vector_store (): if os.path.exists(persistant_directory): vector_store = Chroma( persist_directory = persistant_directory, embedding_function = OpenAIEmbeddings() ) return vector_store return None
The application checks for existing data on startup and loads it automatically.
Text Processing Configuration Chunk Size and Overlap These parameters control how documents are split into chunks:
text_splitter = RecursiveCharacterTextSplitter( chunk_size = 1000 , chunk_overlap = 400 )
Maximum number of characters per chunk
Larger values : More context per chunk, but less granular retrievalSmaller values : More granular retrieval, but less contextRecommended range : 500-2000 characters Number of characters to overlap between chunks
Larger values : Better context preservation across chunk boundariesSmaller values : Less redundancy, faster processingRecommended : 20-40% of chunk_size Customizing Chunking Parameters Larger Chunks
Smaller Chunks
Balanced
For documents where you want more context per chunk: text_splitter = RecursiveCharacterTextSplitter( chunk_size = 2000 , # Larger chunks chunk_overlap = 500 # Proportional overlap )
Use when:
Documents have long, connected narratives
You need more context in each retrieval
Working with complex technical content
For more precise retrieval: text_splitter = RecursiveCharacterTextSplitter( chunk_size = 500 , # Smaller chunks chunk_overlap = 100 # Less overlap )
Use when:
Documents have distinct, separate topics
You want precise, targeted retrieval
Processing FAQ-style content
The default configuration (recommended for most use cases): text_splitter = RecursiveCharacterTextSplitter( chunk_size = 1000 , chunk_overlap = 400 )
Good balance between context and granularity. Experiment with different chunk sizes to find what works best for your specific documents and use case.
System Prompt Customization The system prompt controls how the AI responds to questions:
system_prompt = ''' Use o contexto para responder as perguntas. Se não encontrar uma resposta no contexto, explique que não há informações disponíveis. Responda em formato de markdown e com visualizações elaboradas e interativas. Contexto: {context} '''
Custom System Prompts English Version
Concise Responses
Technical Focus
Educational
Translate the system prompt to English: system_prompt = ''' Use the context to answer the questions. If you cannot find an answer in the context, explain that the information is not available. Respond in markdown format with elaborate and interactive visualizations. Context: {context} '''
For shorter, more direct answers: system_prompt = ''' Answer questions based on the provided context. Be concise and direct. If information is not in the context, say so. Context: {context} '''
For technical documentation: system_prompt = ''' You are a technical documentation assistant. Answer questions based solely on the provided context. Include code examples when relevant. Cite specific sections when possible. If information is not available, state this clearly. Context: {context} '''
For learning and teaching: system_prompt = ''' You are an educational assistant. Answer questions using the provided context. Explain concepts clearly with examples. Break down complex topics into simpler parts. If information is not in the context, state this. Context: {context} '''
Always keep the {context} placeholder in your system prompt - this is where retrieved document chunks are inserted.
Streamlit Configuration Page Configuration st.set_page_config( page_title = 'RAG Chat' , page_icon = '🤖' )
Title & Icon
Layout
Theme
Customize the browser tab title and icon: st.set_page_config( page_title = 'My Custom RAG App' , page_icon = '📚' )
Use wide layout for more screen space: st.set_page_config( page_title = 'RAG Chat' , page_icon = '🤖' , layout = 'wide' )
Configure initial sidebar state: st.set_page_config( page_title = 'RAG Chat' , page_icon = '🤖' , initial_sidebar_state = 'expanded' )
st.header( '🤖 Como posso te ajudar hoje?' )
Customize the main header:
st.header( '📚 Ask me anything about your documents' )
Current Configuration Currently, the .env file only requires:
# Required: OpenAI Configuration OPENAI_API_KEY = sk-your-api-key-here
This is the only environment variable currently used by RAG Chat. Other configuration options require code modification (see Advanced Configuration below).
Advanced Configuration The following customizations require modifying the source code in app.py. These are not built-in features but examples of how you can extend RAG Chat.
Using Environment Variables You can modify app.py to use environment variables for configuration:
import os from dotenv import load_dotenv load_dotenv() # Configurable chunk size chunk_size = int (os.getenv( 'CHUNK_SIZE' , 1000 )) chunk_overlap = int (os.getenv( 'CHUNK_OVERLAP' , 400 )) text_splitter = RecursiveCharacterTextSplitter( chunk_size = chunk_size, chunk_overlap = chunk_overlap ) # Configurable default model default_model = os.getenv( 'DEFAULT_MODEL' , 'gpt-3.5-turbo' )
Embedding Configuration Customize the embedding model:
# Use a specific embedding model embedding_function = OpenAIEmbeddings( model = "text-embedding-3-small" ) vector_store = Chroma( persist_directory = persistant_directory, embedding_function = embedding_function )
Configuration Checklist
API Key
✅ Set up .env file with OPENAI_API_KEY
Storage
✅ Configure persistant_directory for vector store
Chunking
✅ Adjust chunk_size and chunk_overlap if needed
System Prompt
✅ Customize system_prompt for your use case
UI
✅ Configure Streamlit page settings
Troubleshooting API Key Issues
Verify the key in your .env file
Ensure no extra spaces or quotes
Check that the key starts with sk-
Verify the key is active in your OpenAI account
Ensure .env is in the root directory
Restart the application after creating .env
Check that python-dotenv is installed
Storage Issues
Check directory permissions
Ensure sufficient disk space
Verify the path is valid
Vector Store Not Persisting
Confirm persist_directory is set correctly
Check that the directory is not being deleted
Verify write permissions
Next Steps
Uploading Documents Learn how to upload and process documents
API Reference Explore the complete API reference