Skip to main content

​
Docker Deployment

Docker provides a reliable way to deploy NeMo Guardrails with all dependencies packaged in a container. This guide shows you how to build and run NeMo Guardrails using Docker.

​
Prerequisites

  • Docker installed on your system (Get Docker)
  • NeMo Guardrails source code or configuration files
  • Basic familiarity with Docker commands

​
Official Dockerfile

NeMo Guardrails includes an official Dockerfile that sets up the complete environment:

​
Building the Docker Image

1

Clone the Repository

If you haven’t already, clone the NeMo Guardrails repository:
git clone https://github.com/NVIDIA/NeMo-Guardrails.git
cd NeMo-Guardrails
2

Build the Image

Build the Docker image using the provided Dockerfile:
docker build -t nemoguardrails:latest .
This process may take several minutes as it installs all dependencies.
3

Verify the Build

Verify the image was created successfully:
docker images | grep nemoguardrails
The Docker image includes the all-MiniLM-L6-v2 embedding model pre-downloaded for faster startup times.

​
Running the Container

​
Using Example Configurations

The Docker image comes with example bot configurations. Run with default settings: 
docker run -p 8000:8000 nemoguardrails:latest

​
Using Custom Configurations

Mount your own configuration directory:
1

Prepare Your Configuration

Ensure your configuration directory contains the necessary files:
/path/to/your/config/
├── config.yml
├── config.co
└── kb/
2

Run with Volume Mount

Mount your configuration directory to the container:
docker run -p 8000:8000 \
  -v /path/to/your/config:/config \
  nemoguardrails:latest server --config=/config
3

Test the Deployment

Test your guardrails server:
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

​
Docker Compose

For more complex deployments, use Docker Compose: Run with: 
docker-compose up -d

​
Environment Variables

Pass environment variables for API keys and configuration: 
docker run -p 8000:8000 \
  -e OPENAI_API_KEY="your-api-key" \
  -e COHERE_API_KEY="your-cohere-key" \
  -v /path/to/config:/config \
  nemoguardrails:latest server --config=/config
Never hardcode API keys in your Dockerfile or commit them to version control. Use environment variables or secrets management.

​
Resource Limits

Set resource constraints for the container: 
docker run -p 8000:8000 \
  --memory="2g" \
  --cpus="1.5" \
  -v /path/to/config:/config \
  nemoguardrails:latest server --config=/config

​
Container Management

​
View Running Containers

docker ps

​
View Container Logs

docker logs <container-id>

​
Stop the Container

docker stop <container-id>

​
Access Container Shell

docker exec -it <container-id> /bin/bash

​
Production Considerations

When deploying to production:
  1. Use Multi-Stage Builds: Optimize image size by using multi-stage builds
  2. Pin Dependencies: Use specific version tags instead of latest
  3. Health Checks: Add Docker health check configurations
  4. Logging: Configure proper logging drivers
  5. Security: Run as non-root user and scan images for vulnerabilities
See the Production Deployment guide for more details.

​
Troubleshooting

​
Container Exits Immediately

Check the logs for errors: 
docker logs <container-id>

​
Port Already in Use

Use a different host port: 
docker run -p 8080:8000 nemoguardrails:latest

​
Permission Denied for Mounted Volumes

Ensure proper permissions on your host directory: 
chmod -R 755 /path/to/config

​
Next Steps

Production Deployment

Learn production best practices

Configuration

Configure your guardrails

Build docs developers (and LLMs) love

Get started for freeTalk to us