Skip to main content
Aurora uses HashiCorp Vault to securely store user credentials, cloud provider tokens, and API keys.

​
Overview

Vault provides:
  • Secure storage: Encrypted secrets at rest and in transit
  • User scoping: Each user’s secrets are isolated under users/{user_id}/
  • Dynamic secrets: Cloud provider credentials are resolved at runtime
  • Audit logging: Track secret access and modifications

​
Architecture

Application → Vault KV v2 Engine → Encrypted Storage
                ↓
         aurora (mount)
                ↓
         users/{user_id}/
                ↓
    {secret_name} → secret data

​
Configuration

​
Environment Variables

​
VAULT_ADDR
string
default:"http://vault:8200"
required
Vault server address
​
VAULT_TOKEN
string
required
Vault root token for authentication. Auto-generated by make init.On first startup, check the vault-init container logs for the root token.
​
VAULT_KV_MOUNT
string
default:"aurora"
KV v2 secrets engine mount point
​
VAULT_KV_BASE_PATH
string
default:"users"
Base path for user secrets

​
Persistent Storage

Vault data is persisted using Docker volumes:
  • vault-data: Main Vault data storage
  • vault-init: Initialization state and unseal keys

​
Auto-Initialization

The vault-init container automatically:
  1. Initializes Vault on first startup
  2. Stores unseal keys in the vault-init volume
  3. Unseals Vault automatically
  4. Generates a root token

​
First Run Setup

On first startup: 
# Start Aurora
make init
make prod-prebuilt

# Check vault-init logs for the root token
docker logs aurora-vault-init-1 | grep "Root Token"

# Update .env with the root token
VAULT_TOKEN=hvs.XXXXXXXXXXXXXXXXXXXX

# Restart containers
make down
make prod-prebuilt

​
Secret Management

​
Secret Path Format

Secrets are stored in the KV v2 engine with the following structure: 
aurora/                    # KV mount
  └── users/              # Base path
      └── {user_id}/      # User-scoped secrets
          ├── aws/        # Cloud provider credentials
          ├── gcp/
          ├── azure/
          └── {custom}/   # Custom secret names

​
Database References

In the Aurora database, secrets are stored as references rather than actual values: 
vault:kv/data/aurora/users/{user_id}/{secret_name}
At runtime, Aurora resolves these references by:
  1. Parsing the reference format
  2. Authenticating to Vault with VAULT_TOKEN
  3. Fetching the secret data
  4. Using the credentials

​
Example: Storing AWS Credentials

from utils.vault import store_user_secret, get_user_secret

# Store AWS credentials
store_user_secret(
    user_id="user123",
    secret_name="aws",
    data={
        "access_key_id": "AKIAIOSFODNN7EXAMPLE",
        "secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCY"
    }
)

# Retrieve AWS credentials
aws_creds = get_user_secret(user_id="user123", secret_name="aws")
print(aws_creds["access_key_id"])  # AKIAIOSFODNN7EXAMPLE

​
Vault CLI Operations

You can interact with Vault directly using the CLI:

​
Access Vault Shell

# Enter vault container
docker exec -it aurora-vault-1 sh

# Set token
export VAULT_TOKEN=hvs.XXXXXXXXXXXXXXXXXXXX

​
List Secrets

# List all user secret keys
vault kv list aurora/users/

# List secrets for a specific user
vault kv list aurora/users/user123/

​
Read a Secret

# Read secret data
vault kv get aurora/users/user123/aws

# Read in JSON format
vault kv get -format=json aurora/users/user123/aws

​
Write a Secret

# Write a simple secret
vault kv put aurora/users/user123/test-secret value="hello"

# Write multiple fields
vault kv put aurora/users/user123/database \
  host="db.example.com" \
  username="admin" \
  password="secret123"

​
Delete a Secret

# Soft delete (can be recovered)
vault kv delete aurora/users/user123/test-secret

# Permanently delete all versions
vault kv metadata delete aurora/users/user123/test-secret

​
Test Vault Connection

# Check Vault status
vault status

# Test authentication
vault token lookup

# Test KV engine
vault kv put aurora/users/test-user/test-secret value="test"
vault kv get aurora/users/test-user/test-secret
vault kv delete aurora/users/test-user/test-secret

​
API Integration

​
Storing User Secrets

When users connect cloud providers through the UI, Aurora:
  1. Receives OAuth tokens or API keys
  2. Stores them in Vault under the user’s path
  3. Saves a Vault reference in the database
# In server/connectors/gcp_connector.py
from utils.vault import store_user_secret

def save_gcp_credentials(user_id: str, credentials: dict):
    # Store in Vault
    store_user_secret(
        user_id=user_id,
        secret_name="gcp",
        data={
            "token": credentials["access_token"],
            "refresh_token": credentials["refresh_token"],
            "project_id": credentials["project_id"]
        }
    )
    
    # Save reference in database
    save_to_db(
        user_id=user_id,
        provider="gcp",
        credential_path="vault:kv/data/aurora/users/" + user_id + "/gcp"
    )

​
Retrieving Secrets

from utils.vault import get_user_secret

def get_gcp_client(user_id: str):
    # Retrieve from Vault
    creds = get_user_secret(user_id=user_id, secret_name="gcp")
    
    # Use credentials
    return build_gcp_client(
        token=creds["token"],
        project_id=creds["project_id"]
    )

​
Security Best Practices

​
Token Management

The root token has unlimited access to Vault. In production:
  1. Store it securely (password manager, secrets manager)
  2. Rotate it regularly
  3. Use AppRole authentication for services
  4. Never commit it to version control

​
Production Considerations

  1. Use AppRole Authentication: Instead of the root token, create AppRole credentials for Aurora services
  2. Enable Audit Logging: Track all secret access
  3. Implement Secret Rotation: Regularly rotate cloud provider credentials
  4. Backup Vault Data: Regularly backup the vault-data volume
  5. Use TLS: Enable TLS for Vault API communication

​
AppRole Example (Production)

# Enable AppRole authentication
vault auth enable approle

# Create a policy for Aurora
vault policy write aurora-policy - <<EOF
path "aurora/users/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}
EOF

# Create an AppRole
vault write auth/approle/role/aurora \
  secret_id_ttl=0 \
  token_ttl=1h \
  token_max_ttl=4h \
  policies="aurora-policy"

# Get Role ID and Secret ID
vault read auth/approle/role/aurora/role-id
vault write -f auth/approle/role/aurora/secret-id

# Use in Aurora (update .env)
VAULT_ROLE_ID=<role_id>
VAULT_SECRET_ID=<secret_id>

​
Troubleshooting

​
Vault Not Initialized

# Check vault-init logs
docker logs aurora-vault-init-1

# If init failed, remove volumes and restart
make down
docker volume rm aurora-vault-data aurora-vault-init
make prod-prebuilt

​
Vault Sealed

# Check status
docker exec aurora-vault-1 vault status

# Unseal manually (if auto-unseal failed)
docker exec -it aurora-vault-1 vault operator unseal

​
Connection Errors

# Check Vault is running
docker ps | grep vault

# Check Vault logs
docker logs aurora-vault-1

# Test connection from aurora-server
docker exec aurora-server-1 wget -qO- http://vault:8200/v1/sys/health

​
Permission Denied

# Verify token is valid
docker exec -e VAULT_TOKEN=$VAULT_TOKEN aurora-vault-1 vault token lookup

# Check policy permissions
docker exec -e VAULT_TOKEN=$VAULT_TOKEN aurora-vault-1 vault policy read default

​
Migration from Database Storage

If you’re migrating from storing secrets directly in the database:
  1. Export existing secrets from the database
  2. Store in Vault using the format above
  3. Update database records to Vault references
  4. Delete plaintext secrets from the database
# Migration script example
from utils.vault import store_user_secret

def migrate_user_secrets(user_id: str, db_secrets: dict):
    for provider, credentials in db_secrets.items():
        # Store in Vault
        store_user_secret(
            user_id=user_id,
            secret_name=provider,
            data=credentials
        )
        
        # Update DB with Vault reference
        update_db_credential(
            user_id=user_id,
            provider=provider,
            credential_path=f"vault:kv/data/aurora/users/{user_id}/{provider}"
        )
        
        # Delete plaintext from DB
        delete_plaintext_credential(user_id=user_id, provider=provider)

​
Web UI

Vault’s web UI is available at http://localhost:8200/ui
  1. Enter your VAULT_TOKEN
  2. Navigate to aurora/users/
  3. View, create, and manage secrets
The Vault UI is useful for debugging but should be disabled or restricted in production.

Build docs developers (and LLMs) love

Get started for freeTalk to us