Authorization & Security

Overview

The Delta Sharing Reference Server provides basic authorization through bearer token authentication. For production deployments, we strongly recommend placing the server behind a secure proxy with advanced authentication mechanisms.

The reference server is not a production-ready secure web server. It’s designed for development and testing. Always use additional security layers for production deployments.

Bearer Token Authentication

The server supports a simple bearer token authentication mechanism where clients must provide a pre-configured token in the Authorization header.

Configuring Bearer Tokens

Add the authorization section to your server configuration file:

conf/delta-sharing-server.yaml

version: 1

shares:
  - name: "my_share"
    schemas:
      - name: "my_schema"
        tables:
          - name: "my_table"
            location: "s3a://bucket/table"
            id: "00000000-0000-0000-0000-000000000000"

# Authorization configuration
authorization:
  bearerToken: "dapi1234567890abcdefghijklmnopqrstuvwxyz"

host: "localhost"
port: 8080
endpoint: "/delta-sharing"

authorization.bearerToken

string

The secret token that clients must provide to access the server. Generate a strong, random token for production use.

Generating Secure Tokens

Create a cryptographically secure bearer token:

# Generate a random 32-byte token
openssl rand -hex 32

# Or use Python
python3 -c "import secrets; print('dapi' + secrets.token_hex(32))"

Using a dapi prefix is a Databricks convention but not required. Choose any prefix that helps identify your tokens.

Client Authentication

Clients must include the bearer token in all requests:

curl -H "Authorization: Bearer dapi1234567890abcdefghijklmnopqrstuvwxyz" \
  http://localhost:8080/delta-sharing/shares

In profile files, the token is stored in the bearerToken field:

profile.share

{
  "shareCredentialsVersion": 1,
  "endpoint": "http://localhost:8080/delta-sharing",
  "bearerToken": "dapi1234567890abcdefghijklmnopqrstuvwxyz"
}

No Authorization Mode

If you do not configure a bearer token in the server YAML file, all requests will be accepted without authorization.

conf/delta-sharing-server.yaml

version: 1
shares:
  - name: "public_share"
    # ...

host: "localhost"
port: 8080
endpoint: "/delta-sharing"

# No authorization section = no authentication required

Never run the server without authorization on a public network! This mode should only be used for:

Local development
Testing
Networks with external security controls

Using Environment Variables

Avoid hardcoding tokens in configuration files by using environment variables:

Option 1: Bash Substitution

Set the token as an environment variable:

export DELTA_SHARING_TOKEN="dapi1234567890abcdefghijklmnopqrstuvwxyz"

# Use in config with bash substitution
envsubst < conf/delta-sharing-server.yaml.template > conf/delta-sharing-server.yaml

Template file:

conf/delta-sharing-server.yaml.template

authorization:
  bearerToken: "${DELTA_SHARING_TOKEN}"

Option 2: Docker Environment Variables

When using Docker, pass the token as an environment variable:

docker run -p 8080:8080 \
  -e BEARER_TOKEN="dapi1234567890abcdefghijklmnopqrstuvwxyz" \
  --mount type=bind,source=$(pwd)/conf/delta-sharing-server.yaml,target=/config/delta-sharing-server-config.yaml \
  deltaio/delta-sharing-server:0.7.8 -- --config /config/delta-sharing-server-config.yaml

JWT Authentication with NGINX Proxy

For production deployments, use a reverse proxy like NGINX with JWT authentication.

Why Use JWT Authentication?

Token Expiration

JWTs can include expiration times, automatically invalidating old tokens without server-side state.

Fine-Grained Claims

Embed user identity, permissions, and other metadata in the token for access control.

Centralized Auth

Integrate with existing identity providers (Okta, Auth0, Azure AD, etc.).

Audit Logging

Track which users access which resources with embedded user information.

NGINX JWT Configuration

Install NGINX Plus or Use OpenResty

JWT authentication is available in:

NGINX Plus (commercial): Built-in JWT module
OpenResty: Free, open-source with lua-resty-jwt library

For this example, we’ll use OpenResty:

# Ubuntu/Debian
sudo apt-get install openresty

# macOS
brew install openresty/brew/openresty

Install lua-resty-jwt

luarocks install lua-resty-jwt

Configure NGINX

Create an NGINX configuration file:

/etc/openresty/nginx.conf

http {
  server {
    listen 443 ssl;
    server_name delta-sharing.example.com;
    
    ssl_certificate /etc/ssl/certs/server.crt;
    ssl_certificate_key /etc/ssl/private/server.key;
    
    # JWT validation
    location / {
      access_by_lua_block {
        local jwt = require "resty.jwt"
        local jwt_secret = os.getenv("JWT_SECRET")
        
        -- Extract JWT from Authorization header
        local auth_header = ngx.var.http_Authorization
        if not auth_header then
          ngx.status = 401
          ngx.say("Missing Authorization header")
          return ngx.exit(401)
        end
        
        local token = auth_header:match("Bearer%s+(.+)")
        if not token then
          ngx.status = 401
          ngx.say("Invalid Authorization header format")
          return ngx.exit(401)
        end
        
        -- Verify JWT
        local jwt_obj = jwt:verify(jwt_secret, token)
        if not jwt_obj.verified then
          ngx.status = 401
          ngx.say("Invalid or expired token")
          return ngx.exit(401)
        end
        
        -- Optional: Check custom claims
        if jwt_obj.payload.service ~= "delta-sharing" then
          ngx.status = 403
          ngx.say("Token not valid for this service")
          return ngx.exit(403)
        end
      }
      
      # Proxy to Delta Sharing server
      proxy_pass http://localhost:8080;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
    }
  }
}

Start NGINX and Delta Sharing Server

# Start Delta Sharing server (without bearer token)
./bin/delta-sharing-server -- --config conf/delta-sharing-server.yaml

# Start NGINX with JWT secret
export JWT_SECRET="your-jwt-secret-key"
sudo openresty -c /etc/openresty/nginx.conf

NGINX Plus JWT Configuration

If you’re using NGINX Plus, the configuration is simpler:

http {
  # Define JWT key
  map $jwt_claim_sub $jwt_subject {
    default "$jwt_claim_sub";
  }
  
  server {
    listen 443 ssl;
    server_name delta-sharing.example.com;
    
    ssl_certificate /etc/ssl/certs/server.crt;
    ssl_certificate_key /etc/ssl/private/server.key;
    
    location / {
      auth_jwt "Delta Sharing";
      auth_jwt_key_file /etc/nginx/jwt_secret.key;
      
      # Optional: Validate custom claims
      if ($jwt_claim_service != "delta-sharing") {
        return 403;
      }
      
      proxy_pass http://localhost:8080;
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-User-ID $jwt_claim_sub;
    }
  }
}

API Gateway Integration

For cloud deployments, use managed API gateways:

AWS API Gateway
Azure API Management
Google Cloud Endpoints

Create an API Gateway in front of your Delta Sharing server
Configure a Lambda authorizer or Cognito user pool
Use API Gateway’s built-in throttling and monitoring

AWS SAM Template

Resources:
  DeltaSharingApi:
    Type: AWS::ApiGatewayV2::Api
    Properties:
      Name: DeltaSharingAPI
      ProtocolType: HTTP
  
  Authorizer:
    Type: AWS::ApiGatewayV2::Authorizer
    Properties:
      ApiId: !Ref DeltaSharingApi
      AuthorizerType: JWT
      IdentitySource:
        - $request.header.Authorization
      JwtConfiguration:
        Issuer: https://cognito-idp.us-east-1.amazonaws.com/us-east-1_xxxxx
        Audience:
          - your-client-id

Create an API Management instance
Import your Delta Sharing server as a backend
Configure JWT validation policy

<inbound>
  <validate-jwt header-name="Authorization" failed-validation-httpcode="401">
    <openid-config url="https://login.microsoftonline.com/{tenant}/.well-known/openid-configuration" />
    <required-claims>
      <claim name="aud">
        <value>your-app-id</value>
      </claim>
    </required-claims>
  </validate-jwt>
</inbound>

Deploy Delta Sharing server to Cloud Run or GCE
Configure Cloud Endpoints with OpenAPI spec
Enable JWT authentication

openapi.yaml

securityDefinitions:
  firebase:
    authorizationUrl: ""
    flow: "implicit"
    type: "oauth2"
    x-google-issuer: "https://securetoken.google.com/your-project"
    x-google-jwks_uri: "https://www.googleapis.com/robot/v1/metadata/x509/[email protected]"

TLS/SSL Configuration

Always use HTTPS in production to encrypt data in transit.

The reference server does not natively support TLS. Use a reverse proxy (NGINX, Apache, or cloud load balancer) to terminate SSL.

NGINX SSL Termination

server {
  listen 443 ssl http2;
  server_name delta-sharing.example.com;
  
  # SSL certificates
  ssl_certificate /etc/ssl/certs/delta-sharing.crt;
  ssl_certificate_key /etc/ssl/private/delta-sharing.key;
  
  # Modern SSL configuration
  ssl_protocols TLSv1.2 TLSv1.3;
  ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256';
  ssl_prefer_server_ciphers off;
  
  # HSTS
  add_header Strict-Transport-Security "max-age=63072000" always;
  
  location / {
    proxy_pass http://localhost:8080;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
  }
}

# Redirect HTTP to HTTPS
server {
  listen 80;
  server_name delta-sharing.example.com;
  return 301 https://$server_name$request_uri;
}

Security Best Practices

Use Strong Tokens

Generate cryptographically random bearer tokens with at least 128 bits of entropy

Rotate Tokens Regularly

Change bearer tokens periodically and after any security incident

Use HTTPS

Always use TLS/SSL encryption for all communications

Implement Rate Limiting

Use NGINX or API Gateway to prevent abuse and DDoS attacks

Monitor Access

Log all requests and set up alerts for suspicious activity

Network Isolation

Run the server in a private network, only accessible through a secure proxy

Principle of Least Privilege

Grant minimal cloud storage permissions required for operation

Audit Configurations

Regularly review who has access and what data is shared

Monitoring and Logging

Enable comprehensive logging for security auditing:

NGINX Access Logging

log_format delta_sharing '$remote_addr - $jwt_claim_sub [$time_local] '
                        '"$request" $status $body_bytes_sent '
                        '"$http_referer" "$http_user_agent" '
                        '$request_time';

access_log /var/log/nginx/delta-sharing-access.log delta_sharing;
error_log /var/log/nginx/delta-sharing-error.log warn;

Integrate with log aggregation tools:

ELK Stack (Elasticsearch, Logstash, Kibana)
Splunk
Datadog
CloudWatch Logs (AWS)
Azure Monitor (Azure)
Cloud Logging (GCP)

Multi-Tenant Authorization

For sharing different data with different recipients:

Use Separate Server Instances

Deploy multiple server instances, each with its own configuration and bearer token

Or Use JWT Claims

Implement custom authorization logic in NGINX/API Gateway based on JWT claims

NGINX Lua Example

-- Check if user has access to requested share
local allowed_shares = jwt_obj.payload.shares or {}
local requested_share = ngx.var.uri:match("/shares/([^/]+)")

local has_access = false
for _, share in ipairs(allowed_shares) do
  if share == requested_share then
    has_access = true
    break
  end
end

if not has_access then
  ngx.status = 403
  ngx.say("Access denied to this share")
  return ngx.exit(403)
end

Or Use a Custom Proxy

Build a custom proxy service that implements fine-grained access control before forwarding to the Delta Sharing server

Compliance Considerations

For regulated industries:

GDPR Compliance

HIPAA Compliance

Use Business Associate Agreements (BAAs) with cloud providers
Enable comprehensive audit logging
Implement strong authentication (JWT, not just bearer tokens)
Encrypt all data in transit with TLS 1.2+

SOC 2 Compliance

Document security controls and configurations
Implement automated security monitoring
Regular security reviews and token rotation
Incident response procedures

Next Steps

Run with Docker

Deploy the server using Docker containers

Create Profile Files

Generate profile files for recipients

Client Quickstart

Test your server with client libraries

Protocol Reference

Understand the Delta Sharing Protocol

Get Started

Core Concepts

Python Connector

Apache Spark Connector

Reference Server

Community Connectors

​Overview

​Bearer Token Authentication

​Configuring Bearer Tokens

​Generating Secure Tokens

​Client Authentication

​No Authorization Mode

​Using Environment Variables

​Option 1: Bash Substitution

​Option 2: Docker Environment Variables

​JWT Authentication with NGINX Proxy

​Why Use JWT Authentication?

​NGINX JWT Configuration

​NGINX Plus JWT Configuration

​API Gateway Integration

​TLS/SSL Configuration

​NGINX SSL Termination

​Security Best Practices

Use Strong Tokens

Rotate Tokens Regularly

Use HTTPS

Implement Rate Limiting

Monitor Access

Network Isolation

Principle of Least Privilege

Audit Configurations

​Monitoring and Logging

​Multi-Tenant Authorization

​Compliance Considerations

​Next Steps

Run with Docker

Create Profile Files

Client Quickstart

Protocol Reference

Build docs developers (and LLMs) love

Overview

Bearer Token Authentication

Configuring Bearer Tokens

Generating Secure Tokens

Client Authentication

No Authorization Mode

Using Environment Variables

Option 1: Bash Substitution

Option 2: Docker Environment Variables

JWT Authentication with NGINX Proxy

Why Use JWT Authentication?

NGINX JWT Configuration

NGINX Plus JWT Configuration

API Gateway Integration

TLS/SSL Configuration

NGINX SSL Termination

Security Best Practices

Monitoring and Logging

Multi-Tenant Authorization

Compliance Considerations

Next Steps