Security Best Practices

Overview

Operating an Avail node, especially a validator, requires careful attention to security. This guide covers binary verification, key management, network security, and operational best practices.

Binary Verification

GPG Signature Verification

All official Avail releases are GPG signed. Always verify binaries before running them.

Import the public key

Replace vX.Y.Z with the actual release version:

curl -s https://github.com/availproject/avail/releases/download/vX.Y.Z/gpg-public-key.asc | gpg --import

Verify the binary signature

gpg --verify DOWNLOADED_BINARY.tar.gz.sig DOWNLOADED_BINARY.tar.gz

Expected output:

gpg: Good signature from "Your Project Release Key <[email protected]>"

Verify the checksum file

gpg --verify YOUR_DOWNLOADED_FILE.tar.gz.sha256.sig YOUR_DOWNLOADED_FILE.tar.gz.sha256

Verify the checksum

sha256sum -c YOUR_DOWNLOADED_FILE.tar.gz.sha256

Expected output:

YOUR_DOWNLOADED_FILE.tar.gz: OK

Never run unverified binaries from unofficial sources. A compromised binary can lead to complete loss of validator funds.

Key Management

Session Keys

Validators use multiple session keys for different consensus operations:

BABE keys (Sr25519) - Block production
GRANDPA keys (Ed25519) - Finality voting
ImOnline keys - Heartbeat messages
Authority Discovery keys - Peer discovery

Generating Session Keys

Generate a new wallet

avail-node key generate --scheme Sr25519 --output-type json -w 24 > wallet.json

Use -w 24 for a 24-word mnemonic phrase. Store this securely offline.

Extract the secret phrase

cat wallet.json | jq -r '.secretPhrase' > wallet.secret
chmod 600 wallet.secret

Generate Ed25519 keys from the same seed

avail-node key inspect --scheme Ed25519 --output-type json $(cat wallet.secret) > wallet.ed25519.json

Inserting Keys into the Keystore

Insert BABE key (Sr25519)

avail-node key insert \
  --base-path /var/lib/avail \
  --chain mainnet \
  --scheme Sr25519 \
  --suri "$(cat wallet.secret)" \
  --key-type babe

Insert GRANDPA key (Ed25519)

avail-node key insert \
  --base-path /var/lib/avail \
  --chain mainnet \
  --scheme Ed25519 \
  --suri "$(cat wallet.secret)" \
  --key-type gran

Verify keys are inserted

ls /var/lib/avail/chains/*/keystore/

Node Keys (libp2p)

Generate a unique node key for P2P networking:

avail-node key generate-node-key 2> node.public.key 1> node.private.key

Secure the private key:

chmod 600 node.private.key
sudo chown avail:avail node.private.key

Use the node key:

avail-node --node-key-file /path/to/node.private.key

Key Storage Security

Critical Security Practices:

Never expose private keys - Keep all private keys encrypted and offline
Backup mnemonics securely - Store in multiple secure locations (hardware wallets, encrypted USB drives)
Restrict file permissions - chmod 600 for all key files
Use separate keys - Never reuse validator keys across networks
Hardware security modules - Consider using HSMs for production validators

Key Rotation

For validator key rotation:

Generate new session keys on your node:

curl -H "Content-Type: application/json" \
  -d '{"id":1, "jsonrpc":"2.0", "method": "author_rotateKeys"}' \
  http://localhost:9944

Submit the new keys via the session.setKeys extrinsic
Keys will be active in the next session

Network Security

Firewall Configuration

Configure firewalls to restrict access:

# Allow P2P traffic
sudo ufw allow 30333/tcp

# Allow RPC only from localhost (for validators)
sudo ufw deny 9944/tcp

# Allow Prometheus only from monitoring server
sudo ufw allow from MONITORING_IP to any port 9615

# Enable firewall
sudo ufw enable

Validator-Specific Rules

Validators should NEVER expose RPC endpoints publicly.

# Run validator with local RPC only
avail-node --validator \
  --chain mainnet \
  --rpc-port 9944 \
  --port 30333 \
  --prometheus-port 9615

Do NOT use these flags on validators:

--rpc-external
--unsafe-rpc-external
--rpc-cors=all
--rpc-methods=unsafe

Sentry Node Architecture

For production validators, use a sentry node architecture:

┌─────────────┐
│   Public    │
│  Internet   │
└──────┬──────┘
       │
┌──────▼──────────┐
│  Sentry Nodes   │ (Public RPC, P2P)
│  (Full Nodes)   │
└──────┬──────────┘
       │
┌──────▼──────────┐
│   Validators    │ (Private, reserved peers only)
│  (No public     │
│   endpoints)    │
└─────────────────┘

Configure validator to only connect to sentries:

avail-node --validator \
  --chain mainnet \
  --reserved-nodes /ip4/SENTRY_IP/tcp/30333/p2p/SENTRY_PEER_ID \
  --reserved-only \
  --no-mdns

RPC Security

Safe vs Unsafe Methods

Safe RPC methods

system_health, system_name, system_version
chain_getBlock, chain_getHeader
state_getStorage, state_getMetadata
Kate RPC methods (read-only)

Unsafe RPC methods

author_submitExtrinsic - Can submit transactions
author_insertKey - Can insert keys
system_addReservedPeer - Can modify peer list
All author_* methods

RPC Method Restrictions

# Only expose safe methods
avail-node --rpc-methods=safe

# Rate limiting
avail-node --rpc-rate-limit 100  # 100 calls/minute per connection

# Connection limits
avail-node --rpc-max-connections 100

Using RPC Proxy

For public RPC endpoints, use a reverse proxy (nginx, Cloudflare) to:

Filter unsafe methods
Implement rate limiting
Add DDoS protection
Enable TLS/SSL

System Security

User Isolation

Run the node as a dedicated user:

# Create dedicated user
sudo useradd -m -s /bin/bash avail

# Set directory permissions
sudo chown -R avail:avail /var/lib/avail

# Run as avail user
sudo -u avail avail-node --chain mainnet -d /var/lib/avail

Systemd Hardening

Secure systemd service configuration:

[Service]
Type=simple
User=avail
Group=avail
ExecStart=/usr/local/bin/avail-node --validator --chain mainnet -d /var/lib/avail

# Security hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/avail
ProtectKernelTunables=true
ProtectControlGroups=true
RestrictRealtime=true
RestrictNamespaces=true
LimitNOFILE=4096

# Resource limits
MemoryMax=16G
CPUQuota=200%

Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target

System Updates

Keep system updated

sudo apt update && sudo apt upgrade -y

Enable automatic security updates

sudo apt install unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades

Monitor security advisories

Subscribe to Avail security announcements and GitHub release notifications.

Monitoring & Alerting

Security Monitoring

Monitor these security-relevant metrics:

# Failed login attempts
- alert: ExcessiveFailedLogins
  expr: rate(node_auth_failures_total[5m]) > 5

# Unusual peer connections
- alert: SuspiciousPeerCount
  expr: substrate_peers_count > 100 or substrate_peers_count < 3

# Validator missing blocks
- alert: ValidatorOffline
  expr: substrate_block_height - substrate_finalized_height > 10

Log Monitoring

Monitor logs for suspicious activity:

# Watch for connection attempts
journalctl -u avail-node -f | grep -i "connection"

# Monitor for errors
journalctl -u avail-node --since "1 hour ago" | grep -i "error\|warn"

Backup & Recovery

Critical Data to Backup

Session keys - /var/lib/avail/chains/*/keystore/
Node key - Private libp2p key
Wallet mnemonics - Offline encrypted backup
Chain specification - Custom chainspec files

Backup Script

#!/bin/bash
BACKUP_DIR="/secure/backup/avail-$(date +%Y%m%d)"
mkdir -p "$BACKUP_DIR"

# Backup keystore
cp -r /var/lib/avail/chains/*/keystore "$BACKUP_DIR/"

# Backup node key
cp /var/lib/avail/chains/*/network/secret_ed25519 "$BACKUP_DIR/"

# Encrypt backup
tar czf - "$BACKUP_DIR" | gpg --encrypt --recipient [email protected] > "$BACKUP_DIR.tar.gz.gpg"

# Remove unencrypted backup
rm -rf "$BACKUP_DIR"

Incident Response

Compromise Response Checklist

If you suspect a security breach:

Isolate the node

sudo systemctl stop avail-node
sudo ufw deny in

Rotate all keys

Generate new session keys and node keys immediately.

Audit logs

Review system logs, auth logs, and node logs for suspicious activity.

Preserve evidence

Take disk snapshots before making changes for forensic analysis.

Report

Contact the Avail security team if the issue affects the network.

Security Checklist

Additional Resources

Substrate Security Best Practices
Polkadot Validator Security
Avail Security Advisories (GitHub Security tab)

Monitoring & Metrics

Troubleshooting

⌘I

Getting Started

Running a Node

Architecture

Pallets

Operations

Development

​Overview

​Binary Verification

​GPG Signature Verification

​Key Management

​Session Keys

​Generating Session Keys

​Generate a new wallet

​Extract the secret phrase

​Generate Ed25519 keys from the same seed

​Inserting Keys into the Keystore

​Node Keys (libp2p)

​Key Storage Security

​Key Rotation

​Network Security

​Firewall Configuration

​Validator-Specific Rules

​Sentry Node Architecture

​RPC Security

​Safe vs Unsafe Methods

​RPC Method Restrictions

​Using RPC Proxy

​System Security

​User Isolation

​Systemd Hardening

​System Updates

​Monitoring & Alerting

​Security Monitoring

​Log Monitoring

​Backup & Recovery

​Critical Data to Backup

​Backup Script

​Incident Response

​Compromise Response Checklist

​Security Checklist

​Additional Resources

Build docs developers (and LLMs) love

Overview

Binary Verification

GPG Signature Verification

Key Management

Session Keys

Generating Session Keys

Generate a new wallet

Extract the secret phrase

Generate Ed25519 keys from the same seed

Inserting Keys into the Keystore

Node Keys (libp2p)

Key Storage Security

Key Rotation

Network Security

Firewall Configuration

Validator-Specific Rules

Sentry Node Architecture

RPC Security

Safe vs Unsafe Methods

RPC Method Restrictions

Using RPC Proxy

System Security

User Isolation

Systemd Hardening

System Updates

Monitoring & Alerting

Security Monitoring

Log Monitoring

Backup & Recovery

Critical Data to Backup

Backup Script

Incident Response

Compromise Response Checklist

Security Checklist

Additional Resources