MongoDB Database Administration

Overview

NMIS 9 uses MongoDB as its primary database for storing configuration, operational data, events, and performance metrics. This guide covers database administration, maintenance, and optimization.

Database Connection

Configuration

Database connection settings are configured in /usr/local/nmis9/conf/Config.nmis:

'db_server' => 'localhost',
'db_port' => '27017',
'db_name' => 'nmisng',
'db_username' => 'nmisng',
'db_password' => '<encrypted>',
'db_connection_timeout' => 5000,    # milliseconds
'db_query_timeout' => 5000,         # milliseconds
'db_write_concern' => 1,

Connection Timeouts

db_connection_timeout - Maximum time to establish database connection (default: 5000ms)
db_query_timeout - Maximum time for query execution (default: 5000ms)
db_ping_timeout - Timeout for connection health checks (default: 1000ms)

For long-running operations like database cleanup:

'db_query_timeout' => 300000,  # 5 minutes for dbcleanup

Write Concern

db_write_concern controls data durability:

0 - No acknowledgment (fastest, least safe)
1 - Acknowledged writes (default, balanced)
majority - Majority of replica set (safest, slowest)

Database Structure

Collections

NMIS uses the following main collections:

nodes

Stores node configuration and metadata:

{
  "_id": ObjectId("..."),
  "uuid": "<node-uuid>",
  "cluster_id": "<cluster-id>",
  "name": "router1",
  "activated": { "NMIS": true },
  "configuration": {
    "host": "192.168.1.1",
    "group": "core",
    "community": "<encrypted>",
    "collect": true,
    "ping": true
  },
  "lastupdate": ISODate("..."),
  "overrides": {},
  "addresses": [],
  "aliases": []
}

inventory

Stores device inventory data (interfaces, components, etc.):

{
  "_id": ObjectId("..."),
  "cluster_id": "<cluster-id>",
  "node_uuid": "<node-uuid>",
  "concept": "interface",
  "path": ["<path-components>"],
  "path_keys": ["<key-components>"],
  "data": {
    "index": "1",
    "ifDescr": "GigabitEthernet0/0",
    "ifSpeed": 1000000000
  },
  "enabled": true,
  "historic": 0,
  "lastupdate": NumberLong(...)
}

events

Stores active and historical events:

{
  "_id": ObjectId("..."),
  "cluster_id": "<cluster-id>",
  "node_uuid": "<node-uuid>",
  "event": "Node Down",
  "element": "",
  "details": "Node unreachable",
  "level": "Critical",
  "active": true,
  "historic": 0,
  "startdate": NumberLong(...),
  "notify": true,
  "escalate": 0
}

queue

Stores scheduled jobs and operations:

{
  "_id": ObjectId("..."),
  "type": "collect",
  "time": NumberLong(...),
  "priority": 0.5,
  "in_progress": 0,
  "args": {
    "uuid": "<node-uuid>",
    "wantsnmp": true
  }
}

opstatus

Stores operational status data:

{
  "_id": ObjectId("..."),
  "cluster_id": "<cluster-id>",
  "node_uuid": "<node-uuid>",
  "section": "interface",
  "data": {
    "index": "1",
    "operStatus": "up",
    "inputUtil": 45.2
  },
  "time": NumberLong(...)
}

sessions

Stores user authentication sessions (capped collection).

audit

Audit log for system changes and administrative actions.

Database Indexes

NMIS maintains indexes for query performance. View current indexes:

mongo nmisng --eval "db.nodes.getIndexes()"

Ensure Indexes

Create or update all required indexes:

/usr/local/nmis9/bin/nmis-cli act=ensure-indexes

Remove obsolete indexes:

/usr/local/nmis9/bin/nmis-cli act=ensure-indexes drop_unwanted=true

Key Indexes

nodes collection:

uuid (unique)
name (unique per cluster)
cluster_id
configuration.group
configuration.active

inventory collection:

node_uuid, concept
cluster_id, historic
path_keys
enabled, historic

events collection:

node_uuid, active
cluster_id, historic
event, active
startdate
expire_at (TTL index)

queue collection:

type, in_progress
time, priority
args.uuid

Database Setup

Initial Setup

Run the MongoDB setup script to create databases, users, and configure authentication:

Run setup script

sudo /usr/local/nmis9/admin/setup_mongodb.pl

This creates:

NMIS database and collections
Database users with appropriate permissions
Required indexes
Initial configuration

Enable authentication (if not already enabled)

The script will offer to enable MongoDB authentication by modifying /etc/mongod.conf:

security:
  authorization: enabled

Restart MongoDB

sudo systemctl restart mongod

Authentication Setup

If setting up authentication manually:

Create admin user

use admin
db.createUser({
  user: "admin",
  pwd: "<strong-password>",
  roles: ["root"]
})

Create NMIS database user

use nmisng
db.createUser({
  user: "nmisng",
  pwd: "<strong-password>",
  roles: [{role: "dbOwner", db: "nmisng"}]
})

Update NMIS configuration

Update /usr/local/nmis9/conf/Config.nmis with encrypted credentials:

# Use NMIS encryption utility
/usr/local/nmis9/bin/nmis-cli act=encrypt-password password="<password>"

Database Maintenance

Database Cleanup

Remove old records based on retention policies:

# Simulate cleanup (dry run)
/usr/local/nmis9/bin/nmis-cli act=dbcleanup simulate=true info=true

# Perform actual cleanup
/usr/local/nmis9/bin/nmis-cli act=dbcleanup

Purge Old Files

Remove old files and purge completed outages:

# Simulate purge
/usr/local/nmis9/bin/nmis-cli act=purge simulate=true info=true

# Perform purge
/usr/local/nmis9/bin/nmis-cli act=purge

Compact Collections

Reclaim disk space after deletions:

mongo nmisng --eval "db.events.compact()"
mongo nmisng --eval "db.opstatus.compact()"

Compact operations lock the collection and can impact performance. Run during maintenance windows.

Repair Database

If database corruption is suspected:

# Stop NMIS daemons
sudo systemctl stop nmis9d

# Repair database
mongo nmisng --eval "db.repairDatabase()"

# Start NMIS daemons
sudo systemctl start nmis9d

Performance Tuning

Query Performance

Analyze slow queries:

// Enable profiling
db.setProfilingLevel(1, 100)  // Log queries > 100ms

// View slow queries
db.system.profile.find().sort({ts:-1}).limit(10)

// Explain query plan
db.nodes.find({"configuration.group": "core"}).explain("executionStats")

Collection Statistics

View collection statistics:

mongo nmisng --eval "db.nodes.stats()"
mongo nmisng --eval "db.events.stats()"

Database Statistics

mongo nmisng --eval "db.stats(1024*1024)"  # Show in MB

WiredTiger Cache Size

Adjust WiredTiger cache in /etc/mongod.conf:

storage:
  wiredTiger:
    engineConfig:
      cacheSizeGB: 2  # Adjust based on available RAM

Recommended: 50% of RAM minus 1GB for system.

Connection Pooling

NMIS manages connection pooling automatically. For custom applications:

'db_max_pool_size' => 100,
'db_min_pool_size' => 10,

Monitoring

Database Server Status

mongo admin --eval "db.serverStatus()"

Key metrics:

Connections: current, available
Operations: queries, inserts, updates, deletes per second
Network: bytes in/out
Memory: resident, virtual, mapped
WiredTiger cache: bytes in cache, pages read/written

Current Operations

View running operations:

db.currentOp()

// Kill long-running operation
db.killOp(<opid>)

Replication Lag

For replica sets:

rs.printSlaveReplicationInfo()

Disk Usage

Monitor database disk usage:

du -sh /var/lib/mongodb/
df -h /var/lib/mongodb/

Backup and Restore

See Backup and Restore for detailed backup procedures including:

MongoDB dump and restore procedures
Scheduled backup scripts
Backup verification
Disaster recovery

Troubleshooting

Connection Failures

Check MongoDB status:

sudo systemctl status mongod
sudo journalctl -u mongod -n 100

Verify connectivity:

mongo --host localhost --port 27017 --eval "db.serverStatus()"

Authentication Failures

Verify credentials:

mongo nmisng -u nmisng -p --authenticationDatabase nmisng

Check user permissions:

use nmisng
db.getUser("nmisng")

High Memory Usage

Check WiredTiger cache size
Review slow queries and missing indexes
Consider increasing system RAM
Adjust cache size in mongod.conf

Slow Queries

Enable query profiling
Review execution plans
Add missing indexes
Consider query optimization

Disk Space Issues

Run database cleanup: act=dbcleanup
Purge old files: act=purge
Compact collections
Archive or remove old data
Consider increasing disk capacity

Best Practices

Enable authentication in production environments
Use encrypted passwords in NMIS configuration
Schedule regular cleanups to manage database growth
Monitor disk space and set up alerts
Create database indexes for custom queries
Regular backups with tested restore procedures
Monitor query performance and optimize as needed
Keep MongoDB updated to latest stable version
Use replica sets for high availability (production)
Document custom configurations and changes

Getting Started

Core Features

Configuration

Device Models

Administration

Deployment

​Overview

​Database Connection

​Configuration

​Connection Timeouts

​Write Concern

​Database Structure

​Collections

​nodes

​inventory

​events

​queue

​opstatus

​sessions

​audit

​Database Indexes

​Ensure Indexes

​Key Indexes

​Database Setup

​Initial Setup

​Authentication Setup

​Database Maintenance

​Database Cleanup

​Purge Old Files

​Compact Collections

​Repair Database

​Performance Tuning

​Query Performance

​Collection Statistics

​Database Statistics

​WiredTiger Cache Size

​Connection Pooling

​Monitoring

​Database Server Status

​Current Operations

​Replication Lag

​Disk Usage

​Backup and Restore

​Troubleshooting

​Connection Failures

​Authentication Failures

​High Memory Usage

​Slow Queries

​Disk Space Issues

​Best Practices

​See Also

Build docs developers (and LLMs) love

Overview

Database Connection

Configuration

Connection Timeouts

Write Concern

Database Structure

Collections

nodes

inventory

events

queue

opstatus

sessions

audit

Database Indexes

Ensure Indexes

Key Indexes

Database Setup

Initial Setup

Authentication Setup

Database Maintenance

Database Cleanup

Purge Old Files

Compact Collections

Repair Database

Performance Tuning

Query Performance

Collection Statistics

Database Statistics

WiredTiger Cache Size

Connection Pooling

Monitoring

Database Server Status

Current Operations

Replication Lag

Disk Usage

Backup and Restore

Troubleshooting

Connection Failures

Authentication Failures

High Memory Usage

Slow Queries

Disk Space Issues

Best Practices

See Also