Skip to main content
The ns collector retrieves information about namespaces, currently supporting network namespaces.

Overview

The ns collector helps debug containerized and multi-tenant networking by tracking which network namespace packets and events belong to. This is essential for understanding packet flow in container environments.

What Data is Retrieved

The ns collector retrieves:
  • Network namespace inode number: Unique identifier for the network namespace
  • Network namespace cookie: Kernel cookie for the namespace (when available)
These identifiers help correlate events with specific containers, virtual machines, or network isolation boundaries.

Probe Installation

The ns collector does not install any probes. It only retrieves data when namespace information is available in probe arguments.
The collector activates automatically when probes have:
  • struct net * parameters
  • struct sk_buff * parameters (reads namespace from skb->dev)
  • struct net_device * parameters (reads namespace from device)

Command-Line Options

The ns collector has no specific command-line options.

Event Sections Produced

The ns collector produces the netns event section. See netns event documentation for detailed format.

Event Format

netns {inode_number}
With network cookie (when supported):
netns {inode_number} cookie {cookie}
Examples:
netns 4026531840
netns 4026532256 cookie 12345678

Network Namespace Cookies

Network cookies are a kernel feature that provides:
  • Stable namespace identifiers
  • Survives namespace recreation
  • Kernel-managed uniqueness
The collector automatically detects if the kernel supports network cookies by inspecting the struct net BTF type for the net_cookie field.
Network cookies are available in recent kernels. If your kernel doesn’t support them, only the inode number will be reported.

Usage Examples

Basic Namespace Tracking

Track network namespaces:
retis collect -c skb,ns

Filter by Namespace

Monitor specific namespace:
retis collect -c skb,ns -m 'sk_buff.dev.nd_net.net.ns.inum == 4026531840'

Track Namespace Transitions

See packets crossing namespace boundaries:
retis collect -c skb,skb-tracking,ns -o events.json
retis sort events.json

Container Debugging

Debug container networking:
# Combine with device info
retis collect -c skb,ns,dev

# Add connection tracking
retis collect -c skb,ns,ct

# Full container context
retis collect -c skb,skb-tracking,ns,dev -o events.json
retis sort events.json

Cross-Namespace Traffic

Track traffic between containers:
retis collect -c skb,skb-tracking,ns,dev -p 'net:*' -o events.json
retis sort events.json

Example Output

Basic Output

202389123456789 [ping] 12345 [tp] net:netif_receive_skb
  if 2 (eth0) 192.168.1.100 > 192.168.1.1 ttl 64 tos 0x0 id 12345 off 0 len 84 proto ICMP (1) type 8 code 0
  netns 4026531840
Packet in the root network namespace.
202389123456789 [curl] 12345 [tp] net:net_dev_start_xmit
  if 5 (veth1) 10.0.0.100 > 10.0.0.1 ttl 64 tos 0x0 id 12345 off 0 len 60 proto TCP (6) flags [S]
  netns 4026532256 cookie 87654321
Packet in a container’s network namespace.

Tracking Across Namespaces

Using retis sort after collection:
202389123400000 [curl] 1234 [tp] net:net_dev_start_xmit #abc123 (skb 18446629157470561024) n 0
  if 5 (veth1) 10.0.0.100 > 10.0.0.1 ttl 64 tos 0x0 id 12345 off 0 len 60 proto TCP (6) flags [S]
  netns 4026532256 cookie 87654321
  + 202389123410000 [swapper/0] 0 [tp] net:netif_receive_skb #abc123 (skb 18446629157470561024) n 1
    if 4 (veth0) 10.0.0.100 > 10.0.0.1 ttl 64 tos 0x0 id 12345 off 0 len 60 proto TCP (6) flags [S]
    netns 4026531840
Packet moves from container namespace (4026532256) to root namespace (4026531840) via veth pair.

Finding Namespace Information

List Namespaces

Find namespace inode numbers:
# List all network namespaces
ls -li /var/run/netns/

# Show process's network namespace
ls -l /proc/<pid>/ns/net

# List all namespace inodes
lsns -t net

Container Namespaces

For containers:
# Docker
docker inspect -f '{{.State.Pid}}' <container>
ls -l /proc/<pid>/ns/net

# Check with nsenter
nsenter -t <pid> -n ip link show

Root Namespace

The root network namespace typically has inode 4026531840 (but this can vary). Find your system’s root namespace:
ls -li /proc/1/ns/net

Integration with Other Collectors

skb

Essential combination:
retis collect -c skb,ns
Shows packet content and namespace.

dev

Interface and namespace:
retis collect -c skb,dev,ns
Correlate interfaces with namespaces (critical for veth pairs).

skb-tracking

Track cross-namespace flow:
retis collect -c skb,skb-tracking,ns,dev -o events.json
retis sort events.json
See complete packet path including namespace transitions.

skb-drop

Debug namespace drops:
retis collect -c skb,ns,skb-drop
Identify which namespace packets are dropped in.

ct (Conntrack)

Namespace-aware conntrack:
retis collect -c skb,ns,ct
Each namespace has independent conntrack tables.

nft (Netfilter)

Firewall rules per namespace:
retis collect -c skb,ns,nft --allow-system-changes
Different namespaces can have different firewall rules.

ovs (OpenVSwitch)

OVS and namespaces:
retis collect -c skb,ns,dev,ovs --ovs-track
Track packets through OVS across namespaces.

Use Cases

Container Networking Debug

Trace packet in container:
retis collect -c skb,skb-tracking,ns,dev -m 'sk_buff.dev.nd_net.net.ns.inum == <container_ns>' -o events.json
retis sort events.json

Inter-Container Communication

Debug container-to-container traffic:
retis collect -c skb,skb-tracking,ns,dev,ct -o events.json
retis sort events.json
See packets move between container namespaces.

veth Pair Debugging

Trace veth pair operation:
retis collect -c skb,skb-tracking,ns,dev -p 'net:*' -o events.json
retis sort events.json
Watch packets cross veth pair boundaries.

Multi-Tenant Isolation

Verify namespace isolation:
retis collect -c skb,ns,nft --allow-system-changes
Ensure packets stay within expected namespaces.

CNI Plugin Debugging

Debug Kubernetes CNI:
retis collect -c skb,skb-tracking,ns,dev,ovs -o events.json
retis sort events.json
Trace packets through CNI network setup.

Service Mesh Debugging

retis collect -c skb,ns,ct -f 'tcp'
Debug sidecar proxy networking.

Understanding Namespace Inodes

Inode Numbers

Namespace inode numbers:
  • Unique identifier for each namespace
  • Assigned when namespace is created
  • Can be reused after namespace deletion
  • Starts from 4026531840 for initial namespaces

Root vs Container Namespaces

  • Root namespace: Where the system starts
  • Container namespaces: Created for isolation
  • Each container typically gets its own network namespace

Namespace Lifetime

Namespaces exist while:
  • At least one process is in the namespace, or
  • A bind mount exists (e.g., in /var/run/netns/), or
  • A process has an open file descriptor to it

Technical Details

Kernel Types

The ns collector activates when these types appear in probe arguments:
  • struct net *
  • struct sk_buff * (navigates to network namespace)
  • struct net_device * (navigates to network namespace)

Data Extraction

The collector:
  1. Locates struct net pointer
  2. Reads namespace inode number
  3. Checks if kernel supports network cookies (BTF inspection)
  4. Reads cookie if supported
  5. Formats event section
At initialization, the collector:
  1. Uses BTF to inspect struct net
  2. Looks for net_cookie member
  3. Sets flag if found
  4. This determines if cookies are reported in events

Source Code References

  • Collector: retis/src/collect/collector/ns/ns.rs
  • eBPF hook: retis/src/collect/collector/ns/bpf/netns_hook.bpf.c
  • Event factory: Inline in ns.rs

Best Practices

  1. Always combine with dev: Interface names make sense with namespace context
  2. Use tracking: Enable skb-tracking for cross-namespace flows
  3. Filter by namespace: Use -m to focus on specific namespaces
  4. Identify namespaces first: Use lsns or /proc/<pid>/ns/net to find target namespaces
  5. Consider cookies: Use kernel with cookie support for stable identifiers

Performance Considerations

  • No probe overhead: Doesn’t install probes
  • Minimal extraction: Only reads namespace pointers
  • Small data: Adds ~16 bytes per event
  • BTF lookup once: Cookie support checked at initialization
  • Works with any probe: Activates automatically

Troubleshooting

No Namespace Information

If namespace information is missing:
  1. Verify ns collector is enabled
  2. Check probes have appropriate arguments
  3. Verify kernel has namespace support

Can’t Find Namespace

To find a process’s namespace:
# Find process PID
ps aux | grep <process>

# Get namespace inode
ls -li /proc/<pid>/ns/net

# Or use lsns
lsns -t net -p <pid>

Namespace Confusion

If namespaces seem mixed up:
  1. Verify inode numbers: ls -li /proc/*/ns/net
  2. Check namespace relationships
  3. Ensure correct PID namespace context
  4. Remember inodes can be reused

Missing Cookies

If cookies aren’t reported:
  1. Check kernel version (cookies are recent feature)
  2. Verify BTF is available
  3. Update kernel if needed
  4. Inode numbers still work without cookies

Filtering by Namespace

By Inode

retis collect -c skb,ns -m 'sk_buff.dev.nd_net.net.ns.inum == 4026532256'

Root Namespace

retis collect -c skb,ns -m 'sk_buff.dev.nd_net.net.ns.inum == 4026531840'

Non-Root Namespaces

retis collect -c skb,ns -m 'sk_buff.dev.nd_net.net.ns.inum != 4026531840'

Container Platform Examples

Docker

# Get container namespace
CNS=$(docker inspect -f '{{.State.Pid}}' mycontainer | xargs -I{} readlink /proc/{}/ns/net | cut -d'[' -f2 | cut -d']' -f1)

# Trace container traffic
retis collect -c skb,ns,dev -m "sk_buff.dev.nd_net.net.ns.inum == $CNS"

Kubernetes

# Get pod namespace
POD=$(kubectl get pod mypod -o jsonpath='{.status.containerStatuses[0].containerID}' | sed 's/docker:\/\///')
CNS=$(docker inspect -f '{{.State.Pid}}' $POD | xargs -I{} readlink /proc/{}/ns/net | cut -d'[' -f2 | cut -d']' -f1)

# Trace pod traffic
retis collect -c skb,skb-tracking,ns,dev -m "sk_buff.dev.nd_net.net.ns.inum == $CNS" -o events.json
retis sort events.json

LXC/LXD

# Get container PID
PID=$(lxc info mycontainer | grep PID | awk '{print $2}')
CNS=$(readlink /proc/$PID/ns/net | cut -d'[' -f2 | cut -d']' -f1)

# Trace container
retis collect -c skb,ns,dev -m "sk_buff.dev.nd_net.net.ns.inum == $CNS"

See Also

Build docs developers (and LLMs) love