Skip to main content
The nft collector provides insight into Netfilter rules and actions by tracking packet flow through nftables.

Overview

The nft collector helps debug firewall behavior by reporting which nftables rules are evaluated and what actions are taken on packets. This is invaluable for understanding why packets are allowed, dropped, or transformed by your firewall.

What Data is Retrieved

The nft collector retrieves:
  • Table information: Table name and handle
  • Chain information: Chain name and handle
  • Rule handle: Specific rule that matched
  • Verdict: Action taken (accept, drop, jump, etc.)
  • Policy indication: Whether the verdict comes from chain policy

Probe Installation

The nft collector automatically installs a kprobe on the __nft_trace_packet kernel function.
This function is called by the kernel when nftrace is enabled for packets.

Prerequisites

Kernel Configuration

CONFIG_NF_TABLES
kernel config
required
Must be enabled (y) or available as a module (m).If built as a module, the nf_tables module must be loaded.
Check if nftables is available:
# Check kernel config
grep CONFIG_NF_TABLES /boot/config-$(uname -r)

# Check if module is loaded
lsmod | grep nf_tables

# Load module if needed
modprobe nf_tables

NFTables Trace Rule

For the collector to work, a special nftrace rule must be added. This can be done manually or automatically.

Manual Setup

Create a table that enables tracing:
nft add table inet Retis_Table
nft add chain inet Retis_Table Retis_Chain
nft add rule inet Retis_Table Retis_Chain meta nftrace set 1

Automatic Setup

Use the --allow-system-changes flag:
retis collect -c nft --allow-system-changes
This automatically:
  • Creates the Retis_Table table
  • Adds the Retis_Chain chain
  • Installs the nftrace rule
  • Removes the table when Retis stops
Without --allow-system-changes or a manual trace rule, nft events won’t be reported. Retis will warn you if the rule is missing.

Command-Line Options

--nft-verdicts

--nft-verdicts
string
default:"drop,accept"
Comma-separated list of verdicts whose events will be collected.Supported values:
  • accept: Accept verdicts
  • drop: Drop verdicts
  • continue: Continue to next rule
  • break: Break out of current chain
  • jump: Jump to another chain
  • goto: Goto another chain
  • return: Return from chain
  • stolen: Packet was stolen (rarely visible with filters)
  • queue: Queue to userspace
  • repeat: Repeat rule evaluation
  • all: All verdicts
Note: Stolen verdicts might not be visible if a packet filter is specified using -f.

Event Sections Produced

The nft collector produces the nft event section. See nft event documentation for detailed format.

Event Format

table {table_name} ({table_handle}) chain {chain_name} ({chain_handle})
    handle {rule_handle} {verdict} chain {chain_name}
Or for policy verdicts:
table {table_name} ({table_handle}) chain {chain_name} ({chain_handle})
    {verdict} (policy)

Usage Examples

Basic Firewall Tracing

Trace all accept and drop verdicts:
retis collect -c skb,nft --allow-system-changes

Trace All Verdicts

See every firewall decision:
retis collect -c skb,nft --nft-verdicts all --allow-system-changes

Filter Specific Traffic

Trace only SSH traffic:
retis collect -c skb,nft --allow-system-changes -f 'tcp port 22'

With Full Context

Include packet and conntrack information:
retis collect -c skb,ct,nft --skb-sections all --allow-system-changes

Track Through Stack

Follow packets through firewall and stack:
retis collect -c skb,skb-tracking,nft --allow-system-changes -o events.json
retis sort events.json

Save for Analysis

retis collect -c skb,nft --allow-system-changes -o firewall.json --print

Example Output

Basic Output

53529978697438 [swapper/0] 0 [k] __nft_trace_packet
  table firewalld (2) chain filter_PREROUTING (164) accept (policy)

53529978701985 [swapper/0] 0 [k] __nft_trace_packet
  table firewalld (2) chain filter_INPUT (165) handle 169 accept
This shows:
  • First packet hit the PREROUTING chain’s accept policy
  • Second packet matched rule 169 in the INPUT chain and was accepted

With Packet Information

53529978697438 [swapper/0] 0 [k] __nft_trace_packet
  if 2 (eth0) 192.168.1.100 > 192.168.1.1 ttl 64 tos 0x0 id 12345 off 0 len 60 proto TCP (6) 54321 > 22 flags [S]
  table firewalld (2) chain filter_INPUT (165) handle 169 accept
Now we see it’s a TCP SYN to port 22 that was accepted by rule 169.

Drop Example

53529978697438 [swapper/0] 0 [k] __nft_trace_packet
  if 2 (eth0) 203.0.113.100 > 192.168.1.1 ttl 64 tos 0x0 id 12345 off 0 len 60 proto TCP (6) 54321 > 23 flags [S]
  table firewalld (2) chain filter_INPUT (165) handle 180 drop
A packet to port 23 (telnet) was dropped by rule 180.

Linking Events to Rules

To match events to actual firewall rules, dump your ruleset with handles:
nft -a list ruleset

Example Mapping

Given this event:
table firewalld (2) chain filter_INPUT (165) handle 169 accept
Find the rule in your ruleset:
table inet firewalld { # handle 2
    chain filter_INPUT { # handle 165
        type filter hook input priority filter + 10; policy accept;
        ct state { established, related } accept # handle 169  <--
        ct status dnat accept # handle 170
        iifname "lo" accept # handle 171
    }
}
The packet matched the ct state { established, related } accept rule.

Integration with Other Collectors

skb

Essential for seeing what packets are affected:
retis collect -c skb,nft --skb-sections all --allow-system-changes

ct (Conntrack)

See conntrack state alongside firewall decisions:
retis collect -c skb,ct,nft --allow-system-changes
Particularly useful since many rules use conntrack state.

skb-tracking

Follow packets through multiple rules:
retis collect -c skb,skb-tracking,nft --allow-system-changes -o events.json
retis sort events.json

skb-drop

See both firewall drops and kernel drops:
retis collect -c skb,skb-drop,nft --allow-system-changes
Differentiate between firewall drops (nft drop verdict) and other kernel drops.

dev

Include interface information:
retis collect -c skb,dev,nft --allow-system-changes
Useful for multi-interface firewall rules.

ns

Debug namespace-aware firewalls:
retis collect -c skb,ns,nft --allow-system-changes

Use Cases

Debug Blocked Traffic

retis collect -c skb,nft --allow-system-changes -f 'host 10.0.0.1'
Find which rule is blocking a specific host.

Verify Firewall Rules

retis collect -c skb,nft --nft-verdicts all --allow-system-changes -f 'tcp port 80'
See all rules that web traffic traverses.

Performance Analysis

retis collect -c skb,skb-tracking,nft --allow-system-changes -o events.json
retis sort events.json
Count how many rules packets traverse.

NAT Troubleshooting

retis collect -c skb,ct,nft --allow-system-changes -f 'tcp port 80'
See firewall rules and NAT transformations together.

Policy Validation

retis collect -c skb,nft --allow-system-changes
Verify that your firewall policy is applied as expected.

Understanding Verdicts

accept

Packet is accepted and continues processing.

drop

Packet is silently dropped (no ICMP error sent).

jump vs goto

  • jump: Saves position and returns after target chain completes
  • goto: Doesn’t save position, no automatic return

continue vs return

  • continue: Move to next rule in same chain
  • return: Exit current chain and return to caller

queue

Packet is queued to userspace for processing.

stolen

Packet was consumed by a rule (rare).

Technical Details

Kernel Types

The nft collector activates when these types appear in probe arguments:
  • struct nft_traceinfo *

Automatic Probe

The collector automatically installs:
kprobe:__nft_trace_packet
This function is only called when nftrace is enabled.

Parameter Offsets

The collector uses BTF to find parameter offsets:
  • struct nft_chain *: Chain information
  • struct nft_rule_dp *: Rule information
  • struct nft_verdict *: Verdict information
  • enum nft_trace_types: Trace type

Source Code References

  • Collector: retis/src/collect/collector/nft/nft.rs
  • eBPF hook: retis/src/collect/collector/nft/bpf/nft_hook.bpf.c
  • Event factory: retis/src/collect/collector/nft/bpf.rs

Best Practices

  1. Use --allow-system-changes: Easiest way to enable tracing
  2. Combine with skb: See what packets are affected
  3. Filter traffic: Reduce noise for specific investigations
  4. Dump ruleset with handles: Use nft -a list ruleset
  5. Understand verdict flow: Know how jump/goto/return work
  6. Check all verdicts initially: Use --nft-verdicts all then narrow down

Performance Considerations

  • Trace overhead: nftrace has kernel overhead
  • Single probe: Only one kprobe installed
  • Verdict filtering: Reduces events in userspace
  • Efficient extraction: Uses BTF for parameter offsets
  • Production use: Consider performance impact of tracing

Troubleshooting

No NFT Events

If you’re not seeing nft events:
  1. Verify nftrace rule exists:
    nft list table inet Retis_Table
    
  2. Use --allow-system-changes:
    retis collect -c nft --allow-system-changes
    
  3. Check if nf_tables module is loaded:
    lsmod | grep nf_tables
    

Module Not Loaded

# Load the module
modprobe nf_tables

# Make it persistent
echo "nf_tables" >> /etc/modules-load.d/nftables.conf

Can’t Find Rules

If rule handles in events don’t match your ruleset:
  1. Dump ruleset with handles: nft -a list ruleset
  2. Ensure you’re looking at the right table family (inet, ip, ip6)
  3. Rules might have been modified since collection

Binary Not Available

If --allow-system-changes fails:
# Install nftables
apt-get install nftables    # Debian/Ubuntu
yum install nftables         # RHEL/CentOS

See Also

Build docs developers (and LLMs) love