Skip to main content
The skb-drop collector provides information about why packets (skbs) were dropped in the kernel networking stack.

​
Overview

The skb-drop collector is essential for debugging packet loss. It captures drop events and reports the specific reason why a packet was dropped, helping you quickly identify networking issues.

​
What Data is Retrieved

The skb-drop collector retrieves:
  • Drop reason: The specific reason code explaining why the packet was dropped
  • Reason subsystem: Which subsystem dropped the packet (core kernel, OVS, etc.)

​
Probe Installation

The skb-drop collector automatically installs a probe on the skb:kfree_skb tracepoint.
This tracepoint is triggered whenever the kernel drops a packet, providing the drop reason.

​
Command-Line Options

The skb-drop collector has no specific command-line options.

​
Event Sections Produced

The skb-drop collector produces the skb-drop event section. See skb-drop event documentation for detailed format.

​
Event Format

drop (reason {drop_reason})
Example: 
drop (reason NO_SOCKET)

​
Understanding Drop Reasons

The collector acts on enum skb_drop_reason values from the kernel, including:

​
Core Drop Reasons

Common drop reasons from the core kernel:
  • NOT_SPECIFIED: Generic drop (no specific reason)
  • NO_SOCKET: No socket found for packet
  • PKT_TOO_SMALL: Packet smaller than minimum size
  • TCP_CSUM: TCP checksum error
  • TCP_INVALID_SEQUENCE: Invalid TCP sequence number
  • SOCKET_FILTER: Dropped by socket filter (BPF)
  • NETFILTER_DROP: Dropped by Netfilter/nftables
  • IPV4_CSUM: IPv4 checksum error
  • IP_INVALID: Invalid IP packet
  • UNICAST_IN_L2_MULTICAST: Unicast IP in L2 multicast

​
Subsystem-Specific Reasons

The collector also understands non-core drop reasons:
  • OVS: enum ovs_drop_reason values from OpenVSwitch
  • Other subsystems: Additional drop reason enums as they’re added to the kernel

​
Finding Drop Reason Definitions

Drop reasons are defined in the kernel:
  • Core reasons: include/net/dropreason.h
  • OVS reasons: net/openvswitch/drop.h
See the kernel source for the complete list.

​
Usage Examples

​
Basic Drop Monitoring

Monitor all packet drops: 
retis collect -c skb,skb-drop

​
With Filtering

Monitor drops for specific traffic: 
# TCP drops only
retis collect -c skb,skb-drop -f 'tcp'

# Specific host
retis collect -c skb,skb-drop -f 'host 10.0.0.1'

# Port 443
retis collect -c skb,skb-drop -f 'tcp port 443'

​
With Tracking

See the path packets took before being dropped: 
retis collect -c skb,skb-tracking,skb-drop -o events.json
retis sort events.json

​
With Additional Context

Gather more information about drops: 
# Include SKB metadata
retis collect -c skb,skb-drop --skb-sections all

# Include conntrack state
retis collect -c skb,skb-drop,ct

# Include Netfilter context
retis collect -c skb,skb-drop,nft --allow-system-changes

# Include device information
retis collect -c skb,skb-drop,dev

​
Save for Analysis

# Collect drops to file
retis collect -c skb,skb-drop -o drops.json

# Collect and print to console
retis collect -c skb,skb-drop -o drops.json --print

​
Example Output

​
Basic Drop Event

202389123456789 [swapper/0] 0 [tp] skb:kfree_skb
  if 2 (eth0) 10.0.0.100 > 10.0.0.200 ttl 64 tos 0x0 id 54321 off 0 len 60 proto TCP (6) 45678 > 80 flags [S]
  drop (reason NO_SOCKET)
This shows:
  • A TCP SYN packet was dropped
  • Drop reason: No socket listening on port 80
  • Full packet details from the skb collector

​
With Tracking

Using retis sort after collection: 
202389123400000 [ping] 1234 [tp] net:netif_receive_skb #abc123 (skb 18446629157470561024) n 0
  if 2 (eth0) 10.0.0.100 > 10.0.0.200 ttl 64 tos 0x0 id 54321 off 0 len 84 proto ICMP (1) type 8 code 0
  + 202389123410000 [swapper/0] 0 [k] ip_rcv #abc123 (skb 18446629157470561024) n 1
    if 2 (eth0) 10.0.0.100 > 10.0.0.200 ttl 64 tos 0x0 id 54321 off 0 len 84 proto ICMP (1) type 8 code 0
  + 202389123420000 [swapper/0] 0 [tp] skb:kfree_skb #abc123 (skb 18446629157470561024) n 2
    if 2 (eth0) 10.0.0.100 > 10.0.0.200 ttl 64 tos 0x0 id 54321 off 0 len 84 proto ICMP (1) type 8 code 0
    drop (reason NETFILTER_DROP)
This shows the packet’s journey through the stack until it was dropped by Netfilter.

​
Common Drop Scenarios

​
No Socket

drop (reason NO_SOCKET)
Meaning: No application is listening on the destination port. Solutions:
  • Start the application
  • Check the application is listening on the correct port
  • Verify the application is listening on the correct address

​
Netfilter Drop

drop (reason NETFILTER_DROP)
Meaning: Packet was dropped by firewall rules. Solutions:
  • Check firewall rules with nft list ruleset
  • Use the nft collector to see which rule dropped it
  • Verify firewall configuration

​
Checksum Errors

drop (reason TCP_CSUM)
or 
drop (reason IPV4_CSUM)
Meaning: Packet has an invalid checksum. Solutions:
  • Check for network hardware issues
  • Verify offloading settings (ethtool -k)
  • Look for MTU mismatches

​
Socket Filter

drop (reason SOCKET_FILTER)
Meaning: Dropped by a BPF socket filter. Solutions:
  • Check for BPF programs attached to sockets
  • Review application-level filtering
  • Use bpftool to inspect BPF programs

​
Integration with Other Collectors

​
skb

See packet content at drop: 
retis collect -c skb,skb-drop --skb-sections all
Shows full packet details including metadata.

​
skb-tracking

Trace packet path to drop: 
retis collect -c skb,skb-tracking,skb-drop -o events.json
retis sort events.json
Shows everywhere the packet went before being dropped.

​
ct (Conntrack)

See connection tracking state: 
retis collect -c skb,skb-drop,ct
Helps debug conntrack-related drops.

​
nft (Netfilter)

Identify firewall rules: 
retis collect -c skb,skb-drop,nft --allow-system-changes
When drop reason is NETFILTER_DROP, shows which rule dropped it.

​
ovs (OpenVSwitch)

Debug OVS drops: 
retis collect -c skb,skb-drop,ovs --ovs-track
Shows OVS-specific drop reasons.

​
dev

Identify interface: 
retis collect -c skb,skb-drop,dev
Shows which interface the drop occurred on.

​
ns

Namespace context: 
retis collect -c skb,skb-drop,ns
Helps debug namespace-related drops.

​
Use Cases

​
Debugging Connection Failures

retis collect -c skb,skb-drop,ct -f 'tcp and host 10.0.0.1'
Find why connections are failing.

​
Firewall Troubleshooting

retis collect -c skb,skb-drop,nft --allow-system-changes -f 'host 10.0.0.1'
Identify which firewall rule is blocking traffic.

​
Application Not Responding

retis collect -c skb,skb-drop -f 'tcp port 8080'
Check if packets are reaching the host but being dropped.

​
Performance Issues

retis collect -c skb,skb-drop,skb-tracking -o drops.json
retis sort drops.json
Analyze patterns in packet drops.

​
OpenVSwitch Issues

retis collect -c skb,skb-drop,ovs --ovs-track
Debug OVS datapath drops.

​
Technical Details

​
Kernel Types

The skb-drop collector activates when these types appear in probe arguments:
  • struct sk_buff *

​
Automatic Probe

The collector automatically installs: 
tp:skb:kfree_skb
This tracepoint is invoked by the kfree_skb_reason() kernel function.

​
Drop Reason Extraction

The collector:
  1. Hooks the kfree_skb tracepoint
  2. Extracts the drop reason from the tracepoint arguments
  3. Maps reason codes to human-readable strings
  4. Supports both core and subsystem-specific reasons

​
Source Code References

  • Collector: retis/src/collect/collector/skb_drop/skb_drop.rs
  • eBPF hook: retis/src/collect/collector/skb_drop/bpf/skb_drop_hook.bpf.c
  • Event factory: retis/src/collect/collector/skb_drop/bpf.rs

​
Best Practices

  1. Always combine with skb: See what was dropped
  2. Use tracking for context: See the path before the drop
  3. Filter appropriately: Focus on relevant traffic
  4. Combine with nft: For firewall-related drops
  5. Save to file: Enable detailed post-analysis
  6. Check systematically: Work through the stack from bottom to top

​
Performance Considerations

  • Minimal overhead: Only triggers on drops (already rare)
  • Single probe: Only adds one tracepoint
  • Efficient extraction: Drop reason is readily available
  • No filtering impact: Works with any filter

​
Troubleshooting

​
No Drop Events

If you expect drops but see none:
  1. Verify packets are reaching the system (use tcpdump)
  2. Check if packets are dropped in hardware (driver stats)
  3. Confirm filter isn’t too restrictive
  4. Ensure skb-drop collector is enabled

​
Unknown Drop Reasons

If you see numeric reason codes instead of names:
  1. Kernel might have newer drop reasons than Retis supports
  2. Update Retis to the latest version
  3. Report the unknown reason code

​
Missing Context

If drop events lack context:
  1. Enable skb-tracking to see the path
  2. Add relevant collectors (ct, nft, ovs)
  3. Use --skb-sections all for full metadata

​
See Also

Build docs developers (and LLMs) love

Get started for freeTalk to us