2.1 KiB
Controlling TC qdisc TXQ selection via BPF
Use-case
As a policy we don't want any traffic generated by the Linux networking stack, to use transmit queue zero.
This use-case is connected with AF_XDP. The example
../AF_XDP-interaction/ is sending important Real-Time traffic on XDP-socket
queue zero. Some HW and NIC drivers (e.g. igb and igc) don't have enough
hardware TX-queues to allocate seperate queues for XDP. Thus, these queues are
shared between XDP and network stack, and there is a potential lock-contention
and also HW queue usage contention.
Example
The BPF code in this example is rather simple:
- See: tc_txq_policy_kern.c
This BPF program is meant to be loaded in the TC egress hook.
TC-BPF loader
The tc cmdline tool is notorious difficult to use, and have issues (mounting
BPF file-system) on Yocto build.
Thus, tc_txq_policy.c contains a C-code loader, that attach the BPF-prog to
the TC-hook, without depending on tc command util. Furthermore, the loader
uses bpftool skeleton feature (to generate a header file) allowing to create a
binary that contains the BPF-object itself, making it self-contained.
Different ways to view queue_mapping
Notice that queue_mapping set in BPF-prog is like RX-recorded number
(skb_rx_queue_recorded). When reaching TX-layer it will have been decremented
by one (by skb_get_rx_queue()) at the TX netstack processing stage (in
__dev_queue_xmit()).
perf probe
The perf tool can be used for recording and inspecting the skb->queue_mapping.
Remember: BPF-prog queue_mapping setting have been decremented by one at this
TX netstack processing stage.
perf probe -a 'dev_hard_start_xmit skb->dev->name:string skb->queue_mapping skb->hash'
Added new event:
probe:dev_hard_start_xmit (on dev_hard_start_xmit with name=skb->dev->name:string queue_mapping=skb->queue_mapping hash=skb->hash)
You can now use it in all perf tools, such as:
perf record -e probe:dev_hard_start_xmit -aR sleep 1
Afterwards run perf script and see results.