Now that program process all queues it is relevant to know
what queue_id received the packet.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Cannot share/associate the FQ (Fill Queue) and CQ (Completion Q) with
umem container (xsk_umem_info). This is the reason current program
cannot handle packets for each queue ID.
As described in [1] we need more FQ and CQ ring pairs, a pair per for each
unique netdev and queue IP tuple.
Thus, add FQ and CQ for each queue id in struct xsk_socket_info.
[1] https://www.kernel.org/doc/html/latest/networking/af_xdp.html
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
It is confusing to name the option what will wait for packets
on the file descriptor, for --poll, just because the function
call have this name. It confusing as AF_XDP users often want
to busy-poll for packets (for max performance reasons).
Name the new option --wakeup instead of --wait as the effect
of waiting for the FD is that the kernel needs to wakeup
the userspace process.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
The macros XSK_BTF_READ_xxx doesn't handle or report on errors
if the input or field were wrong, which is not uncommon for macros.
The bad behavior is that the macro continue to dereference the
pointer even-when xsk_btf__read_xxx() functions returns an error.
This often leads to crashing the application.
Change macro to only dereference when no errors were reported.
Side-note: The compiler warnings will detect if API user didn't
init the 'dest' value with a default value. As the effect of
the change is that the 'dest' value will not be touch, and thus
contain the value before the macro was used.
Example:
warning: ‘mark’ may be used uninitialized in this function
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Zero init xdp_hints_rx_time, which means xdp_hints_rx_time.btf_type_id
is zero if setup_btf_info() didn't find the struct name.
In print_meta_info_via_btf() we also exit if btf_id is zero, as
this isn't a valid BTF type id.
Thus, we don't need to explicitly handle "error" case of a btf_type_id
being zero, it will just naturally not match when userspace could not
find the struct name.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
The 'struct xdp_hints_rx_time' exist both in kernel-side (af_xdp_kern.c)
and userspace program af_xdp_user.c, but are very different.
The kernel-side is the actual XDP-hints memory layout used in metadata area.
The userspace side uses the same struct name, but contains BTF info
that allow us to access the struct members from kernel-side struct.
The BTF info is extracted on userspace startup, from the BPF-prog
ELF data BTF section.
Caching BTF lookups at startup is done, as target application
is a real-time application.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
The local BTF code in af_xdp_user.c was only used for
debugging the BTF structures while developing and testing
the lib_xsk_extend.c code.
If is confusing for other reading the code, so simply
remove thisi, as the lib_xsk_extend.c should hide
these details fpr API users.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Program fails on newer kernels, with error message:
libbpf: Netlink-based XDP prog detected, please unload it in order to launch AF_XDP prog
ERROR: Can't setup AF_XDP socket "Invalid argument"
Since kernel v5.13 libbpf version, when bpf_link support is
detected then libbpf/xsk require XDP/BPF programs use this feature.
See kernel commit 10397994d30f ("libbpf: xsk: Use bpf_link").
To continue using our netlink-based XDP attach approach,
instruct libbpf/xsk to not be in change of loading the
BPF-prog. As our XDP-prog is special it also makes
to control this ourselves.
This is achived with the flag XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD
xsk_cfg.libbpf_flags = XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD
Remember to update the xskmap manually.
Fixes: 10397994d30f ("libbpf: xsk: Use bpf_link")
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Add a large comments to the most central piece of code that
access the right offset into the metadata based on the BTF
info we have extracted.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
The root cause of the slowdown on the first packet with timestamps,
as primary related to refilling the fill-queue, which happend before
process_packet.
Primary cause, as first packet still see a slowdown of around 4 usec
while it was around 9 usec before.
In commit 5df5332f23 ("AF_XDP-interaction: config AF_XDP frame_headroom")
the fill-queue size was increase to be larger. This caused the
first call to handle_receive_packets() to refill too many frames
into the fill-queue.
Patch split out refill into function restock_receive_fill_queue()
which is now called *after* process_packet step, but before
releasing RX packets via xsk_ring_cons__release(&xsk->rx).
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
This API xsk_btf__read_member() is faster, but it was not the
root cause of the slowdown on the first packet with timestamps.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
With the current xsk_btf__read() we are seeing a slowdown on the
first packet with timestamps. The theory is this is caused
by the allocation for the cached hashmap entry.
meta-time rx_ktime:870394156129518 time_now:870394156139039 diff:9521 ns
meta-time rx_ktime:870396208293894 time_now:870396208295098 diff:1204 ns
meta-time rx_ktime:870398256286553 time_now:870398256287772 diff:1219 ns
Create a new API that can access struct members more directly
via caching the xsk_btf_member offset + size in the C-code API user.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
