Program fails on newer kernels, with error message:
libbpf: Netlink-based XDP prog detected, please unload it in order to launch AF_XDP prog
ERROR: Can't setup AF_XDP socket "Invalid argument"
Since kernel v5.13 libbpf version, when bpf_link support is
detected then libbpf/xsk require XDP/BPF programs use this feature.
See kernel commit 10397994d30f ("libbpf: xsk: Use bpf_link").
To continue using our netlink-based XDP attach approach,
instruct libbpf/xsk to not be in change of loading the
BPF-prog. As our XDP-prog is special it also makes
to control this ourselves.
This is achived with the flag XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD
xsk_cfg.libbpf_flags = XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD
Remember to update the xskmap manually.
Fixes: 10397994d30f ("libbpf: xsk: Use bpf_link")
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Add a large comments to the most central piece of code that
access the right offset into the metadata based on the BTF
info we have extracted.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
The root cause of the slowdown on the first packet with timestamps,
as primary related to refilling the fill-queue, which happend before
process_packet.
Primary cause, as first packet still see a slowdown of around 4 usec
while it was around 9 usec before.
In commit 5df5332f23 ("AF_XDP-interaction: config AF_XDP frame_headroom")
the fill-queue size was increase to be larger. This caused the
first call to handle_receive_packets() to refill too many frames
into the fill-queue.
Patch split out refill into function restock_receive_fill_queue()
which is now called *after* process_packet step, but before
releasing RX packets via xsk_ring_cons__release(&xsk->rx).
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
This API xsk_btf__read_member() is faster, but it was not the
root cause of the slowdown on the first packet with timestamps.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
With the current xsk_btf__read() we are seeing a slowdown on the
first packet with timestamps. The theory is this is caused
by the allocation for the cached hashmap entry.
meta-time rx_ktime:870394156129518 time_now:870394156139039 diff:9521 ns
meta-time rx_ktime:870396208293894 time_now:870396208295098 diff:1204 ns
meta-time rx_ktime:870398256286553 time_now:870398256287772 diff:1219 ns
Create a new API that can access struct members more directly
via caching the xsk_btf_member offset + size in the C-code API user.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
The program segfaulted if using the XSK_BTF_READ_INTO macro
with field as a C-string e.g. "rx_ktime".
Add comments to highlight this happens.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Based on:
Subj: libbpf: Helpers to access XDP hints based on BTF definitions
https://lore.kernel.org/all/20210803010331.39453-15-ederson.desouza@intel.com/
From: Ederson de Souza <ederson.desouza@intel.com>
Ederson says:
A new set of functions to help get the BTF definition of XDP hints
structure and get the information based on it.
`xsk_umem__btf_id` helps retrieve the BTF id of XDP metadata.
`xsk_btf__init` sets up a context based on the BTF, including a hashmap,
so that subsequent queries are faster.
`xsk_btf__read` returns a pointer to the position in the XDP metadata
containing a given field.
`xsk_btf__has_field` checks the presence of a field in the BTF.
`xsk_btf__free` frees up the context.
Besides those, a macro `XSK_BTF_READ_INTO` acts as a convenient helper
to read the field contents into a given variable.
Note that currently, the hashmap used to speed-up offset location into
the BTF doesn't use the field name as a string as key to the hashmap. It
directly uses the pointer value instead, as it is expected that most of
time, field names will be addressed by a shared constant string residing
on read-only memory, thus saving some time. If this assumption is not
entirely true, this optimisation needs to be rethought (or discarded
altogether).
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
This is a copy of the kernels/libbpf hashmap
tools/lib/bpf/hashmap.{h,c}
The plan is to prototype an AF_XDP/xsk userspace API for accessing
BTF information, that should be moved to libbpf (or libxdp). Thus,
this hashmap code will become avail if successful.
Original kernel commit 553db8ba73df ("libbpf: add resizable non-thread
safe internal hashmap") (Author: Andrii Nakryiko). Thus, giving Andrii
author credit in this git commit.
Andrii Nakryiko said:
libbpf: add resizable non-thread safe internal hashmap
There is a need for fast point lookups inside libbpf for multiple use
cases (e.g., name resolution for BTF-to-C conversion, by-name lookups in
BTF for upcoming BPF CO-RE relocation support, etc). This patch
implements simple resizable non-thread safe hashmap using single linked
list chains.
Four different insert strategies are supported:
- HASHMAP_ADD - only add key/value if key doesn't exist yet;
- HASHMAP_SET - add key/value pair if key doesn't exist yet; otherwise,
update value;
- HASHMAP_UPDATE - update value, if key already exists; otherwise, do
nothing and return -ENOENT;
- HASHMAP_APPEND - always add key/value pair, even if key already exists.
This turns hashmap into a multimap by allowing multiple values to be
associated with the same key. Most useful read API for such hashmap is
hashmap__for_each_key_entry() iteration. If hashmap__find() is still
used, it will return last inserted key/value entry (first in a bucket
chain).
For HASHMAP_SET and HASHMAP_UPDATE, old key/value pair is returned, so
that calling code can handle proper memory management, if necessary.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Don't allow loading the default xsk BPF-prog if not specifying
any BPF-prog. We have a need for our own BPf-prog.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
To verify the contents of the incomming packets add a
function print_pkt_info() that decode part of the packet
headers and print IP-header src+dst (both IPv4 and IPv6).
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
The default frame_headroom seems to be zero as
define XSK_UMEM__DEFAULT_FRAME_HEADROOM is zero.
This patch does explicit xsk_umem_config setup.
Comment taken from kernel/samples/bpf/xdpsock_user.c.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
This only works in skb-mode, other modes fail the bpf_xdp_adjust_meta
call and result in XDP_ABORTED. This only occurred on mlx5.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
BPF program title (bpf_program__title) is getting deprecated
in libbpf use bpf_program__section_name() instead.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
In-practice only 'xdp_stats_map' definition gets associated BTF,
as it uses the proper '__type' construct.
The AF_XDP map BPF_MAP_TYPE_XSKMAP doesn't know about BTF yet.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
When not loading an XDP-prog the XSK library bits will load a
default XDP-prog.
In our use-case we always want to load our XDP-prog, which is
stored in af_xdp_kern.o.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Something in fishy as it seems this BPF-object is not getting used.
Use old maps SEC defines until figuring out what is wrong,
but keep new dot-maps defines.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
The BPF-helper is now defined in bpf_helper_defs.h header:
lib/libbpf-install/usr/include/bpf/bpf_helper_defs.h
Update and keep comment as a help for people doing kernel
devel on BPF and want to test BPF-helpers not yet upstream.
Accepted in kernel v5.12
- commit 34b2021cc616 ("bpf: Add BPF-helper for MTU checking")
- https://git.kernel.org/torvalds/c/34b2021cc616
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
This USER_TARGETS_OBJS_DEPS allows makefiles to define
dependencies for USER_TARGETS_OBJS targets.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
This allows including Makefile to define USER_TARGETS_OBJS
that contain code that USER_TARGETS depend on and needs
to be compiled as objects and later linked with.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
