Zero init xdp_hints_rx_time, which means xdp_hints_rx_time.btf_type_id
is zero if setup_btf_info() didn't find the struct name.
In print_meta_info_via_btf() we also exit if btf_id is zero, as
this isn't a valid BTF type id.
Thus, we don't need to explicitly handle "error" case of a btf_type_id
being zero, it will just naturally not match when userspace could not
find the struct name.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
The 'struct xdp_hints_rx_time' exist both in kernel-side (af_xdp_kern.c)
and userspace program af_xdp_user.c, but are very different.
The kernel-side is the actual XDP-hints memory layout used in metadata area.
The userspace side uses the same struct name, but contains BTF info
that allow us to access the struct members from kernel-side struct.
The BTF info is extracted on userspace startup, from the BPF-prog
ELF data BTF section.
Caching BTF lookups at startup is done, as target application
is a real-time application.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
The local BTF code in af_xdp_user.c was only used for
debugging the BTF structures while developing and testing
the lib_xsk_extend.c code.
If is confusing for other reading the code, so simply
remove thisi, as the lib_xsk_extend.c should hide
these details fpr API users.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Program fails on newer kernels, with error message:
libbpf: Netlink-based XDP prog detected, please unload it in order to launch AF_XDP prog
ERROR: Can't setup AF_XDP socket "Invalid argument"
Since kernel v5.13 libbpf version, when bpf_link support is
detected then libbpf/xsk require XDP/BPF programs use this feature.
See kernel commit 10397994d30f ("libbpf: xsk: Use bpf_link").
To continue using our netlink-based XDP attach approach,
instruct libbpf/xsk to not be in change of loading the
BPF-prog. As our XDP-prog is special it also makes
to control this ourselves.
This is achived with the flag XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD
xsk_cfg.libbpf_flags = XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD
Remember to update the xskmap manually.
Fixes: 10397994d30f ("libbpf: xsk: Use bpf_link")
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Add a large comments to the most central piece of code that
access the right offset into the metadata based on the BTF
info we have extracted.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
The root cause of the slowdown on the first packet with timestamps,
as primary related to refilling the fill-queue, which happend before
process_packet.
Primary cause, as first packet still see a slowdown of around 4 usec
while it was around 9 usec before.
In commit 5df5332f23 ("AF_XDP-interaction: config AF_XDP frame_headroom")
the fill-queue size was increase to be larger. This caused the
first call to handle_receive_packets() to refill too many frames
into the fill-queue.
Patch split out refill into function restock_receive_fill_queue()
which is now called *after* process_packet step, but before
releasing RX packets via xsk_ring_cons__release(&xsk->rx).
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
This API xsk_btf__read_member() is faster, but it was not the
root cause of the slowdown on the first packet with timestamps.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
With the current xsk_btf__read() we are seeing a slowdown on the
first packet with timestamps. The theory is this is caused
by the allocation for the cached hashmap entry.
meta-time rx_ktime:870394156129518 time_now:870394156139039 diff:9521 ns
meta-time rx_ktime:870396208293894 time_now:870396208295098 diff:1204 ns
meta-time rx_ktime:870398256286553 time_now:870398256287772 diff:1219 ns
Create a new API that can access struct members more directly
via caching the xsk_btf_member offset + size in the C-code API user.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
The program segfaulted if using the XSK_BTF_READ_INTO macro
with field as a C-string e.g. "rx_ktime".
Add comments to highlight this happens.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Based on:
Subj: libbpf: Helpers to access XDP hints based on BTF definitions
https://lore.kernel.org/all/20210803010331.39453-15-ederson.desouza@intel.com/
From: Ederson de Souza <ederson.desouza@intel.com>
Ederson says:
A new set of functions to help get the BTF definition of XDP hints
structure and get the information based on it.
`xsk_umem__btf_id` helps retrieve the BTF id of XDP metadata.
`xsk_btf__init` sets up a context based on the BTF, including a hashmap,
so that subsequent queries are faster.
`xsk_btf__read` returns a pointer to the position in the XDP metadata
containing a given field.
`xsk_btf__has_field` checks the presence of a field in the BTF.
`xsk_btf__free` frees up the context.
Besides those, a macro `XSK_BTF_READ_INTO` acts as a convenient helper
to read the field contents into a given variable.
Note that currently, the hashmap used to speed-up offset location into
the BTF doesn't use the field name as a string as key to the hashmap. It
directly uses the pointer value instead, as it is expected that most of
time, field names will be addressed by a shared constant string residing
on read-only memory, thus saving some time. If this assumption is not
entirely true, this optimisation needs to be rethought (or discarded
altogether).
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
This is a copy of the kernels/libbpf hashmap
tools/lib/bpf/hashmap.{h,c}
The plan is to prototype an AF_XDP/xsk userspace API for accessing
BTF information, that should be moved to libbpf (or libxdp). Thus,
this hashmap code will become avail if successful.
Original kernel commit 553db8ba73df ("libbpf: add resizable non-thread
safe internal hashmap") (Author: Andrii Nakryiko). Thus, giving Andrii
author credit in this git commit.
Andrii Nakryiko said:
libbpf: add resizable non-thread safe internal hashmap
There is a need for fast point lookups inside libbpf for multiple use
cases (e.g., name resolution for BTF-to-C conversion, by-name lookups in
BTF for upcoming BPF CO-RE relocation support, etc). This patch
implements simple resizable non-thread safe hashmap using single linked
list chains.
Four different insert strategies are supported:
- HASHMAP_ADD - only add key/value if key doesn't exist yet;
- HASHMAP_SET - add key/value pair if key doesn't exist yet; otherwise,
update value;
- HASHMAP_UPDATE - update value, if key already exists; otherwise, do
nothing and return -ENOENT;
- HASHMAP_APPEND - always add key/value pair, even if key already exists.
This turns hashmap into a multimap by allowing multiple values to be
associated with the same key. Most useful read API for such hashmap is
hashmap__for_each_key_entry() iteration. If hashmap__find() is still
used, it will return last inserted key/value entry (first in a bucket
chain).
For HASHMAP_SET and HASHMAP_UPDATE, old key/value pair is returned, so
that calling code can handle proper memory management, if necessary.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Don't allow loading the default xsk BPF-prog if not specifying
any BPF-prog. We have a need for our own BPf-prog.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
To verify the contents of the incomming packets add a
function print_pkt_info() that decode part of the packet
headers and print IP-header src+dst (both IPv4 and IPv6).
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
The default frame_headroom seems to be zero as
define XSK_UMEM__DEFAULT_FRAME_HEADROOM is zero.
This patch does explicit xsk_umem_config setup.
Comment taken from kernel/samples/bpf/xdpsock_user.c.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
This only works in skb-mode, other modes fail the bpf_xdp_adjust_meta
call and result in XDP_ABORTED. This only occurred on mlx5.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
