1
0
mirror of https://github.com/netsampler/goflow2.git synced 2024-05-06 15:54:52 +00:00

59 Commits

Author SHA1 Message Date
7d4c41a161 Merge pull request #71 from mariomac/rmalloc
Avoid unnecessary memory allocations
2022-02-08 20:39:55 -08:00
615b9f697c Avoid unnecessary memory allocations 2022-01-26 17:24:35 +01:00
8d59905c44 Merge pull request #68 from netsampler/bugfix/enricher-2
Bugfix: issues when reading a partial chunk of protobuf from stdin
2022-01-24 20:34:11 -08:00
7acb84835b Bugfix: issues when reading a partial chunk of protobuf from stdin 2022-01-24 20:30:30 -08:00
20e8e56709 Merge pull request #65 from netsampler/bugfix/offset
Bugfix: index out of range in some cases
2022-01-06 22:00:10 +01:00
27863691f0 Bugfix: index out of range in some cases 2022-01-06 21:59:02 +01:00
4308c483a4 Merge pull request #61 from netsampler/feature/upgrade-sarama
upgrade sarama to 1.30.1
2021-12-27 14:20:55 +01:00
05b436277b upgrade sarama to 1.30.1 2021-12-27 14:17:43 +01:00
a8a50f785c Merge pull request #60 from netsampler/feature/add-sctp
Map ProtoName for SCTP
2021-12-20 17:06:40 +01:00
def05e071b Map ProtoName for SCTP 2021-12-20 16:53:00 +01:00
2e1cf5bbfd Merge pull request #48 from netsampler/feature/kafka-flush-control
Add flags to control Kafka Flush parameters
2021-11-13 16:41:38 -08:00
de5e751729 Merge pull request #53 from netsampler/bugfix/optiontemplates
Bugfix: decoding OptionsTemplateSet
2021-11-13 16:41:32 -08:00
43cf8b58d9 Bugfix: decoding OptionsTemplateSet
* Was improperly decoding scope fields
* Start removing custom errors structures (will optimize template not found)
2021-11-13 15:49:27 -08:00
bf66556023 Merge pull request #52 from vincentbernat/fix/non-terminated-stream
utils: copy payload before accepting another UDP packet
2021-11-10 08:10:39 -08:00
0f5528170d utils: copy payload before accepting another UDP packet
In d1e1ace3186d ("Allow Flow Routines to be cancellable (#40)"), the
payload was passed to another goroutine and erased by the next packet
to be received if the goroutine did not process it fast enough. Make
a copy before passing it to the goroutine to fix that.
2021-11-10 15:56:49 +01:00
8fd64006ef Merge pull request #51 from leoluk/remove-kafkastate
Remove unused KafkaState
2021-11-05 09:16:34 -07:00
56a2ffa46b Remove unused KafkaState 2021-11-03 16:59:57 +01:00
d1e1ace318 Allow Flow Routines to be cancellable (#40)
* Allow Flow Routines to be cancellable
2021-10-31 16:42:07 -07:00
c145be66cd Add flags to control Kafka Flush parameters 2021-10-31 10:23:03 -07:00
92043a6233 Merge pull request #47 from netsampler/cleanup/mod-extension
cleanup: bump to go 1.17, sarama to 1.30.0, fix extensions
2021-10-30 19:51:12 -07:00
ec08b786c8 cleanup: bump to go 1.17, sarama to 1.30.0, fix extensions 2021-10-30 19:47:11 -07:00
7baa828267 Merge pull request #45 from shyam334/update-dep-sarama
deps: bump sarama to v1.29.1
2021-10-30 19:40:58 -07:00
a9f9289f6b deps: bump sarama to v1.29.1 2021-10-28 15:06:42 +11:00
17a96d9911 netflow: correctly decode options template set (#39)
netflow: correctly decode options template set
2021-09-23 20:46:39 -07:00
b0b73b2b90 Merge pull request #38 from vincentbernat/fix/defer-unlock
style: defer unlock when possible/not trivial
2021-09-23 20:44:05 -07:00
536b08812f Custom map flow fields (#36)
* adds dataframe link decoding
* can map NetFlow/IPFIX fields and bytes sections from sFlow/packets to any field inside the protobuf
* add CLI argument for loading a mapping yaml file
2021-09-23 20:41:17 -07:00
95945d3042 style: defer unlock when possible/not trivial
Defer unlocking just after taking a lock when possible (when unlock is
done at the very end) and when not trivial (the function body is more
than a couple of lines). This simplifies a bit some functions (no need
to unlock before each return) and for the other, it may avoid a bug in
the future in case a return is inserted into the body of a function.

Use of defer has been optimized a lot in Go and it is believed that
simpler defers have zero overhead since Go 1.14:
https://golang.org/doc/go1.14#runtime

> This release improves the performance of most uses of defer to incur
> almost zero overhead compared to calling the deferred function
> directly. As a result, defer can now be used in performance-critical
> code without overhead concerns.
2021-09-23 10:01:24 +02:00
defd786b2a Merge pull request #34 from jotak/podman
Makefile: parameterized DOCKER_REPO and DOCKER_BIN
2021-09-09 21:51:39 -07:00
13daf3b669 Merge pull request #33 from amorenoz/delta
ipfix: support relative flow timestamps
2021-09-09 21:51:12 -07:00
d3e3fdb0f8 Makefile: parameterized DOCKER_REPO and DOCKER_BIN
e.g. allowing to build with podman

Also add gitignore for dist/

Signed-off-by: Joel Takvorian <jtakvori@redhat.com>
2021-09-09 14:53:26 +02:00
9247511c44 ipfix: support relative flow timestamps
IPFIX supports sending flowEndDeltaMicroseconds (159) and
flowEndDeltaMicroseconds (160) to provide flow timestamps relative to
the exportTime in the IPFIX Message Header.

Use them to calculate flow TimeFlowStart and TimeFlowEnd.

Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
2021-09-08 17:12:31 +02:00
11acc67852 Merge pull request #32 from loganmc10/patch-1
Fix README for SrcPort and DstPort
2021-09-07 08:43:42 -07:00
5ad65d5100 Fix README for SrcPort and DstPort 2021-09-07 07:35:09 -06:00
faca667b19 Merge pull request #30 from netsampler/bug/enricher-decode
bugfix: enricher decoding certain protobuf
2021-08-29 08:00:47 -07:00
0b0f9dd952 Merge pull request #25 from netsampler/feature/enricher-sampling
Enricher can set sampling rate
2021-08-28 10:38:07 -07:00
21c7fee62e bugfix: enricher decoding certain protobuf
* now requires length prefixed messages
* allows custom line separator for text transport output
2021-08-27 10:04:22 -07:00
a645df2da4 Merge pull request #27 from netsampler/feature/golang-117
bump golang version to 1.17
2021-08-16 18:43:49 -07:00
84dc1ff5d7 bump golang version to 1.17 2021-08-16 18:37:42 -07:00
5a42b2b1f7 Enricher can set sampling rate 2021-08-10 22:32:43 -07:00
1ad90a7375 Create LICENSE 2021-08-10 09:34:58 -07:00
8677c8d9ba Merge pull request #23 from s-chekanov/fix_db_start
Fix clickhouse startup after restart
2021-08-02 07:38:41 -07:00
3384a90a29 Fixing clickhouse startup after restart 2021-08-02 15:33:41 +03:00
6465eeb200 Merge pull request #17 from netsampler/feature/arm
feature: add arm builds
v1.0.4
2021-06-11 23:38:57 -07:00
583059057a feature: add arm builds 2021-06-11 23:30:56 -07:00
e52a053c9c formatting improved with selectors and text output (#14)
* Improve format registration
* less interleaving (json does not require importing protobuf)
* generic text renderer
* escape for strings in text/json formatter
v1.0.3
2021-06-11 08:42:32 -07:00
9bdcdf0b22 Merge pull request #16 from netsampler/bugfix/destination-mac-ipfix
Bugfix NetFlow/IPFIX producer
2021-06-10 18:59:33 -07:00
04c8543306 Bugfix NetFlow/IPFIX producer
* Source and destination mac addresses are fed from `NFV9_FIELD_xx_yyy_MAC`
  Fixes a bug where the samples were missing either source or destination Mac.
  This is supposed to be linked to the sampling direction.
2021-06-09 18:08:55 -07:00
88cbf58172 Merge pull request #10 from simPod/mf-lf
Ensure line feed in Makefile
2021-06-07 23:21:29 -07:00
7da5c4da27 Ensure line feed in Makefile 2021-06-08 08:07:52 +02:00
645497bec6 Merge pull request #9 from netsampler/fix/protobuf
Fix protobuf
2021-06-07 22:50:42 -07:00