mirror/netsampler-goflow2
1
0
Fork 0
mirror of https://github.com/netsampler/goflow2.git synced 2024-05-06 15:54:52 +00:00
Code Issues Projects Releases Wiki Activity
125 Commits 6 Branches 21 Tags
v1
Commit Graph
125 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
LouisandGitHub e1c581e031 format: fix renderer (#250) v1.3.7 2023-12-06 04:37:51 -08:00
LouisandGitHub ac5e849a3b decoder: ipfix have max size (#238) 2023-12-01 00:40:04 -08:00
LouisandGitHub 9287e05528 netflow/sflow: add parsing limits for memory safety (#203) v1.3.6 2023-08-26 09:56:57 -07:00
Vincent BernatandGitHub 66b47ee328 decoders: replace binary.Read with a version without reflection and allocations (#141) 
Instead of allocating small slices, we rely on the fact that most call
sites are providing a `bytes.Buffer` and use the `Next()` method. For
sFlow decoding, in my case, we get a 33% speedup.

A `bytes.Reader` would even be more efficient, but unfortunately, they
don't have a `Next()` method.

While Go should be smart enough to make the allocation of `bs` on the
stack, it does not, even when `io.ReadFull()` is inlines.

```
decoders/utils/utils.go:23:13: make([]byte, n) escapes to heap
```
 2023-08-18 19:38:00 -07:00
LouisandGitHub 7c52d81c0d producer: fix fragments (#190) v1.3.5 2023-08-09 19:49:00 -07:00
LouisandGitHub 1298b9408d Update issue templates (#189) 2023-08-09 18:19:18 -07:00
LouisandGitHub 2fe21e11f5 Update issue templates (#188) 2023-08-09 18:13:55 -07:00
dependabot[bot]GitHubietf-drafts
 		33c6845301 build(deps): bump google.golang.org/protobuf from 1.30.0 to 1.31.0 (#183) 
Bumps google.golang.org/protobuf from 1.30.0 to 1.31.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-09 17:50:14 -07:00
dependabot[bot]GitHubietf-drafts
 		3a285dce71 build(deps): bump github.com/oschwald/geoip2-golang from 1.8.0 to 1.9.0 (#181) 
Bumps [github.com/oschwald/geoip2-golang](https://github.com/oschwald/geoip2-golang) from 1.8.0 to 1.9.0.
- [Release notes](https://github.com/oschwald/geoip2-golang/releases)
- [Commits](https://github.com/oschwald/geoip2-golang/compare/v1.8.0...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/oschwald/geoip2-golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-06-22 23:52:06 -07:00
dependabot[bot]GitHubietf-drafts
 		2523abfc9f build(deps): bump github.com/prometheus/client_golang (#180) 
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-06-22 23:41:12 -07:00
dependabot[bot]GitHubietf-drafts
 		e5461ee848 build(deps): bump github.com/sirupsen/logrus from 1.9.2 to 1.9.3 (#176) 
Bumps [github.com/sirupsen/logrus](https://github.com/sirupsen/logrus) from 1.9.2 to 1.9.3.
- [Release notes](https://github.com/sirupsen/logrus/releases)
- [Changelog](https://github.com/sirupsen/logrus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sirupsen/logrus/compare/v1.9.2...v1.9.3)

---
updated-dependencies:
- dependency-name: github.com/sirupsen/logrus
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-06-04 19:25:07 -07:00
dependabot[bot]GitHubietf-drafts
 		dc8d7c5739 build(deps): bump github.com/stretchr/testify from 1.8.3 to 1.8.4 (#177) 
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.3 to 1.8.4.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.3...v1.8.4)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-06-04 19:12:49 -07:00
LouisandGitHub e5696f114b decoders: fix netflow legacy/v5 sampling interval (#171) v1.3.4 2023-05-25 06:10:28 -07:00
dependabot[bot]GitHubietf-draftsLouis
 		e657df6080 build(deps): bump github.com/stretchr/testify from 1.8.2 to 1.8.3 (#173) 
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.2 to 1.8.3.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.2...v1.8.3)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Louis <[email protected]>
 2023-05-21 20:35:03 -07:00
dependabot[bot]GitHubietf-drafts
 		737140c49d build(deps): bump github.com/sirupsen/logrus from 1.9.0 to 1.9.2 (#174) 
Bumps [github.com/sirupsen/logrus](https://github.com/sirupsen/logrus) from 1.9.0 to 1.9.2.
- [Release notes](https://github.com/sirupsen/logrus/releases)
- [Changelog](https://github.com/sirupsen/logrus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sirupsen/logrus/compare/v1.9.0...v1.9.2)

---
updated-dependencies:
- dependency-name: github.com/sirupsen/logrus
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-21 19:36:28 -07:00
dependabot[bot]GitHubietf-drafts
 		f77d883205 build(deps): bump github.com/prometheus/client_golang (#166) 
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.15.0 to 1.15.1.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.15.0...v1.15.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-09 23:36:15 -07:00
dependabot[bot]GitHubietf-drafts
 		ddf5c872e8 build(deps): bump github.com/libp2p/go-reuseport from 0.2.0 to 0.3.0 (#165) 
Bumps [github.com/libp2p/go-reuseport](https://github.com/libp2p/go-reuseport) from 0.2.0 to 0.3.0.
- [Release notes](https://github.com/libp2p/go-reuseport/releases)
- [Commits](https://github.com/libp2p/go-reuseport/compare/v0.2.0...v0.3.0)

---
updated-dependencies:
- dependency-name: github.com/libp2p/go-reuseport
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-04-30 23:07:35 -07:00
LouisandGitHub 69a6eaf99e templates: retrofit to avoid issue when upgrading dep (#160) v1.3.3 2023-04-20 20:05:33 -07:00
dependabot[bot]GitHubietf-drafts
 		e3f58f2564 build(deps): bump github.com/prometheus/client_golang (#159) 
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.14.0...v1.15.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-04-20 19:14:32 -07:00
dependabot[bot]GitHubietf-drafts
 		9061d30a79 build(deps): bump google.golang.org/protobuf from 1.29.1 to 1.30.0 (#158) 
Bumps google.golang.org/protobuf from 1.29.1 to 1.30.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-04-20 19:08:48 -07:00
LouisandGitHub b099a5f313 Create dependabot.yml (#157) 2023-04-20 19:06:44 -07:00
LouisandGitHub b58874c36a app: hotfix (#156) v1.3.2 2023-04-20 06:46:01 -07:00
LouisandGitHub 958109e91c allow unknown records for sFlow (#155) v1.3.1 2023-04-15 21:21:10 -07:00
dependabot[bot]GitHubietf-drafts
 		5d47e60bed build(deps): bump golang.org/x/net from 0.5.0 to 0.7.0 (#154) 
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.5.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-04-15 21:14:27 -07:00
LouisandGitHub 9850000ffa allow unknown record structures in sFlow (#153) 2023-04-15 20:59:53 -07:00
LouisandGitHub c4c4ffb4e4 Refactor template as module (#49) 
* Refactor template as module
* Abstraction to allow custom template storage (eg: file, redis, http...)
* Works similarly to Transport and Format
v1.3.0 		2023-04-15 20:52:32 -07:00
LouisandGitHub 6b3c5f1215 go.mod: update dependencies (#146) v1.2.0 2023-04-15 19:33:39 -07:00
LouisandGitHub d59f209302 mapping: support endianness (#130) 2023-03-28 22:05:47 -07:00
LouisandGitHub 40e5ef08fd mapping: allow lists to be mapped (#149) 2023-03-28 21:55:21 -07:00
LouisandGitHub 5529d4993b makefile: fix typo (#147) 2023-03-14 19:53:48 -07:00
LouisandGitHub d72f05bb0b docker: fix build and push of manifest (#145) 2023-03-14 19:42:14 -07:00
Brooks SwinnertonGitHublspgn
 		d53e5f9b5a compose: update Clickhouse schema to match casing of proto (#133) 
This commit resolves an issue with the mapping between the ClickHouse
schema and the flow.proto schema.

In 3326554, the casing of the proto fields was updated, but the
ClickHouse column names were not also updated, resulting in the
ClickHouse Kafka engine only being able to successfully deserialize
fields that without an underscore.

Also updates the provisioned dashboards. Versions of the tools are also updated.

Co-authored-by: lspgn <[email protected]>
 2023-03-06 20:39:33 -08:00
Vincent BernatandGitHub 176eb8772f docs: fix SrcVlan/VlanId mapping between protocols (#136) 
In source code:

```go
	NFV9_FIELD_SRC_VLAN                     = 58
	NFV9_FIELD_DST_VLAN                     = 59

	IPFIX_FIELD_vlanId                                = 58
	IPFIX_FIELD_postVlanId                            = 59

		case netflow.NFV9_FIELD_SRC_VLAN:
			DecodeUNumber(v, &(flowMessage.VlanId))
			DecodeUNumber(v, &(flowMessage.SrcVlan))
		case netflow.NFV9_FIELD_DST_VLAN:
			DecodeUNumber(v, &(flowMessage.DstVlan))
```
 2023-03-06 19:09:47 -08:00
LouisandGitHub 332655476b proto: change protobuf definition case (#109) 2022-10-08 15:47:31 -07:00
LouisandGitHub f542b64401 clear logging statement (#129) 2022-10-08 15:36:20 -07:00
LouisandGitHub f6a776993f format: improve text formatter (#110) 2022-10-08 15:29:06 -07:00
LouisandGitHub e3471e58c6 fix single ip template case (#106) 2022-10-08 15:16:40 -07:00
Brian LandersGitHubBrian Landerslspgn
 		e0d30b75db Add SASL/SCAM support to Kafka transport. (#121) 
This pull request adds the optional -transport.kafka.scram flag,
which tells the Kafka transport to use SASL/SCRAM for authentication.
This allows goflow2 to work with AWS Managed Service for Kafka (MSK)
which does not support SASL/PLAIN.

Co-authored-by: Brian Landers <[email protected]>
Co-authored-by: lspgn <[email protected]>
v1.1.1 		2022-10-08 15:02:22 -07:00
LouisGitHubArun Cherla
 		3734d83954 Kafka: enable compression (#128) 
Co-authored-by: Arun Cherla <[email protected]>
 2022-10-08 14:17:18 -07:00
LouisandGitHub 7801c6a273 nit: defer after error check (#127) 2022-10-08 08:55:53 -07:00
LouisandGitHub 05a03e2446 socket reuse URI argument (#107) 2022-10-08 08:50:23 -07:00
Vincent BernatandGitHub 57fad2e0c8 Fix SrcAS/DstAS for sFlow (#125) 2022-10-08 08:41:47 -07:00
LouisandGitHub f2a5297b6a Compose: updating ClickHouse and Grafana versions (#126) 
* Fix bug with ClickHouse SQL configuration loading protobuf
* Add official ClickHouse datasource to Grafana
 2022-10-08 08:26:03 -07:00
LouisandGitHub d6caeaacdd Bugfix: fix closing of UDP routine (#118) 2022-08-24 20:38:56 -07:00
IanGitHubIan Pye
 		fc42020d1b Silently drop 0 byte packets without processing as flow (#114) 
Co-authored-by: Ian Pye <[email protected]>
 2022-08-17 20:01:02 -07:00
LouisandGitHub 29075b832d docs: add information about mapping (#108) 2022-08-02 22:28:47 -07:00
LouisGitHubSimon Podlipsky
 		1468ff77c2 BGP Next Hop, Communities and AS-Path fields (#101) 
* Add BGP Next Hop field to flow message (#11)
* Add BGP Communities and AS Path fields (#12)

Co-authored-by: Simon Podlipsky <[email protected]>
 2022-07-20 22:04:43 -07:00
LouisandGitHub a3b3b4759b Bump to go 1.18 (#102) 
* bump to go 1.18
 2022-07-20 22:04:30 -07:00
LouisandGitHub e0abb7ba07 Add MPLS decoding for IPFIX (#95) 
* Add MPLS decoding for IPFIX
Co-authored-by: David Roy <[email protected]>
 2022-07-20 21:40:11 -07:00
LouisGitHubAdrian Moreno
 		0020934574 Observation domain (#98) 
* netflow: Add observation domain and point to message

The ObservationDomainID and ObservationPointID are two IPFIX fields that
identify the entity that is capturing flows and can be used to enrich
the context around a specific sample.

Parse these fields from the sample and add them to the FlowMessage.

Signed-off-by: Adrian Moreno <[email protected]>
Co-authored-by: Adrian Moreno <[email protected]>
 2022-07-20 21:27:51 -07:00