e3f58f2564
build(deps): bump github.com/prometheus/client_golang ( #159 )
...
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang ) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/prometheus/client_golang/releases )
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md )
- [Commits](https://github.com/prometheus/client_golang/compare/v1.14.0...v1.15.0 )
---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-20 19:14:32 -07:00
9061d30a79
build(deps): bump google.golang.org/protobuf from 1.29.1 to 1.30.0 ( #158 )
...
Bumps google.golang.org/protobuf from 1.29.1 to 1.30.0.
---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-20 19:08:48 -07:00
b099a5f313
Create dependabot.yml ( #157 )
2023-04-20 19:06:44 -07:00
b58874c36a
app: hotfix ( #156 )
2023-04-20 06:46:01 -07:00
958109e91c
allow unknown records for sFlow ( #155 )
2023-04-15 21:21:10 -07:00
5d47e60bed
build(deps): bump golang.org/x/net from 0.5.0 to 0.7.0 ( #154 )
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.5.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases )
- [Commits](https://github.com/golang/net/compare/v0.5.0...v0.7.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-15 21:14:27 -07:00
9850000ffa
allow unknown record structures in sFlow ( #153 )
2023-04-15 20:59:53 -07:00
c4c4ffb4e4
Refactor template as module ( #49 )
...
* Refactor template as module
* Abstraction to allow custom template storage (eg: file, redis, http...)
* Works similarly to Transport and Format
2023-04-15 20:52:32 -07:00
6b3c5f1215
go.mod: update dependencies ( #146 )
2023-04-15 19:33:39 -07:00
d59f209302
mapping: support endianness ( #130 )
2023-03-28 22:05:47 -07:00
40e5ef08fd
mapping: allow lists to be mapped ( #149 )
2023-03-28 21:55:21 -07:00
5529d4993b
makefile: fix typo ( #147 )
2023-03-14 19:53:48 -07:00
d72f05bb0b
docker: fix build and push of manifest ( #145 )
2023-03-14 19:42:14 -07:00
d53e5f9b5a
compose: update Clickhouse schema to match casing of proto ( #133 )
...
This commit resolves an issue with the mapping between the ClickHouse
schema and the flow.proto schema.
In 3326554, the casing of the proto fields was updated, but the
ClickHouse column names were not also updated, resulting in the
ClickHouse Kafka engine only being able to successfully deserialize
fields that without an underscore.
Also updates the provisioned dashboards. Versions of the tools are also updated.
Co-authored-by: lspgn <lspgn@users.noreply.github.com>
2023-03-06 20:39:33 -08:00
176eb8772f
docs: fix SrcVlan/VlanId mapping between protocols ( #136 )
...
In source code:
```go
NFV9_FIELD_SRC_VLAN = 58
NFV9_FIELD_DST_VLAN = 59
IPFIX_FIELD_vlanId = 58
IPFIX_FIELD_postVlanId = 59
case netflow.NFV9_FIELD_SRC_VLAN:
DecodeUNumber(v, &(flowMessage.VlanId))
DecodeUNumber(v, &(flowMessage.SrcVlan))
case netflow.NFV9_FIELD_DST_VLAN:
DecodeUNumber(v, &(flowMessage.DstVlan))
```
2023-03-06 19:09:47 -08:00
332655476b
proto: change protobuf definition case ( #109 )
2022-10-08 15:47:31 -07:00
f542b64401
clear logging statement ( #129 )
2022-10-08 15:36:20 -07:00
f6a776993f
format: improve text formatter ( #110 )
2022-10-08 15:29:06 -07:00
e3471e58c6
fix single ip template case ( #106 )
2022-10-08 15:16:40 -07:00
e0d30b75db
Add SASL/SCAM support to Kafka transport. ( #121 )
...
This pull request adds the optional -transport.kafka.scram flag,
which tells the Kafka transport to use SASL/SCRAM for authentication.
This allows goflow2 to work with AWS Managed Service for Kafka (MSK)
which does not support SASL/PLAIN.
Co-authored-by: Brian Landers <brian.landers@meraki.net>
Co-authored-by: lspgn <lspgn@users.noreply.github.com>
2022-10-08 15:02:22 -07:00
3734d83954
Kafka: enable compression ( #128 )
...
Co-authored-by: Arun Cherla <kcherla@gmail.com>
2022-10-08 14:17:18 -07:00
7801c6a273
nit: defer after error check ( #127 )
2022-10-08 08:55:53 -07:00
05a03e2446
socket reuse URI argument ( #107 )
2022-10-08 08:50:23 -07:00
57fad2e0c8
Fix SrcAS/DstAS for sFlow ( #125 )
2022-10-08 08:41:47 -07:00
f2a5297b6a
Compose: updating ClickHouse and Grafana versions ( #126 )
...
* Fix bug with ClickHouse SQL configuration loading protobuf
* Add official ClickHouse datasource to Grafana
2022-10-08 08:26:03 -07:00
d6caeaacdd
Bugfix: fix closing of UDP routine ( #118 )
2022-08-24 20:38:56 -07:00
fc42020d1b
Silently drop 0 byte packets without processing as flow ( #114 )
...
Co-authored-by: Ian Pye <pye@kentik.com>
2022-08-17 20:01:02 -07:00
29075b832d
docs: add information about mapping ( #108 )
2022-08-02 22:28:47 -07:00
1468ff77c2
BGP Next Hop, Communities and AS-Path fields ( #101 )
...
* Add BGP Next Hop field to flow message (#11 )
* Add BGP Communities and AS Path fields (#12 )
Co-authored-by: Simon Podlipsky <simon@podlipsky.net>
2022-07-20 22:04:43 -07:00
a3b3b4759b
Bump to go 1.18 ( #102 )
...
* bump to go 1.18
2022-07-20 22:04:30 -07:00
e0abb7ba07
Add MPLS decoding for IPFIX ( #95 )
...
* Add MPLS decoding for IPFIX
Co-authored-by: David Roy <door7302@gmail.com>
2022-07-20 21:40:11 -07:00
0020934574
Observation domain ( #98 )
...
* netflow: Add observation domain and point to message
The ObservationDomainID and ObservationPointID are two IPFIX fields that
identify the entity that is capturing flows and can be used to enrich
the context around a specific sample.
Parse these fields from the sample and add them to the FlowMessage.
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Co-authored-by: Adrian Moreno <amorenoz@redhat.com>
2022-07-20 21:27:51 -07:00
838505b10b
Merge pull request #86 from OlivierCazade/log-transport-error
...
Add logging of transport error
2022-07-06 20:24:53 -07:00
1834503bb5
Merge pull request #90 from kentik/fix-yaml-mapping
...
Adding field names to allow ipfix yaml mapping field to parse
2022-07-06 20:23:31 -07:00
97d58ccfe2
Adding field names to allow ipfix yaml mapping field to parse
2022-07-06 13:53:57 -07:00
78f2ce561d
Add logging of transport error
2022-06-13 17:36:28 +02:00
5300494e47
Merge pull request #82 from jotak/fix-deltams-convert
...
In IPFIX DeltaMicroseconds case, basetime must be converted to MS
2022-05-09 08:52:30 -07:00
10dc7271d5
In IPFIX DeltaMicroseconds case, basetime must be converted to MS
2022-05-09 11:05:49 +02:00
0c878cde53
Merge pull request #81 from aams-telefonica/feature/timeflow-start-end-ms
...
Add TimeFlowStartMs and TimeFlowEndMs to Netflow v5, v9, v10 (IPFIX)
2022-05-07 10:20:25 -07:00
23e75e85f8
Added TimeFlowStartMs and TimeFlowEndMs to Netflow v10 (IPFIX)
2022-05-07 12:11:47 +02:00
3e6a0510fd
Added TimeFlowStartMs and TimeFlowEndMs to Netflow v5
2022-05-07 12:05:42 +02:00
55bef5b2f0
Added TimeFlowStartMs and TimeFlowEndMs to Netflow v9
2022-05-07 12:00:33 +02:00
d59dad6db5
Merge pull request #77 from netsampler/bug/netflowv9-variable
...
support netflow v9 variable length
2022-04-28 20:53:15 -07:00
99e086f684
support netflow v9 variable length
2022-04-02 10:40:35 -07:00
58f0f97a62
Merge pull request #72 from vincentbernat/feature/netflow-tests
...
Add some tests for Netflow decoding
2022-03-13 22:10:46 -07:00
bd480ebebb
Merge pull request #75 from vincentbernat/fix/counter-type
...
Fix type for a counter when decoding a template flowset
2022-03-03 11:03:25 -08:00
a5fc8ecfb6
Fix type for a counter when decoding a template flowset
2022-03-03 19:45:34 +01:00
e3f8f990ec
Add some tests for Netflow decoding
...
The tests are a bit more expansive than the existing tests for sFlow
or NFv5 as we check the whole structure. I am also testing the
String() function as it is easier to read. It is a bit redundant, but
checking only for the wire format makes it difficult to compare with
Wireshark. Only testing for the textual representation is not totally
good as it is not what is used by users of the decode function.
2022-02-26 16:47:08 +01:00
7d4c41a161
Merge pull request #71 from mariomac/rmalloc
...
Avoid unnecessary memory allocations
2022-02-08 20:39:55 -08:00
615b9f697c
Avoid unnecessary memory allocations
2022-01-26 17:24:35 +01:00
