mirror/netsampler-goflow2
1
0
Fork 0
mirror of https://github.com/netsampler/goflow2.git synced 2024-05-06 15:54:52 +00:00
Code Issues Projects Releases Wiki Activity
netsampler-goflow2/compose/elk
History
Brooks Swinnerton d53e5f9b5a compose: update Clickhouse schema to match casing of proto (#133) 
This commit resolves an issue with the mapping between the ClickHouse
schema and the flow.proto schema.

In 3326554, the casing of the proto fields was updated, but the
ClickHouse column names were not also updated, resulting in the
ClickHouse Kafka engine only being able to successfully deserialize
fields that without an underscore.

Also updates the provisioned dashboards. Versions of the tools are also updated.

Co-authored-by: lspgn <lspgn@users.noreply.github.com>
2023-03-06 20:39:33 -08:00
..
docker-compose.yml
compose: update Clickhouse schema to match casing of proto (#133)
2023-03-06 20:39:33 -08:00
logstash.conf
Add new compose stack (#5)
2021-05-31 22:10:05 -07:00
README.md
Add new compose stack (#5)
2021-05-31 22:10:05 -07:00

README.md

Flows + Logstash + Elastic + Kibana

Clickhouse is a powerful data warehouse.

A sample docker-compose is provided. It's composed of:

  • GoFlow2
  • Logstash
  • Elastic
  • Kibana

To start the containers, use:

$ docker-compose up

This command will automatically build the GoFlow2 container.

GoFlow2 collects NetFlow v9/IPFIX and sFlow packets and logs them into a file (/var/log/goflow/goflow.log). Logstash collects the log messages, parse the JSON and sends to Elastic. Kibana can be used to visualize the data. You can access the dashboard at http://localhost:5601.

This stack requires to create an index pattern. Define the index pattern to be logstash-*. Select @timestamp to be the time filter. You can then visualize flows in the Discover section.