mirror of
https://github.com/netsampler/goflow2.git
synced 2024-05-06 15:54:52 +00:00
80ef166b84
* add compose for ELK * reorg compose directories, bind to docker versions and fix Clickhouse/Grafana configurations
29 lines
911 B
Markdown
29 lines
911 B
Markdown
# Flows + Kafka + Clicklhouse + Grafana + Prometheus
|
|
|
|
Clickhouse is a powerful data warehouse.
|
|
|
|
A sample [docker-compose](./docker-compose.yml) is provided.
|
|
It's composed of:
|
|
* Apache Kafka
|
|
* Apache Zookeeper
|
|
* GoFlow2
|
|
* Prometheus
|
|
* Clickhouse
|
|
* Grafana
|
|
|
|
To start the containers, use:
|
|
```bash
|
|
$ docker-compose up
|
|
```
|
|
|
|
This command will automatically build Grafana and GoFlow2 containers.
|
|
|
|
GoFlow2 collects NetFlow v9/IPFIX and sFlow packets and sends as a protobuf into Kafka.
|
|
Zookeeper coordinates Kafka and can also be used by Clickhouse to ensure replication.
|
|
Prometheus scrapes the metrics of the collector.
|
|
Clickhouse consumes from Kafka, stores raw data and aggregates over specific columns
|
|
using `MATERIALIZED TABLES` and `VIEWS` defined in a [schema file](./clickhouse/create.sh).
|
|
|
|
You can visualize the data in Grafana at http://localhost:3000 (credentials: admin/admin) with the
|
|
pre-made dashboards.
|