Randhir Singh shows how to use open-source tools to monitor Kafka streaming pipelines:
The solution uses the following open-source tools. The solution architecture is illustrated below.
- Apache Kafka Connect is a tool to stream data between Apache Kafka and other components.
- InfluxDB which is a time series database from InfluxData. It will be used to store time series data from Kafka input and output topics.
- Influx sink connector from Datamountaineer. It is a connector and sink to write events from Kafka to InfluxDB.
- Chronograf is an open-source monitoring solution from InfluxData.
Click through for the solution.