In this little proof of concept work, we grab some of these flows process them in Apache NiFi and then store them in Apache Hive 3 tables for analytics. We should probably push the data to HBase for aggregates and Druid for time series. We will see as this expands.
There are also other data access options including the NiFi REST API and the NiFi Python APIs.
- Send notification when the NiFi starts, stops or died unexpectedly
- Two OOTB notifications
- Email notification service
- HTTP notification service
- It’s easy to write a custom notification service
AmbariReportingTask (global, per process group)
MonitorDiskUsage(Flowfile, content, provenance repositories)
Much of this is an overview of the tools and measures available.