Add monitoring of Nifi using Prometheus/Grafana

As a developer/sys admin of Casper I want to know how the pipeline is working, and to help me identify when something stops working so that I can have visibility.

We think we want:

• Grafana for visualization and alerting

• Prometheus for data scraping and querying

  • We might want something like influxdb if the thing being monitored supports sending that data

Some functionality we’d like:

• To know when something stops responding / is down.

• To have visibility of the compute resources (cpu load, memory, etc) of the servers

• To be able to chart the time it takes a message to enter the pipeline, move through “stages” and finally be synced.

• To monitor vital stats on a kafka topic: # of topics and size of each