Monitoring Validators

There is a Grafana dashboard compatible with all the cosmos-sdk and tendermint based blockchains.

Preview#

Let's set it up for our network.

Prerequisites#

First install Grafana and Prometheus on your machine.

Enable Tendermint Metrics#

sed -i 's/prometheus = false/prometheus = true/g' <YOUR-NODE-HOMEDIR>/config/config.toml

After restarting your node, you should be able to access the tendermint metrics(default port is 26660): http://localhost:26660

Configure Prometheus Targets#

Find prometheus.yml file and append the following job under the scrape_configs:

  - job_name: archway    static_configs:      - targets: ['localhost:26660']        labels:          instance: validator

Note: On linux machines this file can be found under this path: /etc/prometheus/prometheus.yml

Resolving Port Conflicts#

If you run your archway network and Prometheus on the same machine, you will see one of them cannot run due to a port conflict. Let's resolve it.

Open the ./node-main/config/app.toml file and look for

[grpc]address = "0.0.0.0:9090"

Then change the port to something else like this:

[grpc]address = "0.0.0.0:9095"

Restart Prometheus#

sudo service prometheus restart

Check the status if the prometheus service is running properly:

systemctl status prometheus.service

If you see some output like this, it means it is running without an issue:

● prometheus.service - Monitoring system and time series database     Loaded: loaded (/lib/systemd/system/prometheus.service; enabled; vendor preset: enabled)     Active: active (running) since Wed 2021-10-06 15:32:48 CEST; 17min ago       Docs: https://prometheus.io/docs/introduction/overview/   Main PID: 457917 (prometheus)      Tasks: 21 (limit: 37746)     Memory: 48.3M     CGroup: /system.slice/prometheus.service             └─457917 /usr/bin/prometheus