Monitor Server with Prometheus and Grafana

Prometheus is used an open source software, that can collect metrics and alerting.

You can download latest version oof Prometheus from

https://prometheus.io/download/

Create a user

Download and Install prometheus

Create a service file

Add following content

Enable prometheus to start on boot

Start prometheus

Prometheus runs on port 9090, you can access promethus at

It will look like

Prometheus have some basic graphing features, but you can’t use it for monitoring. To create dash board and monitor, you need to use grafana.

Collecting Data

Node Exporter is used to collect data from servers. All monitored servers need Node Exporter installed.

Lets create a user for Node Exporter to run

Install Node Exporter

Create a systemd service file for node exporter

Add

Enable and start Node Exporter

Node Exporter run on port 9100 and expose system metrics on url

Node Exporter

Adding Servers to Prometheus

Once Node Exporter installed on a server, you need to tell Prometheus to get data from the Node Exporter you just installed. To do this, edit Prometheus configuration file.

Add following

To monitor multiple servers, you can dd more servers in targets line. Here is an example config

https://gist.github.com/serverok/83a622e7577da36384f87fe60c9930af/raw

Restart prometheus

Grafana

Grafana is used to visualise data collected by Prometheus. You can download Grafana from

https://grafana.com/grafana/download

Grafana offers free cloud hosted version with some limitation (1 user, 5 dashboards). Free version is suitable if you are getting started and don’t want to install your own. You can signup for cloud hosted version at

https://grafana.com/get

If you decide to install your own Grafana, you can run

Enable and start grafana

If you did your own install, grafana runs on port 3000. To access, use url

Default username and passwords are “admin”. Once logged in you will be asked to set password for grafana admin user.

Before you can use Grafana, you need to set a data source and create dash board. In our case, data source is prometheus. To connect Grafana to your Prometheus insallation, go to Settings > Data Sources

Grafana Data Sources

On next page, select Prometheus

Grafana Add data source

On next page, for URL, enter http://PROMETHUS_SERVER_IP:9090, scroll down, click on “Save & Test” button. If grafana can connect to your prometheus installation, you should see success message with “Data source is working”. If not, you need to check your firewall rules.

Creating Grafana Dashboards

Grafana displays data in dash boards. You can create your own or use pre existing dash boards. You can find pre-made dash boards at

https://grafana.com/grafana/dashboards

On my grafana installation, i used dashboard

https://grafana.com/grafana/dashboards/11074

To add this dash board to your Grafana, click on the + button, then select Import. On next screen, you can enter ID for the dash board you need to import. In this case 11074. Click “Load” button to import the dash board.

Here is a dash board for one of the server

grafana dashboard

You can edit Panels in grafana dash board to see how it is created. You can create a new dash board with panel you need. This way your dashboards only show required information.

Related Posts

Server Monitoring

Prometheus Node Exporter on non default port

2 Responses

Add a Comment

Your email address will not be published. Required fields are marked *