DEV Community

Cover image for Monitoring Systems with Prometheus and Grafana: A Comprehensive Guide
Deepanshu Anand
Deepanshu Anand

Posted on

Monitoring Systems with Prometheus and Grafana: A Comprehensive Guide

In this blog we will be walking through a step-by-step guide to build your monitoring system for your devices. In this guide we will be setting up a Prometheus server and Grafana dashboard to analyze your systems resource utilization.
We will be deploying our Prometheus and Grafana in docker containers because they are lightweight and easy to manage.

Why is System Monitoring Important?

System monitoring is crucial for several reasons, as it provides valuable insights and benefits that contribute to the overall health, performance, and security of IT infrastructures and applications. Here are some key reasons why system monitoring is important:

  • Early Detection of Issues
  • Performance Optimization
  • Capacity Planning
  • Security and Compliance
  • Troubleshooting and Root Cause Analysis
  • SLA and KPI Tracking
  • Resource Optimization and Cost Reduction
  • Trend Analysis and Planning

What is Prometheus

Prometheus is a monitoring and alerting toolkit designed to collect and store time-series data from various sources, such as applications, services, and system components. It is particularly well-suited for monitoring highly dynamic and distributed environments. Prometheus scrapes data from designated endpoints at regular intervals, allowing users to monitor the health and performance of their systems. It uses a query language called PromQL to extract and analyze data. Prometheus also supports alerting based on predefined rules, enabling users to receive notifications when certain conditions are met.

What is Grafana

Grafana is a powerful visualization and analytics platform that works seamlessly with Prometheus and other data sources. It allows users to create custom dashboards and visualizations using various data sources, including Prometheus. Grafana provides a user-friendly interface for exploring and understanding complex data sets. It supports a wide range of charts, graphs, and panels, making it easy to display data in a meaningful way. Grafana is commonly used to create real-time monitoring dashboards, performance reports, and executive summaries, helping teams and organizations gain valuable insights into their systems' health and performance

Setting up the environment for Prometheus and Grafan

  • enabling ssh
sudo apt-get install openssh-server
sudo systemctl enable ssh
sudo systemctl start ssh
sudo ufw allow ssh
sudo ufw enable
sudo ufw status
Enter fullscreen mode Exit fullscreen mode
  • installing docker
sudo apt install docker.io
Enter fullscreen mode Exit fullscreen mode

Configuring Prometheus and Grafana server on ubuntu

  • create a directory to store all the configuration files which will be mounted to the docker container
cd /opt/
mkdir prometheus
cd prometheus
touch prometheus.yml
touch alerts.rules
Enter fullscreen mode Exit fullscreen mode

alerts rules and alert manager will be discussed in the next post

  • create docker container for prometheus
docker run -d -p 9090:9090 -v /opt/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml -v /opt/prometheus/alerts.rules:/etc/prometheus/alerts.rules --restart unless-stopped --name prometheus prom/prometheus
Enter fullscreen mode Exit fullscreen mode
  • creating a node exporter a node exporter helps in retrieving the metrics of a Linux based system and send it to prometheus
docker run -d -p 9100:9100 -v --name node-exporter prom/node-exporter
Enter fullscreen mode Exit fullscreen mode
  • create docker container for grafana
docker run -d --name=grafana -p 3000:3000 grafana/grafana
Enter fullscreen mode Exit fullscreen mode

now open any browser and search for ubuntu_ip:9090, ubuntu_ip:3000 and ubuntu_ip:9100/metrics for checking whether the prometheus, grafana and node exporter are running or not
ubuntu_ip is the IP address of the machine where the docker container is created

if you want to monitor your windows machine you can install a windows exporter from here. After installing you can start the service to start the windows exporter

Configuring Prometheus Server

vim /opt/prometheus/prometheus.yml
Enter fullscreen mode Exit fullscreen mode

prometheus.yml

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'node-exporter'
    static_configs:
      - targets: ['ubuntu_id:9100']
  # ubuntu_ip is the IP address where node exporter is installed

  # for windows exporter
  - job_name: 'win-exporter'
    static_configs:
      - targets: ['windows_ip:9182']
  # windows_ip is the IP address of the window machine
Enter fullscreen mode Exit fullscreen mode
docker restart prometheus
Enter fullscreen mode Exit fullscreen mode
  • now you will be able to use PromQL to gain insights of you Linux machine by visiting ubuntu_ip:9090

Prometheus UI

Configuring Grafana Dashboard

open ubuntu_ip:3000 on your browser

Grafana login

now use the default username and password admin and admin and after that add prometheus to your datasource

Grafana Home

Grafana Datasource
After adding the datasource build a dashboard for your node exporter using the following library Grafana Dashboards

Thanks for reading the blog I hope it was helpful for you. In the next blog we will be configuring our Prometheus Server to send alerts based on some rules.

Top comments (1)

Collapse
 
respect17 profile image
Kudzai Murimi

Thanks a lot!