What kind of metrics are captured by Prometheus

Prometheus can capture a wide range of metrics, not only from applications but also from the system itself, including metrics related to memory, CPU, disk I/O, network usage, and more. Here’s a more detailed breakdown of what Prometheus can capture:

1. Application Metrics

These are metrics that you expose from your applications. Examples include:

• HTTP request count, latency, and error rates
• Database query performance
• Custom business logic metrics (e.g., number of processed orders)

To capture these metrics, you need to instrument your application using one of the Prometheus client libraries (e.g., for Go, Java, Python, Node.js, etc.).

2. System Metrics

Prometheus can capture various system-level metrics using exporters. An exporter is a service that exposes metrics in a format that Prometheus can scrape.

Common Exporters:

• Node Exporter: Captures hardware and OS metrics from *nix kernels.
• cAdvisor: Captures container metrics.
• Blackbox Exporter: Allows for blackbox probing of endpoints over HTTP, HTTPS, DNS, TCP, ICMP, and gRPC.
• SNMP Exporter: Scrapes metrics from SNMP (Simple Network Management Protocol) devices.

Example Metrics from Node Exporter:

• CPU Metrics:
  • node_cpu_seconds_total: Total seconds the CPUs spent in each mode (user, system, idle, etc.)
  • node_load1: 1-minute load average
• Memory Metrics:
  • node_memory_MemAvailable_bytes: Available memory in bytes
  • node_memory_MemTotal_bytes: Total memory in bytes
• Disk Metrics:
  • node_disk_read_bytes_total: Total number of bytes read from disk
  • node_disk_write_bytes_total: Total number of bytes written to disk
• Network Metrics:
  • node_network_receive_bytes_total: Total number of bytes received
  • node_network_transmit_bytes_total: Total number of bytes transmitted

Example: Setting Up Node Exporter

To capture system metrics, you can use the Node Exporter. Here’s how to set it up and configure Prometheus to scrape metrics from it:

1. Download and Run Node Exporter:

   wget https://github.com/prometheus/node_exporter/releases/download/v1.2.2/node_exporter-1.2.2.linux-amd64.tar.gz
   tar xvfz node_exporter-1.2.2.linux-amd64.tar.gz
   cd node_exporter-1.2.2.linux-amd64
   ./node_exporter
   

2. Update Prometheus Configuration: Add the Node Exporter target to your prometheus.yml:

   global:
     scrape_interval: 15s
   
   scrape_configs:
     - job_name: 'prometheus'
       static_configs:
         - targets: ['localhost:9090']
   
     - job_name: 'node-exporter'
       static_configs:
         - targets: ['localhost:9100']
   

3. Restart Prometheus to apply the new configuration:

   docker-compose restart prometheus
   

4. Access Node Exporter Metrics: Open your browser and go to http://localhost:9100/metrics to see the metrics exposed by the Node Exporter.

Visualization and Alerting

To visualize these metrics and set up alerts, you can integrate Prometheus with Grafana and Alertmanager:

• Grafana: For creating dashboards and visualizing metrics.
• Alertmanager: For handling alerts sent by Prometheus, such as sending notifications via email, Slack, etc.