How Prometheus works internally

Prometheus operates based on a pull-based model where it scrapes metrics from HTTP endpoints exposed by applications and services. Here’s a detailed look at how Prometheus works internally and how it integrates with applications like an Express.js app running on port 8080:

Internal Architecture of Prometheus

1. Data Model:

   • Prometheus stores all metrics as time-series data.
   • Each time-series is uniquely identified by a metric name and a set of key-value pairs called labels.
   • Metrics are collected and stored locally in a time-series database.

2. Components:

   • Prometheus Server: Central component that retrieves metrics from configured targets at regular intervals.
   • Time-Series Database: Stores metric samples and metadata.
   • Query Engine: Provides a powerful language (PromQL) to query and aggregate metrics.
   • Alerting Rules: Allows defining alert conditions based on metrics.
   • Web UI and API: Provides a web-based interface for visualization and querying metrics data.

3. Scraping:

   • Prometheus scrapes metrics from HTTP endpoints exposed by applications and services.
   • Each target (e.g., an application instance) is defined in the Prometheus configuration file (prometheus.yml), specifying the endpoint URL and other parameters.
   • Scraping intervals are configurable (scrape_interval), typically set to 15 to 60 seconds.

4. Integration with Applications:

   • Applications expose metrics via HTTP endpoints typically at /metrics.
   • Metrics are formatted in Prometheus exposition format, a text-based format that Prometheus understands.
   • The metrics endpoint (/metrics) provides a snapshot of current metric values when queried by Prometheus.

Integration with an Express.js Application

To integrate Prometheus with an Express.js application running on port 8080:

1. Instrumentation:

   • Use a Prometheus client library like prom-client in your Node.js application to instrument your code.
   • Define and register metrics (counters, histograms, gauges) to track different aspects of your application's behavior (e.g., HTTP request counts, latency, error rates).

2. Exposing Metrics:

   • Create an endpoint (/metrics) in your Express.js application to expose metrics in Prometheus exposition format.
   • Ensure this endpoint is accessible to Prometheus (e.g., by allowing traffic from Prometheus server IP or using a reverse proxy).

3. Prometheus Configuration:

   • Configure Prometheus (prometheus.yml) to scrape metrics from your application’s /metrics endpoint.
   • Specify the target (localhost:8080 in this case) and other parameters like scrape interval.

4. Running Prometheus:

   • Start Prometheus server with the configured prometheus.yml.
   • Prometheus will start scraping metrics from the defined targets (including your Express.js application) at regular intervals.

5. Visualization and Alerting:

   • Use Prometheus’s web UI to query and visualize metrics data.
   • Define alerting rules based on metric thresholds or conditions.
   • Integrate with Grafana for more advanced visualization and dashboarding capabilities.

Example Prometheus Configuration (prometheus.yml)

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'example-app'
    static_configs:
      - targets: ['localhost:8080']