In Prometheus, you can enrich your monitoring setup by assigning specific labels to each target in your static_configs. This allows for better organization and filtering of metrics. Below is an example of how to achieve this:

# Global configuration for scrape intervals
# This sets the default interval for scraping metrics from targets
# Adjust as necessary for your monitoring needs

global:
  scrape_interval: 15s  # Default scrape interval for all targets

# Scrape configuration for monitoring static targets
scrape_configs:
  - job_name: 'development'  # Unique identifier for this job
    static_configs:
      - targets:
          - '192.168.1.1:9100'  # Target for service 1
          - '192.168.1.1:9101'  # Target for service 2
          - '192.168.1.1:9102'  # Target for service 3
        labels:
          service: '1'  # Label for the first target
      - targets:
          - '192.168.1.1:9101'
        labels:
          service: '2'  # Label for the second target
      - targets:
          - '192.168.1.1:9102'
        labels:
          service: '3'  # Label for the third target

Explanation:

  • job_name: This is a unique name that identifies the group of targets being scraped. It is useful for querying and organizing metrics.
  • targets: This section lists the endpoints that Prometheus will scrape. Each target is defined by its IP address and port.
  • labels: You can specify additional metadata for each target. In this example, each target has a service label that differentiates them based on their role.

After updating your prometheus.yml, remember to reload the configuration for the changes to take effect. You can do this by sending a SIGHUP signal to the Prometheus process:

kill -HUP $(pidof prometheus)

This method allows you to dynamically adjust your monitoring setup without needing to restart the Prometheus server.