Prometheus Helm Chart: How to configure blackbox exporter?

Question

I'm using the kube-prometheus-stack. In addition to it I installed the Prometheus Blackbox Exporter.

When I want to add a scrape target to my kube-prometheus-stack, I usually need to add a ServiceMonitor. I have added the following ServiceMonitor for my blackbox exporter:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: blackbox-exporter-servicemonitor
  labels:
    app: blackbox-exporter-app
    release: prometheus
spec:
  endpoints:
  - interval: 30s
    path: /metrics
    port: http
    scrapeTimeout: 10s
  selector:
    matchLabels:
      app: blackbox-exporter-app

This works so far, meaning the blackbox exporter shows up in my prometheus targets.

However to configure a blackbox exporter I would have to add a scrape target like this (in the prometheus config file, when not using the kube-prometheus-stack):

- job_name: 'blackbox'
  metrics_path: /probe
  params:
    module: [http_2xx]  # Look for a HTTP 200 response.
  static_configs:
    - targets:
      - http://localhost:5000
      - http://localhost:3000
      - http://localhost:9090
  relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: 127.0.0.1:9115  # The blackbox exporter's real hostname:port.

Which is where I'm stuck. How do I add a config like this to my service monitor such that I can add static_configs to add the probe targets for the blackbox exporter?

Mateusz Szymczyk Mateusz Szymczyk · Accepted Answer · 2020-12-14T11:33:33

Blackbox exporter works in a different way. You don't need to create ServiceMonitor to scrape it and provide a static_configs to it. Instead, you create a ServiceMonitor with parameters passed in an HTTP request.

For example:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    app: blackbox-exporter-kibana
  name: blackbox-exporter-kibana
  namespace: ops-system
spec:
  endpoints:
    - interval: 10s
      path: /probe
      port: web
      params:
        module: 
        - http_2xx
        target: 
        - "https://localhost:5000"
      metricRelabelings:
      - action: replace
        regex: (.*)
        replacement: my_local_service
        sourceLabels:
        - __param_target
        targetLabel: target
  selector:
    matchLabels:
      app: prometheus-blackbox-exporter

Just make sure the labels match.

Prometheus Helm Chart: How to configure blackbox exporter?

1 Answers