slo-generator

Easily setup a service level objective using prometheus, based on lessons from the SRE workbook.

slo-generator -slo.path=slo_example.yml -rule.output rule.yml

Generating

Look the file slo_example.yml to see how to parametrize SLOs and generate Prometheus rules by running the following command:

Alert methods currently supported

• 1. Target Error Rate ≥ SLO Threshold, using alertMethod: simple
• 2. Increased Alert Window, using alertMethod: simple
• 3. Incrementing Alert Duration, using alertMethod: simple
• 4. Alert on Burn Rate alertMethod: simple and burnRate: <rate>
• 5. Multiple Burn Rate Alerts, using alertMethod: multi-window and shortWindow: false
• 6. Multiwindow, Multi-Burn-Rate Alerts, using alertMethod: multi-window

alertMethod: simple

Look the file slo_simple_example.yml to see a full example of usage. the simple alert method require two params:

1. alertWindow: how far back in time will used to alerting. supported values: 5m, 30m, 1h, 2h, 6h, 1d and 3d.
2. alertWait: for long time will begin fire an alert.

alertMethod: multi-window

The philosofy of this alert is described on the section of book: (https://landing.google.com/sre/workbook/chapters/alerting-on-slos#6-multiwindow-multi-burn-rate-alerts)

SLOs at scale

The Workbook suggests to create classes to simplify how to set a SLO for your services, read details about concepts here

Look at slo_example_with_classes.yml and slo_classes.yml to see how to define classes and associate with your services.

After that, you can run the command specifying the classes:

slo-generator -slo.path=slo_example_with_classes.yml -classes.path slo_classes.yml -rule.output rule.yml

Kubernetes integration

We support to export SLOs as a well known PrometheusRule resources managed by prometheus-operator, just use -kubernetes flag, example:

slo-generator -kubernetes -slo.path=slo_example.yml > slo_manifest.yml
cat slo_manifest.yml
kubectl apply -f slo_manifest.yml

Grafana integration

All generated SLOs are visible by grafana:

Import dashboard using following JSON files