• This repository has been archived on 06/Mar/2023
  • Stars
    star
    1,106
  • Rank 41,969 (Top 0.9 %)
  • Language Jinja
  • License
    MIT License
  • Created almost 7 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Deploy Prometheus monitoring system

DEPRECATED

This role has been deprecated in favor of a the prometheus-community/ansible collection.

Ansible Role: prometheus

CircleCI License Ansible Role GitHub tag

Description

Deploy Prometheus monitoring system using ansible.

Upgradability notice

When upgrading from <= 2.4.0 version of this role to >= 2.4.1 please turn off your prometheus instance. More in 2.4.1 release notes

Requirements

  • Ansible >= 2.7 (It might work on previous versions, but we cannot guarantee it)
  • jmespath on deployer machine. If you are using Ansible from a Python virtualenv, install jmespath to the same virtualenv via pip.
  • gnu-tar on Mac deployer host (brew install gnu-tar)

Role Variables

All variables which can be overridden are stored in defaults/main.yml file as well as in table below.

Name Default Value Description
prometheus_version 2.27.0 Prometheus package version. Also accepts latest as parameter. Only prometheus 2.x is supported
prometheus_skip_install false Prometheus installation tasks gets skipped when set to true.
prometheus_binary_local_dir "" Allows to use local packages instead of ones distributed on github. As parameter it takes a directory where prometheus AND promtool binaries are stored on host on which ansible is ran. This overrides prometheus_version parameter
prometheus_config_dir /etc/prometheus Path to directory with prometheus configuration
prometheus_db_dir /var/lib/prometheus Path to directory with prometheus database
prometheus_read_only_dirs [] Additional paths that Prometheus is allowed to read (useful for SSL certs outside of the config directory)
prometheus_web_listen_address "0.0.0.0:9090" Address on which prometheus will be listening
prometheus_web_config {} A Prometheus web config yaml for configuring TLS and auth.
prometheus_web_external_url "" External address on which prometheus is available. Useful when behind reverse proxy. Ex. http://example.org/prometheus
prometheus_storage_retention "30d" Data retention period
prometheus_storage_retention_size "0" Data retention period by size
prometheus_config_flags_extra {} Additional configuration flags passed to prometheus binary at startup
prometheus_alertmanager_config [] Configuration responsible for pointing where alertmanagers are. This should be specified as list in yaml format. It is compatible with official <alertmanager_config>
prometheus_alert_relabel_configs [] Alert relabeling rules. This should be specified as list in yaml format. It is compatible with the official <alert_relabel_configs>
prometheus_global { scrape_interval: 60s, scrape_timeout: 15s, evaluation_interval: 15s } Prometheus global config. Compatible with official configuration
prometheus_remote_write [] Remote write. Compatible with official configuration
prometheus_remote_read [] Remote read. Compatible with official configuration
prometheus_external_labels environment: "{{ ansible_fqdn | default(ansible_host) | default(inventory_hostname) }}" Provide map of additional labels which will be added to any time series or alerts when communicating with external systems
prometheus_targets {} Targets which will be scraped. Better example is provided in our demo site
prometheus_scrape_configs defaults/main.yml#L58 Prometheus scrape jobs provided in same format as in official docs
prometheus_config_file "prometheus.yml.j2" Variable used to provide custom prometheus configuration file in form of ansible template
prometheus_alert_rules defaults/main.yml#L81 Full list of alerting rules which will be copied to {{ prometheus_config_dir }}/rules/ansible_managed.rules. Alerting rules can be also provided by other files located in {{ prometheus_config_dir }}/rules/ which have *.rules extension
prometheus_alert_rules_files defaults/main.yml#L78 List of folders where ansible will look for files containing alerting rules which will be copied to {{ prometheus_config_dir }}/rules/. Files must have *.rules extension
prometheus_static_targets_files defaults/main.yml#L78 List of folders where ansible will look for files containing custom static target configuration files which will be copied to {{ prometheus_config_dir }}/file_sd/.

Relation between prometheus_scrape_configs and prometheus_targets

Short version

prometheus_targets is just a map used to create multiple files located in "{{ prometheus_config_dir }}/file_sd" directory. Where file names are composed from top-level keys in that map with .yml suffix. Those files store file_sd scrape targets data and they need to be read in prometheus_scrape_configs.

Long version

A part of prometheus.yml configuration file which describes what is scraped by prometheus is stored in prometheus_scrape_configs. For this variable same configuration options as described in prometheus docs are used.

Meanwhile prometheus_targets is our way of adopting prometheus scrape type file_sd. It defines a map of files with their content. A top-level keys are base names of files which need to have their own scrape job in prometheus_scrape_configs and values are a content of those files.

All this mean that you CAN use custom prometheus_scrape_configs with prometheus_targets set to {}. However when you set anything in prometheus_targets it needs to be mapped to prometheus_scrape_configs. If it isn't you'll get an error in preflight checks.

Example

Lets look at our default configuration, which shows all features. By default we have this prometheus_targets:

prometheus_targets:
  node:  # This is a base file name. File is located in "{{ prometheus_config_dir }}/file_sd/<<BASENAME>>.yml"
    - targets:              #
        - localhost:9100    # All this is a targets section in file_sd format
      labels:               #
        env: test           #

Such config will result in creating one file named node.yml in {{ prometheus_config_dir }}/file_sd directory.

Next this file needs to be loaded into scrape config. Here is modified version of our default prometheus_scrape_configs:

prometheus_scrape_configs:
  - job_name: "prometheus"    # Custom scrape job, here using `static_config`
    metrics_path: "/metrics"
    static_configs:
      - targets:
          - "localhost:9090"
  - job_name: "example-node-file-servicediscovery"
    file_sd_configs:
      - files:
          - "{{ prometheus_config_dir }}/file_sd/node.yml" # This line loads file created from `prometheus_targets`

Example

Playbook

---
- hosts: all
  roles:
  - cloudalchemy.prometheus
  vars:
    prometheus_targets:
      node:
      - targets:
        - localhost:9100
        - demo.cloudalchemy.org:9100
        labels:
          env: demosite

Demo site

Prometheus organization provide a demo site for full monitoring solution based on prometheus and grafana. Repository with code and links to running instances is available on github.

Defining alerting rules files

Alerting rules are defined in prometheus_alert_rules variable. Format is almost identical to one defined in Prometheus 2.0 documentation. Due to similarities in templating engines, every templates should be wrapped in {% raw %} and {% endraw %} statements. Example is provided in defaults/main.yml file.

Local Testing

The preferred way of locally testing the role is to use Docker and molecule (v2.x). You will have to install Docker on your system. See "Get started" for a Docker package suitable to for your system. We are using tox to simplify process of testing on multiple ansible versions. To install tox execute:

pip3 install tox

To run tests on all ansible versions (WARNING: this can take some time)

tox

To run a custom molecule command on custom environment with only default test scenario:

tox -e py35-ansible28 -- molecule test -s default

For more information about molecule go to their docs.

If you would like to run tests on remote docker host just specify DOCKER_HOST variable before running tox tests.

CircleCI

Combining molecule and CircleCI allows us to test how new PRs will behave when used with multiple ansible versions and multiple operating systems. This also allows use to create test scenarios for different role configurations. As a result we have a quite large test matrix which will take more time than local testing, so please be patient.

Contributing

See contributor guideline.

Troubleshooting

See troubleshooting.

License

This project is licensed under MIT License. See LICENSE for more details.

More Repositories

1

ansible-node-exporter

Provision basic metrics exporter for prometheus monitoring tool
Python
504
star
2

ansible-grafana

Platform for analytics and monitoring
Jinja
502
star
3

ansible-alertmanager

Deploy Prometheus Alertmanager service
Jinja
141
star
4

ansible-blackbox-exporter

Blackbox prober exporter
Python
59
star
5

ansible-coredns

Ansible provisioning of CoreDNS
Python
27
star
6

ansible-mysqld_exporter

Provision mysqld exporter for prometheus monitoring tool
Jinja
21
star
7

ansible-fluentd

Provision fluentd log collector
Shell
20
star
8

ansible-pushgateway

Install Prometheus pushgateway service
Python
19
star
9

ansible-process_exporter

Provision process exporter for prometheus monitoring tool
Python
18
star
10

ansible-snmp-exporter

Provision SNMP metrics exporter for prometheus monitoring
Python
17
star
11

skeleton

Ansible role skeleton used for new cloud alchemy roles
Shell
16
star
12

demo-site

Demo site auto-deployed with Ansible and Travis CI.
HTML
7
star
13

ansible-systemd-exporter

Deploy https://github.com/povilasv/systemd_exporter
Python
7
star
14

presentation

Presentation about prometheus deployment
HTML
7
star
15

ansible-smokeping_prober

Deploy smokeping prober
Python
7
star
16

ansible-ebpf_exporter

Ansible provisioning of eBPF exporter
Shell
5
star
17

ansible-bind_exporter

Ansible Role for the Prometheus Community bind_exporter
Python
4
star
18

ansible-cortex

Python
4
star
19

ansible-ipmi_exporter

IPMI Exporter
Python
3
star
20

auto-maintenance

Org wide automated repo maintenance
Shell
3
star
21

ansible-alertmanager-irc-relay

Ansible role for the alertmanager-irc-relay
Jinja
2
star
22

ansible-pushprox

PushProxy
Shell
1
star
23

dockerfiles

dockerfiles used in ansible role testing
Dockerfile
1
star