• Stars
    star
    132
  • Rank 265,009 (Top 6 %)
  • Language
    Shell
  • License
    MIT License
  • Created about 9 years ago
  • Updated about 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Collect and forward metrics using portable shell scripts

metrics.sh

metrics.sh is a lightweight metrics collection and forwarding daemon implemented in portable POSIX compliant shell scripts. A transparent interface based on hooks enables writing custom collectors and reporters in an elegant way.

Project philosophy

  • Flat learning curve
  • Easily extensible
  • Low impact on system resources
  • No compilation, custom interpreters and runtimes required
  • Freedom to choose any service for storage, charting and alerting
  • Open source, no proprietary parts
  • Community-driven development, growing number of included metrics over time

Usage

$ ./metrics.sh --help

  Usage: ./metrics.sh [-d] [-h] [-v] [-c] [-m] [-r] [-i] [-C] [-u]

  Options:

    -c, --config   <file>      path to config file
    -m, --metrics  <metrics>   comma-separated list of metrics to collect
    -r, --reporter <reporter>  use specified reporter (default: stdout)
    -i, --interval <seconds>   collect metrics every n seconds (default: 2)
    -v, --verbose              enable verbose mode
    -C, --print-config         print output to be used in a config file
    -u, --update               pull the latest version (requires git)
    -d, --docs                 show documentation
    -h, --help                 show this text

Installation

$ git clone https://github.com/pstadler/metrics.sh.git

See this guide how to run metrics.sh as a service on Linux. Or here for instructions to set metrics.sh up for systemd.

Requirements

metrics.sh has been tested on Ubuntu 14.04 and Mac OS X but is supposed to run on most Unix-like operating systems. Some of the provided metrics require procfs to be available when running on *nix. POSIX compliancy means that metrics.sh works with minimalistic command interpreters such as dash. Built-in metrics do not require root privileges.

Metrics

Metric Description
cpu CPU usage in %
memory Memory usage in %
swap Swap usage in %
network_io Network I/O in kB/s, collecting two metrics: network_io.in and network_io.out
disk_io Disk I/O in MB/s
disk_usage Disk usage in %
heartbeat System heartbeat
ping Check whether a remote host is reachable

Reporters

Reporter Description
stdout Write to standard out (default)
file Write to a file or named pipe
udp Send data to any service via UDP
statsd Send data to StatsD
influxdb Send data to InfluxDB
prometheus Provide HTTP endpoint for Prometheus
keen_io Send data to Keen IO
stathat Send data to StatHat
logentries_com Send data to Logentries

Configuration

metrics.sh can be configured on the fly by passing along options when calling it:

$ ./metrics.sh --help              # print help
$ ./metrics.sh -m cpu,memory -i 1  # report cpu and memory usage every second

Some of the metrics and reporters are configurable or require some variables to be defined in order to work. Documentation is available with the --docs option.

$ ./metrics.sh --docs | less

As an example, the disk_usage metric has a configuration variable DISK_USAGE_MOUNTPOINT which is set to a default value depending on the operating system metrics.sh is running on. Setting the variable before starting will overwrite it.

$ DISK_USAGE_MOUNTPOINT=/dev/vdb ./metrics.sh -m disk_usage
# reports disk usage of /dev/vdb

Configuration files

Maintaining all these options can become a cumbersome job, but metrics.sh provides functionality for creating and reading configuration files.

$ ./metrics.sh -C > metrics.ini  # write configuration to metrics.ini
$ ./metrics.sh -c metrics.ini    # load configuration from metrics.ini

By default most lines in the configuration are commented out:

;[metric network_io]
;Network traffic in kB/s.
;NETWORK_IO_INTERFACE=eth0

To enable a metric, simply remove comments and modify values where needed:

[metric network_io]
;Network traffic in kB/s.
NETWORK_IO_INTERFACE=eth1

Multiple metrics of the same type

Configuring and reporting multiple metrics of the same type is possible through the use of aliases:

[metric network_io:network_eth0]
NETWORK_IO_INTERFACE=eth0

[metric network_io:network_eth1]
NETWORK_IO_INTERFACE=eth1

network_eth0 and network_eth1 are aliases of the network_io metric with specific configurations for each of them. Data of both network interfaces will now be collected and reported independently:

network_eth0.in: 0.26
network_eth0.out: 0.14
network_eth1.in: 0.08
network_eth1.out: 0.03
...

Writing custom metrics and reporters

metrics.sh provides a simple interface based on hooks for writing custom metrics and reporters. Each hook is optional and only needs to be implemented if necessary. In order for metrics.sh to find and load custom metrics, they have to be placed in ./metrics/custom or wherever CUSTOM_METRICS_PATH is pointing to. The same applies to custom reporters, whose default location is ./reporters/custom or any folder specified by CUSTOM_REPORTERS_PATH.

Custom metrics

# Hooks for metrics in order of execution
defaults () {}  # setting default variables
start () {}     # called at the beginning
collect () {}   # collect the actual metric
stop () {}      # called before exiting
docs () {}      # used for printing docs and creating output for configuration

Metrics run within an isolated scope. It's generally safe to create variables and helper functions within metrics.

Below is an example script for monitoring the size of a specified folder. Assuming this script is located at ./metrics/custom/dir_size.sh, it can be invoked by calling ./metrics.sh -m dir_size.

#!/bin/sh

# Set default values. This function should never fail.
defaults () {
  if [ -z $DIR_SIZE_PATH ]; then
    DIR_SIZE_PATH="."
  fi
  if [ -z $DIR_SIZE_IN_MB ]; then
    DIR_SIZE_IN_MB=false
  fi
}

# Prepare the collector. Create helper functions to be used during collection
# if needed. Returning 1 will disable this metric and report a warning.
start () {
  if [ $DIR_SIZE_IN_MB = false ]; then
    DU_ARGS="-s -k $DIR_SIZE_PATH"
  else
    DU_ARGS="-s -m $DIR_SIZE_PATH"
  fi
}

# Collect actual metric. This function is called every N seconds.
collect () {
  # Calling `report $val` will check if the value is a number (int or float)
  # and then send it over to the reporter's report() function, together with
  # the name of the metric, in this case "dir_size" if no alias is used.
  report $(du $DU_ARGS | awk '{ print $1 }')
  # If report is called with two arguments, the first one will be appended
  # to the metric name, for example `report "foo" $val` would be reported as
  # "dir_size.foo: $val". This is helpful when a metric is collecting multiple
  # values like `network_io`, which reports "network_io.in" / "network_io.out".
}

# Stop is not needed for this metric, there's nothing to clean up.
# stop () {}

# The output of this function is shown when calling `metrics.sh`
# with `--docs` and is even more helpful when creating configuration
# files with `--print-config`.
docs () {
  echo "Monitor size of a specific folder in Kb or Mb."
  echo "DIR_SIZE_PATH=$DIR_SIZE_PATH"
  echo "DIR_SIZE_REPORT_MB=$DIR_SIZE_IN_MB"
}

Custom reporters

# Hooks for reporters in order of execution
defaults () {}  # setting default variables
start () {}     # called at the beginning
report () {}    # report the actual metric
stop () {}      # called before exiting
docs () {}      # used for printing docs and creating output for configuration

Below is an example script for sending metrics as JSON data to an API endpoint. Assuming this script is located at ./reporters/custom/json_api.sh, it can be invoked by calling ./metrics.sh -r json_api.

#!/bin/sh

# Set default values. This function should never fail.
defaults () {
  if [ -z $JSON_API_METHOD ]; then
    JSON_API_METHOD="POST"
  fi
}

# Prepare the reporter. Create helper functions to be used during collection
# if needed. Returning 1 will result in an error and execution will be stopped.
start () {
  if [ -z $JSON_API_ENDPOINT ]; then
    echo "Error: json_api requires \$JSON_API_ENDPOINT to be specified"
    return 1
  fi
}

# Report metric. This function is called whenever there's a new value
# to report. It's important to know that metrics don't call this function
# directly, as there's some more work to be done before. You can safely assume
# that arguments passed to this function are sanitized and valid.
report () {
  local metric=$1 # the name of the metric, e.g. "cpu", "cpu_alias", "cpu.foo"
  local value=$2  # int or float
  curl -s -H "Content-Type: application/json" $JSON_API_ENDPOINT \
       -X $JSON_API_METHOD -d "{\"metric\":\"$metric\",\"value\":$value}"
}

# Stop is not needed here, there's nothing to clean up.
# stop () {}

# The output of this function is shown when calling `metrics.sh`
# with `--docs` and is even more helpful when creating configuration
# files with `--print-config`.
docs () {
  echo "Send data as JSON to an API endpoint."
  echo "JSON_API_ENDPOINT=$JSON_API_ENDPOINT"
  echo "JSON_API_METHOD=$JSON_API_METHOD"
}

More Repositories

1

keybase-gpg-github

Step-by-step guide on how to create a GPG key on keybase.io, adding it to a local GPG setup and using it with Git and GitHub.
2,609
star
2

flightplan

Run sequences of shell commands against local and remote hosts.
JavaScript
1,815
star
3

ticker.sh

Real-time stock tickers from the command-line.
Shell
502
star
4

battery.js

A tiny wrapper for the HTML5 Battery Status API.
JavaScript
118
star
5

octofolders

The missing GitHub folder icons for Mac OS X.
97
star
6

dotfiles

For my very own convenience.
Vim Script
62
star
7

alfred-screensharing

Connect to a host in Alfred with automatic network discovery.
53
star
8

alfred-cask

Manage Homebrew Casks with Alfred.
Shell
46
star
9

alfred-mount

Use Alfred to connect to your network shares with ease.
Makefile
41
star
10

non-terminating-bash-processes

Taming non-terminating Bash processes.
Shell
40
star
11

optometrist

Your optometrists favorite color theme for Terminal.app and iTerm2.
34
star
12

alfred-top

Process management in Alfred with top and kill.
Shell
20
star
13

candy-node

Deploy Candy as a node.js application
JavaScript
20
star
14

plex-duplicates

Find duplicate movie files in Plex Media Server.
Ruby
17
star
15

dtox

Extensible data transfer objects in JavaScript.
JavaScript
11
star
16

pagerank-service

A web service providing Google PageRank results.
Ruby
10
star
17

the-cube

The Cube is an experiment with CSS3 transitions.
HTML
10
star
18

alfred-play

Intelligent global hotkeys for Music.app and Spotify with Alfred.
Makefile
9
star
19

rethinkdb-udp-charts

Real-time charts using RethinkDB changefeeds.
JavaScript
5
star
20

ff-pagerank-client

A simple Firefox extension for displaying the Google PageRank of the websites you're visiting.
JavaScript
5
star
21

octoprowl

GitHub + Prowl = Octoprowl
Ruby
4
star
22

ticker-rs

Real-time stock tickers from the command-line. Written in Rust.
Rust
4
star
23

cloudflare-webfinger

Bring your own domain to Mastodon using Cloudflare Workers.
JavaScript
4
star
24

redirectcheck-service

A web service to check the HTTP status code behind a URL.
Ruby
3
star
25

candy-go

Deploy Candy as a Go application.
Go
3
star
26

candy-gae

Candy on Google App Engine (experimental)
Python
1
star
27

pfsense-traffic

Command-line tool for displaying traffic data from pfSense
Shell
1
star
28

candy-rack

Deploy Candy as a Rack application.
Ruby
1
star
29

rancheros-uefi

UEFI bootable RancherOS image.
1
star
30

ff-redirectcheck-client

A simple Firefox extension for retrieving and displaying the HTTP status code right beside the link on a page.
JavaScript
1
star