Docker images for this exporter are available at the quay.io and pkg.github.com registries:

• quay.io/martinhelmich/prometheus-nginxlog-exporter:v1

• docker.pkg.github.com/martin-helmich/prometheus-nginxlog-exporter/exporter:v1

Have a look at the releases page to see the available versions and how to pull their images. In general, I would recommend using the v1 tag instead of latest.

Run the exporter as follows (adjust paths like /path/to/logs and /path/to/config to your own needs):

Command-line flags and arguments can simply be appended to the docker run command, for example to use a configuration file:

Each release from 1.5.1 or newer provides both DEB and RPM packages.

DEB:

RPM:

The package come with a dependency on systemd and configure the exporter to be running automatically:

The packages drop a configuration file to /etc/prometheus-nginxlog-exporter.hcl which you can adjust to your own needs.

If you do not want to use one of the pre-built packages, you can download the binary itself and manually configure systemd to start it. You can find an example unit file for this service in this repository. Simply copy the unit file to /etc/systemd/system:

The shipped unit file expects the binary to be located in /usr/sbin/prometheus-nginxlog-exporter (if you sideload the exporter without using your package manager, you might want to put it to /usr/local, instead) and the configuration file in /etc/prometheus-nginxlog-exporter.hcl. Adjust to your own needs.

If you run a logfile-generating service (be it NGINX, or anything that generates similar access log files) in Kubernetes, you can run the exporter as a sidecar along your "main" container within the same pod.

The following example shows you how to deploy the exporter as a sidecar, accepting logs from the main container via syslog:

In this example, the configuration file is passed via the exporter-config ConfigMap. This might look like follows:

The config file instructs the exporter to accept log input via syslog. To forward logs to the exporter, just instruct your main container to send its access logs via syslog to 127.0.0.1:5531 (which works, since the main container and the sidecar share their network namespace).

To build the exporter from source, simply build it with go get:

Alternatively, clone this repository and just run go build:

For historic reasons, this exporter exports separate metrics for different namespaces (because the namespace is part of the metric name). However, in many (most) cases, it’s more convenient to have the same metric name across different namespaces (with different log formats and names).

This can be done in two steps:

1. Override Prometheus metrics namespace to some common prefix (metrics_override)

2. Set label name for nginxlog-exporter’s config namespace (namespace_label)

Exported metrics will have the following format:

• prefix can be set to "", resulting metrics like http_response_count_total{…​}

• namespace_label can be omitted - so you have full control on metric format

Some details and history on this can be found in issue #13.

Partial case of [Dynamic-re-labeling]:

Exported metrics will have upstream_addr and country labels.

Currently, the exporter supports reading log data from

1. files

2. syslog

All log sources can be configured on a per-namespace basis using the source property.

Reading from files

namespace "test" {
  source {
    files = ["/var/log/nginx/access.log"]
    // ...
  }
}

Reading from syslog

namespace "test" {
  source {
    syslog {
      listen_address = "udp://127.0.0.1:8514" (1)
      format = "rfc3164" (2)
      tags = ["nginx"] (3)
    }

    // ...
  }
}

Re-labeling lets you add arbitrary fields from the parsed log line as labels to your metrics. To add a dynamic label, add a relabel statement to your configuration file:

1. The whitelist property is optional; if set, only the supplied values will be added as label. All other values will be subsumed under the "other" label value. See #16 for a more detailed discussion around the reasoning.

Dynamic relabeling also allows you to aggregate your metrics by request path (which replaces the experimental feature originally introduced in #23). The following example splits the content of the request variable at every space (using split) and return the second element (index 1) of the resulting list which is the base for the regex):

1. The separator property is optional; if omitted, the space character (" ") will be assumed as separator.

If a match is found, the replacement replaces each occurrence of the corresponding match in the original value. Otherwise the processing continues to check the following match statements.

The YAML configuration for relabelings works similar to the HCL configuration:

If your regular expression contains groups, you can also use the matched values of those in the replacement value:

You can specify one or more wildcards in the source file names, in which case the wildcards will be resolved to the corresponding list of files at startup of the exporter.

Be aware that the list of matches is only evaluated at the start of the program. If a new file is added with a match of one glob filter, you’ll have to restart the program for it to be monitored.

Given a config like this:

And a folder containing these files:

The list of files monitored by this namespace will be /var/log/nginx/main_access.log,/var/log/nginx/virtualhost1_access.log.

You can use the JSON parser by setting the --parser command line flag or parser config file property to json.