• This repository has been archived on 02/Aug/2022
  • Stars
    star
    146
  • Rank 252,769 (Top 5 %)
  • Language
    Java
  • License
    Apache License 2.0
  • Created almost 6 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

๐Ÿ“ˆ OpenDistro Performance Analyzer

Java CI CD codecov Documentation Chat PRs welcome!

Performance Analyzer

Performance Analyzer exposes a REST API that allows you to query numerous performance metrics for your cluster, including aggregations of those metrics, independent of the Java Virtual Machine (JVM). PerfTop is the default command line interface (CLI) for displaying those metrics.

Setup

Performance Analyzer API

Performance Analyzer uses a single HTTP method and URI for all requests:

GET <endpoint>/_opendistro/_performanceanalyzer/metrics

Then you provide parameters for metrics, aggregations, dimensions, and nodes (optional):

?metrics=<metrics>&agg=<aggregations>&dim=<dimensions>&nodes=all"
  • metrics - comma separated list of metrics you are interested in. For a full list of metrics, see Metrics Reference.
  • agg - comma separated list of agg to be used on each metric. Possible values are sum, avg, min and max. Length of the list should be equal to the number of metrics specified.
  • dim - comma separated list of dimensions. For the list of dimensions supported by each metric, see Metrics Reference.
  • nodes - If the string all is passed, metrics from all nodes in the cluster are returned. For any other value, metrics from only the local node is returned.

SAMPLE REQUEST

GET _opendistro/_performanceanalyzer/metrics?metrics=Latency,CPU_Utilization&agg=avg,max&dim=ShardID&nodes=all

Batch Metrics API

While the metrics api associated with performance analyzer provides the last 5 seconds worth of metrics, the batch metrics api provides more detailed metrics and from longer periods of time. See the design doc for more information.

In order to access the batch metrics api, first enable it using one of the following HTTP request:

POST localhost:9200/_opendistro/performanceanalyzer/batch/config -H โ€˜Content-Type: application/jsonโ€™ -d โ€˜{"enabled": true}โ€™
POST localhost:9200/_opendistro/performanceanalyzer/batch/cluster/config -H โ€˜Content-Type: application/jsonโ€™ -d โ€˜{"enabled": true}โ€™

The former enables batch metrics on a single node, while the latter enables it on nodes across the entire cluster. Batch metrics can be disabled using analogous queries with {"enabled": false}.

You can then query either the config or cluster config apis to see how many minutes worth of batch metrics data will be retained by nodes in the cluster (batchMetricsRetentionPeriodMinutes):

GET localhost:9200/_opendistro/_performanceanalyzer/config

{"performanceAnalyzerEnabled":true,"rcaEnabled":false,"loggingEnabled":false,"shardsPerCollection":0,"batchMetricsEnabled":true,"batchMetricsRetentionPeriodMinutes":7}

GET localhost:9200/_opendistro/_performanceanalyzer/cluster/config

{"currentPerformanceAnalyzerClusterState":9,"shardsPerCollection":0,"batchMetricsRetentionPeriodMinutes":7}

The default retention period is 7 minutes. However, the cluster owner can adjust this by setting batch-metrics-retention-period-minutes in performance-analyzer.properties (note, setting this value will require a restart so that the cluster can read the new value upon startup). The value must be between 1 and 60 minutes (inclusive) โ€” the range is capped like so in order to prevent excessive data retention on the cluster, which would eat up a lot of storage.

You can then access the batch metrics available at each node via queries of the following format:

GET localhost:9600/_opendistro/_performanceanalyzer/batch?metrics=<metrics>&starttime=<starttime>&endtime=<endtime>&samplingperiod=<samplingperiod>
  • metrics - Comma separated list of metrics you are interested in. For a full list of metrics, see Metrics Reference.
  • starttime - Unix timestamp (difference between the current time and midnight, January 1, 1970 UTC) in milliseconds determining the oldest data point to return. starttime is inclusive โ€” data points from at or after the starttime will be returned. Note, the starttime and endtime supplied by the user will both be rounded down to the nearest samplingperiod. starttime must be no less than now - retention_period and it must be less than the endtime (after the rounding).
  • endtime - Unix timestamp in milliseconds determining the freshest data point to return. endtime is exclusive โ€” only datapoints from before the endtime will be returned. endtime must be no greater than the system time at the node, and it must be greater than the startime (after being rounded down to the nearest samplingperiod).
  • samplingperiod - Optional parameter indicating the sampling period in seconds (default is 5s). The requested time range will be partitioned according to the sampling period, and data from the first available 5s interval in each partition will be returned to the user. Must be at least 5s, must be less than the retention period, and must be a multiple of 5.

Note, the maximum number of datapoints that a single query can request for via API is capped at 100,800 datapoints (in order to prevent excessive memory consumption by the datapoints). If a query exceeds this limit, an error is returned. The query parameters can be adjusted on such queries to request for fewer datapoints at a time.

Note, unlike with the metrics api, there is no nodes=all parameter for the batch metrics api. You must query a specific node in order to obtain metrics from that node.

Note, the default retention period is 7 minutes because a typical use-case would be to query for 5 minutes worth of data from the node. In order to do this, a client would actually select a starttime of now-6min and an endtime of now-1min (this one minute offset will give sufficient time for the metrics in the time range to be available at the node). Atop this 6 minutes of retention, we need an extra 1 minute of retention to account for the time that would have passed by the time the query arrives at the node, and for the fact that starttime and endtime will be rounded down to the nearest samplingperiod.

SAMPLE REQUEST

GET _opendistro/_performanceanalyzer/batch?metrics=CPU_Utilization,IO_TotThroughput&starttime=1594412250000&endtime=1594412260000&samplingperiod=5

See the design doc for the expected response.

Documentation

Please refer to the technical documentation for detailed information on installing and configuring Performance Analyzer.

Code of Conduct

This project has adopted an Open Source Code of Conduct.

Security issue notifications

If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our vulnerability reporting page. Please do not create a public GitHub issue.

Licensing

See the LICENSE file for our project's licensing. We will ask you to confirm the licensing of your contribution.

Copyright

Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.

More Repositories

1

sql

๐Ÿ” Open Distro SQL Plugin
Java
620
star
2

opendistro-build

๐Ÿงฐ Open Distro Build Scripts
Shell
343
star
3

alerting

๐Ÿ“Ÿ Open Distro Alerting Plugin
Kotlin
279
star
4

sample-code

๐Ÿ‘‹ Welcome to the Open Distro sample-code area. Share your great ideas and code samples with the Open Distro Community.
Python
278
star
5

k-NN

๐Ÿ†• A machine learning plugin which supports an approximate k-NN search algorithm for Open Distro.
Java
276
star
6

alerting-kibana-plugin

๐Ÿ“Ÿ Open Distro Kibana Alerting Plugin
JavaScript
140
star
7

index-management

๐Ÿ—ƒ Open Distro Index Management
Kotlin
115
star
8

perftop

๐Ÿ“ˆ PerfTop: A client for the Open Distro Performance Analyzer
JavaScript
94
star
9

anomaly-detection

A machine learning plugin in Open Distro for real time anomaly detection on streaming data.
Java
78
star
10

job-scheduler

๐Ÿ•“ Open Distro Job Scheduler
Java
47
star
11

deprecated-security-advanced-modules

[DO NOT USE - DEPRECATED as of v1.4.0] Advanced modules for the Open Distro security plugin; Merged into security repo.
Java
47
star
12

anomaly-detection-kibana-plugin

A Kibana plugin providing visualizations for anomaly detection in Open Distro.
TypeScript
44
star
13

index-management-kibana-plugin

๐Ÿ—ƒ Open Distro Index Management Kibana UI plugin
TypeScript
42
star
14

kibana-reports

Kibana Reports
TypeScript
39
star
15

performance-analyzer-rca

The Performance Analyzer RCA is a framework that builds on the Performance Analyzer engine to support root cause analysis (RCA) of performance and reliability problems for Elasticsearch instances.
Java
38
star
16

data-prepper

This repository is archived. Please migrate to the active project: https://github.com/opensearch-project/data-prepper
Java
37
star
17

deprecated-security-ssl

[DO NOT USE - DEPRECATED AS OF v1.0.0] SSL module for Open Distro security plugin
Java
31
star
18

odfe-cli

A full-featured command line interface (CLI) for Open Distro.
Go
24
star
19

asynchronous-search

โ–ถ๏ธ Asynchronous search makes it possible for users to run queries in the background, allowing users to track the progress, and retrieve partial results as they become available.
Java
23
star
20

kibana-notebooks

Open Distro Kibana Notebooks
TypeScript
21
star
21

deprecated-security-parent

[DO NOT USE - DEPRECATED as of v1.4.0] Parent repo for Open Distro Security plugin; Merged into security repo.
19
star
22

security

Java
17
star
23

cross-cluster-replication

Kotlin
15
star
24

trace-analytics

TypeScript
7
star
25

kibana-visualizations

TypeScript
4
star
26

common-utils

Open Distro Common-Utils
Java
4
star
27

security-kibana-plugin

TypeScript
3
star
28

notifications

Notifications plugin for Open Distro enables other plugins to send notifications via Email, Slack, Amazon Chime, Custom web-hook etc channels
Kotlin
3
star
29

pipe-processing-language

Piped Processing Language (PPL) for Elasticsearch
3
star