• Stars
    star
    1,436
  • Rank 31,481 (Top 0.7 %)
  • Language
    Java
  • License
    Apache License 2.0
  • Created almost 9 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

MySQL performance monitoring and analysis.

MySQL Performance Analyzer

MySQL Performance Analyzer is an open source project for MySQL performance monitoring and analysis. This repository includes two sub projects:

  • Java web application project myperf
  • Java web server jetty wrapper

Build

MySQL Performance Analyzer is a Java Maven project. JDK and Maven 3.0 or later are required to build it.

Although JDK 8 is specified in pom.xml, this project does not use Java 8 specific features, so the user can modify pom.xml files of the two sub projects to use different JDK version. For example, if the build host only has JDK 7, modify the file myperf/pom.xml, change the lines

  	<source>1.8</source>
	<target>1.8</target>

to

	<source>1.7</source>
	<target>1.7</target>

The build will create a zip file named as myperfserver-server.zip under directory perfJettyServer/target. To build it, at top level, run

  mvn clean package

Installation and Usage Instructions

  1. Requirement: Java JDK 8, or the one specified by in pom.xml if changed during build time.

  2. Unzip myperfserver-server.zip to the desired installation directory. If you intend to install on Windows host, please review two shell scripts and create Windows equivalent.

    For future updates, most of the time you only need to copy myperf/target/myperf.war to webapps directory under the installation directory, then restart.

  3. For a more formal installation, it is recommended to have a MySQL database server as back store to store the metrics. Otherwise, use the built-in derby db.

    • Create a database, for example, named as metrics, with the MySQL database server.
    • Create a MySQL user (for example, 'metrics'@'my_host' -> here my_host is the machine where you MySQL perf analyzer) with all privileges on above schema.
    • The above information will be required when you first login to the analyzer to setup metrics gathering.
  4. Review script start_myperf.sh to see if you need to modify any command line settings. Usually, port number is the only one you need change

    • -f: config, configuration file path. If used, the values inside configuration file will be loaded first, but can be overwritten by command line agruments.
    • -j: jettyHome, leave it as it is
    • -p: http port to be used, 9092 by default
    • -w: war (web archive) file, has to be myperf.war
    • -k: working directory, if not specified, it will use ./work
    • -c: url context, default to /myperf, leave it as is.

    The following are the list allowed in configuration file, which uses Java property file format (name=value pairs)

    • jettyHome
    • useHttps: value yes/y/no/n/true/false
    • port
    • webcontextroot
    • workdir
    • logpath
    • warfile
    • certKeyStorePath
    • certKeyStorePassword

    To support https, SSL certificate should be provided (either self signed or signed by third party) and stored in Java keystore using Java keytool (see https://wiki.eclipse.org/Jetty/Howto/Configure_SSL). If encounter login issue when switch between SSL and non SSL, you have to remove the browser cookie originated from the server host this application, using any cookie tool provided by the browser.

    Modify java command path inside start_myperf.sh, if needed, for example, when there are multiple JDK packages installed.

  5. Start up:

       ./start_myperf.sh
    

    Check nohup.out and logs directory for any error logs.

  6. Shutdown:

       ./stop_myperf.sh
    
  7. First time Login and Setup

After startup, point your browser to http://your_host:9092/myperf (or the port number you changed). Or https://your_host:9092/myperf if https is correctly configured.

The initial login user and credential are myperf/change.

After login, you will be directed to setup page:

  • You can add an email address for notifications. The email uses OS "mailx" command.

  • Configure the metrics storage database, using the one that you created in the earlier steps.

    A metrics scan interval of 1 or 5 minutes should be good enough.

  • If use built-in derbydb, choose short retention days.

  • After configuration is done, you need to start the scanner ("Start Scanner" button on top of the page).

    Everytime you change the configuration, you need to restart the scanner.

  • If the scanner does not work as expected, restart the analyzer

       ./stop_myperf.sh
    

    then

       ./start_myperf.sh
    
  • If your SNMP uses non default community, or not version v2c, use "SNMP Settings" tab to configure SNMP. Currently only Linux based SNMP is supported. If your MySQL servers are not running on Linux, use "Alert Settings" tab to disable SNMP metrics polling.

  1. For each database server you want to monitor, you need to create a MySQL user with the following privileges:

    • process
    • replication client
    • show databases
    • show view
    • select on all (if you want to use it to check data dictionary or run explain plans)
  2. The analyzer relies on Linux SNMP to gather OS level data. Check snmpd service status.

  3. Hipchat integration: if you use hipchat for work related collaboration, you can create a hipchat room to receive alerts from MySQL Perf Analyzer. Check https://developer.atlassian.com/hipchat/tutorials/building-a-simple-add-on-using-a-wizard regarding how to setup hipchat integration. After you have hipchat notification URL and auth token, you can configure MySQL Perf Analyzer to send notification to your room, using Settings page.

Known Limitations

  1. SNMP is based on the Linux specification.
  2. Email notification uses Linux's "mailx" command.

License

This code licensed under the Apache license. See the LICENSE file for terms.

More Repositories

1

CMAK

CMAK is a tool for managing Apache Kafka clusters
Scala
11,676
star
2

open_nsfw

Not Suitable for Work (NSFW) classification using deep neural network Caffe models.
Python
5,791
star
3

TensorFlowOnSpark

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.
Python
3,860
star
4

serialize-javascript

Serialize JavaScript to a superset of JSON that includes regular expressions and functions.
JavaScript
2,785
star
5

gryffin

Gryffin is a large scale web security scanning platform.
Go
2,075
star
6

fluxible

A pluggable container for universal flux applications.
JavaScript
1,815
star
7

AppDevKit

AppDevKit is an iOS development library that provides developers with useful features to fulfill their everyday iOS app development needs.
Objective-C
1,439
star
8

squidb

SquiDB is a SQLite database library for Android and iOS
Java
1,313
star
9

CaffeOnSpark

Distributed deep learning on Hadoop and Spark clusters.
Jupyter Notebook
1,262
star
10

react-stickynode

A performant and comprehensive React sticky component.
JavaScript
1,227
star
11

blink-diff

A lightweight image comparison tool.
JavaScript
1,191
star
12

egads

A Java package to automatically detect anomalies in large scale time-series data
Java
1,152
star
13

elide

Elide is a Java library that lets you stand up a GraphQL/JSON-API web service with minimal effort.
Java
985
star
14

vssh

Go Library to Execute Commands Over SSH at Scale
Go
930
star
15

webseclab

set of web security test cases and a toolkit to construct new ones
Go
915
star
16

kubectl-flame

Kubectl plugin for effortless profiling on kubernetes
Go
746
star
17

streaming-benchmarks

Benchmarks for Low Latency (Streaming) solutions including Apache Storm, Apache Spark, Apache Flink, ...
Jupyter Notebook
621
star
18

lopq

Training of Locally Optimized Product Quantization (LOPQ) models for approximate nearest neighbor search of high dimensional data in Python and Spark.
Python
558
star
19

redislite

Redis in a python module.
Python
556
star
20

HaloDB

A fast, log structured key-value store.
Java
486
star
21

hecate

Automagically generate thumbnails, animated GIFs, and summaries from videos
C++
468
star
22

fetchr

Universal data access layer for web applications.
JavaScript
447
star
23

storm-yarn

Storm-yarn enables Storm clusters to be deployed into machines managed by Hadoop YARN.
Java
418
star
24

react-i13n

A performant, scalable and pluggable approach to instrumenting your React application.
JavaScript
383
star
25

FEL

Fast Entity Linker Toolkit for training models to link entities to KnowledgeBase (Wikipedia) in documents and queries.
Java
334
star
26

monitr

A Node.js process monitoring tool.
C++
312
star
27

Oak

A Scalable Concurrent Key-Value Map for Big Data Analytics
Java
266
star
28

TDOAuth

A BSD-licensed single-header-single-source OAuth1 implementation.
Swift
250
star
29

routr

A component that provides router related functionalities for both client and server.
JavaScript
246
star
30

mysql_partition_manager

MySQL Partition Manager
SQLPL
210
star
31

l3dsr

Direct Server Return load balancing across Layer 3 boundaries.
Shell
190
star
32

dnscache

dnscache for Node
JavaScript
184
star
33

object_relation_transformer

Implementation of the Object Relation Transformer for Image Captioning
Python
174
star
34

check-log4j

To determine if a host is vulnerable to log4j CVE‐2021‐44228
Shell
173
star
35

fili

Easily make RESTful web services for time series reporting with Big Data analytics engines like Druid and SQL Databases.
Java
171
star
36

sherlock

Sherlock is an anomaly detection service built on top of Druid
Java
149
star
37

YMTreeMap

High performance Swift treemap layout engine for iOS and macOS.
Swift
129
star
38

maha

A framework for rapid reporting API development; with out of the box support for high cardinality dimension lookups with druid.
Scala
127
star
39

covid-19-data

COVID-19 datasets are constructed entirely from primary (government and public agency) sources
110
star
40

subscribe-ui-event

Subscribe-ui-event provides a cross-browser and performant way to subscribe to browser UI Events.
JavaScript
109
star
41

jafar

🌟!(Just another form application renderer)
JavaScript
109
star
42

panoptes

A Global Scale Network Telemetry Ecosystem
Python
98
star
43

reginabox

Registry In A Box
JavaScript
97
star
44

preceptor

Test runner and aggregator
JavaScript
85
star
45

hive-funnel-udf

Hive UDFs for funnel analysis
Java
85
star
46

SparkADMM

Generic Implementation of Consensus ADMM over Spark
Python
82
star
47

react-cartographer

Generic component for displaying Yahoo / Google / Bing maps.
JavaScript
82
star
48

graphkit

A lightweight Python module for creating and running ordered graphs of computations.
Python
80
star
49

storm-perf-test

A simple storm performance/stress test
Java
76
star
50

UDPing

UDPing measures latency and packet loss across a link.
C++
72
star
51

bgjs

TypeScript
66
star
52

YMCache

YMCache is a lightweight object caching solution for iOS and Mac OS X that is designed for highly parallel access scenarios.
Objective-C
63
star
53

ycb

A multi-dimensional configuration library that builds bundles from resource files describing a variety of values.
JavaScript
63
star
54

ariel

Ariel is an AWS Lambda designed to collect, analyze, and make recommendations about Reserved Instances for EC2.
Python
62
star
55

validatar

Functional testing framework for Big Data pipelines.
Java
58
star
56

imapnio

Java imap nio client that is designed to scale well for thousands of connections per machine and reduce contention when using large number of threads and cpus.
Java
54
star
57

serviceping

A ping like utility for tcp services
Python
50
star
58

express-busboy

A simple body-parser like module for express that uses connect-busboy under the hood.
JavaScript
44
star
59

covid-19-api

Yahoo Knowledge COVID-19 API provides JSON-API and GraphQL interfaces to access COVID-19 publicly sourced data
JavaScript
43
star
60

proxy-verifier

Proxy Verifier is an HTTP replay tool designed to verify the behavior of HTTP proxies. It builds a verifier-client binary and a verifier-server binary which each read a set of YAML or JSON files that specify the HTTP traffic for the two to exchange.
C++
42
star
61

panoptes-stream

A cloud native distributed streaming network telemetry.
Go
41
star
62

yql-plus

The YQL+ parser, execution engine, and source SDK.
Java
40
star
63

context-parser

A robust HTML5 context parser that parses HTML 5 web pages and reports the execution context of each character.
HTML
40
star
64

cocoapods-blocklist

A CocoaPods plugin used to check a project against a list of pods that you do not want included in your build. Security is the primary use, but keeping specific pods that have conflicting licenses is another possible use.
Ruby
39
star
65

covid-19-dashboard

Source code for the Yahoo Knowledge Graph COVID-19 Dashboard
JavaScript
38
star
66

FmFM

Python
36
star
67

ember-gridstack

Ember components to build drag-and-drop multi-column grids powered by gridstack.js
JavaScript
36
star
68

VerizonVideoPartnerSDK-controls-ios

Public iOS implementation of the OneMobileSDK default custom controls interface... demonstrating how customers can implement their own custom video player controls.
Swift
35
star
69

k8s-namespace-guard

K8s - Admission controller for guarding namespace
Go
34
star
70

fluxible-action-utils

Utility methods to aid in writing actions for fluxible based applications.
JavaScript
34
star
71

parsec

A collection of libraries and utilities to simplify the process of building web service applications.
Java
34
star
72

mod_statuspage

Simple express/connect middleware to provide a status page with following details of the nodejs host.
JavaScript
32
star
73

bftkv

A distributed key-value storage that's tolerant to Byzantine fault.
JavaScript
30
star
74

protractor-retry

Use protractor features to automatically re-run failed tests with a specific configurable number of attempts.
JavaScript
28
star
75

cubed

Data Mart As A Service
Java
27
star
76

spivak

Python
27
star
77

jsx-test

An easy way to test your React Components (`.jsx` files).
JavaScript
27
star
78

ycb-java

YCB Java
Java
27
star
79

fluxible-immutable-utils

A mixin that provides a convenient interface for using Immutable.js inside react components.
JavaScript
25
star
80

SubdomainSleuth

Scanner to identify dangling DNS records and subdomain takeovers
Go
25
star
81

maaf

Modality-Agnostic Attention Fusion for visual search with text feedback
Python
25
star
82

node-limits

Simple express/connect middleware to set limit to upload size, set request timeout etc.
JavaScript
24
star
83

GitHub-Security-Alerts-Workflow

Automation to Incorporate GitHub Security Alerts Into your Business Workflow
Python
23
star
84

bandar-log

Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.
Scala
21
star
85

fumble

Simple error objects in node. Created specifically to be used with https://github.com/yahoo/fetchr and based on https://github.com/hapijs/boom
JavaScript
21
star
86

express-csp

Express extension for Content Security Policy
JavaScript
19
star
87

elide-js

Elide is a library that makes it easy to talk to a JSON API compliant backend.
JavaScript
18
star
88

Zake

A python package that works to provide a nice set of testing utilities for the kazoo library.
Python
18
star
89

npm-auto-version

Automatically generate new NPM versions based on Git tags when publishing
JavaScript
18
star
90

httpmi

An HTTP proxy for IPMI commands.
Python
17
star
91

hodman

Selenium object library
JavaScript
17
star
92

cerebro

JavaScript
17
star
93

SongbirdCharts

Allows for other apps to render accessible audio charts
Kotlin
17
star
94

Override

In app feature flag management
Swift
16
star
95

ychaos

YChaos - The Resilience Framework by Yahoo!
Python
16
star
96

elide-spring-boot-example

Spring Boot example using the Elide framework.
Java
15
star
97

parsec-libraries

Tools to simplify deploying web services with Parsec.
Java
15
star
98

node-info

Node environment information
JavaScript
14
star
99

NetCHASM

An Automated health checking and server status verification system.
C++
13
star
100

invirtualenv

Tool to deploy python virtualenvs
Python
13
star