mjprof

mjprof is a command line monadic java profiler.

Introduction

mjprof is a monadic thread dump analysis tool set. It is a fancy way to say it analyzes jstack output using a series of simple composable building blocks (monads).

Motivation

So, You are out there in the wild vs a production machine. All you have in hand is the "poor man's profiler" jstack. You take one, two, three jstack thread dumps and then you need to look at them manually inside an editor, vi, less, etc.... If you have done it enough times, you probably know that it is a lot of manual work. Especially when you have thousands of threads in your process.

Running mjprof

mjprof reads thread dumps from one or more data sources and writes to the standard output. An example of a data source can be the standard input.

The following will filter out all the threads which are not in RUNNABLE state:
jstack -l pid | ./mjprof.sh contains/state,RUNNABLE/

The commands passed to mjprof consists of several building blocks, monads from now on, concatenated with , (comma) While the monads are relatively simple, mixing and matching them can yield a very thorough analysis, while still allowing you to focus on the data you need. Monad parameters are wrapped with / (instead of () {} or [] which are special chars in the shell) and seperated by ,

Monads

Data Sources

Data sources generate thread dumps and feed them into mjprof. The default data source is stdin. When no data source is specified stdin will be used. When more than one data source is specified the thread dumps generated by all of them will be fed into mjprof.

• jmx/host:port|MainClass|pid,[count],[sleep],[username],[passwd]/ - Generate dumps via JMX
• jmxc/host:port|MainClass|pid,[count],[sleep],[username],[passwd]/ - Generate thread dumps via JMX and collect sper thread CPU
• path/path/ - Read thread dump from file
• stdin - Read thread dumps from standard input
• visualvm/path/ - Read profiling session from xml export of VisualVM

Output

• gui/[title],[maxInvocations]/ - Display current thread dump in a GUI window
• snapshot/[filename]/ - Write to a file
• stdout - Writes current stream of thread dumps to stdout

Filters

• contains/attr,value/ - Returns only threads which contain the string in certain attribute (regexp not supported)
• -contains/attr,value/ - Returns only threads which do not contain the string (regexp not supported)

Single thread mappers

• -at - Eliminates the 'at' from the beginning of stack frames
• bottom/int/ - Returns at most n bottom stack frames of the stack
• -fn - Eliminates file name and line from stack frames
• frame/string/ - Eliminates stack frames from all stacks which do not contain string.
• -frame/string/ - Eliminates stack frames from all stacks which contain string.
• -namesuffix - Trim the last number from thread names helps to group thread pool threads together
• noop - Does nothing
• -pkg - Eliminates package name from stack frames
• -prop/attr/ - Removes a certain attribute e.g. eliminate/stack/
• top/int/ - Returns at most n top stack frames of the stack
• trimbelow/string/ - Trim all stack frames below the first occurrence of string

Full dump mappers:

• group/[attr]/ - Group a single thread dump by an attribute. If not attribute is specified all dump is merged
• merge/attribute/ - Combine all dumps to a single one merge based on thread id attribute
• sort/attr/ - Sorts based on an attribute
• -sort/string/ - Sorts based on an attribute (descending order)

Terminals:

• count - counts number of threads

• ctree - combine all stack traces with colors (UNIX Terminal)

• flat - Shows flat histogram of the profiles

• list - lists the possible stack trace attributes

• tree - combine all stack traces

• help -Prints this message

Macros:

• blocked - contains/state,BLOCKED/
• jvm - ncontains/stack,at/
• -jvm - contains/stack,at/.ncontains/name,RMI TCP /.ncontains/name,Reference Handle/.ncontains/name,Finalizer/.ncontains/name,JMX server connection timeout/.ncontains/name,RMI Scheduler/
• locks - stackkeep/- /
• -locks - stackelim/- /
• parking - contains/state,TIMED_WAITING/.contains/stack,sun.misc.Unsafe.park/
• running - contains/state,RUNNABLE/
• sleeping - contains/state,TIMED_WAITING/
• waiting - contains/state,WAITING/
• withstack - contains/stack,at/

Properties

Properties may change from one dump to another and the can also be eliminated by mjstack. Following is the list of usual properties

• status - The status of the thread
• nid - Native thread id ( a number)
• name - Name of thread
• state - State of thread
• los - The locked ownable synchronizers part of the stack trace
• daemon - Whether the thread is a daemon or not
• tid - The thread id (a number)
• prio - Thread priority, a number
• stack - The actual stack trace
• cpu_ns - cpu consumed in nano seconds
• wall_ms - wall time in ms for the time frame cpu was recorded
• %cpu - %cpu of the thread info

You can also get the actual list of properties bu using the list monad.
jstack -l pid | ./mjprof list

Examples

jstack Original output:
jstack -l 38515 > mystack.txt
Keep only thread which their names contain ISCREAM:
jstack -l 38515 | mjprof contains/name,ISCREAM/
Sort them by state
cat mystack.txt | mjprof contains/name,ISCREAM/.sort/state/
Eliminate the Locked Ownable Synchronizers Section
jstack -l 38515 | mjprof contains/name,ISCREAM/.sort/state/.eliminate/los/
Shorten stack traces to include only 10 last stack frames
mjprof jstack/MyAppMainClass/.contains/name,ISCREAM/.sort/state/.eliminate/los/.keeptop/10/
Count threads
mjprof jstack/38515/.contains/name,ISCREAM/.sort/state/.eliminate/los/.keeptop/10/.count

Building MJProf

mjprof uses maven for compilation. In order to build mjprof use the following command line:
mvn clean package This will create a zip file in target/mjprof1.0-bin.zip which contains everything you need.

Writing a plugin

mjprof monadic capabilities can be extended with plugins. If you feel something is missing you can write your own plugin. We will appreciate if you will contribute it back to the community. A plugin is a Java class which is annotated with the com.performizeit.mjprof.plugin.Plugin. The annotation accepts three parameters. "name" the plugin name (must be unique) paramTypes the expected parameter types in the plugin. description one liner that will be used for synopsis. For example:

import com.performizeit.mjprof.api.Plugin;

@Plugin(name="group", paramTypes={String.class},description="Group by an attribute")

There are several plugin types mappers filters terminals etc... for each type you will need to implement a different interface.

Mapper

Implement JStackMapper interface which includes a single method ThreadInfo map(ThreadInfo stck)

Filter

Implement JStackFilter interface which includes a single method boolean filter(ThreadIndo stck)

Terminal

Implement JStackFilter interface which includes a single method void addStackDump(JStackDump jsd)

Comparator

Implement JStackComparator interface which includes a single method int compare(ThreadInfo o1, ThreadInfo o2)

Installing a plugin

In order to install the plugin just drop your jar which contains plugin implementation into the 'plugins' directory inside mjprof installation directory.