-
Designed for rapid generation of flamegraphs - no need to edit source code or recompile!
-
Generates output in several common formats - perf and Brendan Gregg’s
-
Interface inspired by recon_trace to automatically stop flamegraph generation
-
Can automatically open data in your flamegraph viewer of choice
This tool is designed to be the first profiling tool you reach for during development. It is designed for rapid visualization of program execution. It generates one or more flamegraphs of any function you specify. You can easily select function(s) to profile in the shell without having to alter your code or restart your application. This tool has two distinct advantages over existing flamegraph generators in Erlang:
-
It doesn’t require the user to insert profiling code into the functions they want to visualize as flamegraphs. Everything can be done on the fly without restarting your application.
-
It generates output in formats used by most flamegraph viewers. No manual post-processing is needed.
-
No Perl code.
There are two ways to profile code with eflambè. You can either invoke the code directly using eflambe:apply/1,2 or you can "capture" trace data from a function when it is invoked while your application is running with eflambe:capture/1,2,3.
This is useful when you want to profile a function as it is invoked normally inside of an application.
eflambe:capture({Module :: atom(), Function :: atom(), Arity :: integer}, NumberOfCallsToProfile :: integer(), Options :: list()).Example:
1> eflambe:capture({lists, seq, 2}, 1, [{output_format, brendan_gregg}]).
% Wait for program to invoke `lists:seq/2`For profiling a function that you want to invoke directly.
eflambe:apply({Module :: atom(), Function :: atom(), Args :: list()}, Options :: list()).
eflambe:apply({Fun :: fun(), Args :: list()}, Options :: list()).Example:
1> eflambe:apply({module, fun, Args}, [{output_format, brendan_gregg}]).
ok-
{return, value | flamegraph | filename}- What return to the calling code. Defaults tovalueforapply/2function andfilenameforcapture/3function. -
value- return the return value of the function being profiled. Implies the flamegraph data will be written to a file. -
flamegraph- return the flamegraph data in the format specified instead of writing it to a file. -
filename- return the filename of the file the flamegraph data was written to. Implies the flamegraph data will be written to a file. -
{output_directory, Dir}- Specify the output directory to write trace data to. Default is project root, and relative paths are relative to the project root. -
{output_format, brendan_gregg | svg}- Specify the output format. Default isbrendan_gregg. -
{open, Program}- Specify the program to load the trace output in after output is generated.Programmust be one ofspeedscopeorhotspot. By default this option is not set.hotspotis not useful right now as it cannot load files generated by thebrendan_greggformatter.
While designing this tool there were several different approaches I considered. Because I wanted the user to be able to dynamically select functions to profile at runtime, I’d need to start tracing any process that invoked the selected function. Unfortunately it’s hard to turn tracing on at an exact point in time for a specific process. All the existing tracing libraries do not support this. An ideal implementation would have been an erl_tracer behavior that blocked execution when the selected function was invoked for just long enough to start a trace of all the internal function calls. But unfortunately this wasn’t possible due to the fact that erl_tracer callbacks must be NIFs and NIFs cannot call Erlang functions directly. There are some hacky ways to get around this, but it didn’t feel like a good solution.
The solution I settled on is unfortunately no less hacky but is a bit more straightforward. Instead of trying to halt execution at a specific point in time, we use meck to create a mock of the module containing the function the user wants to profile with the passthrough option enabled. This results in a new module that is an exact copy, but it allows us to wrap the function the user specified in eflambe profiling code. Once profiling is complete the mock is removed and the original module is restored. The downside to this is that we’ve injected code into the module the user wants profiled, so the data collected doesn’t provide an accurate picture of the performance of the function itself. It should however provide a fairly accurate picture of the performance of all the code inside the function.
Heavily inspired by:
Feedback is welcome! This was started as a hackathon project but I plan on maintaining it for my teams every day use, so if you see something that could be improved please open an issue!