linux-test-project/lcov

-------------------------------------------------
- README file for the LTP GCOV extension (LCOV) -
- Last changes: 2023-05-22
-------------------------------------------------

Description
-----------
  LCOV is an extension of GCOV, a GNU tool which provides information about
  what parts of a program are actually executed (i.e. "covered") while running
  a particular test case. The extension consists of a set of Perl scripts
  which build on the textual GCOV output to implement the following enhanced
  functionality:

    * HTML based output: coverage rates are additionally indicated using bar
      graphs and specific colors.

    * Support for large projects: overview pages allow quick browsing of
      coverage data by providing a hierarchical directory structure
      view, a flat list of all source files in the project,  or a three-level
      detail view: directory, file and source code view.

  LCOV was initially designed to support Linux kernel coverage measurements,
  but works as well for coverage measurements on standard user space
  applications.

  LCOV has recently been enhanced to support differential coverage, as well
  as date- and owner-binning.  See https://arxiv.org/abs/2008.07947 or
  https://ieeexplore.ieee.org/document/9438597 for
  a detailed explanation of the concepts and several possible use models.
  A video presentation of the basic ideas can be found at
  http://doi.org/10.5281/zenodo.4653252

  In addition, several other features and capabilities have been added.  See
  section 6, below, for a brief description - and also see the man pages and
  the test cases.


Further README contents
-----------------------
  1. Included files
  2. Installing LCOV
  3. Dependencies
  4. An example of how to access kernel coverage data
  5. An example of how to access coverage data for a user space program
  6. New features in lcov 2.0
  7. Questions and Comments



1. Important files
------------------
  README             - This README file
  CHANGES            - List of changes between releases
  bin/lcov           - Tool for capturing LCOV coverage data
  bin/genhtml        - Tool for creating HTML output from LCOV data
  bin/gendesc        - Tool for creating description files as used by genhtml
  bin/geninfo        - Internal tool (creates LCOV data files)
  bin/genpng         - Internal tool (creates png overviews of source files)
  man                - Directory containing man pages for included tools
  example            - Directory containing an example to demonstrate LCOV
  lcovrc             - LCOV configuration file
  Makefile           - Makefile providing 'install' and 'uninstall' targets


2. Installing LCOV
------------------
The LCOV package is available as either RPM or tarball from:

  https://github.com/linux-test-project/lcov/releases

To install the tarball, unpack it to a directory and run:

  make install

Use Git for the most recent (but possibly unstable) version:

  git clone https://github.com/linux-test-project/lcov.git

Change to the resulting lcov directory and type:

  make install

The default install location is /usr/local.  Note that you may need to
have superuser permissions to write into system directories.

To install in a different location - for example, your home directory, run:

  make PREFIX=$HOME/my_lcov install


3. Dependencies:
----------------

The lcov module is implemented primarily in Perl - and requires both a
moderately up-to-date Perl installation and multiple Perl packages.

These perl packages include:

  - Capture::Tiny
  - DateTime
  - Devel::Cover
  - Digest::MD5
  - File::Spec
  - at least one flavor of JSON module.
    In order of performance/preference:
       - JSON::XS
       - Cpanel::JSON::XS
       - JSON::PP
       - JSON
 - Memory::Process
 - Time::HiRes

If your system is missing any of these, then you may be able to install them
via:

   $ perl -MCPAN -e 'install "packageName"'

You will very likely need superuser access to be able to install Perl
modules.

Some of the applications provided with the lcov module are written
in Python - and may require additional Python packages.
In particular, 'xlsxwriter' is required in order to generate any
of the spreadsheet reports.

To measure Python code coverage, users will need Python packages:

  - Coverage.py

In addition, contributors will need:

  - perltidy

Your platform may support other mechanisms to install and/or update
required packages.



4. An example of how to access Linux kernel coverage data
---------------------------------------------------------
Requirements: Follow the Linux kernel coverage setup instructions at:

  https://docs.kernel.org/dev-tools/gcov.html

As root, do the following:

  a) Resetting counters

     lcov --zerocounters

  b) Capturing the current coverage state to a file

     lcov --capture --output-file kernel.info

  c) Getting HTML output

     genhtml kernel.info

Point the web browser of your choice to the resulting index.html file.


5. An example of how to access coverage data for a user space program
---------------------------------------------------------------------
Requirements: compile the program in question using GCC with the options
'-fprofile-arcs -ftest-coverage' or '--coverage'. During linking, make sure
to specify '-lgcov' or '--coverage'.

Assuming the compile directory is called "appdir", do the following:

  a) Resetting counters

     lcov --directory appdir --zerocounters

  b) Capturing the current coverage state to a file

     lcov --directory appdir --capture --output-file app.info

     Note that this step only works after the application has
     been started and stopped at least once. Otherwise lcov will
     abort with an error mentioning that there are no data/.gcda files.

     The GCC runtime emits coverage data (the .gcda files) in an atexit
     callback.  If your application exits abnormally or crashes before
     the callback is executed, then no coverage data will be available.

  c) Getting HTML output

     genhtml app.info

  d) Generating a differential coverage report

     See the examples in .../tests/gendiffcov.

Point the web browser of your choice to the resulting index.html file.

Please note that independently of where the application is installed or
from which directory it is run, the --directory statement needs to
point to the directory in which the application was compiled.

For further information on the gcc profiling mechanism, please also
consult the gcov man page.


6. New features in lcov 2.0:
----------------------------

New features and capabilities fall into 7 major categories:

  a) Categorization

     This refers primarily to differential coverage categorization as
     well as date- and owner-binning.  See https://arxiv.org/abs/2008.07947
     or https://ieeexplore.ieee.org/document/9438597 for a detailed
     description of the concepts.

     Differential categorization and binning are orthogonal in the sense
     that you can generate differential report without binning as well
     as 'vanilla' coverage reports with binning.  See the above papers
     and the genhtml man page for details.

     Related options:
        --baseline-file, --diff-file, --annotate-script, --date-bins,
        --new-file-as-baseline, --elide-path-mismtach

  b) Error handling

     A generic - but very simple - error handler has been added to the
     lcov tool suite.  The error handler is used to report exceptions,
     and provides a mechanism for the user to ignore the particular
     message if desired.
     See the genhtml/lcov/geninfo man pages for details.

     Note that some errors are unrecoverable - and cannot be suppressed or
     ignored.

     Related options:  --ignore-error, --keep-going

  c) Navigation and display:

     Navigation aids such as hyperlinks to the first uncovered region,
     to the next uncovered region, etc. have been implemented.  Similarly,
     new tables, new columns, and new links between tables enable the
     user to identify the author of particular code (covered or not
     covered), as well as the time period when the code was written.

     Collectively, these features help the user to quickly identify the
     cause of code coverage issues, and to then decide what to do.

     An option to generate a 'hierarchical' coverage report (which follows
     the source code directory structure) or 'flat' (all files in top level
     of two-level report) as well as various other small features (tooltip
     popups, user-specified HTML header, footer, and table labels, etc.) are
     also available.

     See the genhtml man page for some details, as well as the
     'gendiffcov/simple' testcases for some examples.

      Related options:
          --baseline-title, --baseline-date, --current-date,
          --flat, --hierarchical,
          --show-owners, --show-noncode, --show-navigation, --show-proportion,
          --suppress-aliases

  d) Data manipulation

     Filters are used to suppress or remove certain coverage artifacts -
     for example, branches generated by the compiler (e.g., for exception
     handling).  These artifacts can overwhelm the user code and obscure
     coverage features that are interesting to the user.

     Other options are used to focus on or to exclude certain sections
     of code, as well as to do regexp replacement of file names - possibly
     using case-insensitve comparison.
     (Path munging is useful primarily when the build structure does
     not exactly match the layout in your revision control system; this
     is common in large projects with reusable components.)

     See the lcov/geninfo/genhtml man pages for a detailed description of
     the available filters and manipulation features.

     Related options:
        --include, --exclude, --erase-functions, --omit-lines,
        --substitute, --filter

  e) Callbacks/customization

     The user can supply callbacks which are used to:

        i) interface with the revision control system
           Sample scripts:
             - Perforce:  see 'p4diff' and 'p4annotate'
             - Git: see 'gitdiff' and 'gitblame'
        ii) verify that source code versions are compatible, and
            Sample scripts: see 'get_signature', 'getp4version'
        iii) enforce a desired code coverage criteria
             Sample script: criteria

     See the genhtml/lcov/geninfo man pages for details.

     Note that the various sample scripts are found in the source code 'bin'
     directory, but are installed in the $LCOV_HOME/share/lcov/support-scripts
     directory of the release.

     Related options:
       --annotate-script, --criteria-script, --version-script

  f) Performance

     lcov/genhtml/geninfo have been refactored to parallelize computation
     across multiple cores, if requested.
     In general, this provides speedup that is nearly linear in the number
     of cores.
     There is also an option to throttle parallelism to not exceed peak
     memory consumption contraints, as well as options to enable simple
     profile data collection - so you can see where time is going and
     thus to hint at potential optimizations.  The 'spreadsheet.py'
     script can be used to view generated profile data..

     See the genhtml/lcov/geninfo man pages for details

     Related options: --parallel, --memory, --profile

  g) Language support

     Added 'py2lcov' script, which translates python Coverage.py XML data
     to lcov format.

     See the Coverage.py documentation at https://coverage.readthedocs.io,
     as well as ".../py2lcov --help"

In general, the new features and options are implemented uniformly in lcov,
genhtml, and geninfo.  Most of the features can be enabled/disabled
using either command line options or by setting defaults in your 'lcovrc'
file.  See the lcovrc man page for details.


7. Questions and comments
-------------------------
See the included man pages for more information on how to use the LCOV tools.

In case of further questions, feel free to open a new issue using the issue
tracker on the LCOV code repository site at:

  https://github.com/linux-test-project/lcov