• Stars
    star
    220
  • Rank 180,422 (Top 4 %)
  • Language
    Java
  • License
    Apache License 2.0
  • Created over 10 years ago
  • Updated 8 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A sub microsecond java logger, supporting standard logging APIs such as Slf & Log4J

Chronicle Logger

About

Version

badge

Overview

Today most programs require the logging of large amounts of data, especially in trading systems where this is a regulatory requirement. Loggers can affect your system performance, therefore logging is sometimes kept to a minimum. With Chronicle Logger we aim to minimize logging overhead, freeing your system to focus on the business logic.

Chronicle logger supports most of the standard logging API’s including:

Chronicle Logger is able to aggregate all your logs to a central store. It has built-in resilience, so you will never lose messages.

At the moment, Chronicle Logger only supports binary logs, which is beneficial for write speed but requires extra tools to read them. We provide some basic #tools[tools] for that and an API to develop your own.

Note
chronicle-logger-log4j-2 does not suffer from the log4shell vulnerability as it does not reuse the Log4J2 message formatting machinery

How it works

Chronicle Logger is built on top of Chronicle Queue. It provides multiple logging frameworks' adapters and is a low latency, high throughput synchronous writer. Unlike asynchronous writers, you will always see the last message before the application dies, as usually it is the last message that is the most valuable.

Performance

We have run a benchmark to compare Chronicle Logger with normal file appender of Log4J2 (the quickest of mainstream logging frameworks). Results below:

Benchmark Mode Samples Score Score error Units

Chronicle Logger, simple message

avgt

5

784.761

68.018

ns/op

Chronicle Logger, message with Exception

avgt

5

12801.245

417.695

ns/op

Log4J2, simple message

avgt

5

2427.177

454.057

ns/op

Log4J2, message with Exception

avgt

5

17173.369

3193.413

ns/op

Test Hardware:

Intel Core i7-6700K
32GB DDR4 RAM
512GB M.2 PCI-e 3.0 x 4 NVMe SSD

Bindings

All config files for bindings support limited variable interpolation where the variables are replaced with the corresponding values from the same configuration file or the system properties. We have one predefined variable, pid, so ${pid} will replaced by current process id. System properties have the precedence in placeholder replacement so they can be overriden.

The following can be configured for each logger:

Property Description Values Per-Logger

path

the base directory of a Chronicle

yes

level

default log level

trace, debug, info, warn, error

yes

Additionally, underlying Chronicle Queue can be tweaked by providing the following optional config properties:

  • bufferCapacity

  • blockSize

  • rollCycle

If set, these will override the default Chronicle Queue configuration. Use with caution!

Please Note

  • Loggers are not hierarchically grouped so my.domain.package.MyClass1 and my.domain are two distinct entities.

  • The path is used to track the underlying Chronicle Queue so having two loggers configured with the same path is unsupported

chronicle-logger-slf4j

The chronicle-logger-slf4j is an implementation of SLF4J API > 1.7.x.

To configure this sl4j binding you need to specify the location of a properties files (file-system or classpath) via system properties:

-Dchronicle.logger.properties=${pathToYourPropertiesFile}

Alternatively, you could use one of the default locations: chronicle-logger.properties or config/chronicle-logger.properties located in the classpath.

The default configuration is build using properties with chronicle.logger.root as prefix but you can also set per-logger settings i.e. chronicle.logger.L1.*

Config Example

# shared properties
chronicle.base                        = ${java.io.tmpdir}/chronicle-logs/${pid}

# logger : default
chronicle.logger.root.path            = ${slf4j.chronicle.base}/main
chronicle.logger.root.level           = debug

# optional tweaks
chronicle.logger.root.cfg.bufferCapacity = 128
chronicle.logger.root.cfg.blockSize      = 256

# logger : L1
chronicle.logger.L1.path              = ${slf4j.chronicle.base}/L1
chronicle.logger.L1.level             = info

chronicle-logger-logback

The chronicle-logger-logback module provides appender for Logback: net.openhft.chronicle.logger.logback.ChronicleAppender

Config Example

<appender name  = "ChronicleAppender"
        class = "net.openhft.chronicle.logger.logback.ChronicleAppender">

  <!-- Path used by the underlying ChronicleQueue -->
  <path>${java.io.tmpdir}/ChronicleAppender</path>

  <!--
  Configure the underlying ChronicleQueue tweaks
  -->
  <chronicleConfig>
      <blockSize>128</blockSize>
  </chronicleConfig>
</appender>

chronicle-logger-log4j-1

We provide log4j1 appender net.openhft.chronicle.logger.log4j1.ChronicleAppender

Config Example

<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>

    <!-- ******************************************************************* -->
    <!--                                                                     -->
    <!-- ******************************************************************* -->

    <appender name  = "CHRONICLE"
              class = "net.openhft.chronicle.logger.log4j1.ChronicleAppender">
        <param name="path" value="${java.io.tmpdir}/chronicle-log4j1/chronicle"/>
    </appender>

    <!-- ******************************************************************* -->
    <!-- STDOUT                                                              -->
    <!-- ******************************************************************* -->

    <appender name  = "STDOUT"
              class = "org.apache.log4j.ConsoleAppender">
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%-4r [%t] %-5p %c %x - %m%n" />
        </layout>
    </appender>

    <!-- ******************************************************************* -->
    <!--                                                                     -->
    <!-- ******************************************************************* -->

    <logger name="chronicle" additivity="false">
        <level value="trace"/>
        <appender-ref ref="CHRONICLE"/>
    </logger>

    <!-- ******************************************************************* -->
    <!--                                                                     -->
    <!-- ******************************************************************* -->

    <logger name="net.openhft" additivity="false">
        <level value="warn"/>
        <appender-ref ref="STDOUT"/>
    </logger>

    <!-- ******************************************************************* -->
    <!--                                                                     -->
    <!-- ******************************************************************* -->

    <root>
        <level value="debug" />
        <appender-ref ref="STDOUT" />
    </root>

</log4j:configuration>

chronicle-logger-log4j-2

Use <Chronicle/> element in <appenders/> to create Chronicle appender. Optional <chronicleCfg/> element can be used to tweak underlying Chronicle Queue.

Config Example

<?xml version="1.0" encoding="UTF-8"?>
<configuration packages="net.openhft.chronicle.logger,net.openhft.chronicle.logger.log4j2">

    <!-- ******************************************************************* -->
    <!-- APPENDERS                                                           -->
    <!-- ******************************************************************* -->

    <appenders>

        <Console name="STDOUT" target="SYSTEM_OUT">
            <PatternLayout pattern="[CHRONOLOGY] [%-5p] %c - %m%n%throwable{none}"/>
        </Console>

        <Chronicle name="CHRONICLE">
            <path>${sys:java.io.tmpdir}/chronicle-log4j2/binary-chronicle</path>
            <chronicleCfg>
                <blockSize>128</blockSize>
                <bufferCapacity>256</bufferCapacity>
            </chronicleCfg>
        </Chronicle>

    </appenders>

    <!-- ******************************************************************* -->
    <!-- LOGGERS                                                             -->
    <!-- ******************************************************************* -->

    <loggers>

        <root level="all">
            <appender-ref ref="STDOUT"/>
        </root>

        <logger name="chronicle" level="trace" additivity="false">
            <appender-ref ref="CHRONICLE"/>
        </logger>

        <!-- *************************************************************** -->
        <!--                                                                 -->
        <!-- *************************************************************** -->

        <logger name="net.openhft" level="warn"/>

    </loggers>

</configuration>

chronicle-logger-jul

Use net.openhft.chronicle.logger.jul.ChronicleHandler as a handler

Config Example

handlers=java.util.logging.ConsoleHandler, net.openhft.chronicle.logger.jul.ChronicleHandler

.level=ALL

java.util.logging.ConsoleHandler.level=ALL
java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter

net.openhft.level=WARNING
net.openhft.handlers=java.util.logging.ConsoleHandler

net.openhft.chronicle.logger.jul.ChronicleHandler.path = ${java.io.tmpdir}/chronicle-jul
net.openhft.chronicle.logger.jul.ChronicleHandler.level = ALL

chronicle.level=INFO
chronicle.handlers=net.openhft.chronicle.logger.jul.ChronicleHandler
chronicle.useParentHandlers=false

chronicle-logger-jcl

Similar to slf4j, to configure this binding you need to specify the location of a properties files (file-system or classpath) via system properties:

-Dchronicle.logger.properties=${pathToYourPropertiesFile}

Alternatively, you could use one of the default locations: chronicle-logger.properties or config/chronicle-logger.properties located in the classpath.

Config Example

chronicle.logger.base             = ${java.io.tmpdir}/chronicle-jcl
chronicle.logger.root.path        = ${chronicle.logger.base}/root
chronicle.logger.root.level       = debug

# logger : Logger1
chronicle.logger.logger_1.path    = ${chronicle.logger.base}/logger_1
chronicle.logger.logger_1.level   = info

Tools

  • net.openhft.chronicle.logger.tools.ChroniCat - tool to dump log contents to STDOUT

    ---
    ChroniCat [-w <wireType>] <path>
        <wireType> - wire format, default BINARY_LIGHT
        <path>     - base path of Chronicle Logs storage

mvn exec:java -Dexec.mainClass="net.openhft.chronicle.logger.tools.ChroniCat" -Dexec.args="…​" ---

  • net.openhft.chronicle.logger.tools.ChroniTail - same as ChroniCat but waits for more data, similar to *nix tail utility

ChroniTail [-w <wireType>] <path>
    <wireType> - wire format, default BINARY_LIGHT
    <path>     - base path of Chronicle Logs storage

mvn exec:java -Dexec.mainClass="net.openhft.chronicle.logger.tools.ChroniTail" -Dexec.args="..."
  • We also provide generic interface to interact with logs, net.openhft.chronicle.logger.tools.ChronicleLogReader, allowing arbitrary operations with decoded log lines. Please refer to javadocs.

More Repositories

1

Chronicle-Queue

Micro second messaging that stores everything to disk
Java
3,130
star
2

Chronicle-Map

Replicate your Key Value Store across your network, with consistency, persistance and performance.
Java
2,669
star
3

Java-Thread-Affinity

Bind a java thread to a given core
Java
1,724
star
4

Zero-Allocation-Hashing

Zero-allocation hashing for Java
Java
762
star
5

Java-Runtime-Compiler

Java Runtime Compiler
Java
617
star
6

OpenHFT

Parent module to include active modules
Shell
609
star
7

Chronicle-Core

Low level access to native memory, JVM and OS.
Java
528
star
8

Chronicle-Wire

A Low Garbage Java Serialisation Library that supports multiple formats
Java
464
star
9

Chronicle-Bytes

Chronicle Bytes has a similar purpose to Java NIO's ByteBuffer with many extensions
Java
382
star
10

Chronicle-Engine

A high performance, low latency, reactive processing framework
338
star
11

Java-Lang

Java Language support
Java
286
star
12

Chronicle-Network

A High Performance Network ( TCP/IP ) Library
Java
243
star
13

Chronicle-Threads

Java
167
star
14

Chronicle-Values

Java
102
star
15

Chronicle-Algorithms

Java
77
star
16

JLBH

JLBH
Java
68
star
17

Chronicle-Queue-Demo

Sample programs for Chronicle Queue
Java
65
star
18

Chronicle-Accelerate

HFT meets Blockchain in Java platform
Java
59
star
19

Chronicle-TimeSeries

Multi-Threaded Time Series library
Java
59
star
20

Chronicle-Decentred

Framework for building Secure Scalable Microservices on Distributed Ledger Technology
Java
50
star
21

Chronicle-Salt

Chronicle wrapper for the NaCl library
Java
30
star
22

Chronicle-Test-Framework

Java
24
star
23

Chronicle-Ticker

A time ticker as a light weight clock
Java
23
star
24

RFC

RFC's used by OpenHFT
21
star
25

Chronicle-Websocket-Jetty

Provide Websocket access via Jetty
Java
17
star
26

Puzzles

OpenHFT Puzzles
Java
15
star
27

jvm-micro-benchmarks

Microbenchmarks for JVM code.
Java
14
star
28

Spoon

Java
11
star
29

Chronicle-Engine-GUI

CSS
11
star
30

Exception-Handler

Open a browser with a message about an Exception.
Java
10
star
31

Stage-Compiler

Java
8
star
32

Binary-Compatibility-Enforcer-Plugin

Wraps the Java API ComplianceChecker into a maven plugin
Java
7
star
33

Chronicle-Common

Java
6
star
34

Chronicle-Coder

Encode and decode data as symbols and readable words
Java
5
star
35

Posix

Java
5
star
36

Microservice-Benchmark

Open Microservices Benchmark
Java
5
star
37

Chronicle-Analytics

Support for Analytics
Java
4
star