• Stars
    star
    186
  • Rank 207,316 (Top 5 %)
  • Language
    Java
  • License
    Apache License 2.0
  • Created almost 7 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Logback Appender

Build Status License

中文版README

Aliyun Log Logback Appender

Logback is intended as a successor to the popular log4j project. You can control the destination of the log through logback. It can be console, file, GUI components, socket, NT event log, syslog. You can control the output format for each log as well. You can control the generation process of the log through log level. The most interesting thing is you can complete the above things through a configuration file and without any code modification.

You can set the destination of your log to AliCloud Log Service through Aliyun Log Logback Appender, The format of the log in AliCloud Log Service is as follows:

level: ERROR
location: com.aliyun.openservices.log.logback.example.LogbackAppenderExample.main(LogbackAppenderExample.java:18)
message: error log
throwable: java.lang.RuntimeException: xxx
thread: main
time: 2018-01-02T03:15+0000
log: 2018-01-02 11:15:29,682 ERROR [main] com.aliyun.openservices.log.logback.example.LogbackAppenderExample: error log
__source__: xxx
__topic__: yyy

Field Specifications:

  • level stands for log level
  • location is logs's output position
  • message is the content of the log
  • throwable is exception of the log (this field will appear only if the exception is recorded)
  • thread stands for thread name
  • time is the log's generation time (you can configure it's format through timeFormat and timeZone)
  • log is custom log format (this field will appear only if you configure the encoder)
  • __source__ is the log's source, you can specify its value in conf file
  • __topic__ is the log's topic, you can specify its value in conf file

Advantage

  • Disk Free: the generation data will be send to AliCloud Log Service in real time through network.
  • Without Refactor: if your application already use logback, you can just add logback appender to your configuration file.
  • Asynchronous and High Throughput: the data will be send to AliCloud Log Service asynchronously. It is suitable for high concurrent write.
  • Context Query: at server side, in addition to searching log with keywords, you can obtain the context information of original log as well.

Supported Version

  • logback 1.2.3
  • aliyun-log-producer 0.3.8
  • protobuf-java 2.5.0

Configuration Steps

1. Adding the Dependencies in pom.xml

<dependency>
    <groupId>com.google.protobuf</groupId>
    <artifactId>protobuf-java</artifactId>
    <version>2.5.0</version>
</dependency>
<dependency>
    <groupId>com.aliyun.openservices</groupId>
    <artifactId>aliyun-log-logback-appender</artifactId>
    <version>0.1.18</version>
</dependency>

2. Modify the Configuration File

Take logback.xml as an example, you can configure the appender and logger related to AliCloud Log Services as follows:

  <!-- To prevent data loss when the process exits, please remember to add this configuration -->
  <shutdownHook class="ch.qos.logback.core.hook.DelayingShutdownHook"/>

  <appender name="aliyun" class="com.aliyun.openservices.log.logback.LoghubAppender">
    <!-- Required parameters -->
    <!-- Configure account and network  -->
    <endpoint>your project endpoint</endpoint>
    <accessKeyId>your accesskey id</accessKeyId>
    <accessKeySecret>your accesskey</accessKeySecret>

    <!-- Configure sls -->
    <project>your project</project>
    <logStore>your logStore</logStore>
    <!-- Required parameters(end) -->

    <!-- Optional parameters -->
    <topic>your topic</topic>
    <source>your source</source>

    <!-- Optional parameters -->
    <totalSizeInBytes>104857600</totalSizeInBytes>
    <maxBlockMs>0</maxBlockMs>
    <ioThreadCount>8</ioThreadCount>
    <batchSizeThresholdInBytes>524288</batchSizeThresholdInBytes>
    <batchCountThreshold>4096</batchCountThreshold>
    <lingerMs>2000</lingerMs>
    <retries>10</retries>
    <baseRetryBackoffMs>100</baseRetryBackoffMs>
    <maxRetryBackoffMs>50000</maxRetryBackoffMs>
    
    <!-- Optional parameters -->
    <encoder>
        <pattern>%d %-5level [%thread] %logger{0}: %msg</pattern>
    </encoder>
    
    <!--  Optional parameters -->
    <timeFormat>yyyy-MM-dd'T'HH:mmZ</timeFormat>
    <!--  Optional parameters -->
    <timeZone>UTC</timeZone>
  </appender>

  <!-- This listener will print the status in StatusManager to console
  <statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener"/>
  -->

Note

  • To prevent data loss when the process exits, please remember to add label DelayingShutdownHook.
  • The LoghubAppender will catch the exceptions in the process of running and put them into BasicStatusManager, you can obtain the exception information through OnConsoleStatusListener or other means. Reference: https://logback.qos.ch/access.html

Parameter Description

The Aliyun Log Logback Appender provides following parameters.

# Specify the project name of your log services, required
project = [your project]
# Specify the logstore of your log services, required
logStore = [your logStore]
# Specify the HTTP endpoint of your log services, required
endpoint = [your project endpoint]
# Specify the account information of your log services, required
accessKeyId = [your accesskey id]
accessKeySecret = [your accessKeySecret]

# The upper limit log size that a single producer instance can hold, default is 100MB.
totalSizeInBytes=104857600
# If the producer has insufficient free space, the caller's maximum blocking time on the send method, defaults is 60 seconds. In order not to block the log printing thread, it is strongly recommended to set this value to 0.
maxBlockMs=0
# The thread pool size for executing log sending tasks, defaults is the number of processors available.
ioThreadCount=8
# When the size of the cached log in a Producer Batch is greater than or equal batchSizeThresholdInBytes, the batch will be send, default is 512KB, maximum can be set to 5MB.
batchSizeThresholdInBytes=524288
# When the number of log entries cached in a ProducerBatch is greater than or equal to batchCountThreshold, the batch will be send.
batchCountThreshold=4096
# A ProducerBatch has a residence time from creation to sending, defaulting is 2 seconds and a minimum of 100 milliseconds.
lingerMs=2000
# The number of times a Producer Batch can be retried if it fails to send for the first time, default is 10.
retries=10
# The backoff time for the first retry, default 100 milliseconds.
baseRetryBackoffMs=100
# The maximum backoff time for retries, default is 50 seconds.
maxRetryBackoffMs=50000

# Specify the topic of your log, default is "", optional
topic = [your topic]

# Specify the source of your log, default is host ip, optional
source = [your source]

# Specify time format of the field time, default is yyyy-MM-dd'T'HH:mmZ, optional
timeFormat = yyyy-MM-dd'T'HH:mmZ

# Specify timezone of the field time, default is UTC, optional
timeZone = UTC

Sample Code

LogbackAppenderExample.java

logback-example.xml

Contributors

@lionbule @zzboy made a great contribution to this project.

Thanks for the excellent work by @lionbule @zzboy.

More Repositories

1

oss-browser

OSS Browser 提供类似windows资源管理器功能。用户可以很方便的浏览文件,上传下载文件,支持断点续传等。
JavaScript
3,175
star
2

aliyun-openapi-java-sdk

Alibaba Cloud SDK for Java
Java
1,379
star
3

aliyun-oss-java-sdk

Aliyun OSS SDK for Java
Java
1,216
star
4

alibaba-cloud-sdk-go

Alibaba Cloud SDK for Go
Go
1,104
star
5

alicloud-android-demo

Java
990
star
6

aliyun-openapi-python-sdk

Alibaba Cloud SDK for Python
Python
980
star
7

aliyun-oss-php-sdk

Aliyun OSS SDK for PHP
PHP
975
star
8

aliyun-oss-go-sdk

Aliyun OSS SDK for Go
Go
951
star
9

aliyun-oss-python-sdk

Aliyun OSS SDK for Python
Python
935
star
10

darabonba

Darabonba 是一种用于 OpenAPI 的 DSL 语言,可以用来生成多语言的 SDK、Code Sample、Test Case 等代码
JavaScript
894
star
11

alibabacloud-alfa

阿里云微前端解决方案
TypeScript
845
star
12

aliyun-oss-android-sdk

Android SDK for aliyun object storage service
Java
793
star
13

aliyun-cli

Alibaba Cloud CLI
Go
770
star
14

ossfs

Export s3fs for aliyun oss.
C++
735
star
15

aliyun-openapi-php-sdk

[Abandoned] Open API SDK for PHP developers
PHP
605
star
16

terraform-provider-alicloud

Terraform AliCloud provider
Go
590
star
17

aliyun-openapi-net-sdk

Alibaba Cloud SDK for .NET
C#
535
star
18

rds_dbsync

围绕 PostgreSQL Greenplum ,实现易用的数据的互迁功能项目
C
528
star
19

openapi-sdk-php

Alibaba Cloud SDK for PHP
PHP
501
star
20

iotkit-embedded

高速镜像: https://code.aliyun.com/linkkit/c-sdk
C
492
star
21

ossutil

A user friendly command line tool to access AliCloud OSS.
Go
456
star
22

aliyun-oss-ios-sdk

iOS SDK for aliyun object storage service
Objective-C
450
star
23

aliyun-odps-python-sdk

ODPS Python SDK and data analysis framework
Python
433
star
24

alicloud-ios-demo

Demos for AMS iOS SDKs
Objective-C
431
star
25

alibabacloud-microservice-demo

An Alibaba Cloud native microservice demo powered by Apache Dubbo and Spring Cloud Alibaba
Java
379
star
26

api-gateway-demo-sign-java

aliyun api gateway request signature demo by java
Java
371
star
27

NeWCRFs

Python
365
star
28

aliyun-oss-csharp-sdk

Aliyun OSS SDK for C#
C#
360
star
29

surftrace

surftrace is a tool that allows you to surf the linux kernel
Python
332
star
30

conditional-lane-detection

Python
328
star
31

aliyun-log-jaeger

Go
294
star
32

coolbpf

C
240
star
33

tablestore-examples

Example code for aliyun tablestore.
Java
238
star
34

tablestore-timeline

TableStore-Timeline Model for Social scene
Java
236
star
35

aliyun-log-c-sdk

Aliyun LOG Producer for C/C++
C
215
star
36

openapi-sdk-php-client

Official repository of the Alibaba Cloud Client for PHP
PHP
214
star
37

aliyun-log-android-sdk

Java
179
star
38

alibabacloud-jindodata

alibabacloud-jindodata
176
star
39

openapi-core-nodejs-sdk

OpenAPI POP core SDK for Node.js
JavaScript
175
star
40

aliyun-emapreduce-datasources

Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.
Scala
168
star
41

aliyun-log-python-sdk

Use python to manage, produce and consume data with Aliyun Log Service.
Python
166
star
42

data-mapping-component

A React Component which focus on Data-Mapping & Table-Field-Mapping.(基于React的数据/表字段映射组件)
JavaScript
155
star
43

aliyun-oss-react-native

Objective-C
148
star
44

aliyun-apsaradb-hbase-demo

C++
146
star
45

django-oss-storage

Django storage backends for AliCloud OSS.
Python
144
star
46

aliyun-oss-c-sdk

Aliyun OSS SDK for C
C
144
star
47

react-visual-modeling

A DAG React Component for visualization modeling, suitable for UML, database modeling, data warehouse construction.(一个基于React的数据可视化建模的DAG图,适用于UML,数据库建模,数据仓库建设等业务)
JavaScript
138
star
48

aliyun-oss-ruby-sdk

Aliyun OSS SDK for Ruby
Ruby
138
star
49

ram-policy-editor

AliCloud RAM Policy Editor for OSS
JavaScript
136
star
50

aliyun-log-java-sdk

Java
135
star
51

serverless-aliyun-function-compute

Serverless Alibaba Cloud Function Compute Plugin – Add Alibaba Cloud Function Compute support to the Serverless Framework
JavaScript
134
star
52

alibabacloud-console-components

阿里云企业云管理平台 UI 组件库
TypeScript
133
star
53

aliyun-log-java-producer

Aliyun LOG Java Producer
Java
131
star
54

fc-nodejs-sdk

The Node.js SDK of FunctionCompute.
JavaScript
130
star
55

aliyun-cms-grafana

JavaScript
127
star
56

aliyun-odps-jdbc

JDBC Driver for ODPS
Java
125
star
57

alibabacloud-quantization-networks

alibabacloud-quantization-networks
Python
122
star
58

aliyun-emapreduce-demo

Java
121
star
59

aliyun-maxcompute-data-collectors

Java
119
star
60

alicloud-ams-demo

C#
117
star
61

alibabacloud-iot-device-sdk

alibaba cloud for iot device javascript SDK , connect with linkplatform , run at node/broswer/winxin min program /ali min program. 阿里云IoT物联网平台javascript版本sdk,可以运行在node/broswer/winxin min program /ali min program. 阿里云IoT物联网平台javascript版本sdk,可以运行在node/broswer/winxin min program /ali min program
JavaScript
110
star
62

MaxCompute-Spark

MaxCompute spark demo for building a runnable application.
Scala
106
star
63

api-gateway-nodejs-sdk

The API Gateway SDK for Node.js
JavaScript
104
star
64

cloud-design

阿里云前端组件库,由专有云&公有云前端团队共建
CSS
99
star
65

gm-jsse

开源国密通信纯 Java JSSE 实现
Java
95
star
66

aliyun-odps-console

ODPS Console Source Code.
Java
93
star
67

aliyun-openapi-cpp-sdk

Alibaba Cloud SDK for C++
C++
90
star
68

aliyun-odps-java-sdk

ODPS SDK for Java Developers
Java
89
star
69

aliyun-tablestore-nodejs-sdk

Aliyun TableStore(原OTS) SDK for Node.js
JavaScript
88
star
70

algorithm-base

让算法工程化更简单
Python
86
star
71

aliyun-log-ios-sdk

Aliyun LOG iOS SDK
Swift
84
star
72

iotx-api-demo

PHP
82
star
73

plugsched

Live upgrade Linux kernel scheduler subsystem
Python
82
star
74

DCT-Mask

Python
81
star
75

aliyun-openapi-nodejs-sdk

Alibaba Cloud SDK for Node.js
JavaScript
80
star
76

aliyun-specs

Aliyun Mobile Service CocoaPods specs.
Ruby
77
star
77

alibabacloud-console-design

阿里云管平台研发解决方案
TypeScript
77
star
78

alibabacloud-redis-training-demo

Java
76
star
79

aliyun-oss-php-sdk-laravel

A Laravel service provider for the AliCloud OSS SDK for PHP
PHP
75
star
80

aliyun-tablestore-go-sdk

TableStore SDK for Golang
Go
75
star
81

alibabacloud-sdk

Tea
75
star
82

fc-docker

Dockerfiles for local building or running function of FC
Dockerfile
74
star
83

elasticsearch-repository-oss

Java
74
star
84

dro-sfm

Python
74
star
85

packagist-mirror

Alibaba Cloud Packagist Mirror
Go
73
star
86

react-monitor-dag

A React-based operation/monitoring DAG diagram.(基于React的运维/监控DAG图)
JavaScript
69
star
87

aliyun_assist_client

Aliyun Assist Client 阿里云 云助手
Go
67
star
88

aliyun-log-php-sdk

PHP
67
star
89

alibabacloud-hologres-connectors

alibabacloud-hologres-connectors
Java
66
star
90

aliyun-tsdb-java-sdk

Aliyun TSDB SDK for Java
Java
64
star
91

aliyun-log-log4j-appender

aliyun-log-log4j-appender
Java
63
star
92

fc-java-sdk

The Java SDK of FunctionCompute.
Java
61
star
93

oss-ftp

The ftp proxy for Aliyun OSS.
Python
61
star
94

react-lineage-dag

JavaScript
61
star
95

aliyun-log-cli

Command Line Interface for Aliyun Log Service
Python
60
star
96

csb-sdk

The CSB-SDK is a client-side invocation SDK for HTTP or Web Service API opened by the CSB (Cloud Service Bus) product. It is responsible for invoking the open API and signing the request information.
Java
58
star
97

ossimport

Data migration tool
58
star
98

aliyun-log-flink-connector

flink log connector
Java
58
star
99

oss-emulator

OSS Emulator
Ruby
58
star
100

aliyun-log-dotnetcore-sdk

C#
55
star