• This repository has been archived on 19/Jul/2021
  • Stars
    star
    964
  • Rank 47,452 (Top 1.0 %)
  • Language
    Go
  • License
    MIT License
  • Created almost 11 years ago
  • Updated over 3 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

[ARCHIVED] Port of perl5 File::RotateLogs to Go

file-rotatelogs

Provide an io.Writer that periodically rotates log files from within the application. Port of File::RotateLogs from Perl to Go.

Build Status

GoDoc

WARNINGS

THIS PROJECT HAS BEEN ARCHIVED. IT WILL NOT RECEIVE UPDATES, THE AUTHOR DOES NOT WISH TO MAINTAIN OR SUPPORT IT. IN SHORT, DO NOT USE THIS PROJECT.

SYNOPSIS

import (
  "log"
  "net/http"

  apachelog "github.com/lestrrat-go/apache-logformat"
  rotatelogs "github.com/lestrrat-go/file-rotatelogs"
)

func main() {
  mux := http.NewServeMux()
  mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { ... })

  logf, err := rotatelogs.New(
    "/path/to/access_log.%Y%m%d%H%M",
    rotatelogs.WithLinkName("/path/to/access_log"),
    rotatelogs.WithMaxAge(24 * time.Hour),
    rotatelogs.WithRotationTime(time.Hour),
  )
  if err != nil {
    log.Printf("failed to create rotatelogs: %s", err)
    return
  }

  // Now you must write to logf. apache-logformat library can create
  // a http.Handler that only writes the approriate logs for the request
  // to the given handle
  http.ListenAndServe(":8080", apachelog.CombinedLog.Wrap(mux, logf))
}

DESCRIPTION

When you integrate this to into your app, it automatically write to logs that are rotated from within the app: No more disk-full alerts because you forgot to setup logrotate!

To install, simply issue a go get:

go get github.com/lestrrat-go/file-rotatelogs

It's normally expected that this library is used with some other logging service, such as the built-in log library, or loggers such as github.com/lestrrat-go/apache-logformat.

import(
  "log"
  "github.com/lestrrat-go/file-rotatelogs"
)

func main() {
  rl, _ := rotatelogs.New("/path/to/access_log.%Y%m%d%H%M")

  log.SetOutput(rl)

  /* elsewhere ... */
  log.Printf("Hello, World!")
}

OPTIONS

Pattern (Required)

The pattern used to generate actual log file names. You should use patterns using the strftime (3) format. For example:

  rotatelogs.New("/var/log/myapp/log.%Y%m%d")

Clock (default: rotatelogs.Local)

You may specify an object that implements the roatatelogs.Clock interface. When this option is supplied, it's used to determine the current time to base all of the calculations on. For example, if you want to base your calculations in UTC, you may specify rotatelogs.UTC

  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.WithClock(rotatelogs.UTC),
  )

Location

This is an alternative to the WithClock option. Instead of providing an explicit clock, you can provide a location for you times. We will create a Clock object that produces times in your specified location, and configure the rotatelog to respect it.

LinkName (default: "")

Path where a symlink for the actual log file is placed. This allows you to always check at the same location for log files even if the logs were rotated

  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.WithLinkName("/var/log/myapp/current"),
  )
  // Else where
  $ tail -f /var/log/myapp/current

Links that share the same parent directory with the main log path will get a special treatment: namely, linked paths will be RELATIVE to the main log file.

Main log file name Link name Linked path
/path/to/log.%Y%m%d /path/to/log log.YYYYMMDD
/path/to/log.%Y%m%d /path/to/nested/log ../log.YYYYMMDD
/path/to/log.%Y%m%d /foo/bar/baz/log /path/to/log.YYYYMMDD

If not provided, no link will be written.

RotationTime (default: 86400 sec)

Interval between file rotation. By default logs are rotated every 86400 seconds. Note: Remember to use time.Duration values.

  // Rotate every hour
  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.WithRotationTime(time.Hour),
  )

MaxAge (default: 7 days)

Time to wait until old logs are purged. By default no logs are purged, which certainly isn't what you want. Note: Remember to use time.Duration values.

  // Purge logs older than 1 hour
  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.WithMaxAge(time.Hour),
  )

RotationCount (default: -1)

The number of files should be kept. By default, this option is disabled.

Note: MaxAge should be disabled by specifing WithMaxAge(-1) explicitly.

  // Purge logs except latest 7 files
  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.WithMaxAge(-1),
    rotatelogs.WithRotationCount(7),
  )

Handler (default: nil)

Sets the event handler to receive event notifications from the RotateLogs object. Currently only supported event type is FiledRotated

  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.WithHandler(rotatelogs.HandlerFunc(func(e rotatelogs.Event) {
      if e.Type() != rotatelogs.FileRotatedEventType {
        return
      }

      // Do what you want with the data. This is just an idea:
      storeLogFileToRemoteStorage(e.(*rotatelogs.FileRotatedEvent).PreviousFile())
    })),
  )

ForceNewFile

Ensure a new file is created every time New() is called. If the base file name already exists, an implicit rotation is performed.

  rotatelogs.New(
    "/var/log/myapp/log.%Y%m%d",
    rotatelogs.ForceNewFile(),
  )

Rotating files forcefully

If you want to rotate files forcefully before the actual rotation time has reached, you may use the Rotate() method. This method forcefully rotates the logs, but if the generated file name clashes, then a numeric suffix is added so that the new file will forcefully appear on disk.

For example, suppose you had a pattern of '%Y.log' with a rotation time of 86400 so that it only gets rotated every year, but for whatever reason you wanted to rotate the logs now, you could install a signal handler to trigger this rotation:

rl := rotatelogs.New(...)

signal.Notify(ch, syscall.SIGHUP)

go func(ch chan os.Signal) {
  <-ch
  rl.Rotate()
}()

And you will get a log file name in like 2018.log.1, 2018.log.2, etc.

More Repositories

1

jwx

Implementation of various JWx (Javascript Object Signing and Encryption/JOSE) technologies
Go
1,908
star
2

libxml2

Interface to libxml2, with DOM interface
Go
230
star
3

server-starter

Go port of start_server utility (Server::Starter)
Go
215
star
4

backoff

Backoff mechanics for Go
Go
187
star
5

strftime

Fast strftime for Go
Go
110
star
6

test-mysqld

Create real MySQL server instance for testing
Go
88
star
7

slack

Slack client for go
Go
85
star
8

jsschema

JSON Schema for Go
Go
66
star
9

xslate

Powerful Template Engine for Go (port Perl5's Text::Xslate)
Go
65
star
10

jsval

Validator toolset, aimed to be used with JSON Schema
Go
54
star
11

apache-logformat

Port of Perl5's Apache::LogFormat::Compiler to golang
Go
48
star
12

sharaq

Image Transformer
Go
35
star
13

fluent-client

A fluentd client
Go
34
star
14

xmlsec

xmlsec1 binding for golang
Go
29
star
15

openapi

[WIP] OpenAPI for Go
Go
24
star
16

xstrings

Unicode-aware string utilities for Go
Go
22
star
17

jsref

JSON Reference Implementation for Go
Go
20
star
18

ngram

Ngram for golang
Go
19
star
19

cron

Dispatch jobs cron-style
Go
19
star
20

pdebug

Utilities for my print debugging fun. YMMV
Go
17
star
21

scripting

Handy toolset when using Go as a shell script replacement
Go
17
star
22

httprc

Quasi Up-to-date HTTP In-memory Cache
Go
16
star
23

msgpack

A `msgpack` serializer and deserializer
Go
16
star
24

tcputil

Some Utilities To Help Your TCP-Related testing
Go
16
star
25

hsup

Generate scaffold web app from JSON Hyper Schema files
Go
15
star
26

multifs

Create an fs.FS instance that "mounts" other fs.FS
Go
15
star
27

ical

Work with ical formatted data
Go
12
star
28

gcp-auto-lb-clean

Delete Dangling GCP Load Balancers Created By GKE
Go
12
star
29

channels

Channel patterns
Go
11
star
30

urlenc

Marshal/Unmarshal interface for structs that can encode/decode themselves to URL query strings
Go
11
star
31

tcptest

Start A Network Server On Random Local Port (Port of Perl5's TCP::Test)
Go
11
star
32

helium

(Work In Progress) An exercise rewriting libxml2 in Go
Go
10
star
33

naivebayes

Yet Another Naive-Bayesian filter algorithm
Go
9
star
34

echo-middleware-jwx

Echo (labstack/echo) middleware for using github.com/lestrrat-go/jwx
Go
9
star
35

option

Base option type
Go
9
star
36

gettext

Go
8
star
37

config

Libraries to aid configuration for golang applications
8
star
38

iter

Iterator tools
Go
8
star
39

jshschema

JSON Hyper Schema for Go
Go
8
star
40

lex

Simple lexer for Go
Go
8
star
41

jspointer

JSON pointer for Go
Go
8
star
42

fsnotify

fsnotify for Go
Go
7
star
43

sqllib

Maintain a library of prepared SQL statements (*sql.Stmt)
Go
6
star
44

structinfo

Tools to inspect Go structs
Go
6
star
45

rotating

A representation of a file that knows how to rotate itself based on size/interval
Go
6
star
46

blackmagic

Reflect-based black magic
Go
5
star
47

mux

A minimalistic HTTP Router for Go
Go
5
star
48

bufferpool

Very simple bytes.Buffer pool using sync.Pool
Go
5
star
49

pubsub

Simple broadcast pattern
Go
5
star
50

fear-of-go

4
star
51

sandbox

My tests, benchmarks
Go
4
star
52

tmplbox

Yet Another (text|html)/template wrapper
Go
4
star
53

packasset

Lightweight resource embedding utility for Go
Go
4
star
54

runcmd

Wrapper for "os/exec".Command
Go
4
star
55

sketch

Generate JSON (De)serializable Object From Go Schema
Go
4
star
56

merror

Error representing multiple errors
Go
4
star
57

envload

Restore and load environment variables
Go
3
star
58

lmdb

[WIP] Go Binding to LMDB
Go
3
star
59

strcursor

Tools for when you need to perform character-wise parsing
Go
3
star
60

httpcc

HTTP/1.1 Cache-Control Header Parser
Go
3
star
61

astv

An Alternate Go AST Visitor
Go
2
star
62

openscad

OpenSCAD Code Generator and Linter written in Go
Go
2
star
63

byteslice

Easy to handle `[]byte` type in your Go JSON structs
Go
2
star
64

codegen

Utilities for generating Go code
Go
2
star
65

dataurl

Parse/Encode RFC2397 "data" URL scheme
Go
2
star
66

contributions

2
star
67

rungroup

Control execution of multiple goroutines
Go
2
star
68

openscad-mcad

Port of openscad/mcad to github.com/lestrrat-go/openscad
Go
1
star
69

spanner-model

Go
1
star
70

rescue

Go
1
star
71

jsonptr

[WIP] RFC6901
Go
1
star
72

spanner-emulator-driver

Start/Stop Spanner Emulator from Go code
Go
1
star
73

gnap

Move along, there's nothing to see here yet
Go
1
star
74

jwx-aws-kms

Use KMS services as signer/verifier for github.com/lestrrat-go/jwx
Go
1
star
75

epd

[WIP] Port of ePaper HAT binding
Go
1
star
76

accesslog

HTTP middleware to log access logs based on `log/slog` for Go
Go
1
star
77

scriptor

Framework to construct a scripted sequence of actions, such as for testing
Go
1
star