• Stars
    star
    368
  • Rank 115,958 (Top 3 %)
  • Language
    Python
  • License
    BSD 2-Clause "Sim...
  • Created over 11 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Django middleware and log filter to attach a unique ID to every log message generated as part of a request

django-log-request-id

Django middleware and log filter to attach a unique ID to every log message generated as part of a request.

Author: Jamie Matthews, @j4mie

Build Status

Example

DEBUG [33031a43fc244539895fef70c433337e] myproject.apps.myapp.views: Doing something in a view
DEBUG [33031a43fc244539895fef70c433337e] myproject.apps.myapp.forms: The form validated successfully!
DEBUG [33031a43fc244539895fef70c433337e] myproject.apps.myapp.models: Doing some model magic
DEBUG [33031a43fc244539895fef70c433337e] myproject.apps.myapp.views: Redirecting to form success page

Why?

So you can grep (or otherwise search) a set of logs for a high-traffic application to isolate all messages associated with a single request.

How?

The request ID is stored in a thread local. Use of thread locals is not generally considered best practice for Django applications, but seems to be the only viable approach in this case. Pull requests with better ideas are welcome.

Any other neat features?

In some cases, components further up the HTTP stack such as load balancers or proxies may generate request IDs. For example, Heroku's http-request-id feature adds a header to the request called X_REQUEST_ID. If such a header is present (and configured in your settings, see below), this ID will be used (instead of generating one). You can configure your settings to use a generated ID or return a default request_id when you expect the ID in the request header but it is not available.

The ID also gets added to the HttpRequest object that is handed to your views (as request.id), in case you need to use it in your application.

Installation and usage

First, install the package: pip install django-log-request-id

Add the middleware to your MIDDLEWARE_CLASSES setting. It should be at the very top.

MIDDLEWARE_CLASSES = (
    'log_request_id.middleware.RequestIDMiddleware',
    # ... other middleware goes here
)

Add the log_request_id.filters.RequestIDFilter to your LOGGING setting. You'll also need to update your formatters to include a format with the new request_id variable, add a handler to output the messages (eg to the console), and finally attach the handler to your application's logger.

If none of the above made sense, study Django's logging documentation.

An example LOGGING setting is below:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
        'request_id': {
            '()': 'log_request_id.filters.RequestIDFilter'
        }
    },
    'formatters': {
        'standard': {
            'format': '%(levelname)-8s [%(asctime)s] [%(request_id)s] %(name)s: %(message)s'
        },
    },
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'filters': ['request_id'],
            'formatter': 'standard',
        },
    },
    'loggers': {
        'myapp': {
            'handlers': ['console'],
            'level': 'DEBUG',
            'propagate': False,
        },
    }
}

You can then output log messages as usual:

import logging
logger = logging.getLogger(__name__)
logger.debug("A wild log message appears!")

If you wish to use an ID provided in a request header, add the following setting:

LOG_REQUEST_ID_HEADER = "HTTP_X_REQUEST_ID"

Setting this value as above will enable requests having the header X-Request-Id to be logged with the header value supplied.

Note that this value must conform to the format for Django META keys, otherwise it will be ignored.

If you wish to fall back to a generated ID when you have the LOG_REQUEST_ID_HEADER set but it was not provided in the request, add the following setting:

GENERATE_REQUEST_ID_IF_NOT_IN_HEADER = True

If you wish to include the request id in the response headers, add the following setting, where RESPONSE_HEADER_NAME is the name of the custom header you are going to use:

REQUEST_ID_RESPONSE_HEADER = "RESPONSE_HEADER_NAME"

If you wish to change the default request_id in the log output, the the following settings, where none (default) is the value you want to be the default value in case it's missing.

NO_REQUEST_ID = "none"

Logging all requests

The RequestIDMiddleware also has the ability to log all requests received by the application, including some useful information such as the user ID (if present). To enable this feature, add LOG_REQUESTS = True to your settings. The messages are sent to the log_request_id.middleware logger at INFO level.

Logging other user attributes

If you would like to log another user attribute instead of user ID, this can be specified with the LOG_USER_ATTRIBUTE setting. Eg. to log the username, use: LOG_USER_ATTRIBUTE = "username". If this setting is set to None, no user attribute will be logged.

License

Copyright © 2012, DabApps.

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Code of conduct

For guidelines regarding the code of conduct when contributing to this repository please review https://www.dabapps.com/open-source/code-of-conduct/

More Repositories

1

django-zen-queries

Explicit control over database query execution in Django applications
Python
366
star
2

django-email-as-username

DEPRECATED: User authentication with email addresses instead of usernames.
Python
261
star
3

django-readers

A lightweight function-oriented toolkit for better organisation of business logic and efficient selection and projection of data in Django projects.
Python
184
star
4

django-db-queue

Simple database-backed job queue
Python
160
star
5

django-user-roles

DEPRECATED: Simple role-based user permissions for Django.
Python
125
star
6

django-forms-dynamic

Resolve form field arguments dynamically when a form is instantiated
Python
117
star
7

django-user-streams

DEPRECATED: Simple, fast user news feeds for Django
Python
52
star
8

periodical

NO LONGER MAINTAINED A library for working with time and date series in Python
Python
47
star
9

django-reusable-app

DEPRECATED
Python
45
star
10

django-geosimple

DEPRECATED/NO LONGER MAINTAINED
Python
38
star
11

django-private-views

DEPRECATED: Site-wide login protection
Python
37
star
12

crab

🦀 a simple unix toolkit for working with local development environments.
Python
32
star
13

postcodeserver

A tiny JSON server for UK postcode lookups
Python
22
star
14

django-enforce-host

Middleware to redirect requests to a canonical host
Python
17
star
15

betta

Simple Bootstrap Variable Editor
JavaScript
16
star
16

django-rest-framework-serialization-spec

DEPRECATED, see https://github.com/dabapps/django-readers instead
Python
11
star
17

django-wrapwith

A Django template tag for wrapping a template block in a reusable enclosing template
Python
5
star
18

django-s3-file-upload-server

Upload files from the browser to S3 - server side implementation
Python
5
star
19

roe

DabApps' Project Development Kit
TypeScript
4
star
20

django-auth-base-template

NO LONGER MAINTAINED Switchable logged in/logged out base templates for Django
Python
4
star
21

csv-wrangler

Statically typed Python 3 CSV generation helpers
Python
3
star
22

django-s3-file-upload-client

Upload files from the browser to S3 - client side implementation
TypeScript
3
star
23

react-set-props

Store arbitrary state in redux with a similar API to setState
TypeScript
2
star
24

django-simple-robots

Simple robots.txt file serving for web applications
Python
2
star
25

django-topography

EXPERIMENTAL: Map the URL structure of a Django application
Python
1
star
26

dablab

DEPRECATED - DabApps Laboratories
HTML
1
star
27

heroku-buildpack-catfish

Compatibility shim to help build catfish-compatible repositories on Heroku
Shell
1
star
28

simple-records

Simple Readonly TypeScript Records
TypeScript
1
star
29

heroku-buildpack-cleanup

Cleanup many things from a Heroku slug after build
Shell
1
star
30

create-webpack-config

A utility for creating webpack configs with common settings
JavaScript
1
star
31

brightonbrains-2011

DEAD ☠
CSS
1
star
32

redux-create-reducer

A utility to create redux reducers from a set of handlers
TypeScript
1
star
33

heroku-buildpack-rds-certificate

Heroku buildpack to download the RDS SSL certificate and store it in the root of your app
Shell
1
star
34

django-readers-debug

A pretty-printer for debugging django-readers queryset functions
Python
1
star