• Stars
    star
    7,081
  • Rank 5,504 (Top 0.2 %)
  • Language
    Ruby
  • License
    MIT License
  • Created about 15 years ago
  • Updated 29 days ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

help to kill N+1 queries and unused eager loading

Bullet

Main workflow Gem Version AwesomeCode Status for flyerhzm/bullet Coderwall Endorse

The Bullet gem is designed to help you increase your application's performance by reducing the number of queries it makes. It will watch your queries while you develop your application and notify you when you should add eager loading (N+1 queries), when you're using eager loading that isn't necessary and when you should use counter cache.

Best practice is to use Bullet in development mode or custom mode (staging, profile, etc.). The last thing you want is your clients getting alerts about how lazy you are.

Bullet gem now supports activerecord >= 4.0 and mongoid >= 4.0.

If you use activerecord 2.x, please use bullet <= 4.5.0

If you use activerecord 3.x, please use bullet < 5.5.0

External Introduction

Install

You can install it as a gem:

gem install bullet

or add it into a Gemfile (Bundler):

gem 'bullet', group: 'development'

enable the Bullet gem with generate command

bundle exec rails g bullet:install

The generate command will auto generate the default configuration and may ask to include in the test environment as well. See below for custom configuration.

Note: make sure bullet gem is added after activerecord (rails) and mongoid.

Configuration

Bullet won't enable any notification systems unless you tell it to explicitly. Append to config/environments/development.rb initializer with the following code:

config.after_initialize do
  Bullet.enable = true
  Bullet.sentry = true
  Bullet.alert = true
  Bullet.bullet_logger = true
  Bullet.console = true
  Bullet.xmpp = { :account  => '[email protected]',
                  :password => 'bullets_password_for_jabber',
                  :receiver => '[email protected]',
                  :show_online_status => true }
  Bullet.rails_logger = true
  Bullet.honeybadger = true
  Bullet.bugsnag = true
  Bullet.appsignal = true
  Bullet.airbrake = true
  Bullet.rollbar = true
  Bullet.add_footer = true
  Bullet.skip_html_injection = false
  Bullet.stacktrace_includes = [ 'your_gem', 'your_middleware' ]
  Bullet.stacktrace_excludes = [ 'their_gem', 'their_middleware', ['my_file.rb', 'my_method'], ['my_file.rb', 16..20] ]
  Bullet.slack = { webhook_url: 'http://some.slack.url', channel: '#default', username: 'notifier' }
end

The notifier of Bullet is a wrap of uniform_notifier

The code above will enable all of the Bullet notification systems:

  • Bullet.enable: enable Bullet gem, otherwise do nothing
  • Bullet.alert: pop up a JavaScript alert in the browser
  • Bullet.bullet_logger: log to the Bullet log file (Rails.root/log/bullet.log)
  • Bullet.console: log warnings to your browser's console.log (Safari/Webkit browsers or Firefox w/Firebug installed)
  • Bullet.xmpp: send XMPP/Jabber notifications to the receiver indicated. Note that the code will currently not handle the adding of contacts, so you will need to make both accounts indicated know each other manually before you will receive any notifications. If you restart the development server frequently, the 'coming online' sound for the Bullet account may start to annoy - in this case set :show_online_status to false; you will still get notifications, but the Bullet account won't announce it's online status anymore.
  • Bullet.rails_logger: add warnings directly to the Rails log
  • Bullet.honeybadger: add notifications to Honeybadger
  • Bullet.bugsnag: add notifications to bugsnag
  • Bullet.airbrake: add notifications to airbrake
  • Bullet.appsignal: add notifications to AppSignal
  • Bullet.rollbar: add notifications to rollbar
  • Bullet.sentry: add notifications to sentry
  • Bullet.add_footer: adds the details in the bottom left corner of the page. Double click the footer or use close button to hide footer.
  • Bullet.skip_html_injection: prevents Bullet from injecting code into the returned HTML. This must be false for receiving alerts, showing the footer or console logging.
  • Bullet.skip_http_headers: don't add headers to API requests, and remove the javascript that relies on them. Note that this prevents bullet from logging warnings to the browser console or updating the footer.
  • Bullet.stacktrace_includes: include paths with any of these substrings in the stack trace, even if they are not in your main app
  • Bullet.stacktrace_excludes: ignore paths with any of these substrings in the stack trace, even if they are not in your main app. Each item can be a string (match substring), a regex, or an array where the first item is a path to match, and the second item is a line number, a Range of line numbers, or a (bare) method name, to exclude only particular lines in a file.
  • Bullet.slack: add notifications to slack
  • Bullet.raise: raise errors, useful for making your specs fail unless they have optimized queries
  • Bullet.always_append_html_body: always append the html body even if no notifications are present. Note: console or add_footer must also be true. Useful for Single Page Applications where the initial page load might not have any notifications present.
  • Bullet.skip_user_in_notification: exclude the OS user (whoami) from notifications.

Bullet also allows you to disable any of its detectors.

# Each of these settings defaults to true

# Detect N+1 queries
Bullet.n_plus_one_query_enable     = false

# Detect eager-loaded associations which are not used
Bullet.unused_eager_loading_enable = false

# Detect unnecessary COUNT queries which could be avoided
# with a counter_cache
Bullet.counter_cache_enable        = false

Note: When calling Bullet.enable, all other detectors are reset to their defaults (true) and need reconfiguring.

Safe list

Sometimes Bullet may notify you of query problems you don't care to fix, or which come from outside your code. You can add them to a safe list to ignore them:

Bullet.add_safelist :type => :n_plus_one_query, :class_name => "Post", :association => :comments
Bullet.add_safelist :type => :unused_eager_loading, :class_name => "Post", :association => :comments
Bullet.add_safelist :type => :counter_cache, :class_name => "Country", :association => :cities

If you want to skip bullet in some specific controller actions, you can do like

class ApplicationController < ActionController::Base
  around_action :skip_bullet, if: -> { defined?(Bullet) }

  def skip_bullet
    previous_value = Bullet.enable?
    Bullet.enable = false
    yield
  ensure
    Bullet.enable = previous_value
  end
end

Log

The Bullet log log/bullet.log will look something like this:

  • N+1 Query:
2009-08-25 20:40:17[INFO] USE eager loading detected:
  Post => [:comments]ยท
  Add to your query: .includes([:comments])
2009-08-25 20:40:17[INFO] Call stack
  /Users/richard/Downloads/test/app/views/posts/index.html.erb:8:in `each'
  /Users/richard/Downloads/test/app/controllers/posts_controller.rb:7:in `index'

The first log entry is a notification that N+1 queries have been encountered. The remaining entry is a stack trace so you can find exactly where the queries were invoked in your code, and fix them.

  • Unused eager loading:
2009-08-25 20:53:56[INFO] AVOID eager loading detected
  Post => [:comments]ยท
  Remove from your query: .includes([:comments])
2009-08-25 20:53:56[INFO] Call stack

These lines are notifications that unused eager loadings have been encountered.

  • Need counter cache:
2009-09-11 09:46:50[INFO] Need Counter Cache
  Post => [:comments]

XMPP/Jabber and Airbrake Support

see https://github.com/flyerhzm/uniform_notifier

Growl Support

Growl support is dropped from uniform_notifier 1.16.0, if you still want it, please use uniform_notifier 1.15.0.

Important

If you find Bullet does not work for you, please disable your browser's cache.

Advanced

Work with ActiveJob

Include Bullet::ActiveJob in your ApplicationJob.

class ApplicationJob < ActiveJob::Base
  include Bullet::ActiveJob if Rails.env.development?
end

Work with other background job solution

Use the Bullet.profile method.

class ApplicationJob < ActiveJob::Base
  around_perform do |_job, block|
    Bullet.profile do
      block.call
    end
  end
end

Work with sinatra

Configure and use Bullet::Rack.

configure :development do
  Bullet.enable = true
  Bullet.bullet_logger = true
  use Bullet::Rack
end

If your application generates a Content-Security-Policy via a separate middleware, ensure that Bullet::Rack is loaded before that middleware.

Run in tests

First you need to enable Bullet in test environment.

# config/environments/test.rb
config.after_initialize do
  Bullet.enable = true
  Bullet.bullet_logger = true
  Bullet.raise = true # raise an error if n+1 query occurs
end

Then wrap each test in Bullet api.

# spec/rails_helper.rb
if Bullet.enable?
  config.before(:each) do
    Bullet.start_request
  end

  config.after(:each) do
    Bullet.perform_out_of_channel_notifications if Bullet.notification?
    Bullet.end_request
  end
end

Debug Mode

Bullet outputs some details info, to enable debug mode, set BULLET_DEBUG=true env.

Contributors

https://github.com/flyerhzm/bullet/contributors

Demo

Bullet is designed to function as you browse through your application in development. To see it in action, you can follow these steps to create, detect, and fix example query problems.

1. Create an example application

$ rails new test_bullet
$ cd test_bullet
$ rails g scaffold post name:string
$ rails g scaffold comment name:string post_id:integer
$ bundle exec rails db:migrate

2. Change app/models/post.rb and app/models/comment.rb

class Post < ApplicationRecord
  has_many :comments
end

class Comment < ApplicationRecord
  belongs_to :post
end

3. Go to rails c and execute

post1 = Post.create(:name => 'first')
post2 = Post.create(:name => 'second')
post1.comments.create(:name => 'first')
post1.comments.create(:name => 'second')
post2.comments.create(:name => 'third')
post2.comments.create(:name => 'fourth')

4. Change the app/views/posts/index.html.erb to produce a N+1 query

<% @posts.each do |post| %>
  <tr>
    <td><%= post.name %></td>
    <td><%= post.comments.map(&:name) %></td>
    <td><%= link_to 'Show', post %></td>
    <td><%= link_to 'Edit', edit_post_path(post) %></td>
    <td><%= link_to 'Destroy', post, :confirm => 'Are you sure?', :method => :delete %></td>
  </tr>
<% end %>

5. Add the bullet gem to the Gemfile

gem "bullet"

And run

bundle install

6. enable the Bullet gem with generate command

bundle exec rails g bullet:install

7. Start the server

$ rails s

8. Visit http://localhost:3000/posts in browser, and you will see a popup alert box that says

The request has unused preload associations as follows:
None
The request has N+1 queries as follows:
model: Post => associations: [comment]

which means there is a N+1 query from the Post object to its Comment association.

In the meantime, there's a log appended into log/bullet.log file

2010-03-07 14:12:18[INFO] N+1 Query in /posts
  Post => [:comments]
  Add to your finder: :include => [:comments]
2010-03-07 14:12:18[INFO] N+1 Query method call stack
  /home/flyerhzm/Downloads/test_bullet/app/views/posts/index.html.erb:14:in `_render_template__600522146_80203160_0'
  /home/flyerhzm/Downloads/test_bullet/app/views/posts/index.html.erb:11:in `each'
  /home/flyerhzm/Downloads/test_bullet/app/views/posts/index.html.erb:11:in `_render_template__600522146_80203160_0'
  /home/flyerhzm/Downloads/test_bullet/app/controllers/posts_controller.rb:7:in `index'

The generated SQL is:

Post Load (1.0ms)   SELECT * FROM "posts"
Comment Load (0.4ms)   SELECT * FROM "comments" WHERE ("comments".post_id = 1)
Comment Load (0.3ms)   SELECT * FROM "comments" WHERE ("comments".post_id = 2)

9. To fix the N+1 query, change app/controllers/posts_controller.rb file

def index
  @posts = Post.includes(:comments)

  respond_to do |format|
    format.html # index.html.erb
    format.xml  { render :xml => @posts }
  end
end

10. Refresh http://localhost:3000/posts. Now there's no alert box and nothing new in the log.

The generated SQL is:

Post Load (0.5ms)   SELECT * FROM "posts"
Comment Load (0.5ms)   SELECT "comments".* FROM "comments" WHERE ("comments".post_id IN (1,2))

N+1 query fixed. Cool!

11. Now simulate unused eager loading. Change app/controllers/posts_controller.rb and app/views/posts/index.html.erb

def index
  @posts = Post.includes(:comments)

  respond_to do |format|
    format.html # index.html.erb
    format.xml  { render :xml => @posts }
  end
end
<% @posts.each do |post| %>
  <tr>
    <td><%= post.name %></td>
    <td><%= link_to 'Show', post %></td>
    <td><%= link_to 'Edit', edit_post_path(post) %></td>
    <td><%= link_to 'Destroy', post, :confirm => 'Are you sure?', :method => :delete %></td>
  </tr>
<% end %>

12. Refresh http://localhost:3000/posts, and you will see a popup alert box that says

The request has unused preload associations as follows:
model: Post => associations: [comment]
The request has N+1 queries as follows:
None

Meanwhile, there's a line appended to log/bullet.log

2009-08-25 21:13:22[INFO] Unused preload associations: PATH_INFO: /posts;    model: Post => associations: [comments]ยท
Remove from your finder: :include => [:comments]

13. Simulate counter_cache. Change app/controllers/posts_controller.rb and app/views/posts/index.html.erb

def index
  @posts = Post.all

  respond_to do |format|
    format.html # index.html.erb
    format.xml  { render :xml => @posts }
  end
end
<% @posts.each do |post| %>
  <tr>
    <td><%= post.name %></td>
    <td><%= post.comments.size %></td>
    <td><%= link_to 'Show', post %></td>
    <td><%= link_to 'Edit', edit_post_path(post) %></td>
    <td><%= link_to 'Destroy', post, :confirm => 'Are you sure?', :method => :delete %></td>
  </tr>
<% end %>

14. Refresh http://localhost:3000/posts, then you will see a popup alert box that says

Need counter cache
  Post => [:comments]

Meanwhile, there's a line appended to log/bullet.log

2009-09-11 10:07:10[INFO] Need Counter Cache
  Post => [:comments]

Copyright (c) 2009 - 2022 Richard Huang ([email protected]), released under the MIT license

More Repositories

1

rails_best_practices

a code metric tool for rails projects
Ruby
4,163
star
2

switch_user

Easily switch current user
Ruby
732
star
3

chinese_pinyin

translate chinese hanzi to pinyin
Ruby
430
star
4

activemerchant_patch_for_china

A rails plugin to add an active_merchant patch for china online payment platform including alipay (ๆ”ฏไป˜ๅฎ), 99bill (ๅฟซ้’ฑ) and tenpay (่ดขไป˜้€š)
Ruby
305
star
5

css_sprite

automatically css sprite
Ruby
242
star
6

uniform_notifier

uniform notifier for rails logger, customized logger, javascript alert, javascript console, growl and xmpp
Ruby
235
star
7

rails-bestpractices.com

HTML
200
star
8

redis-sentinel

another redis automatic master/slave failover solution for ruby by using built-in redis sentinel (deprecated)
Ruby
188
star
9

eager_group

fix n+1 aggregate sql functions for rails
Ruby
122
star
10

seo_checker

check your website if it is seo.
Ruby
117
star
11

simple_cacheable

a simple cache implementation for rails
Ruby
91
star
12

code_analyzer

code analyzer tool which is extracted from rails_best_practices
Ruby
86
star
13

resque-restriction

resque-restriction is an extension to resque queue system that restricts the execution number of certain jobs in a period time.
Ruby
86
star
14

rfetion

rfetion is a ruby gem for China Mobile fetion service that you can send SMS free.
Ruby
61
star
15

chinese_regions

provides all chinese regions, cities and districts
Ruby
60
star
16

mongoid-eager-loading

eager loading for mongoid (DEPRECATED)
Ruby
54
star
17

rails-brakeman.com

online security check for rails projects
Ruby
52
star
18

contactlist

java api to retrieve contact list of email(hotmail, gmail, yahoo, sohu, sina, 163, 126, tom, yeah, 189 and 139) and im(msn)
Java
49
star
19

regexp_crawler

A crawler which uses regular expression to catch data from website.
Ruby
45
star
20

chinese_permalink

This plugin adds a capability for AR model to create a seo permalink with your chinese text. It will translate your chinese text to english url based on google translate.
Ruby
41
star
21

apis-bench

Ruby
34
star
22

sitemap

This plugin will generate a sitemap.xml from sitemap.rb whose format is very similar to routes.rb
Ruby
32
star
23

twitter_connect

facebook connect style twitter oauth
Ruby
30
star
24

taobao

Ruby Client Library for Taobao Open Platform
Ruby
27
star
25

huangzhimin.com

my homepage
HTML
24
star
26

railsbp.com

railsbp.com
JavaScript
23
star
27

multiple_mailers

extend actionmailer to allow one smtp account per mailer class.
Ruby
23
star
28

contactlist-client

The contactlist-client gem is a ruby client to contactlist service which retrieves contact list of email(hotmail, gmail, yahoo, sohu, sina, 163, 126, tom, yeah, 189 and 139) and im(msn)
Ruby
20
star
29

donatecn

demo for activemerchant_patch_for_china
Ruby
17
star
30

monitor

Monitor gem can display ruby methods call stack on browser based on unroller
JavaScript
17
star
31

authlogic_renren_connect

Extension of the Authlogic library to add Renren Connect support built upon the renren plugin
Ruby
5
star
32

rails3-template

rails3 template includes a lot of useful plugins/gems
Ruby
5
star
33

nodeslide

node.js related slideshows [deprecated], move to nodeslide.heroku.com
JavaScript
4
star
34

rubyslide.com

collect ruby rails related presentations [deprecated], moved to rubyslide.heroku.com
Ruby
4
star
35

visual_partial

This plugin provides a way that you can see all the partial pages rendered. So it can prevent you from using partial page too much, which hurts the performance.
Ruby
4
star
36

codelinestatistics

The code line statistics takes files and directories from GUI, counts the total files, total sizes of files, total lines, lines of codes, lines of comments and lines of blanks in the files, displays the results and can also export results to html file.
Ruby
4
star
37

clock_chrome_extension

google chrome extension to display multiple clock analogs for multiple timezones
2
star
38

dotfiles

Vim Script
2
star
39

showoff-understanding-mongoid

My understanding mongoid showoff presentation
Ruby
2
star
40

enough_fields

only select specified fields used
Ruby
2
star
41

skype_archive

company hackathon
Ruby
1
star
42

bullet_test

Ruby
1
star
43

play_skype

JavaScript
1
star
44

test_code_analyzer

test code for code_analyzer gem
Ruby
1
star
45

blog.huangzhimin.com

HTML
1
star
46

test_error

raise an error to test if exception_notification really works.
Ruby
1
star
47

try-ripper

code mirror of try-ripper.heroku.com
CSS
1
star