• Stars
    star
    136
  • Rank 267,670 (Top 6 %)
  • Language
    Ruby
  • Created almost 14 years ago
  • Updated about 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Cache based on arguments AND object state; store in memcached, redis, or in-process. Like alias_method, but it's cache_method! One step beyond memoization.

cache_method

It's like alias_method, but it's cache_method!

Lets you cache the results of calling methods given their arguments. Like memoization, but stored in Memcached, Redis, etc. so that the cached results can be shared between processes and hosts.

Real-world usage

Brighter Planet logo

We use cache_method for data science at Brighter Planet and in production at

Rationale

  • It should be easy to cache instance methods
  • It should be easy to cache methods that depend on object state
  • It should be easy to uncache a method without clearing the whole cache
  • It should be easy to do all that using a default in-process store, memcached, dalli (if you're on heroku), redis, etc. (all supported by this gem through the cache gem)

Example

require 'cache_method'
class Blog

  attr_reader :name, :url

  def initialize(name, url)
    @name = name
    @url = url
  end

  def entries(date)
    # ...
  end

  # cache that slow method!
  cache_method :entries

  def update(stuff)
    # ...
  end

  # automatically clear cache for #entries when #update is called...
  cache_method_clear_on :update, :entries

  # custom cache key - not always required!
  def as_cache_key
    { :name => name, :url => url }
  end
end

Then you can do

my_blog.entries(Date.today) => first time won't be cached
my_blog.entries(Date.today) => second time will come from cache

And clear them too

my_blog.cache_method_clear :entries

(which doesn't delete the rest of your cache)

Configuration (and supported cache clients)

By default, an in-process, non-shared cache is used.

You can set where the cache will be stored:

CacheMethod.config.storage = Memcached.new '127.0.0.1:11211'

or

CacheMethod.config.storage = Redis.new

or this might even work...

CacheMethod.config.storage = Rails.cache

See Config for the full list of supported caches.

Cache keys

Caching a method involves getting cache keys for

  1. the object where the method is defined - for example, my_blog.as_cache_key
  2. the arguments passed to the method - for example, Marshal.dump(Date.today), because Date#as_cache_key is not defined

Then the cache keys are SHA-1 hashed and combined for an overall key:

method signature + obj digest                                                                           + args digest
Blog#entries     + SHA1(Marshal.dump({:name="Seamus's blog",:url=>"http://numbers.brighterplanet.com"}) + SHA1(Marshal.dump(Date.today))

Technically the full cache key is

# when caching class methods
method signature + generation + args digest
# when caching instance methods
method signature + obj digest + generation + args digest

(see "Generational caching" below for an explanation of the generation part)

#as_cache_key

As above, you can define a custom cache key for an object:

class Blog
  # [...]
  def as_cache_key
    { :name => name, :url => url }
  end
  # [...]
end

If you don't define #as_cache_key, the default is to Marshal.dump the whole object. (That's not as terrible as it seems - marshalling is fast!)

#to_cache_key (danger!)

There's another way to define a cache key, but it should be used with caution because it gives you total control.

The key is to make sure your #to_cache_key method identifies both the class and the instance!

Comparison

Method Must uniquely identify class Must uniquely identify instance
#as_cache_key N† Y
#to_cache_key Y Y

† The class name is automatically inserted for you by calling object.class.name, which is what causes all the trouble with ActiveRecord::Associations::CollectionProxy, etc.

ActiveRecord

If you're caching methods on instances of ActiveRecord::Base, and/or using them as arguments to other cached methods, then you should probably define something like:

class ActiveRecord::Base
  def as_cache_key
    attributes
  end
end

If you find yourself passing association proxies as arguments to cached methods, this might be helpful:

# For use in ActiveRecord 3.0.x
class ActiveRecord::Associations::AssociationCollection
  # rare use of to_cache_key
  def to_cache_key
    [
      'ActiveRecord::Associations::AssociationCollection',
      proxy_owner.class.name,
      proxy_owner.id,
      proxy_reflection.name,
      conditions
    ].join('/')
  end
end

# For use in ActiveRecord 3.2.x
class ActiveRecord::Associations::CollectionProxy
  # rare use of to_cache_key
  def to_cache_key
    owner = proxy_association.owner
    [
      'ActiveRecord::Associations::CollectionProxy',   # [included because we're using #to_cache_key instead of #as_cache_key ]
      owner.class.name,                                # User
      owner.id,                                        # 'seamusabshere'
      proxy_association.reflection.name,               # :comments
      scoped.where_sql                                 # "WHERE `comments`.`user_id` = 'seamusabshere'" [maybe a little bit redundant, but play it safe]
    ].join('/')
  end
end

Otherwise, cache_method will call user.comments.class.name which causes the proxy to load the target, i.e. load all the Comment objects into memory. You probably don't want to load 1000 AR objects just to generate a cache key.

Generational caching

Generational caching allows clearing the cached results for only one method, for example

my_blog.cache_method_clear :entries

You can disable it to get a little speed boost

CacheMethod.config.generational = false

Generational caching stores a separate generation key that never expires by default. To set a default TTL for these

CacheMethod.config.default_generational_ttl = 120

Debug

CacheMethod can warn you if your obj or args cache keys are suspiciously long.

require 'cache_method'
require 'cache_method/debug'

Then watch your logs.

Module methods

You can put #cache_method right into your module declarations:

module MyModule
  def my_module_method(args)
    # [...]
  end
  cache_method :my_module_method
end

class Tiger
  extend MyModule
end

class Lion
  extend MyModule
end

Rest assured that Tiger.my_module_method and Lion.my_module_method will be cached correctly and separately. This, on the other hand, won't work:

class Tiger
  extend MyModule
  # wrong - will raise NameError Exception: undefined method `my_module_method' for class `Tiger'
  # cache_method :my_module_method
end

Contributors

Copyright

Copyright 2012 Seamus Abshere

More Repositories

1

fuzzy_match

Find a needle (a document or record) in a haystack using string similarity and (optionally) regular expression rules. Uses Dice's Coefficient (aka Pair Similiarity) and Levenshtein Distance internally.
Ruby
668
star
2

upsert

Upsert on MySQL, PostgreSQL, and SQLite3. Transparently creates functions (UDF) for MySQL and PostgreSQL; on SQLite3, uses INSERT OR IGNORE.
Ruby
652
star
3

data_miner

Download, unpack from a ZIP/TAR/GZ/BZ2 archive, parse, correct, convert units and import Google Spreadsheets, XLS, ODS, XML, CSV, HTML, etc. into your ActiveRecord models. Uses RemoteTable gem internally.
Ruby
301
star
4

unix_utils

Like FileUtils, but provides zip, unzip, bzip2, bunzip2, tar, untar, sed, du, md5sum, shasum, cut, head, tail, wc, unix2dos, dos2unix, iconv, curl, perl, etc.
Ruby
227
star
5

remote_table

Open local or remote XLSX, XLS, ODS, CSV (comma separated), TSV (tab separated), other delimited, fixed-width files, and Google Docs. Returns an enumerator of Arrays or Hashes, depending on whether there are headers.
HTML
225
star
6

lock_and_cache

Most caching libraries don't do locking, meaning that >1 process can be calculating a cached value at the same time. Since you presumably cache things because they cost CPU, database reads, or money, doesn't it make sense to lock while caching?
Ruby
134
star
7

mysql2xxxx

Gives you binaries like mysql2csv, mysql2json, and mysql2xml, and Ruby classes to match.
Ruby
83
star
8

cache

Defines a simple interface to multiple cache-like storage engines by wrapping common Ruby client libraries like memcached, redis, memcache-client, dalli. Handles each underlying library's weirdnesses, including forking.
Ruby
69
star
9

eat

A (better?) replacement for open-uri. Lets you open local and remote files by immediately returning their contents as a string.
Ruby
32
star
10

to_regexp

Provides String#to_regexp
Ruby
27
star
11

errata

Define an errata in table format (CSV) and then apply it to an arbitrary source. Inspired by RFC Errata, lets you keep your own errata in a transparent way.
Ruby
21
star
12

cacheable

DEPRECATED. Use cache_method instead.
Ruby
20
star
13

py-upsert

Python library to make it easy to upsert on MySQL, PostgreSQL, and SQLite3.
Python
18
star
14

report

DSL for creating clean CSV, XLSX, and PDF reports in Ruby. Uses xlsx_writer, prawn and pdftk internally.
Ruby
16
star
15

database_url

Convert back and forth between Heroku-style ENV['DATABASE_URL'] and Rails/ActiveRecord-style config/database.yml hashes.
Ruby
16
star
16

lock_method

Like alias_method, but it's lock_method! (lockfiles)
Ruby
12
star
17

common_name

Helps you stop using chains of humanize/downcase/underscore/pluralize/to_sym/etc everywhere in your models, your views, your controllers, etc.
Ruby
11
star
18

engineyard-metadata

Presents a simple, unchanging interface to get metadata about your EngineYard AppCloud instances running on Amazon EC2.
Ruby
10
star
19

cohort_analysis

TBD
Ruby
10
star
20

create_table

Analyze and inspect CREATE TABLE SQL statements and translate across databases. Uses Ragel internally for parsing.
Ruby
10
star
21

ruby_ragel_examples

Examples of using ragel and ruby together
Ruby
9
star
22

fuzzy_infer

Fuzzy set analysis - predicts one or more unknown characteristics of an input case by comparing its known characteristics to a reference dataset whose records contain both the known and unknown characteristics.
Ruby
8
star
23

the_geom_geojson

For PostGIS/PostgreSQL and ActiveRecord, provides "the_geom_geojson" getter and setter that update "the_geom" and "the_geom_webmercator" columns.
Ruby
8
star
24

validates_decency_of

Rails plugin that uses George Carlin's list of seven dirty words (aka swear words, aka cuss words, aka bad words) to check for "decency" on ActiveRecord model attributes.
Ruby
6
star
25

weighted_average

Aircraft.average(:seats) versus Aircraft.weighted_average(:seats, :weighted_by => :takeoffs)
Ruby
6
star
26

loose_tight_dictionary

DEPRECATED: use fuzzy_match. Find a needle in a haystack using string similarity and (optionally) regexp rules.
Ruby
6
star
27

pg_trgm

Ruby trigram similarity that is identical to Postgres's (almost)
Ruby
5
star
28

redirect_routing

Ruby
5
star
29

hash_digest

Generates non-cryptographic digests of Hashes (and Arrays) indifferent to key type (string or symbol) and ordering.
Ruby
5
star
30

ey_cloud_awareness

DEPRECATED: use engineyard-metadata. Make your EngineYard cloud instances aware of each other.
Ruby
4
star
31

xml_split

Split XML files on an element, yielding (streaming, so constant memory usage) each node in turn. Uses sgrep2 internally; future versions should use a pure-Ruby SAX parser.
Ruby
3
star
32

characterizable

DEPRECATED. Use charisma instead.
Ruby
3
star
33

has_handle_fallback

Make it easy to use handles (callsigns/monikers/usernames) in URLs, even if they might be blank.
Ruby
3
star
34

table_warnings

Warn yourself of problems with your ActiveRecord tables.
Ruby
3
star
35

to_json_fix

TODO: one-line summary of your gem
Ruby
3
star
36

honeypot

TODO: one-line summary of your gem
Ruby
3
star
37

cohort_scope

DEPRECATED. Use cohort_analysis. Provides cohorts (in the form of ActiveRecord scopes) that dynamically widen until they contain a certain number of records.
Ruby
3
star
38

vector_embed

Vector embedding of strings, booleans, numerics, and arrays into LIBSVM / LIBLINEAR format.
Ruby
3
star
39

switches

Turn on and off parts of your code based on yaml files.
Ruby
3
star
40

flights1percent

1% flights
JavaScript
2
star
41

zmq

Drop-in replacement for zmq gem with included binaries
Ruby
2
star
42

json_to_csv_to_json

csv_to_json and json_to_csv
Ruby
2
star
43

cvg

Like jq or grep for csv. Combine one or more CSVs while filtering on fields with regular expressions, whitelists, presence, missing, etc.
Ruby
2
star
44

has_timestamps

Rails plugin to add named timestamps to ActiveRecord models.
Ruby
2
star
45

zip5

Convert United States zip codes to their correct Zip5 representation, even if they're missing a leading zero and/or they have the +4 suffix.
Ruby
2
star
46

string_enumerator

Given a string containing placeholders (like [color]), enumerate all of the possible strings resulting from filling those placeholders with replacements (like red, blue).
Ruby
2
star
47

nonrandomapp

Ruby
1
star
48

mini_record

mini_record-compat is DEPRECATED. Use original mini_record OR active_record_inline_schema instead.
Ruby
1
star
49

string_replacer

DEPRECATED/POINTLESS - use sed or augeas. Replace text in a file without disturbing the rest of the file.
Ruby
1
star
50

geocode_records

As long as you do very specific things... quickly re-geocode tables.
Ruby
1
star
51

force_schema

[DEPRECATED - use mini_record] Declare a table structure like an ActiveRecord migration and run 'force_schema!' whenever you want. For when you don't need up and down migrations.
Ruby
1
star
52

fast_timestamp

Rapidly and arbitrarily timestamp ActiveRecord records.
Ruby
1
star