Dynosaur
An auto-scaling engine for Heroku web dynos using pluggable API connections. The first API is Google Analytics Live, which uses the number of active users on the site to decide how many dynos to run. There is also a New Relic plugin that uses the requests per minute.
Methodology
Each plugin generates a 'value' at some set interval (e.g. polls the GA API
every 10s) and provides a conversion from the value ('active users on the site')
to estimated dynos required. (e.g. 100 active users per dyno). By default this
has some built in hysteresis, i.e. the plugins are conservative about
decreasing their estimate and report the highest estimate of the past N
seconds.
Dynosaur takes the maximum of all plugin estimates, applies a global min/max constraint and scales your Heroku app to match.
For example, see this plot of a dummy 'Sine wave' plugin (purple line) and the resulting estimated dynos (orange line).
The plugin estimates 'units' per dyno, and Dynosaur is configured with a minimum of 2 dynos. You can see that the estimated dynos rapidly tracks the upward slopes, but trails the downward slope by about 30s.
Installation and Usage
See below about how to deploy to Heroku.
$ gem install dynosaur
Create a YAML config file (see config.sample.yaml and info below) and run with:
$ dynosaur config.yaml
Global Autoscaler Configuration
The 'scaler' section of the config file configures the main parameters of the autoscaler.
heroku_app_name(string): The name of the heroku app you want to autoscaleheroku_api_key(string): Heroku API key can be retrieved from the Heroku account settings page.dry_run(boolean): If enabled, the scaler does not actually change anything in Heroku, just simulates the values it would choose.interval(int): The autoscaler sleeps for this many seconds before checking for activity. Note that each plugin is configured with an API polling interval too, so this does not increase the frequency of API polling.
Dynosaur will run indefinitely, with info output to stdout at intervals.
If multiple plugins are configured, the scaler will use the maximum of all plugins results (i.e. if your New Relic plugin returns 3 dynos, and your GA plugin returns 5, you should scale to 5 dynos.
Statistics
Dynosaur can optionally use report statistics either to the command line or to Librato.
The following stats are sent every interval seconds.
- combined estimate of dynos required (includes min/max constraints)
- Actual number we are using (includes hysteresis)
For each plugin we send
- value (e.g. 'active users')
- plugin dyno estimate
Plugin Configuration
Dynosaur can scale different layers of your app, each layer is documented separately:
- Web dynos (based on Google Analytics Live API and/or NewRelic RPM data)
- RedisCloud addon (based on number of connections and/or memory usage)
- Papertrail addon (based on amount of log data generated per month)
Common Plugin Configuration
All input plugins have the following config values
name: unique identifier for the plugin instance. Freeform.type: the name of the plugin class e.g.Dynosaur::Inputs::GoogleAnalyticsPlugininterval(default 60s) : how often to poll the respective API. (i.e. the retrieved value is cached for 'interval' seconds.)hysteresis_period(default 300s) : the current estimate is based on the maximum value observed within this interval. i.e. If the maximum value of active users observed in the last 5 minutes is 127, we will base our estimate on 127 active users.
Statistics Plugins
Librato Stats Reporter Configuration
You can sign up for a free account
type: Dynosaur::Stats::Libratoapi_email(string): Email of Librato account user.api_key(string): API key.
Console Stats Reporter Configuration
type: Dynosaur::Stats::Console
Error Reporting Plugins
We've added pluggable error handling, with three implementations available so far
Console (Default)
Logs errors to the console. No config.
Bugsnag
Send exceptions to Bugsnag. Configured like this in config.yaml:
error_handlers:
-
type: Dynosaur::ErrorHandler::Bugsnag
api_key: <%= ENV['BUGSNAG_API_KEY'] %>
Email via AWS Simple Email Service
Configured like this in config.yaml
error_handlers:
-
type: Dynosaur::ErrorHandler::Ses
from: [email protected]
to: [email protected]
aws_access_key_id: <%= ENV['AWS_ACCESS_KEY_ID'] %>
aws_secret_access_key: <%= ENV['AWS_SECRET_ACCESS_KEY'] %>
Warning: this has the tendency to send a lot of email due to minor blips in API reliability.
Success Plugins
We have included a pinger for our "dead man's switch" server Cronut. It will ping every minute or so, and we get an alert when Cronut hasn't received a ping for 10 minutes.
Deploying Dynosaur to Heroku
At Harry's, we run Dynosaur as a Heroku app. Here's the steps to get it deployed:
- Create heroku app.
- Clone heroku git repository.
- Create a Gemfile and add dynosaur as a gem (see
doc/Gemfile). - Write your config file (see
doc/config.sample.yaml) - We use environment variables for secrets, so set them locally and in Heroku
- Check that it works
bundle installbundle exec dynosaur config.yaml
- Copy the Procfile (from
doc/Procfile) to the root of your project. - Commit it all and deploy to heroku
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
Writing a Plugin
You'll need to implement the following methods:
retrieve(): connect to an API (or wherever) and retrieve a new value. This is
wrapped in a caching layer by the plugin base class.
value_to_dynos(): calculate the estimated number of dynos based on the recent
maximum value returned by retrieve() (e.g. we use users_per_dyno in the GA
plugin). For more fine-grained control, you can override estimate_dynos()
instead.
initialize(config): You can pull any configuration you require from the config hash passed in.
See the Google Analytics plugin or the toy Random plugin for an example.
Contributors
- New Relic plugin by Jan Lindblom
