There are no reviews yet. Be the first to send feedback to the community and the maintainers!
NAME forkoff SYNOPSIS brain-dead simple parallel processing for ruby URI http://rubyforge.org/projects/codeforpeople http://github.com/ahoward/forkoff INSTALL gem install forkoff DESCRIPTION forkoff works for any enumerable object, iterating a code block to run in a child process and collecting the results. forkoff can limit the number of child processes which is, by default, 2. SAMPLES <========< samples/a.rb >========> ~ > cat samples/a.rb # forkoff makes it trivial to do parallel processing with ruby, the following # prints out each word in a separate process # require 'forkoff' %w( hey you ).forkoff!{|word| puts "#{ word } from #{ Process.pid }"} ~ > ruby samples/a.rb hey from 7907 you from 7908 <========< samples/b.rb >========> ~ > cat samples/b.rb # for example, this takes only 4 seconds or so to complete (8 iterations # running in two processes = twice as fast) # require 'forkoff' a = Time.now.to_f results = (0..7).forkoff do |i| sleep 1 i ** 2 end b = Time.now.to_f elapsed = b - a puts "elapsed: #{ elapsed }" puts "results: #{ results.inspect }" ~ > ruby samples/b.rb elapsed: 4.19184589385986 results: [0, 1, 4, 9, 16, 25, 36, 49] <========< samples/c.rb >========> ~ > cat samples/c.rb # forkoff does *NOT* spawn processes in batches, waiting for each batch to # complete. rather, it keeps a certain number of processes busy until all # results have been gathered. in otherwords the following will ensure that 3 # processes are running at all times, until the list is complete. note that # the following will take about 3 seconds to run (3 sets of 3 @ 1 second). # require 'forkoff' pid = Process.pid a = Time.now.to_f pstrees = %w( a b c d e f g h i ).forkoff! :processes => 3 do |letter| sleep 1 { letter => ` pstree -l 2 #{ pid } ` } end b = Time.now.to_f puts puts "pid: #{ pid }" puts "elapsed: #{ b - a }" puts require 'yaml' pstrees.each do |pstree| y pstree end ~ > ruby samples/c.rb pid: 7922 elapsed: 3.37899208068848 --- a: | -+- 07922 ahoward ruby -Ilib samples/c.rb |-+- 07923 ahoward ruby -Ilib samples/c.rb |-+- 07924 ahoward (ruby) \-+- 07925 ahoward ruby -Ilib samples/c.rb --- b: | -+- 07922 ahoward ruby -Ilib samples/c.rb |-+- 07923 ahoward ruby -Ilib samples/c.rb |-+- 07924 ahoward ruby -Ilib samples/c.rb \-+- 07925 ahoward ruby -Ilib samples/c.rb --- c: | -+- 07922 ahoward ruby -Ilib samples/c.rb |-+- 07923 ahoward ruby -Ilib samples/c.rb |-+- 07924 ahoward (ruby) \-+- 07925 ahoward ruby -Ilib samples/c.rb --- d: | -+- 07922 ahoward ruby -Ilib samples/c.rb |-+- 07932 ahoward ruby -Ilib samples/c.rb |--- 07933 ahoward ruby -Ilib samples/c.rb \--- 07934 ahoward ruby -Ilib samples/c.rb --- e: | -+- 07922 ahoward ruby -Ilib samples/c.rb |--- 07932 ahoward (ruby) |-+- 07933 ahoward ruby -Ilib samples/c.rb \-+- 07934 ahoward (ruby) --- f: | -+- 07922 ahoward ruby -Ilib samples/c.rb |--- 07932 ahoward (ruby) |-+- 07933 ahoward ruby -Ilib samples/c.rb \-+- 07934 ahoward ruby -Ilib samples/c.rb --- g: | -+- 07922 ahoward ruby -Ilib samples/c.rb |-+- 07941 ahoward ruby -Ilib samples/c.rb |--- 07942 ahoward ruby -Ilib samples/c.rb \--- 07943 ahoward ruby -Ilib samples/c.rb --- h: | -+- 07922 ahoward ruby -Ilib samples/c.rb |-+- 07941 ahoward (ruby) |-+- 07942 ahoward ruby -Ilib samples/c.rb \--- 07943 ahoward ruby -Ilib samples/c.rb --- i: | -+- 07922 ahoward ruby -Ilib samples/c.rb |--- 07942 ahoward (ruby) \-+- 07943 ahoward ruby -Ilib samples/c.rb <========< samples/d.rb >========> ~ > cat samples/d.rb # forkoff supports two strategies of reading the result from the child: via # pipe (the default) or via file. you can select which to use using the # :strategy option. # require 'forkoff' %w( hey you guys ).forkoff :strategy => :file do |word| puts "#{ word } from #{ Process.pid }" end ~ > ruby samples/d.rb hey from 7953 you from 7954 guys from 7955 HISTORY 1.1.0 - move to a model with one work queue and signals sent from consumers to producer to noitify ready state. this let's smaller jobs race through a single process even while a larger job may have one sub-process bound up. incorporates a fix from http://github.com/fredrikj/forkoff which meant some processes would lag behind when jobs didn't have similar execution times. 1.0.0 - move to github 0.0.4 - code re-org - add :strategy option - default number of processes is 2, not 8 0.0.1 - updated to use producer threds pushing onto a SizedQueue for each consumer channel. in this way the producers do not build up a massize parllel data structure but provide data to the consumers only as fast as they can fork and proccess it. basically for a 4 process run you'll end up with 4 channels of size 1 between 4 produces and 4 consumers, each consumer is a thread popping of jobs, forking, and yielding results. - removed use of Queue for capturing the output. now it's simply an array of arrays which removed some sync overhead. - you can configure the number of processes globally with Forkoff.default['proccess'] = 4 - you can now pass either an options hash forkoff( :processes => 2 ) ... or plain vanilla number forkoff( 2 ) ... to the forkoff call - default number of processes is 8, not 2 0.0.0 initial version
sekrets
sekrets is a command line tool and library used to securely manage encrypted files and settings in your rails' applications and git repositories.main
a class factory and dsl for generating command line programs real quickopen4
open child process with handles on pid, stdin, stdout, and stderr: manage child processes and their io handles easily.map
the ruby container you've always wanted: an ordered string/symbol indifferent hashsystemu
univeral capture of stdout and stderr and handling of child process pid for windows, *nix, etc.testy
a BDD testing framework for ruby that's mad at the world and plans to kick it's ass in 78 freakin lines of codeconfiguration
pure ruby scoped configuration files.fattr
fattr.rb is a "fatter attr" for ruby and borrows heavily from the metakoans.rb ruby quizsession
session offers a persistent way to drive the shell (/bin/sh) via rubydao
sa-weet data access object library for rails. top secret.lockfile
a ruby library for creating NFS safe lockfilesmacaddr
cross platform mac address determination for rubymiddleman-gibberish
password protect middleman pages - even on s3threadify
threadify.rb makes it stupid easy to process a bunch of data using 'n' worker threadsshared
shared.rb provides a super easy way to share code between classes or modules in a simple way.bj
Backgroundjob (Bj) is a brain dead simple, zero admin, background priority queue for Rails.tagz
tagz.rb generates html, xml, or any sgml variant like a small ninja running across the backs of a herd of giraffes swatting of heads like a mark-up weedwacker.demon
demon.rb - the ruby daemon library you've been waiting forslave
easy management of child process works over pipes and drbrails_build
A very small, very simple, very fast, and bullet proof static site generator built as a Rails 5 engine.objectpool
a simple, robust, generic thread-safe object pool for rubyrq
ruby queue is a zero-admin zero-configuration tool used to create instant unix clustersraptcha
low drain bamage, storage-less, session-less, plugin-less, zero admin, single-source-file secure captcha system for ruby and/or rails.arrayfields
allow keyword access to array instances.tumblr
a command line utility and library for the excellent tumblr blogging platformhelene
helene is a plugin for writing rails applications on top of amazon's aws platform including sdb, s3, and sqsrego
run arbitrary commands easily when files changefbomb
fbomb is the dangerous flowdock botfucking_favicons
fucking favicons fucking suckmongoid-haystack
a mongoid 3 zero-config, zero-integration, POLS pure mongo fulltext solutioncoxswain
encapsulate pre-forking master / worker pattern for rubyjquery.bires
bandwidth limited progressive image enhancementhashish
awesome data access layer for rails/ruby projectsrails_current
track current_STUFF mo bettaassassin
no zombies ever, not even on `exit!` or `kill -9`default_url_options
all relative urls in rails all the time. even in mailers.forkhandle
a teeny library / design pattern for managing connections in a process and thread safe fashionrememberthemilk
simple (162 loc), json only, interface to the excellent RememberTheMilk APIwrap
non-sucky :before and :after callbacks for any ruby classconducer
a model+view component for rails that combines the conductor and presenter patternro
ro is library for managing your site's content in git, as god intended.rails_default_url_options
you really can have default_url_options everywhere. even in mailers.options
options.rb handles keyword options in ruby in a simple and robust wayirbcp
irbcp gives access to your system's clipboard (copy and paste) from irbrails_errors2html
simple and sane active_model error html renderingfukung
perhaps the most important ruby code EVAAARRR! gets random images from http://fukung.net.rails_nav
objectified navigation for railscssjs
a zero learning curve zero contraints dsl for writing css stylesheets in javascriptpork
pork supports parallel programming in ruby using forked actors and durable sqlite message queuesisolation
a small rails app to demonstrate what *you* don't understand about RDBMS transactionsmongoid-fts
enable mongodb's new fulltext simply and quickly on your mongoid models, including pagination.lru_cache
a simple but efficient implementation of a size limited least recently used cache in rubyterminator
an external timeout mechanism based on processes and signalscdc
uber simple cross domain communication for javascript/iframescoerce
a ruby library full of common cast/coercion operationsrails_view
render views from anywhere. even without a controller contextrails_helper
helper = Helper.new and helper.link_to(:foo)ggeocode
simple wrapper on google's new geocoding apisuck
gem to show issues with gem/minigem load orderingalpo
a library and design pattern for building sane web applications on top of the rails' stackbackup.rake
rails' rake task for backup up and loading data+assets as yaml+filesid3rename
id3rename is a program to do simple renaming of mp3 filesnfsutils
Ruby FileUtils for NFSbookify
development moved to https://github.com/everlater/bookifysenv
the 12-factor environment tool your mother told you to usecodeforpeople
billions and billions of libsledis
a K.I.S.S auto-rotating redis logger for ruby/railskgb
ultra lightweight javascript decision tree builderimage_cache
a small utility library to facility caching image uploads between form validation failures.gnip-ruby
Ruby library for utilizing Gnip services.linked_list
a simple linked list implementation for rubybucket
bucket is a command-line interface for amazon's s3 servicetmpdir_block
extends ruby's built-in Dir.tmpdir to accept a block.bestofyoutube
simple ruby library to grab some good video urls from http://bestofyoutube.comsuperhash
A general mechanism for defining attribute inheritance structures among objects of any type, not just classestesting.rb
adds the minimal features required in order to make test/unit not suckopenobject
a simple property based container that's much more capable than a blankslate but far less polluted than ruby's built-in OpenStructcandy_store
hybrid session store that combines rails' built-in cookie based session store with its database backed oneslug
a simple slug library. unicode prepared.markdown
my markdown scriptmongoid-bolt
mongoid-bolt is a concrete lock implementation and mixin.growltdf
growltdf is the greatest program evar. it let's you scrape http://cyclingnews.com for TDF updates and dump them into growl to you can keep up with the race while working.nmap
narray + mmap = wicked fast persistent numerical arrays for rubymob
background jobs for mongoidfoobar
ydb
mo-betta the yaml/store.hrs
tracking teh hours from the climp3scrape
download shit-loads of mp3s from web pageswapp
golden.imageupload_cache
a small utility library to facility caching http file uploads between form validation failures. designed for rails, but usable anywhere.forkify
forkify.rb makes it easy to process a bunch of data using 'n' worker processesansible
magic unicorns for your deployzcampfire
a command line script for using 37signal's campfire chat roomcast
a collection of casting methods for rubyfifo
a very simple javascript fifo queue / cacheey-cloud-recipes
A starter repo for custom chef recipes on EY's cloud platformtest
one-click-hugo-cms
attributes
the implementation of attributes.rb borrows many of the best ideas from the metakoans.rb ruby quiz (ps. fattr >= attributes)gnip-expander
relay a gnip publisher stream to another, expanding shortened uris in the processLove Open Source and this site? Check out how you can help us