• Stars
    star
    107
  • Rank 323,587 (Top 7 %)
  • Language
    Python
  • License
    Other
  • Created over 11 years ago
  • Updated over 11 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A green threading library with automated concurrency for python based on a processes, signal handlers and greenlets

Conpig Threading Library

Conpig is a green threading library using green threads from Gevent which are based on greenlets. along side a signal based scheduler. This allows one true concurrency using green threads instead of OS processes or OS threads such that spawning a new thread is still cheap.

Want

  • Fast, cheap, easy, traditional concurrency.

Have

  • Cpython's GIL prevents OS threads from running concurrently and switching context for any reason but to perform IO.

  • OS threads are bulky and slow. Processes are even worse.

  • Current green thread implementations require manual context switches. This is tedious. This is also bug prone for programmers who are used to having a scheduler.

  • Processes based libraries like parallel python prevent you from communicating between "threads" with the datastructures you'd usually use. Again, processes are also slow and buggy.

Solution

  • Create a global scheduler using signals to schedule the next context switch.

Drawbacks & Limitations

  • A solution to concurrency, not parallelism.

  • Conpig threads still can only run on one core of a processor.

  • Context switches still only occur every CPython tick.

  • You still need cooperative yields if you want true fairness for programs that don't do as much IO.

Installing (pip to come)

  1. Install Greenlet with pip.

  2. Install Gevent

    • The old version depends on libevent. The new github version depends on libev
    • On OSX the install path for libevent is wrong even if you use Mac Ports.
    • The easiest install method is to download v.1 from github, and install from source using the provided "setup.py".

Usage

  • You can run from a main method
import conpig

def test(arg):
    for i in range(0,4000):
        print arg


conpig.spawn(test, "X")    
conpig.spawn(test, "O")

conpig.wait_all()

Authors

  • Matthew Mirman

  • Eric Faust

License

See LICENSE file.