• Stars
    star
    1,462
  • Rank 32,167 (Top 0.7 %)
  • Language
    Python
  • License
    MIT License
  • Created over 9 years ago
  • Updated over 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Magic decorator syntax for asynchronous code in Python

Codeship Status for madisonmay/Tomorrow

Tomorrow

Magic decorator syntax for asynchronous code in Python 2.7.

Please don't actually use this in production. It's more of a thought experiment than anything else, and relies heavily on behavior specific to Python's old style classes. Pull requests, issues, comments and suggestions welcome.

Installation

Tomorrow is conveniently available via pip:

pip install tomorrow

or installable via git clone and setup.py

git clone [email protected]:madisonmay/Tomorrow.git
sudo python setup.py install

To ensure Tomorrow is properly installed, you can run the unittest suite from the project root:

nosetests -v 

Usage

The tomorrow library enables you to utilize the benefits of multi-threading with minimal concern about the implementation details.

Behind the scenes, the library is a thin wrapper around the Future object in concurrent.futures that resolves the Future whenever you try to access any of its attributes.

Enough of the implementation details, let's take a look at how simple it is to speed up an inefficient chunk of blocking code with minimal effort.

Naive Web Scraper

You've collected a list of urls and are looking to download the HTML of the lot. The following is a perfectly reasonable first stab at solving the task.

For the following examples, we'll be using the top sites from the Alexa rankings.

urls = [
    'http://google.com',
    'http://facebook.com',
    'http://youtube.com',
    'http://baidu.com',
    'http://yahoo.com',
]

Right then, let's get on to the code.

import time
import requests

def download(url):
    return requests.get(url)

if __name__ == "__main__":

    start = time.time()
    responses = [download(url) for url in urls]
    html = [response.text for response in responses]
    end = time.time()
    print "Time: %f seconds" % (end - start)

More Efficient Web Scraper

Using tomorrow's decorator syntax, we can define a function that executes in multiple threads. Individual calls to download are non-blocking, but we can largely ignore this fact and write code identically to how we would in a synchronous paradigm.

import time
import requests

from tomorrow import threads

@threads(5)
def download(url):
    return requests.get(url)

if __name__ == "__main__":
    start = time.time()
    responses = [download(url) for url in urls]
    html = [response.text for response in responses]
    end = time.time()
    print "Time: %f seconds" % (end - start)

Awesome! With a single line of additional code (and no explicit threading logic) we can now download websites ~10x as efficiently.

You can also optionally pass in a timeout argument, to prevent hanging on a task that is not guaranteed to return.

import time

from tomorrow import threads

@threads(1, timeout=0.1)
def raises_timeout_error():
    time.sleep(1)

if __name__ == "__main__":
    print raises_timeout_error()

How Does it Work?

Feel free to read the source for a peek behind the scenes -- it's less than 50 lines of code.

More Repositories

1

CommonRegex

A collection of common regular expressions bundled with an easy to use interface.
Python
1,568
star
2

BlackWidow

Visualizing Python Project Import Graphs
Python
108
star
3

tracker

A time machine for debugging pesky stateful errors.
Python
35
star
4

SemiSync

Semi-synchronous programming in python
Python
34
star
5

Runtype

Runtime type checking for python
Python
7
star
6

docai

Structured information extraction from documents
Python
7
star
7

coplay

Collaborative music at its finest.
JavaScript
3
star
8

MotorControl

Simple arduino code using AdaFruit MotorShield for PD control of brushed motor.
Arduino
3
star
9

TheanoAutoencoder

A generic autoencoder in Theano.
Python
2
star
10

visual-lastfm

Last.fm statistics with D3.js
JavaScript
2
star
11

ScalingLawsCalculator

Calculator to compute scaling laws for neural language models
Python
2
star
12

UnicodePlease

Utilities to prevent you from tearing your hair out dealing with unicode and text encodings in python 2.7.
Python
2
star
13

dldsl

Deep Learning Domain Specific Language
Python
2
star
14

Spokes-Extension

A chrome extension for Spokes -- an intelligent, unified search engine
JavaScript
1
star
15

siren-haiku

Implementation of SIREN described in https://arxiv.org/abs/2006.09661
Python
1
star
16

sublime-settings

Sublime Text 3 settings for python hackers with an appreciation for visual minimalism
Python
1
star
17

CrashCourse

Crash course for Olin College ReadML cocurricular.
1
star
18

Algorithms

Basic algorithms in Python, Javascript, and C++
C++
1
star
19

Decorative

Handy python decorators
Python
1
star
20

SensorDebugger

Simple android project to aid with sensor debugging
Java
1
star