• Stars
    star
    135
  • Rank 269,297 (Top 6 %)
  • Language
    Python
  • License
    Other
  • Created over 7 years ago
  • Updated about 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

๐ŸŽ’ Simple schedule optimization library for Python

Taskpacker Logo

Taskpacker is a generic schedule optimization and visualization library for Python. For instance, below is an optimized schedule where 20 processes (each comprised of several tasks and represented by one color) are ran 24/7 in a factory:

[dna_assembly.png]

Such plots enable you to spot the bottlenecks of your factory. In this example, it appears that ovens are the limiting elements (the only machines packed full with no downtime) and that buying a third oven will increase your factory's throughput.

Main features

Taskpacker was built as a toy project to have an easily-extensible scheduling tool in Python. Only Python2 is supported right now (sorry for that, there is a complex bug with Numberjack in Python3). It is pretty simple and limited (the core code is ~200 lines) but comes with enough features to cover many cases:

  • Supports resources (typically, people or robots) and resource capacity (= how much jobs a resource can do at the same time)
  • Supports tasks dependencies (some tasks must be finished before other tasks can be started) and maximum waiting time (i.e. some tasks must be started at the latest X minutes after their parents are completed)
  • Supports pre-scheduled tasks (such as breaks for human operators, scheduled robotic maintenance etc.)

Work in progress - contribute !

Taskpacker is an open-source software originally written to optimize the robot-operated DNA assembly operations at the Edinburgh Genome Foundry. It is released on Github under the MIT licence (ยข Edinburgh Genome Foundry), with no warranties: this is an experimental piece of software which we hope will be as useful for you as it was for us. And everyone is welcome to contribute !

Installation

Taskpacker can be installed by unzipping the source code in one directory and using this command:

sudo python setup.py install

You can also install it directly from the Python Package Index with this command:

sudo pip taskpacker install

It is probable that you will need some dependencies to build Numberjack. On Ubuntu you can install these with:

sudo apt install libxml2-dev swig

Basic Example

In this example two labbies have been assigned a list of chores. Alice will visit the GMO plants, cook the hamsters, and feed the gremlins. Bob will clean the scalpels, dice the hamsters once they are cooked, then assist Alice in gremlins feeding (a task that takes two people). Certain tasks can only be done after other tasks have been completed. Alice has a stereotypical predisposition to multitasking: she can do 2 jobs at the same time, while Bob can't.

Here is how you would use Taskpacker to find when they will do each task so as to finish as early as possible:

from taskpacker import Task, Resource, numberjack_scheduler, plot_schedule
alice = Resource("Alice", capacity=2)
bob = Resource("Bob", capacity=1)



clean_scalpels = Task("Clean the scalpels", resources=[bob], duration=20,
                      color="white")
visit_plants = Task("Visit the plants", resources=[alice], duration=60,
                     color="yellow")
cook_hamsters = Task("Cook the hamsters", resources=[alice], duration=30,
                     color="red")
dice_hamsters = Task("Dice the hamsters", resources=[bob], duration=40,
                     color="blue", follows=[cook_hamsters, clean_scalpels])
feed_gremlins = Task("Feed the gremlins", resources=[alice, bob], duration=50,
                     color="orange", follows=[dice_hamsters])


all_tasks = [clean_scalpels, visit_plants, cook_hamsters, dice_hamsters,
             feed_gremlins]
scheduled_tasks = numberjack_scheduler(all_tasks)
fig, ax = plot_schedule(scheduled_tasks)
ax.figure.set_size_inches(7, 3)
ax.figure.savefig("alice_and_bod.png", bbox_inches="tight")

Modeling tasks and reources with spreadsheets

Assume that you have a process consisting in several tasks, each task depending on some resources to be available, and possibly on other tasks. Such process can be summarized in a spreadsheet like this one `this file <>`_, which is loaded in Taskpacker as follows:

from taskpacker import (get_resources_from_spreadsheet,
                        get_process_from_spreadsheet)

resources = get_resources_from_spreadsheet(
    spreadsheet_path="path/to/spreadsheet.xls", sheetname="resources")

process_tasks = get_process_from_spreadsheet(
    spreadsheet_path="path/to/spreadsheet.xls",
    sheetname="process",
    resources_dict=resources
)

Then you can for instance plot the dependency graph of the tasks:

from taskpacker import plot_tasks_dependency_graph
plot_tasks_dependency_graph(process_tasks)

[logo]

Or simply schedule the tasks:

from taskpacker import numberjack_scheduler
scheduled_tasks = numberjack_scheduler(process_tasks)

Throughput estimations

Given a list of tasks forming a process, you might ask "how many of these processes can my factory run in a day ?". The following code loads 20 of these processes and asks Taskpacker to stack them one by one as compactly as possible:

from taskpacker import (get_process_from_spreadsheet,
                        get_resources_from_spreadsheet,
                        schedule_processes_series,
                        plot_tasks_dependency_tree,
                        plot_schedule, Task)
import matplotlib.cm as cm


colors = [cm.Paired(0.21 * i % 1.0) for i in range(30)]

resources = get_resources_from_spreadsheet(
    spreadsheet_path="path/to/spreadsheet.xls", sheetname="resources")

processes = [
    get_process_from_spreadsheet(spreadsheet_path="path/to/spreadsheet.xls",
                                 sheetname="process",
                                 resources_dict=resources,
                                 tasks_color=colors[i],
                                 task_name_prefix="WU%d_" % (i + 1))
    for i in range(20)
]

# OPTIMIZE THE SCHEDULE
new_processes = schedule_processes_series(
    processes, est_process_duration=5000, time_limit=5)

# PLOT THE OPTIMIZED SCHEDULE

all_tasks = [t for process in new_processes for t in process]
fig, ax = plot_schedule(all_tasks)
ax.set_xlabel("time (min)")
ax.figure.savefig("dna_assembly_schedule.png", bbox_inches="tight")

[dna_assembly.png]

Note that it is also possible to add scheduled breaks (here we make them appear as white rectangles) so that your Igor can rest:

scheduled_breaks = [
    Task("break_%03d" % i,
         resources=[resources["igor"]],
         scheduled_resource={resources["igor"]: 1},
         duration=12 * 60, # The break lasts 12H
         scheduled_start=24 * 60 * i, # The break happens every 24H
         color='white')
    for i in range(6)
]

new_processes = schedule_processes_series(
    processes, est_process_duration=5000, time_limit=5,
    scheduled_tasks=scheduled_breaks)

[dna_assembly_with_breaks.png]

More Repositories

1

DnaFeaturesViewer

๐Ÿ‘๏ธ Python library to plot DNA sequence features (e.g. from Genbank files)
Python
491
star
2

pdf_reports

๐Ÿ“• Python library and CSS theme to generate PDF reports from HTML/Pug
Python
190
star
3

DnaChisel

โœ๏ธ A versatile DNA sequence optimizer
Python
176
star
4

Flametree

๐Ÿ”ฅ Python file and zip operations made easy
Python
147
star
5

blabel

๐Ÿท๏ธ Python label/sticker PDF generation. HTML templates, built-in barcodes, qr codes, and other goodies
Python
135
star
6

Proglog

๐Ÿ“ Logs and progress bars manager for Python
Python
93
star
7

lala

๐ŸŒŽ Analyze and generate reports of web logs (NGINX)
Python
60
star
8

DnaCauldron

โš—๏ธ Simple cloning simulator (Golden Gate etc.) for single and combinatorial assemblies
Python
44
star
9

sequenticon

๐Ÿ‘พ Generate identicons for DNA sequences with Python
Python
36
star
10

Plateo

๐Ÿค– Python biolab automation library: parsers, reports generators, picklists simulators, and more
Python
35
star
11

Primavera

๐ŸŒธ Python library for primer-based verification of DNA assemblies: primer selection, data analyis, etc.
Python
32
star
12

codon-usage-tables

๐Ÿ“Š Codon usage tables in code-friendly format + Python bindings
Python
31
star
13

Geneblocks

๐Ÿ’  Find common blocks and differences between DNA sequences
Python
28
star
14

crazydoc

Read DNA sequences from colourful Microsoft Word documents
Python
24
star
15

DnaWeaver

A route planner for DNA assembly
Python
22
star
16

Caravagene

๐ŸŽจ Python library to plot multi-part genetic constructs
Python
21
star
17

genome_collector

๐ŸŒ  Easily download genomes and build BLAST/Bowtie indexes in Python
Python
20
star
18

CAB

๐Ÿš– The friendly Computational App Boilerplate. Django + Vue.JS + Redis queues + NginX
Vue
20
star
19

CUBA

๐Ÿ–๏ธ The EGF Collection of Useful Bio Apps - Web demos of EGF software
Vue
19
star
20

BandWagon

๐ŸŽบ Plot DNA digestion band patterns with Python
Python
17
star
21

BandWitch

๐Ÿ’ซ Computer-aided DNA assembly validation and identification from restriction digests.
Python
15
star
22

SBOL-Visual-CSS

โžฐ Draw genetic elements with HTML & CSS
HTML
15
star
23

genedom

Batch domestication of genetic parts with Python
Python
13
star
24

Sequeduct

Sequencing analysis pipeline
Nextflow
12
star
25

bioprinter

๐Ÿ–จ๏ธ Print pictures with living micro-organisms !
Python
12
star
26

Minotaor

An amino acid sequence annotator
Python
11
star
27

tatapov

๐Ÿพ DNA overhang misannealing data for Python
Python
9
star
28

GoldenHinges

๐Ÿ”— Short overhangs design for DNA assembly
Python
9
star
29

zymp

โœ‚๏ธ Design compact restriction sites arrays (python utility)
Python
8
star
30

kappagate

๐Ÿ”ฎ Predict DNA assembly clone validity rates - powered by Kappa
Python
8
star
31

dab

๐Ÿ‘‰ EGF Design and Build, the Foundry's DNA design ordering portal
Vue
8
star
32

egf-shared-documents

๐Ÿ“š Shared slideshows, courses, etc. from the EGF
HTML
7
star
33

easy_dna

๐ŸŽ’ Python library to read, write, edit DNA sequences
Python
6
star
34

icebreaker

โ„๏ธ Python API for the JBEI-ICE sample manager
Python
6
star
35

Examples

Collection of Python modules and Jupyter notebook examples
HTML
5
star
36

topkappy

๐ŸŒ” Pythonic bindings for the Kappa model simulation language
Python
5
star
37

crecombio

A simple Cre, Flp and other site-specific recombination simulator
Python
4
star
38

HowTo

๐Ÿ“’ Short opinionated recipes, mostly for us.
Python
4
star
39

genedeals

Data on different commercial offers for gene-sized DNA
4
star
40

igem-registry-downloader

Download the full iGEM database of parts
Python
3
star
41

saboteurs

๐Ÿ’ฃ Identify elements impairing success accross group experiments.
Python
3
star
42

Ediacara

Python package for interpreting sequencing data of assembled DNA constructs (plasmids)
Python
3
star
43

DnaWeaver-online

A web app for DNA Weaver
Vue
2
star
44

trellab

Thin layer on top of pytrello for automating Trello organization-based tasks
Python
2
star
45

Polymera

Polymera is a Python package for representing ambiguous sequences written with complement alphabets.
Python
2
star
46

OT2Metclo

MSc project
Python
2
star
47

GeneAlloy

Python
2
star
48

Edinburgh-Genome-Foundry.github.io

๐Ÿ“š The EGF Software Website
HTML
1
star
49

tatapov_data

Data for the EGF Tatapov package
1
star
50

egf-codons-website

๐Ÿ“š Sources of the EGF Codons website, written with Vue.js
Vue
1
star
51

Overhang

Compendium of overhangs
Python
1
star
52

Plasmid_assessor

Plasmid assessment for Golden Gate cloning
Python
1
star
53

ImagesAnnotator

Simple web app to annotate images by hand - built with Vue.js
JavaScript
1
star
54

EGF_Docker_Jupyter

Docker Jupyter images with (almost) all EGF packages
Dockerfile
1
star