• Stars
    star
    152
  • Rank 244,685 (Top 5 %)
  • Language
    Python
  • License
    MIT License
  • Created almost 11 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Unified, simple data access python library for data & facts about C. elegans anatomy

Build Status Docs Coverage Status

owmeta

pyow_in_overview

A data access layer in Python which integrates disparate structures and representations for C. elegans anatomy and physiology. Enables a simple Python API for asking various questions about the cells of the C. elegans and enabling data sharing for the purpose of building a data-to-model pipeline for the OpenWorm project.

Overview

owmeta_logo

The data and models required to simulate C. elegans are highly heterogeneous. Consequently, from a software perspective, a variety of underlying representations are needed to store different aspects of the relevant anatomy and physiology. For example, a NetworkX representation of the connectome as a complex graph enables questions to be asked about nearest neighbors of a given neuron. An RDF semantic graph representation is useful for reading and writing annotations about multiple aspects of a neuron, such as what papers have been written about it, properties it may have such as ion channels and neurotransmitter receptors, etc. A NeuroML representation is useful for answering questions about model morphology and simulation parameters. A Blender representation is a full 3D shape definition that can be used for calculations in 3D space.

The diversity of underlying representations required for OpenWorm presents a challenge for data integration and consolidation. owmeta solves this challenge with a unified data access layer whereby different representations are encapsulated into an abstract view. This allows the user to work with objects related to the biological reality of the worm, and forget about which representation is being used under the hood. The worm itself has a unified sense of neurons, networks, muscles, ion channels, etc. and so should our code.

Relationship to ChannelWorm2

ChannelWorm2 is the sub-project of OpenWorm which houses ion channel models. In the future, we expect ChannelWorm2 to be a "consumer" of owmeta. An owmeta database will house physical models, the digitized plots they are derived from (there is a Plot type in owmeta), and provide code to put those models into enumerated formats along with auxiliary files or comments. However, because these projects were not developed sequentially, there is currently some overlap in functionality, and owmeta itself houses a fairly substantial amount of physiological information about C. elegans. Ultimately, the pure core of owmeta, which is meant to be a data framework for storing metadata and provenance (i.e. parameters and trajectories associated with simulations), will be separated out into standalone functionality.

Versioning data as code

A library that attempts to reliably expose dynamic data can often be broken because the underlying data sets that define it change over time. This is because data changes can cause queries to return different answers than before, causing unpredictable behavior.

As such, to create a stable foundational library for others to reuse, the version of the owmeta library guarantees the user a specific version of the data behind that library. In addition, unit tests are used to ensure basic sanity checks on data are maintained. As data are improved, the maintainers of the library can perform appropriate regression tests prior to each new release to guarantee stability.

Installation

See INSTALL.md

Quickstart

To get started, you'll need to connect to a database. The OpenWorm owmeta "project" is currently hosted at https://github.com/openworm/OpenWormData.git. This project holds a working-copy of the database. You can retrieve it by executing the following command line after owmeta installation:

owm clone https://github.com/openworm/OpenWormData.git --branch owmeta

This command should create a directory .owm in your current working directory. Then, in Python, from the same working directory:

>>> from owmeta_core.command import OWM
>>> conn = OWM().connect()

This creates a connection to the project stored under the .owm directory.

Then you can try out a few things:

# Make the context
>>> from owmeta_core.context import Context
>>> ctx = conn(Context)(ident='http://openworm.org/data')

# Grab the representation of the neuronal network
>>> from owmeta.worm import Worm
>>> net = ctx.stored(Worm).query().neuron_network()

# Grab a specific neuron
>>> from owmeta.neuron import Neuron
>>> aval = ctx.stored(Neuron).query(name='AVAL')

# Get the neuron's type
>>> aval.type.one()
'interneuron'

# Count how many connections come from AVAL
>>> aval.connection.count('pre')
86

More examples

Return information about individual neurons:

>>> aval.name()
'AVAL'

# List all known receptors
>>> sorted(aval.receptors())
['GGR-3', 'GLR-1', ... 'NPR-4', 'UNC-8']

# Show how many chemical synapses go in and out of AVAL
>>> aval.connection.count('either', syntype='send')
105

Return the list of all neurons:

>>> len(set(net.neuron_names()))
302
>>> sorted(net.neuron_names())
['ADAL', 'ADAR', ... 'VD8', 'VD9']

Return a set of all muscles:

>>> muscles = ctx.stored(Worm).query().muscles()
>>> len(muscles)
158

Because the ultimate aim of OpenWorm is to be a platform for biological research, the physiological data in owmeta should be uncontroversial and well supported by evidence. Using the Evidence type, it is possible to link data and models to corresponding articles from peer-reviewed literature:

>>> from owmeta.document import Document
>>> from owmeta.evidence import Evidence

# Make a context for evidence (i.e., statements about other groups of statements)
>>> evctx = conn(Context)(ident='http://example.org/evidence/context')

# Make a context for defining domain knowledge
>>> dctx = evctx(Context)(ident='http://example.org/data/context')
>>> doc = evctx(Document)(key="Sulston83", author='Sulston et al.', date='1983')
>>> e = evctx(Evidence)(key="Sulston83", reference=doc)
>>> avdl = dctx(Neuron)(name="AVDL")
>>> avdl.lineageName("AB alaaapalr")
owmeta_core.statement.Statement(subj=Neuron(ident=rdflib.term.URIRef('http://data.openworm.org/sci/bio/Neuron#AVDL')), prop=owmeta.cell.Cell_lineageName(owner=Neuron(ident=rdflib.term.URIRef('http://data.openworm.org/sci/bio/Neuron#AVDL'))), obj=owmeta_core.dataobject_property.ContextualizedPropertyValue(rdflib.term.Literal('AB alaaapalr')), context=owmeta_core.context.Context(ident="http://example.org/data/context"))
>>> e.supports(dctx.rdf_object)
owmeta_core.statement.Statement(subj=Evidence(ident=rdflib.term.URIRef('http://data.openworm.org/Evidence#Sulston83')), prop=owmeta.evidence.Evidence_supports(owner=Evidence(ident=rdflib.term.URIRef('http://data.openworm.org/Evidence#Sulston83'))), obj=ContextDataObject(ident=rdflib.term.URIRef('http://example.org/data/context')), context=owmeta_core.context.Context(ident="http://example.org/evidence/context"))
>>> with conn.transaction_manager:
...     dctx.save_context()
...     evctx.save_context()

Retrieve evidence:

>>> doc = evctx.stored(Document)(author='Sulston et al.', date='1983')
>>> e0 = evctx.stored(Evidence)(reference=doc)
>>> supported_ctx = e0.supports()

# is the neuron's presence asserted?
>>> dctx.identifier == supported_ctx.identifier
True

Query for neurons in C. elegans:

>>> from owmeta.network import Network

# The default Worm() is for C. elegans
>>> with ctx.stored(Worm, Neuron, Network) as cctx:
...     w = cctx.Worm()
...     net = cctx.Network()
...     w.neuron_network(net)
owmeta_core.statement.Statement(subj=Worm(ident=rdflib.term.URIRef('http://data.openworm.org/sci/bio/Worm#a8020ed8519038a6bbc98f1792c46c97b')), prop=owmeta.worm.Worm_neuron_network(owner=Worm(ident=rdflib.term.URIRef('http://data.openworm.org/sci/bio/Worm#a8020ed8519038a6bbc98f1792c46c97b'))), obj=Network(ident=rdflib.term.URIRef('http://data.openworm.org/sci/bio/Network#a5859bb1e51537f60e506c283401fcd84')), context=owmeta_core.context.QueryContext(ident="http://openworm.org/data"))

...     neur = cctx.Neuron.query()
...     net.neuron(neur)
...     neur.count()
302

See what neurons express a given neuropeptide:

>>> n = ctx.stored(Neuron).query()
>>> n.neuropeptide("INS-26")
owmeta_core.statement.Statement(subj=Neuron(ident=rdflib.term.Variable('aNeuron_...')), prop=owmeta.neuron.Neuron_neuropeptide(owner=Neuron(ident=rdflib.term.Variable('aNeuron_...'))), obj=owmeta_core.dataobject_property.ContextualizedPropertyValue(rdflib.term.Literal('INS-26')), context=owmeta_core.context.QueryContext(ident="http://openworm.org/data"))

>>> sorted(x.name() for x in n.load())
['ASEL', 'ASER', 'ASIL', 'ASIR']

Get direct access to the RDFLib graph:

>>> conn.rdf.query("SELECT ?y WHERE { ?x rdf:type ?y }")
<rdflib.plugins.sparql.processor.SPARQLResult object at ...>

Modeling data

As described above, ultimately, ion channel models will be part of the ChannelWorm2 repository. As the project evolves, other models, such as for reproduction and development, may be housed in their own repositories. But for the time being, the owmeta repository contains specific models as well. These models will eventually be transferred to an appropriate and independent data repository within the OpenWorm suite of tools.

# Get data for a subtype of voltage-gated potassium channels
>>> from owmeta.channel import Channel
>>> kv1 = ctx(Channel)(subfamily='Kv1.1')
>>> kv1.models()

The same type of operation can be used to obtain the experimental data a given model was derived from.

# Get experiment(s) that back up the data model
>> some_model = mods[0]
>> some_model.references.get()

Finally, when you're done accessing the database, be sure to disconnect from it:

>>> conn.disconnect()

More examples can be found in the owmeta-core documentation and in the ./examples directory of the owmeta Git repository.

Documentation

Further documentation is available online.

Contributing

We happily welcome pull requests and bug reports. If, you are not sure how you can contribute, fill out this (short) form, and you'll receive an invite to our Slack chat where you can initiate more in-depth conversations.

Questions/Concerns?

You can ask questions, leave bug reports, or propose features on our issue tracker.

More Repositories

1

OpenWorm

Repository for the main Dockerfile with the OpenWorm software stack and project-wide issues
Python
2,659
star
2

sibernetic

This is a C++/OpenCL implementation of the PCISPH algorithm supplemented with a set of biomechanics related features applied to C. elegans locomotion
C
354
star
3

org.geppetto

Geppetto is an open-source platform to build web-based applications to visualize and simulate neuroscience data and models.
Python
209
star
4

CElegansNeuroML

NeuroML based C elegans model, contained in a neuroConstruct project, as well as c302
Jupyter Notebook
134
star
5

c302

The c302 framework
Python
68
star
6

wormbrowser

The Worm Browser -- a 3D browser of the cellular anatomy of the c. elegans
JavaScript
49
star
7

open-worm-analysis-toolbox

A testing pipeline that allows us to run a behavioural phenotyping of our virtual worm running the same test statistics the Schafer Lab used on their worm data.
Python
48
star
8

muscle_model

Model of C elegans body wall muscle based on Boyle & Cohen 2008
Jupyter Notebook
47
star
9

openworm_docs

Documentation for OpenWorm, i.e. docs.openworm.org
Python
44
star
10

hodgkin_huxley_tutorial

A repository containing code for a number of tutorials on the Hodgkin Huxley model, including an interactive Jupyter notebook
Python
36
star
11

bionet

Artificial biological neural network
C++
31
star
12

CyberElegans

Neuromechanical model of C. Elegans
C++
31
star
13

org.geppetto.frontend

Geppetto Java frontend bundle
Java
29
star
14

robots

C. elegans robots
C
24
star
15

openworm.github.io

OpenWorm main website
HTML
24
star
16

org.geppetto.core

Geppetto core bundle
Java
23
star
17

openwormbrowser-ios

OpenWorm Browser for iOS, based on the open-3d-viewer, which was based on Google Body Browser
Objective-C
22
star
18

Blender2NeuroML

Conversion script to bring neuron models created in Blender into NeuroML format
Python
17
star
19

pygeppetto

Python Library to create, load, edit and save a Geppetto Model
Python
14
star
20

ChannelWorm

Tools and curated datasets to build quantitative models of C. elegans ion channels
JavaScript
12
star
21

tracker-commons

Compilation of information and code bases related to open-source trackers for C. elegans
Scala
11
star
22

ChannelWorm2

Tools and curated datasets to build quantitative models of C. elegans ion channels
Jupyter Notebook
10
star
23

org.geppetto.model.neuroml

NeuroML Model Bundle for Geppetto
Java
9
star
24

org.geppetto.simulation

Generic simulation bundle for Geppetto
Java
9
star
25

org.geppetto.solver.sph

PCI SPH Solver bundle for Geppetto
Java
9
star
26

skeletonExtraction

Transforms Sibernetic output into a COLLADA animation
C++
7
star
27

WormWorx

C. elegans simulator
C
6
star
28

behavioral_syntax

behavioral syntax analysis based on the paper of Andre Brown
Jupyter Notebook
5
star
29

movement_cloud

Movement Analysis on the cloud
Python
5
star
30

SegWorm

SegWorm is Matlab code from Dr. Eviatar Yemini built as part of the WormBehavior database (http://wormbehavior.mrc-lmb.cam.ac.uk/)
MATLAB
5
star
31

org.geppetto.frontend.jupyter

Geppetto Jupyter Notebook Extension
Python
5
star
32

sibernetic_NEURON

Interface between Sibernetic and NEURON simulator
Python
5
star
33

org.geppetto.sibernetic

PCI SPH Model Bundle for Geppetto
Java
5
star
34

org.geppetto.recording

Python project allowing to create a recording for Geppetto
Python
5
star
35

neuronal-analysis

Tools to produce, analyse and compare both simulated and recorded neuronal datasets
Jupyter Notebook
4
star
36

OpenData

Repo for Google Summer of Code Open Data Project 2019
Jupyter Notebook
4
star
37

org.geppetto.docs

Repository to host Geppetto documentation
Python
4
star
38

org.geppetto.simulator.sph

SPH Simulator bundle for Geppetto
Java
4
star
39

sibernetic_config_gen

Generates starting scene configuration for the Sibernetic simulation engine
Python
4
star
40

geppetto-client

The web client of a Geppetto application
JavaScript
4
star
41

org.geppetto.simulator.jlems

jLEMS based simulator for Geppetto
Java
3
star
42

YAROM

Yet Another RDF-Object Mapper
Python
3
star
43

OpenWormData

TeX
3
star
44

org.geppetto.model.swc

SWC Model Interpreter for Geppetto
Java
3
star
45

org.geppetto.model

Java
3
star
46

HeuristicWorm

C++
3
star
47

org.geppetto.maven

Maven repository for Geppetto
3
star
48

org.geppetto.simulator.external

Geppetto bundle to add support for external neuronal simulators (NEURON, etc.)
Python
3
star
49

owmeta-core

Core library for owmeta
Python
3
star
50

org.geppetto.model.nwb

Java
3
star
51

geppetto-application

Sample Geppetto application
JavaScript
3
star
52

WormsenseLab_ASH

Electrophysiological recordings of neuron ASH from Wormsense Lab
Jupyter Notebook
3
star
53

NeuroPAL

Some tests on using the NeuroPAL datasets
Jupyter Notebook
3
star
54

org.geppetto.samples

Sample simulations for Geppetto
JavaScript
3
star
55

pharyngeal_muscle_model

C. elegans pharyngeal muscle cell (pm3) model, capable of generating Ca2+ slow action potential (NEURON, NMODL).
C
3
star
56

tests

OpenWorm tests across various repos
Jupyter Notebook
3
star
57

org.wormsim.frontend

WormSim frontend bundle
JavaScript
3
star
58

owmeta-movement

DataSources and utilities for movement data in OpenWorm
Python
2
star
59

org.geppetto.persistence

Persistance bundle for Geppetto
Java
2
star
60

org.geppetto.templatebundle

Java
2
star
61

org.geppetto.datasources

Java
2
star
62

JohnsonMailler_MuscleModel

C. elegans muscle model from Johnson & Mailler 2015
HTML
2
star
63

org.geppetto.simulator.libroadrunner

libRoadRunner simulator bundle for Geppetto
Java
2
star
64

owmeta-modeldb

Python
2
star
65

openworm-scholar

A service providing enhanced notifications of published research in Open Worm
Python
2
star
66

org.geppetto.bower

Geppetto repository for packaged bower components
JavaScript
2
star
67

org.geppetto.testbackend

Java
2
star
68

owmeta-bundles

Bundles for owmeta
2
star
69

org.wormsim.bower

Wormsim repository for packaged bower UI components
2
star
70

owmeta-pytest-plugin

Pytest plugin for testing with owmeta-core
Python
2
star
71

worm-math-book

An online book focusing on the mathematical concepts and models of the OpenWorm project.
Jupyter Notebook
2
star
72

owmeta-sciunit

owmeta types for SciUnit and NeuronUnit
Python
2
star
73

org.geppetto.frontend.nodejs

Geppetto Node.js backend
JavaScript
2
star
74

jenkins

Groovy
1
star
75

multi-dev-sibernetic

Multi devices sibernetic engine
C++
1
star
76

.github

The OpenWorm organisation template repository.
Python
1
star
77

bt-gsoc-2019

End to end process of the Framework including a BitTorrent Client, Access Control and Data Integrity checks.
Python
1
star
78

Newsletter

A repository containing a newsletter with regular OpenWorm updates
1
star
79

owmeta-core-data

owmeta repository for the owmeta-core schema
1
star