• Stars
    star
    248
  • Rank 162,588 (Top 4 %)
  • Language
    MATLAB
  • License
    GNU General Publi...
  • Created over 9 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Complete Matlab pipeline for large scale calcium imaging data analysis

Join the chat at https://gitter.im/epnev/ca_source_extraction

CaImAn-MATLAB

A Computational toolbox for large scale Calcium Imaging data Analysis. The code implements the CNMF algorithm [1] for simultaneous source extraction and spike inference from large scale calcium imaging movies. Many more features are included (see below). The code is suitable for the analysis of somatic imaging data. Improved implementation for the analysis of dendritic/axonal imaging data will be added in the future.

Features and methods included

New: Renaming to CaImAn-MATLAB

We moved the code into the Flatiron Institute github account and renamed the repository to CaImAn-MATLAB to bring it more in touch with the CaImAn Python package. Everything else is the same. The old link https://github.com/epnev/ca_source_extraction redirects here.

Citation

If you use this code please cite the corresponding papers where original methods appeared (see References below), as well as:

[1] Giovannucci A., Friedrich J., Gunn P., Kalfon J., Koay S.A., Taxidis J., Najafi F., Gauthier J.L., Zhou P., Tank D.W., Chklovskii D.B., Pnevmatikakis E.A. (2018). CaImAn: An open source tool for scalable Calcium Imaging data Analysis. bioarXiv preprint. [paper]

References

The following references provide the theoretical background and original code for the included methods.

Deconvolution and demixing of calcium imaging data

[1] Pnevmatikakis, E.A., Soudry, D., Gao, Y., Machado, T., Merel, J., ... & Paninski, L. (2016). Simultaneous denoising, deconvolution, and demixing of calcium imaging data. Neuron 89(2):285-299, [paper].

[2] Pnevmatikakis, E.A., Gao, Y., Soudry, D., Pfau, D., Lacefield, C., ... & Paninski, L. (2014). A structured matrix factorization framework for large scale calcium imaging data analysis. arXiv preprint arXiv:1409.2903. [paper].

[3] Friedrich J. and Paninski L. Fast active set methods for online spike inference from calcium imaging. NIPS, 29:1984-1992, 2016. [paper], [Github repository - Python], [Github repository - MATLAB].

[4] Pnevmatikakis, E. A., Merel, J., Pakman, A., & Paninski, L. Bayesian spike inference from calcium imaging data. In Signals, Systems and Computers, 2013 Asilomar Conference on (pp. 349-353). IEEE, 2013. [paper], [Github repository - MATLAB].

Motion Correction

[5] Pnevmatikakis, E.A., and Giovannucci A. (2017). NoRMCorre: An online algorithm for piecewise rigid motion correction of calcium imaging data. Journal of Neuroscience Methods, 291:83-92 [paper], [Github repository - MATLAB].

Code description

The best way to start is by looking at the various demos.

  • demo_script.m: A simple demo with a small dataset included in the repo to display the notation and basic operations
  • demo_script_class.m: Replicates the demo_script.m file in a cleaner way using a CNMF object.
  • demo_patches.m: A larger demo displaying the process of memory mapping and spliting the field of view in patches to be processed in parallel and then combined.
  • demo_patches_class.m: Similar to demo_patches.m but using the CNMF object.
  • run_pipeline.m: Demo for the complete pipeline of motion correction, source separation and spike extraction for large datasets. More details about the pipeline can be found here.
  • 3D/demo_3D.m: Demo for processing of 3D volumetric imaging data.

Python

A complete analysis Python pipeline including motion correction, source extraction and activity deconvolution is performed through the package CaImAn. This package also includes method for online processing of calcium imaging data and elements of behavioral analysis in head fixed mice.

Usage and Documentation

Check the demo scripts and the wiki to get started.

Dependencies

The following matlab toolboxes are needed for the default parameter settings:

  • Statistics and Machine Learning Toolbox
  • Image processing toolbox

Depending on the settings the following toolboxes may also be required

  • Neural networks toolbox (required for component classifier)
  • Signal processing toolbox (recommended but not required)
  • Parallel computing toolbox (recommended for large datasets but not required)
  • Optimization toolbox (not required)

Depending on the settings the following packages may also be required

  • The CVX library which can be downloaded from http://cvxr.com/cvx/download/ (after unpacking CVX open Matlab and run cvx_setup from inside the CVX directory to properly install and add CVX to the Matlab path). CVX is no longer required.
  • SPGL1 package from https://github.com/mpf/spgl1 (for solving constrained_foopsi using SPGL1)

Developers

This package is mainly developed and maintained by Eftychios A. Pnevmatikakis (Flatiron Institute, Simons Foundation) with help from a lot of contributors.

Acknowledgements

Special thanks to the following people for letting us use their datasets for our various demo files:

  • Weijian Yang, Darcy Peterka, Rafael Yuste, Columbia University
  • Sue Ann Koay, David Tank, Princeton University
  • Diego Pacheco Pinedo, Mala Murthy, Princeton University
  • Clay Lacefied, Randy Bruno, Columbia University

Questions, comments, issues

Please use the gitter chat room (use the button above) for questions and comments and create an issue for any bugs you might encounter.

License

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.

More Repositories

1

CaImAn

Computational toolbox for large scale Calcium Imaging Analysis, including movie handling, motion correction, source extraction, spike deconvolution and result visualization.
Python
620
star
2

DeepFRI

Deep functional residue identification
Python
291
star
3

finufft

Non-uniform fast Fourier transform library of types 1,2,3 in dimensions 1,2,3
C++
280
star
4

NoRMCorre

Matlab routines for online non-rigid motion correction of calcium imaging data
MATLAB
139
star
5

deepblast

Neural Networks for Protein Sequence Alignment
Python
107
star
6

FMM3D

Flatiron Institute Fast Multipole Libraries --- This codebase is a set of libraries to compute N-body interactions governed by the Laplace and Helmholtz equations, to a specified precision, in three dimensions, on a multi-core shared-memory machine.
Fortran
90
star
7

cufinufft

Nonuniform fast Fourier transforms of types 1 and 2, in 1D, 2D, and 3D, on the GPU
Cuda
83
star
8

jax-finufft

JAX bindings to the Flatiron Institute Non-uniform Fast Fourier Transform (FINUFFT) library
Python
77
star
9

sparse_dot

Python wrapper for Intel Math Kernel Library (MKL) matrix multiplication
Python
73
star
10

nemos

NEural MOdelS, a statistical modeling framework for neuroscience.
Python
73
star
11

sciware

Learning materials for scientific software development
HTML
61
star
12

mountainsort

Spike sorting software
C++
48
star
13

inferelator

Task-based gene regulatory network inference using single-cell or bulk gene expression data conditioned on a prior network.
Python
45
star
14

mountainlab-js

MountainLab is data processing, sharing and visualization software for scientists. It is built around MountainSort, spike sorting software, but is designed to be more generally applicable.
JavaScript
43
star
15

bayes-kit

Bayesian inference and posterior analysis for Python
Python
41
star
16

neurosift

Browser-based NWB visualization and DANDI exploration
TypeScript
40
star
17

online_psp

A collection of computationally efficient algorithms for online subspace learning and principal component analysis
Python
38
star
18

spikeforest2

SpikeForest -- spike sorting analysis for website -- version 2
Python
34
star
19

mcmc-monitor

Monitor MCMC runs in the browser
TypeScript
34
star
20

nixpack

nix+spack = nixpack (spanix?)
Python
34
star
21

figurl

Shareable, interactive scientific figures in the cloud
Python
31
star
22

disBatch

Tool to distribute a list of computational tasks over a pool of compute resources. The pool can grow or shrink.
Python
31
star
23

mountainsort5

MountainSort spike sorting algorithm, version 5
Python
30
star
24

ironclust

Spike sorting software being developed at Flatiron Institute, based on JRCLUST (Janelia Rocket Cluster)
Jupyter Notebook
28
star
25

st_gridnet

A Python implementation of the model described in our publication "A convolutional neural network for common-coordinate registration of high-resolution histology images" developed principally for applications to registration of spatial transcriptomics image data.
Python
25
star
26

mountainsort_examples

Examples of using MountainSort spike sorting software.
Jupyter Notebook
25
star
27

SkellySim

Hydrodynamic Cytoskeleton Simulator
C++
23
star
28

catvae

Categorical Variational Autoencoders
Jupyter Notebook
22
star
29

spikeforest

Spike sorting benchmarking system
Python
22
star
30

online_psp_matlab

Benchmark of online PCA algorithms
MATLAB
22
star
31

fmm2d

This codebase is a set of libraries to compute N-body interactions governed by the Laplace and Helmholtz equations, to a specified precision, in two dimensions, on a multi-core shared-memory machine.
Fortran
21
star
32

nifty-ls

A fast Lomb-Scargle periodogram. It's nifty, and uses a NUFFT!
Python
20
star
33

slurm-prometheus-exporter

Prometheus exporter for slurm job/node data
Haskell
20
star
34

kachery-cloud

Python
20
star
35

dendro

Analyze neuroscience data in the cloud
TypeScript
19
star
36

comptools24

Computational Tools for PDEs with Complicated Geometries and Interfaces workshop materials, June 10-14, 2024.
MATLAB
18
star
37

aLENS

a Living ENsemble Simulator -- a lens to help you watch biophysics
C++
17
star
38

spikeforest_old

SpikeForest -- spike sorting analysis for website
Python
16
star
39

baobzi

An adaptive fast function approximator based on tree search
C++
14
star
40

isosplit5

ISO-SPLIT clustering (stand-alone version)
C++
14
star
41

bio-sfa

Code for reproducing the experiment from the paper "A biologically plausible neural network for Slow Feature Analysis"
Jupyter Notebook
12
star
42

pytorch-finufft

Pytorch wrappers for the FINUFFT library
Python
11
star
43

sf_benchmarks

Special function benchmarks
C++
10
star
44

mantis

Manifold-tiling Localized Receptive Fields are Optimal in Similarity-preserving Neural Networks
Python
10
star
45

flathub

A simple elasticsearch frontend for serving astrophysical simulation catalog data
Haskell
10
star
46

ccq-software-build

CCQ software build scripts
Shell
10
star
47

public_www

Basic templates for individual user pages in the public/_www directory
SCSS
9
star
48

inferelator-prior

Gene regulatory network inference using DNA-binding motifs and chromatin accessibility data.
Python
9
star
49

Ensemble-reweighting-using-Cryo-EM-particles

Jupyter Notebook
9
star
50

boxcodes3d

This repository contains box codes for evaluating volume potentials for Laplace, Helmholtz, Maxwell, and Stokes
Fortran
9
star
51

bio-nica

Code for reproducing the experiments in the paper "Bio-NICA: A biologically inspired single-layer network for Nonnegative Independent Component Analysis"
Jupyter Notebook
9
star
52

cppdlr

Discrete Lehmann representation of imaginary time Green's functions
C++
8
star
53

nix-modules

nixpkgs-based module builds of cluster packages
Nix
8
star
54

bio-cca

Code accompanying the paper "A biologically plausible neural network for multi-channel Canonical Correlation Analysis"
Jupyter Notebook
8
star
55

reactopya

React components paired with Python classes for backend computation
JavaScript
8
star
56

PointCloud_Regression

Point Cloud regression with new algebraic representation on ModelNet dataset (ICCV 2023)
Python
8
star
57

clair

Clair (Clang Introspection and Reflection) tools. A set of clang tools developed for Flatiron/CCQ
C++
8
star
58

adapol

Adaptive Pole Fitting for Quantum Many-Body Physics
Jupyter Notebook
8
star
59

DLR_DMFT_scripts

Python
7
star
60

kbucket

System for sharing data for scientific research
JavaScript
7
star
61

ccn-template

Template repository for CCN software projects
Python
7
star
62

neutorch

neuron segmentation and synapse detection using PyTorch
Python
7
star
63

SlurmUtil

slurm monitoring tools and interface
Python
7
star
64

q2-matchmaker

A qiime2 plugin for case-control differential abundance analysis
Jupyter Notebook
6
star
65

mountainlab

Scientific data analysis, sharing, and visualization
C++
6
star
66

least_absolute_regression

Least absolute error regression implemented using Linear Programming, primarily to illustrate repository structure conventions.
Jupyter Notebook
6
star
67

neuropixels-data-sep-2020

Example neuropixels datasets for purposes of developing spike sorting algorithms
Python
6
star
68

ephys-viz

Neurophysiology visualization components deployable to the notebook, web, or desktop
JavaScript
6
star
69

inverse-obstacle-scattering2d

This repository contains codes for solving inverse obstacle scattering problems to recover the shape of the obstacle for sound soft, sound hard, and penetrable objects
Fortran
6
star
70

cryoSBI

Python
6
star
71

kachery-p2p

Peer-to-peer content-addressable file sharing using kachery
TypeScript
5
star
72

gp-shootout

Benchmark and compare large-scale Gaussian process regression methods in 1D, 2D, and 3D, from MATLAB
MATLAB
5
star
73

ccn-software-fens-2024

Materials for CCN software workshop at FENS 2024
Python
5
star
74

looking_glass_3d_test

Test of displaying 3d genomic data using webgl in a "Looking glass" holographic display
5
star
75

caiman_central

Central hub for resources related to Caiman calcium imaging analysis package.
Jupyter Notebook
5
star
76

labbox-ephys

TypeScript
4
star
77

FREYA

Canine breast cancer analysis pipeline
Jupyter Notebook
4
star
78

spikesortercomparison

Notes on metrics and datasets for community spike sorting comparison and validation
TeX
4
star
79

hither

Run batches of Python functions in containers and on remote servers
Python
4
star
80

ILCnetworks

Gene regulatory networks for intestinal innate lymphoid cells
Jupyter Notebook
4
star
81

Fermi_surface_visualization

Interactive visualization of Fermi surfaces representing electron energies and velocities in crystal structures.
Python
4
star
82

flatiron-coffee

Meet your colleagues across the Institute
Python
4
star
83

spike-front

Front end tool for benchmarking spike sorting algorithms.
JavaScript
4
star
84

ccm_widgets

Reactopya widgets of relevance to the Center for Computational Mathematics, Flatiron Institute
JavaScript
4
star
85

Cryo-EM-Heterogeneity-Challenge-1

The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge
Jupyter Notebook
4
star
86

kvsstcp

Key value storage server. Light weight (single python module with no non-standard module dependencies). Simple protocol minimizes effort needed to write a client (example for C and FORTRAN provided).
Python
4
star
87

bio-lda

Code accompanying the paper "A linear discriminant analysis model of imbalanced associative learning in the mushroom body compartment"
Jupyter Notebook
3
star
88

Baobzi.jl

Wrapper library for Baobzi interpolator library
Julia
3
star
89

ftk

Factorization of the translation kernel for fast rigid image alignment
Python
3
star
90

sciware-testing-cpp

C++
3
star
91

binary_classification_metrics

Visualizations and other code for exploring binary classification metrics.
Jupyter Notebook
3
star
92

inverse-volume-scattering2d

This repository contains beta MATLAB codes for data generation of forward medium problems using a hybrid HPS-HBS solver, and an inverse medium solver
MATLAB
3
star
93

stan-playground

Run Stan models in the browser
TypeScript
3
star
94

qt-mountainview

Visualization of spike sorting experiments to be used as a plugin package to mountainlab-js
C++
3
star
95

fgt2d

This repository contains codes for evaluating sums and integrals of gaussians in two dimensions
Fortran
3
star
96

spikeforest_recordings

Ephys recordings for the SpikeForest project
Python
3
star
97

quantum_rosetta_private

Playing with quantum computing algorithms for protein and peptide design and structure prediction, with interfaces for the Rosetta software suite. (Repository created by Vikram K. Mulligan, [email protected]).
C#
3
star
98

mcmc-monitor-old

Monitor MCMC runs
Python
3
star
99

h5_to_json

Represent hdf5 files via json, separating out large binary content into individual files
Python
3
star
100

mountainsort_examples-old

Examples for getting started with MountainSort spike sorting software
MATLAB
2
star