• Stars
    star
    142
  • Rank 258,443 (Top 6 %)
  • Language
    MATLAB
  • License
    GNU General Publi...
  • Created almost 8 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Matlab routines for online non-rigid motion correction of calcium imaging data

Join the chat at https://gitter.im/epnev/ca_source_extraction

NoRMCorre: Non-Rigid Motion Correction

This package provides a Matlab implementation of the NoRMCorre algorithm [1], and can be used for online piecewise rigid motion correction of 2d (planar) or 3d (volumetric) calcium imaging data.

Citation

If you find this package useful please cite the companion paper [1]:

@article{pnevmatikakis2017normcorre,
  title={NoRMCorre: An online algorithm for piecewise rigid motion correction of calcium imaging data},
  author={Pnevmatikakis, Eftychios A and Giovannucci, Andrea},
  journal={Journal of neuroscience methods},
  volume={291},
  pages={83--94},
  year={2017},
  publisher={Elsevier}
}

Synopsis

The algorithm operates by splitting the field of view into a set of overlapping patches. For each patch and each frame a rigid translation is estimated by aligning the patch against a template using an efficient, FFT based, algorithm for subpixel registration [2]. The estimated set of translations is further upsampled to a finer resolution to create a smooth motion field that is applied to a set of smaller overlapping patches. Extra care is taken to avoid smearing caused by interpolating overlapping patches with drastically different motion vectors. The registered frame is used to update the template in an online fashion by calculating a running/mean of past registered frames. The pipeline is summarized in the figure below.

Alt text

Code details

See the function demo.m for an example of the code. The algorithm is implemented in the function normcorre.m. If you have access to the parallel computing toolbox, then the function normcorre_batch.m can offer speed gains by enabling within mini-batch parallel processing. The user gives a dataset (either as 3D or 4D tensor loaded in RAM or memory mapped, or a pointer to a .tiff stack or .hdf5 file), and a parameters struct options. Optionally, an initial template can also be given. The algorithm can also be used for motion correction of 1p micro-endoscopic data, by estimating the shifts on high pass spatially filtered version of the data. See the script demo_1p.m for an example.

The algorithm can also be ran using the MotionCorrection object. See demo_mc_class.m for an example on how to use the object for 2p and 1p data.

The options struct can be set either manually or by using the function NoRMCorreSetParms.m. Some parameters of the options struct are the following:

Parameter name Description
d1,d2,d3 dimensions of field of view
grid_size size of non-overlapping portion of each patch the grid in each direction (x-y-z)
overlap_pre size of overlapping region in each direction before upsampling
mot_uf upsampling factor for smoothing and refinement of motion field
overlap_post size of overlapping region in each direction after upsampling
max_shift maximum allowed shift for rigid translation
max_dev maximum deviation of each patch from estimated rigid translation
upd_template update the template online after registering some frames
bin_width length of bin over which the registered frames are averaged to update the template
init_batch number of frames to be taken for computing initial template
iter number of times to go over the dataset
output_type type of output registered file
phase_flag flag for using phase correlation
correct_bidir check for offset due to bidirectional scanning (default: true)

The performance of registration can be evaluated using the function motion_metrics.m. The function simply computes the correlation coefficient of each (registered) frame, with the mean (registered) frame across time, the mean registered frame, and its crispness.

Developers

Eftychios A. Pnevmatikakis, Flatiron Institure, Simons Foundation

External packages

This package includes functions from the following packages

Integrations

This package will be integrated with the Matlab code for source extraction and deconvolution using CNMF.

A python version of this algorithm developed from Andrea A. Giovannuci is included as part of the CaImAn package that provides a complete pipeline for calcium imaging data pre-processing.

Although the two implementations give almost identical results for the same input file, there are some slight differences in the way they are called and their capabilities. These differences are highlighted here.

More details, contact information, and citing NoRMCorre

Check the wiki for more details and some frequently asked questions.

Please use the gitter chat room for questions and comments, and create an issue for any bugs you might encounter.

If you find this package useful please cite the following paper:

Eftychios A. Pnevmatikakis and Andrea Giovannucci, NoRMCorre: An online algorithm for piecewise rigid motion correction of calcium imaging data, Journal of Neuroscience Methods, vol. 291, pp 83-94, 2017; doi: https://doi.org/10.1016/j.jneumeth.2017.07.031

Acknowledgements

The 2p example dataset is kindly provided from Andrea Giovannucci, taken at Wang lab (Princeton University). The 1p example dataset is kindly provided by Daniel Aharoni and Peyman Golshani (UCLA, Miniscope project).

References

[1] Eftychios A. Pnevmatikakis and Andrea Giovannucci, NoRMCorre: An online algorithm for piecewise rigid motion correction of calcium imaging data, Journal of Neuroscience Methods, vol. 291, pp 83-94, 2017; doi: https://doi.org/10.1016/j.jneumeth.2017.07.031

[2] Guizar-Sicairos, M., Thurman, S. T., & Fienup, J. R. (2008). Efficient subpixel image registration algorithms. Optics letters, 33(2), 156-158. Matlab implementation available here.

More Repositories

1

CaImAn

Computational toolbox for large scale Calcium Imaging Analysis, including movie handling, motion correction, source extraction, spike deconvolution and result visualization.
Python
620
star
2

DeepFRI

Deep functional residue identification
Python
300
star
3

finufft

Non-uniform fast Fourier transform library of types 1,2,3 in dimensions 1,2,3
C++
293
star
4

CaImAn-MATLAB

Complete Matlab pipeline for large scale calcium imaging data analysis
MATLAB
248
star
5

deepblast

Neural Networks for Protein Sequence Alignment
Python
110
star
6

FMM3D

Flatiron Institute Fast Multipole Libraries --- This codebase is a set of libraries to compute N-body interactions governed by the Laplace and Helmholtz equations, to a specified precision, in three dimensions, on a multi-core shared-memory machine.
Fortran
96
star
7

cufinufft

Nonuniform fast Fourier transforms of types 1 and 2, in 1D, 2D, and 3D, on the GPU
Cuda
83
star
8

nemos

NEural MOdelS, a statistical modeling framework for neuroscience.
Python
78
star
9

jax-finufft

JAX bindings to the Flatiron Institute Non-uniform Fast Fourier Transform (FINUFFT) library
Python
77
star
10

sparse_dot

Python wrapper for Intel Math Kernel Library (MKL) matrix multiplication
Python
73
star
11

sciware

Learning materials for scientific software development
HTML
61
star
12

mountainsort

Spike sorting software
C++
48
star
13

inferelator

Task-based gene regulatory network inference using single-cell or bulk gene expression data conditioned on a prior network.
Python
46
star
14

mountainlab-js

MountainLab is data processing, sharing and visualization software for scientists. It is built around MountainSort, spike sorting software, but is designed to be more generally applicable.
JavaScript
43
star
15

bayes-kit

Bayesian inference and posterior analysis for Python
Python
41
star
16

neurosift

Browser-based NWB visualization and DANDI exploration
TypeScript
40
star
17

online_psp

A collection of computationally efficient algorithms for online subspace learning and principal component analysis
Python
38
star
18

spikeforest2

SpikeForest -- spike sorting analysis for website -- version 2
Python
34
star
19

mcmc-monitor

Monitor MCMC runs in the browser
TypeScript
34
star
20

nixpack

nix+spack = nixpack (spanix?)
Python
34
star
21

nifty-ls

A fast Lomb-Scargle periodogram. It's nifty, and uses a NUFFT!
Python
33
star
22

figurl

Shareable, interactive scientific figures in the cloud
Python
31
star
23

disBatch

Tool to distribute a list of computational tasks over a pool of compute resources. The pool can grow or shrink.
Python
31
star
24

mountainsort5

MountainSort spike sorting algorithm, version 5
Python
30
star
25

ironclust

Spike sorting software being developed at Flatiron Institute, based on JRCLUST (Janelia Rocket Cluster)
Jupyter Notebook
28
star
26

st_gridnet

A Python implementation of the model described in our publication "A convolutional neural network for common-coordinate registration of high-resolution histology images" developed principally for applications to registration of spatial transcriptomics image data.
Python
25
star
27

mountainsort_examples

Examples of using MountainSort spike sorting software.
Jupyter Notebook
25
star
28

SkellySim

Hydrodynamic Cytoskeleton Simulator
C++
23
star
29

fmm2d

This codebase is a set of libraries to compute N-body interactions governed by the Laplace and Helmholtz equations, to a specified precision, in two dimensions, on a multi-core shared-memory machine.
Fortran
23
star
30

catvae

Categorical Variational Autoencoders
Jupyter Notebook
22
star
31

online_psp_matlab

Benchmark of online PCA algorithms
MATLAB
22
star
32

spikeforest

Spike sorting benchmarking system
Python
22
star
33

slurm-prometheus-exporter

Prometheus exporter for slurm job/node data
Haskell
20
star
34

kachery-cloud

Python
20
star
35

dendro

Analyze neuroscience data in the cloud
TypeScript
19
star
36

comptools24

Computational Tools for PDEs with Complicated Geometries and Interfaces workshop materials, June 10-14, 2024.
MATLAB
18
star
37

aLENS

a Living ENsemble Simulator -- a lens to help you watch biophysics
C++
17
star
38

spikeforest_old

SpikeForest -- spike sorting analysis for website
Python
16
star
39

baobzi

An adaptive fast function approximator based on tree search
C++
14
star
40

isosplit5

ISO-SPLIT clustering (stand-alone version)
C++
14
star
41

bio-sfa

Code for reproducing the experiment from the paper "A biologically plausible neural network for Slow Feature Analysis"
Jupyter Notebook
12
star
42

pytorch-finufft

Pytorch wrappers for the FINUFFT library
Python
11
star
43

sf_benchmarks

Special function benchmarks
C++
10
star
44

mantis

Manifold-tiling Localized Receptive Fields are Optimal in Similarity-preserving Neural Networks
Python
10
star
45

flathub

A simple elasticsearch frontend for serving astrophysical simulation catalog data
Haskell
10
star
46

ccq-software-build

CCQ software build scripts
Shell
10
star
47

Ensemble-reweighting-using-Cryo-EM-particles

Jupyter Notebook
10
star
48

PointCloud_Regression

Point Cloud regression with new algebraic representation on ModelNet dataset (ICCV 2023)
Python
10
star
49

public_www

Basic templates for individual user pages in the public/_www directory
SCSS
9
star
50

inferelator-prior

Gene regulatory network inference using DNA-binding motifs and chromatin accessibility data.
Python
9
star
51

boxcodes3d

This repository contains box codes for evaluating volume potentials for Laplace, Helmholtz, Maxwell, and Stokes
Fortran
9
star
52

bio-nica

Code for reproducing the experiments in the paper "Bio-NICA: A biologically inspired single-layer network for Nonnegative Independent Component Analysis"
Jupyter Notebook
9
star
53

cppdlr

Discrete Lehmann representation of imaginary time Green's functions
C++
8
star
54

nix-modules

nixpkgs-based module builds of cluster packages
Nix
8
star
55

bio-cca

Code accompanying the paper "A biologically plausible neural network for multi-channel Canonical Correlation Analysis"
Jupyter Notebook
8
star
56

reactopya

React components paired with Python classes for backend computation
JavaScript
8
star
57

clair

Clair (Clang Introspection and Reflection) tools. A set of clang tools developed for Flatiron/CCQ
C++
8
star
58

adapol

Adaptive Pole Fitting for Quantum Many-Body Physics
Jupyter Notebook
8
star
59

DLR_DMFT_scripts

Python
7
star
60

kbucket

System for sharing data for scientific research
JavaScript
7
star
61

ccn-template

Template repository for CCN software projects
Python
7
star
62

neutorch

neuron segmentation and synapse detection using PyTorch
Python
7
star
63

SlurmUtil

slurm monitoring tools and interface
Python
7
star
64

q2-matchmaker

A qiime2 plugin for case-control differential abundance analysis
Jupyter Notebook
6
star
65

mountainlab

Scientific data analysis, sharing, and visualization
C++
6
star
66

least_absolute_regression

Least absolute error regression implemented using Linear Programming, primarily to illustrate repository structure conventions.
Jupyter Notebook
6
star
67

neuropixels-data-sep-2020

Example neuropixels datasets for purposes of developing spike sorting algorithms
Python
6
star
68

ephys-viz

Neurophysiology visualization components deployable to the notebook, web, or desktop
JavaScript
6
star
69

inverse-obstacle-scattering2d

This repository contains codes for solving inverse obstacle scattering problems to recover the shape of the obstacle for sound soft, sound hard, and penetrable objects
Fortran
6
star
70

cryoSBI

Python
6
star
71

kachery-p2p

Peer-to-peer content-addressable file sharing using kachery
TypeScript
5
star
72

gp-shootout

Benchmark and compare large-scale Gaussian process regression methods in 1D, 2D, and 3D, from MATLAB
MATLAB
5
star
73

ccn-software-fens-2024

Materials for CCN software workshop at FENS 2024
Python
5
star
74

looking_glass_3d_test

Test of displaying 3d genomic data using webgl in a "Looking glass" holographic display
5
star
75

caiman_central

Central hub for resources related to Caiman calcium imaging analysis package.
Jupyter Notebook
5
star
76

labbox-ephys

TypeScript
4
star
77

FREYA

Canine breast cancer analysis pipeline
Jupyter Notebook
4
star
78

spikesortercomparison

Notes on metrics and datasets for community spike sorting comparison and validation
TeX
4
star
79

hither

Run batches of Python functions in containers and on remote servers
Python
4
star
80

ILCnetworks

Gene regulatory networks for intestinal innate lymphoid cells
Jupyter Notebook
4
star
81

Fermi_surface_visualization

Interactive visualization of Fermi surfaces representing electron energies and velocities in crystal structures.
Python
4
star
82

flatiron-coffee

Meet your colleagues across the Institute
Python
4
star
83

spike-front

Front end tool for benchmarking spike sorting algorithms.
JavaScript
4
star
84

ccm_widgets

Reactopya widgets of relevance to the Center for Computational Mathematics, Flatiron Institute
JavaScript
4
star
85

Cryo-EM-Heterogeneity-Challenge-1

The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge
Jupyter Notebook
4
star
86

kvsstcp

Key value storage server. Light weight (single python module with no non-standard module dependencies). Simple protocol minimizes effort needed to write a client (example for C and FORTRAN provided).
Python
4
star
87

bio-lda

Code accompanying the paper "A linear discriminant analysis model of imbalanced associative learning in the mushroom body compartment"
Jupyter Notebook
3
star
88

Baobzi.jl

Wrapper library for Baobzi interpolator library
Julia
3
star
89

ftk

Factorization of the translation kernel for fast rigid image alignment
Python
3
star
90

sciware-testing-cpp

C++
3
star
91

binary_classification_metrics

Visualizations and other code for exploring binary classification metrics.
Jupyter Notebook
3
star
92

inverse-volume-scattering2d

This repository contains beta MATLAB codes for data generation of forward medium problems using a hybrid HPS-HBS solver, and an inverse medium solver
MATLAB
3
star
93

stan-playground

Run Stan models in the browser
TypeScript
3
star
94

qt-mountainview

Visualization of spike sorting experiments to be used as a plugin package to mountainlab-js
C++
3
star
95

fgt2d

This repository contains codes for evaluating sums and integrals of gaussians in two dimensions
Fortran
3
star
96

spikeforest_recordings

Ephys recordings for the SpikeForest project
Python
3
star
97

quantum_rosetta_private

Playing with quantum computing algorithms for protein and peptide design and structure prediction, with interfaces for the Rosetta software suite. (Repository created by Vikram K. Mulligan, [email protected]).
C#
3
star
98

mcmc-monitor-old

Monitor MCMC runs
Python
3
star
99

h5_to_json

Represent hdf5 files via json, separating out large binary content into individual files
Python
3
star
100

mountainsort_examples-old

Examples for getting started with MountainSort spike sorting software
MATLAB
2
star