fastMRI reproducible benchmark
The idea of this repository is to have a way to rapidly benchmark new solutions against existing reconstruction algorithms on the fastMRI dataset single-coil track. The reconstruction algorithms implemented or adapted to the fastMRI dataset include to this day:
- zero filled reconstruction
- LORAKS, using the LORAKS Matlab toolbox
- Wavelet-based reconstruction (i.e. solving an L1-based analysis formulation optimization problem with greedy FISTA), using pysap-mri
- U-net
- DeepCascade net
- KIKI net
- Learned Primal Dual, adapted to MRI reconstruction
- XPDNet, a modular unrolled reconstruction algorithm, in which you can plug your best denoiser.
- NCPDNet, an unrolled reconstruction algorithm for non-Cartesian acquisitions, with density-compensation.
All the neural networks are implemented in TensorFlow with the Keras API. The older ones (don't judge this was the beginning of my thesis) are coded using the functional API. The more recent ones are coded in the subclassed API and are more modular. For the LORAKS reconstruction, you will not be able to reconstruct the proper fastMRI data because of the A/D oversampling.
Examples
Various examples are available in the examples folder.
- The
mri_reconstruction_intro.ipynbnotebook compares the performance of classical wavelet-based reconstruction (using the pysap-mri package) and deep learning based approaches like the CascadeNet and the UPDNet on a slice from the fastMRI single coil dataset in a 2D Cartesian setting. - The
non_cartesian_reconstruction.ipynbnotebook showcases the use of the NCPDNet.
These examples can be run in binder.
Pretrained model checkpoints
Some model checkpoints are available in the HuggingFace Hub.
You can either download them manually, clone the model repository, or use the huggingface-hub package like:
from huggingface_hub import hf_hub_download
REPO_ID_BASE = 'zaccharieramzi/{model_name}'
repo_id = REPO_ID_BASE.format(model_name='... the model name')
model_weights_path = hf_hub_download(
repo_id=repo_id,
filename='model_weights.h5',
)Use cases for each model are explained in the respective model cards on the HuggingFace Hub under the "How to use" section.
Reconstruction settings
The main reconstruction settings are Cartesian single-coil and multi-coil 2D reconstruction with random and "periodic sampling". These settings are covered by almost all the networks in this repo, mainly because they are the settings of the fastMRI challenge.
Other reconstruction settings
Non-cartesian: you can reconstruct non-Cartesian data using the NCPDNet.
It relies on the TensorFlow implementation of the NUFFT, tfkbnufft.
This network will allow you to work on 2D single-coil and multi-coil data, as well as 3D single-coil data.
Deep Image Prior: you can also reconstruct non-Cartesian data in an untrained fashion using the DIP model. This idea originated from the Deep Image Prior paper, and was later adapted to MRI reconstruction by different works (Accelerated MRI with untrained Neural networks, Time-Dependent Deep Image Prior for Dynamic MRI). It currently is only used for 2D non-Cartesian data (primarily for computation time reasons), but you can extend it easily to 2D Cartesian data and 3D (PRs welcome).
How to train the neural networks
The scripts to train the neural networks are located in fastmri_recon/training_scripts/.
You just need to install the package and its dependencies:
pip install . &&\
pip install -r requirements.txt
How to write a new neural network for reconstruction
The simplest and most versatile way to write a neural network for reconstruction is to subclass the CrossDomainNet class.
An example is the PDnet
Reproducing the results of the paper
To reproduce the results of the paper for the fastMRI dataset, run the following script (here for the PD contrast):
python fastmri_recon/evaluate/scripts/paper_eval.py --contrast CORPD_FBK
To reproduce the results of the paper for the OASIS dataset, run the following script (here for fewer samples):
python fastmri_recon/evaluate/scripts/paper_eval_oasis.py --n-samples 100
Finally, to reproduce the figures of the paper, you will need to use the qualitative_validation_for_net notebook.
Downloading the model checkpoints
The model checkpoints are stored in the HuggingFace Hub. You can download them using the following script, which will automatically put them in the correct directory (for example here the fastMRI models):
python fastmri_recon/evaluate/scripts/download_checkpoints.py
Data requirements
fastMRI
The fastMRI data must be located in a directory whose path is stored in the FASTMRI_DATA_DIR environment variable.
It can be downloaded on the official website after submitting a request (bottom of the page).
The package currently supports public single coil and multi coil knee data.
OASIS
The OASIS data must be located in a directory whose path is stored in the OASIS_DATA_DIR environment variable.
It can be downloaded on the XNAT store after creating an account.
You can for example use the ZIP download available on this page.
The project is OASIS3.
The whole list of sessions that need to be downloaded is available in OASIS_list.csv.
Citation
This work will be presented at the International Symposium on Biomedical Imaging (ISBI) in April 2020. An extended version has been published in MDPI Applied sciences. If you use this package or parts of it, please cite one of the following work:
- Benchmarking Deep Nets MRI Reconstruction Models on the FastMRI Publicly Available Dataset
- Benchmarking MRI Reconstruction Neural Networks on Large Public Datasets
- XPDNet for MRI Reconstruction: an Application to the fastMRI 2020 Brain Challenge
- Density Compensated Unrolled Networks for Non-Cartesian MRI Reconstruction
Applications
This package has been used to perform MRI reconstruction in the following projects (in addition to the ones mentioned above):