xbatcher: Batch Generation from Xarray Datasets
Xbatcher is a small library for iterating Xarray DataArrays and Datasets in
batches. The goal is to make it easy to feed Xarray objects to machine
learning libraries such as PyTorch or TensorFlow. View the for more
info.
Installation
Xbatcher can be installed from PyPI as:
python -m pip install xbatcher
Or via Conda as:
conda install -c conda-forge xbatcher
Or from source as:
python -m pip install git+https://github.com/xarray-contrib/xbatcher.git
Note
The required dependencies installed with Xbatcher are Xarray, Dask, and NumPy. You will need to separately install TensorFlow or PyTorch to use those data loaders or Xarray accessors. Review the installation instructions for more details.
Documentation
Documentation is hosted on ReadTheDocs: https://xbatcher.readthedocs.org
License
Apache License 2.0, see LICENSE file.
Acknowledgements
This work was funded in part by:
NASA ACCESS19-0049: Pangeo ML: Open Source Tools and Pipelines for Scalable Machine Learning Using NASA Earth Observation Data
This work was motivated by many conversations in the Pangeo community and Pangeo ML working group