• Stars
    star
    175
  • Rank 218,059 (Top 5 %)
  • Language
    Python
  • License
    GNU General Publi...
  • Created about 4 years ago
  • Updated 5 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Numerical integration in arbitrary dimensions on the GPU using PyTorch / TF / JAX

torchquad

Read the Docs (version) Tests GitHub last commit GitHub Conda (channel only) PyPI PyPI - Python Version

GitHub contributors GitHub issues GitHub pull requests Conda PyPI - Downloads JOSS


Logo

High-performance numerical integration on the GPU with PyTorch, JAX and Tensorflow
Explore the docs Β»

Report Bug Β· Request Feature

Table of Contents
  1. About The Project
  2. Goals
  3. Getting Started
  4. Usage
  5. Roadmap
  6. Contributing
  7. License
  8. FAQ
  9. Contact

About The Project

The torchquad module allows utilizing GPUs for efficient numerical integration with PyTorch and other numerical Python3 modules. The software is free to use and is designed for the machine learning community and research groups focusing on topics requiring high-dimensional integration.

Built With

This project is built with the following packages:

  • autoray, which means the implemented quadrature supports NumPy and can be used for machine learning with modules such as PyTorch, JAX and Tensorflow, where it is fully differentiable
  • conda, which will take care of all requirements for you

If torchquad proves useful to you, please consider citing the accompanying paper.

Goals

  • Supporting science: Multidimensional numerical integration is needed in many fields, such as physics (from particle physics to astrophysics), in applied finance, in medical statistics, and others. torchquad aims to assist research groups in such fields, as well as the general machine learning community.
  • Withstanding the curse of dimensionality: The curse of dimensionality makes deterministic methods in particular, but also stochastic ones, computationally expensive when the dimensionality increases. However, many integration methods are embarrassingly parallel, which means they can strongly benefit from GPU parallelization. The curse of dimensionality still applies but the improved scaling alleviates the computational impact.
  • Delivering a convenient and functional tool: torchquad is built with autoray, which means it is fully differentiable if the user chooses, for example, PyTorch as the numerical backend. Furthermore, the library of available and upcoming methods in torchquad offers high-effeciency integration for any need.

Getting Started

This is a brief guide for how to set up torchquad.

Prerequisites

We recommend using conda, especially if you want to utilize the GPU. With PyTorch it will automatically set up CUDA and the cudatoolkit for you, for example. Note that torchquad also works on the CPU; however, it is optimized for GPU usage. torchquad's GPU support is tested only on NVIDIA cards with CUDA. We are investigating future support for AMD cards through ROCm.

For a detailed list of required packages and packages for numerical backends, please refer to the conda environment files environment.yml and environment_all_backends.yml. torchquad has been tested with JAX 0.2.25, NumPy 1.19.5, PyTorch 1.10.0 and Tensorflow 2.7.0 on Linux; other versions of the backends should work as well but some may require additional setup on other platforms such as Windows.

Installation

The easiest way to install torchquad is simply to

conda install torchquad -c conda-forge

Alternatively, it is also possible to use

pip install torchquad

The PyTorch backend with CUDA support can be installed with

conda install "cudatoolkit>=11.1" "pytorch>=1.9=*cuda*" -c conda-forge -c pytorch

Note that since PyTorch is not yet on conda-forge for Windows, we have explicitly included it here using -c pytorch. Note also that installing PyTorch with pip may not set it up with CUDA support. Therefore, we recommend to use conda.

Here are installation instructions for other numerical backends:

conda install "tensorflow>=2.6.0=cuda*" -c conda-forge
pip install "jax[cuda]>=0.2.22" --find-links https://storage.googleapis.com/jax-releases/jax_cuda_releases.html # linux only
conda install "numpy>=1.19.5" -c conda-forge

More installation instructions for numerical backends can be found in environment_all_backends.yml and at the backend documentations, for example https://pytorch.org/get-started/locally/, https://github.com/google/jax/#installation and https://www.tensorflow.org/install/gpu, and often there are multiple ways to install them.

Test

After installing torchquad and PyTorch through conda or pip, users can test torchquad's correct installation with:

import torchquad
torchquad._deployment_test()

After cloning the repository, developers can check the functionality of torchquad by running the following command in the torchquad/tests directory:

pytest

Usage

This is a brief example how torchquad can be used to compute a simple integral with PyTorch. For a more thorough introduction please refer to the tutorial section in the documentation.

The full documentation can be found on readthedocs.

# To avoid copying things to GPU memory,
# ideally allocate everything in torch on the GPU
# and avoid non-torch function calls
import torch
from torchquad import MonteCarlo, set_up_backend

# Enable GPU support if available and set the floating point precision
set_up_backend("torch", data_type="float32")

# The function we want to integrate, in this example
# f(x0,x1) = sin(x0) + e^x1 for x0=[0,1] and x1=[-1,1]
# Note that the function needs to support multiple evaluations at once (first
# dimension of x here)
# Expected result here is ~3.2698
def some_function(x):
    return torch.sin(x[:, 0]) + torch.exp(x[:, 1])

# Declare an integrator;
# here we use the simple, stochastic Monte Carlo integration method
mc = MonteCarlo()

# Compute the function integral by sampling 10000 points over domain
integral_value = mc.integrate(
    some_function,
    dim=2,
    N=10000,
    integration_domain=[[0, 1], [-1, 1]],
    backend="torch",
)

To change the logger verbosity, set the TORCHQUAD_LOG_LEVEL environment variable; for example export TORCHQUAD_LOG_LEVEL=DEBUG.

You can find all available integrators here.

Roadmap

See the open issues for a list of proposed features (and known issues).

Performance

Using GPUs torchquad scales particularly well with integration methods that offer easy parallelization. For example, below you see error and runtime results for integrating the function f(x,y,z) = sin(x * (y+1)Β²) * (z+1) on a consumer-grade desktop PC.

Runtime results of the integration. Note the far superior scaling on the GPU (solid line) in comparison to the CPU (dashed and dotted) for both methods.

Convergence results of the integration. Note that Simpson quickly reaches floating point precision. Monte Carlo is not competitive here given the low dimensionality of the problem.

Contributing

The project is open to community contributions. Feel free to open an issue or write us an email if you would like to discuss a problem or idea first.

If you want to contribute, please

  1. Fork the project on GitHub.
  2. Get the most up-to-date code by following this quick guide for installing torchquad from source:
    1. Get miniconda or similar
    2. Clone the repo
    git clone https://github.com/esa/torchquad.git
    1. With the default configuration, all numerical backends with CUDA support are installed. If this should not happen, comment out unwanted packages in environment_all_backends.yml.
    2. Set up the environment. This creates a conda environment called torchquad and installs the required dependencies.
    conda env create -f environment_all_backends.yml
    conda activate torchquad

Once the installation is done, you are ready to contribute. Please note that PRs should be created from and into the develop branch. For each release the develop branch is merged into main.

  1. Create your Feature Branch (git checkout -b feature/AmazingFeature)
  2. Commit your Changes (git commit -m 'Add some AmazingFeature')
  3. Push to the Branch (git push origin feature/AmazingFeature)
  4. Open a Pull Request on the develop branch, not main (NB: We autoformat every PR with black. Our GitHub actions may create additional commits on your PR for that reason.)

and we will have a look at your contribution as soon as we can.

Furthermore, please make sure that your PR passes all automated tests. Review will only happen after that. Only PRs created on the develop branch with all tests passing will be considered. The only exception to this rule is if you want to update the documentation in relation to the current release on conda / pip. In that case you may ask to merge directly into main.

License

Distributed under the GPL-3.0 License. See LICENSE for more information.

FAQ

  1. Q: Error enabling CUDA. cuda.is_available() returned False. CPU will be used.
    A: This error indicates that PyTorch could not find a CUDA-compatible GPU. Either you have no compatible GPU or the necessary CUDA requirements are missing. Using conda, you can install them with conda install cudatoolkit. For more detailed installation instructions, please refer to the PyTorch documentation.

Contact

Created by ESA's Advanced Concepts Team

  • Pablo GΓ³mez - pablo.gomez at esa.int
  • Gabriele Meoni - gabriele.meoni at esa.int
  • HΓ₯vard Hem Toftevaag

Project Link: https://github.com/esa/torchquad

More Repositories

1

pagmo2

A C++ platform to perform parallel computations of optimisation tasks (global and local) via the asynchronous generalized island model.
C++
812
star
2

pygmo2

A Python platform to perform parallel computations of optimisation tasks (global and local) via the asynchronous generalized island model.
C++
428
star
3

pykep

PyKEP is a scientific library providing basic tools for research in interplanetary trajectory design.
C++
332
star
4

pagmo

A C++ / Python platform to perform parallel computations of optimisation tasks (global and local) via the asynchronous generalized island model. State of the art optimization algorithms are included. A common interface is provided to other optimization frameworks/algorithms such as NLOPT, SciPy, SNOPT, IPOPT, GSL
C++
269
star
5

nanosat-mo-framework

A software framework for small satellites based on CCSDS MO services
Java
87
star
6

opengeode

OpenGEODE - a free SDL editor
Python
69
star
7

snerf

Code for ''Shadow Neural Radiance Fields for Multi-View Photogrammetry'' Derksen Dawa, Dario Izzo
Python
69
star
8

gr-opssat

This repository contains the UHF specifications and example applications for demodulating and decoding the received signal.
Python
68
star
9

pygmo

Contains the sphinx documentation for PyGMO. To see the actual code, please use the PaGMO repository (https://github.com/esa/pagmo) as PyGMO is the python bindings to PaGMO (c++)
48
star
10

NIDN

Neural Inverse Design of Nanostructures
Jupyter Notebook
36
star
11

pypogs

An automated closed-loop satellite tracker for optical links with portable telescopes.
Python
35
star
12

SpOC2

Here you can find the information you need in order to take part in the GECCO 2023 Space Optimisation Competition (SpOC) organised by the Advanced Concepts Team (ACT).
Python
26
star
13

SpOC3

The official GECCO 2024 Space Optimisation Competition (SpOC) organised by the Advanced Concepts Team (ACT).
Jupyter Notebook
25
star
14

cascade

Propagate the evolution of large number of orbiting objects while detecting reliably conjunctions and collisions.
C++
22
star
15

nmf-mission-ops-sat

NanoSat MO Framework - NMF Mission: OPS-SAT
Java
21
star
16

polyhedral-gravity-model

Implementation of a polyhedral gravity model in C++17 with a Python Binding
Jupyter Notebook
21
star
17

SpaceAMPL

AMPL transcriptions of optimal control problems of space relevance (requires ampl commercial software)
AMPL
18
star
18

auromat

AUROra MApping Toolkit - Python library / CLI tools for creating and working with georeferenced images for aurora research.
Python
17
star
19

dSGP4

dSGP4: differentiable SGP4. Supports differentiability, ML integration & embarassingly parallel computations
Python
15
star
20

GUSTO

A Java library for space astronomy
Java
15
star
21

jupiter_tour

A javascript crowdsourcing experiment to prove you can fly a spacecraft through planetary systems
JavaScript
13
star
22

CCSDS_MO

Repository for holding all CCSDS MO documentation, examples, tutorials and issue lists.
13
star
23

neuralg

Neural network approximators of linear algebra operations on GPU with PyTorch
Jupyter Notebook
12
star
24

mo-services-java

ESA's Java implementation of the CCSDS MO services
Java
12
star
25

NASA-breakup-model-cpp

A validated modern C++ implementation of the NASA Standard Breakup Model
C++
11
star
26

CCSDS_MO_TRAINING

Repository for holding all CCSDS MO Training documentation, examples, tutorials and issue lists
Java
11
star
27

pagmo_plugins_nonfree

A pagmo affiliated package (https://github.com/esa/pagmo2) adding commercial solvers to the list of solvers/algorithms: SNOPT7, WORHP
C
10
star
28

gaia-bhthree

Jupyter Notebook
10
star
29

str4sd

Star tracker acquisitions from SWARM mission taken since mission launch up to end of 2018
JavaScript
10
star
30

dcgp-web

Web interface to use dcgp and inspect its chromosomes
JavaScript
8
star
31

CCSDS_MO_XML

REPOSITORY ARCHIVED - for the latest version please go to https://github.com/esa/mo-services-java
7
star
32

sleapi-j

Implementation of CCSDS Space Link Extension Services
Java
6
star
33

LADDS

Large-scale Deterministic Debris Simulation - Codebase for the ARIADNA Study between TU Munich and ESA's Advanced Concepts Team.
Jupyter Notebook
5
star
34

kep3

C++
5
star
35

thermonets

Code to train and test ThermoNets: a neural, differentiable, model for the thermosphere density.
Jupyter Notebook
5
star
36

dcgp.js

JavaScript bindings for the differential Cartesian Genetic Programming library
JavaScript
5
star
37

CCSDS_MO_StubGenerator

Generates technology bindings for CCSDS MAL service specifications
Java
4
star
38

mo.viewer.web

A HTML/JS viewer for CCSDS MO Services
JavaScript
4
star
39

kazoo

C
3
star
40

CCSDS_MO_TRANS

REPOSITORY ARCHIVED - for the latest version please go to https://github.com/esa/mo-services-java
Java
3
star
41

CCSDS_MO_POM

ESA top level POM for CCSDS MO based artifacts
3
star
42

lisa-orbit-files

A repository to share the official LISA science orbit files with the LISA consortium
3
star
43

hipparcosJava

legacy Java applets/applications from 1997 Hippacos Catalogue website. Slightly updgraded to run as standalone java applications in newere JDKs (1.6 up)
HTML
3
star
44

anomaly-dataset

ESA's anomaly dataset of real satellite telemetry from three ESA missions
3
star
45

pyoptgra

A gradient descent algorithm specifically designed for near-linear constrained problems
Fortran
2
star
46

CCSDS_MO_TESTBEDS

REPOSITORY ARCHIVED - for the latest version please go to https://github.com/esa/mo-services-java
Java
2
star
47

CCSDS_MO_APPS

Example MO applications in Java
Java
2
star
48

CCSDS_MO_APIS

Holds the generated Java APIs for the CCSDS MO services
Java
2
star
49

ccsds-css-frm-editor

CCSDS CSS Functional Resource Model Editor
Java
2
star
50

ccsds-css-frm-editor-p2

1
star
51

CCSDS_MO_SUPPORT_LIBS

MO support libraries in Java
Java
1
star
52

CCSDS_MO_MAL_IMPL

CCSDS MO MAL implementation in Java
Java
1
star
53

lxrmount

Tracking satellites and celestial objects using LX RoCon and telescope mount
C
1
star