• Stars
    star
    273
  • Rank 150,780 (Top 3 %)
  • Language
    Python
  • License
    Other
  • Created over 3 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Represent trained machine learning models as Pyomo optimization formulations
OMLT CI Status https://codecov.io/gh/cog-imperial/OMLT/branch/main/graph/badge.svg?token=9U7WLDINJJ Documentation Status COIN

OMLT: Optimization and Machine Learning Toolkit

OMLT is a Python package for representing machine learning models (neural networks and gradient-boosted trees) within the Pyomo optimization environment. The package provides various optimization formulations for machine learning models (such as full-space, reduced-space, and MILP) as well as an interface to import sequential Keras and general ONNX models.

Please reference the preprint of this software package as:

@misc{ceccon2022omlt,
     title={OMLT: Optimization & Machine Learning Toolkit},
     author={Ceccon, F. and Jalving, J. and Haddad, J. and Thebelt, A. and Tsay, C. and Laird, C. D. and Misener, R.},
     year={2022},
     eprint={2202.02414},
     archivePrefix={arXiv},
     primaryClass={stat.ML}
}

Documentation

The latest OMLT documentation can be found at the readthedocs page. Additionally, much of the current functionality is demonstrated using Jupyter notebooks available in the notebooks folder.

Example

import tensorflow
import pyomo.environ as pyo
from omlt import OmltBlock, OffsetScaling
from omlt.neuralnet import FullSpaceNNFormulation, NetworkDefinition
from omlt.io import load_keras_sequential

#load a Keras model
nn = tensorflow.keras.models.load_model('tests/models/keras_linear_131_sigmoid', compile=False)

#create a Pyomo model with an OMLT block
model = pyo.ConcreteModel()
model.nn = OmltBlock()

#the neural net contains one input and one output
model.input = pyo.Var()
model.output = pyo.Var()

#apply simple offset scaling for the input and output
scale_x = (1, 0.5)       #(mean,stdev) of the input
scale_y = (-0.25, 0.125) #(mean,stdev) of the output
scaler = OffsetScaling(offset_inputs=[scale_x[0]],
                    factor_inputs=[scale_x[1]],
                    offset_outputs=[scale_y[0]],
                    factor_outputs=[scale_y[1]])

#provide bounds on the input variable (e.g. from training)
scaled_input_bounds = {0:(0,5)}

#load the keras model into a network definition
net = load_keras_sequential(nn,scaler,scaled_input_bounds)

#multiple formulations of a neural network are possible
#this uses the default NeuralNetworkFormulation object
formulation = FullSpaceNNFormulation(net)

#build the formulation on the OMLT block
model.nn.build_formulation(formulation)

#query inputs and outputs, as well as scaled inputs and outputs
model.nn.inputs
model.nn.outputs
model.nn.scaled_inputs
model.nn.scaled_outputs

#connect pyomo model input and output to the neural network
@model.Constraint()
def connect_input(mdl):
    return mdl.input == mdl.nn.inputs[0]

@model.Constraint()
def connect_output(mdl):
    return mdl.output == mdl.nn.outputs[0]

#solve an inverse problem to find that input that most closely matches the output value of 0.5
model.obj = pyo.Objective(expr=(model.output - 0.5)**2)
status = pyo.SolverFactory('ipopt').solve(model, tee=False)
print(pyo.value(model.input))
print(pyo.value(model.output))

Development

OMLT uses tox to manage development tasks:

  • tox -av to list available tasks
  • tox to run tests
  • tox -e lint to check formatting and code styles
  • tox -e format to automatically format files
  • tox -e docs to build the documentation
  • tox -e publish to publish the package to PyPi

Contributors

GitHub Name Acknowledgements
jalving Jordan Jalving This work was funded by Sandia National Laboratories, Laboratory Directed Research and Development program
fracek Francesco Ceccon This work was funded by an Engineering & Physical Sciences Research Council Research Fellowship [GrantNumber EP/P016871/1]
carldlaird Carl D. Laird Initial work was funded by Sandia National Laboratories, Laboratory Directed Research and Development program. Current work supported by Carnegie Mellon University.
tsaycal Calvin Tsay This work was funded by an Engineering & Physical Sciences Research Council Research Fellowship [GrantNumber EP/T001577/1], with additional support from an Imperial College Research Fellowship.
thebtron Alexander Thebelt This work was supported by BASF SE, Ludwigshafen am Rhein.

More Repositories

1

romodel

Modeling robust optimization problems in Pyomo
Python
80
star
2

entmoot

Multiobjective black-box optimization using gradient-boosted trees
Python
56
star
3

galini

An extensible MINLP solver
Python
43
star
4

GPdoemd

Design of experiments for model discrimination using Gaussian process surrogate models
Python
36
star
5

suspect

Special Structure Detection for Pyomo
Python
27
star
6

SnAKe

Repository for paper: "SnAKe: Bayesian Optimization with Pathwise Exploration".
Python
15
star
7

pooling-network

Python
8
star
8

tree_kernel_gp

Python
6
star
9

min_matches_heuristics

Source code of the methods proposed in "Dimitrios Letsios, Georgia Kouyialis, Ruth Misener. Heuristics with Performance Guarantees for the Minimum Number of Matches Problem in Heat Recovery Network Design. Computers and Chemical Engineering 113:57-85, 2018".".
GAMS
6
star
10

pypopt

Bindings to Ipopt C++ library for Python. šŸ
Python
3
star
11

rogp

Modeling Gaussian Processes in Pyomo
Python
3
star
12

PartitionedFormulations_NN

Implementation of partition-based formulations for mixed-integer optimization of trained ReLU neural networks
Python
3
star
13

two_stage_scheduling

Exact lexicographic scheduling methods and approximate recovery strategies for two-stage makespan scheduling
Python
3
star
14

GNN_MIP_CAMD

Repository for paper: "Optimizing over trained GNNs via symmetry breaking".
Python
3
star
15

OMLT_CAMD

Repository for paper: "Augmenting optimization-based molecular design with graph neural networks".
Python
3
star
16

galini-dashboard

Dashboard for the GALINI Solver
JavaScript
2
star
17

DCBO

Repository for paper: "Dependence in constrained Bayesian optimization: When do we need it and how does it help?".
Python
2
star
18

Partial-Lasserre-relaxation-sparse-maxcut

C++
2
star
19

drill-scheduling

Pyomo implementation of a drill scheduling case study
Python
1
star
20

moo_trees

Python
1
star