• Stars
    star
    123
  • Rank 290,145 (Top 6 %)
  • Language
    Java
  • License
    Apache License 2.0
  • Created about 8 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Industrial Benchmark

Industrial Benchmark

The "Industrial Benchmark" is a realistic benchmark for offline RL and online RL, used to find those RL algorithms that are best suited for real-world applications.

The Industrial Benchmark includes a variety of aspects that we have identified as essential in industrial applications. It is designed to have the same difficulty and complexity as real RL applications. State- and action-space are continuous, the state-space is rather high-dimensional and only partially observable. The actions consist of three continuous components and act on three steerings. There are delayed effects.

The optimization task is multi-criterial in the sense that there are two reward components, which have opposing dependencies on the actions. The dynamical behavior is heteroskedastic with state-dependent observation noise and state-dependent probability distributions, based on latent variables. The industrial benchmark is designed in such a way that the optimal policy does not approach a fixed operating point in the three steerings. Each specific choice is based on our experience with industrial challenges.

Requires: Java 8 and Apache Maven 3.x or Python 3.7

For the Python Version, the industrial benchmark environment is contained in industrial_benchmark_python/IDS.py, and there is an OpenAI Gym compliant wrapper in industrial_benchmark_python/IBGym.py

You can install the Benchmark as a package after cloning, using:

pip install dist/industrial_benchmark_python-2.0-py3-none-any.whl

Or directly from PyPI:

pip install industrial_benchmark_python

To test whether it works you can run

cd industrial_benchmark_python
python test_gym_wrapper.py

and to check out how current RL methods implemented in the stable_baselines package do on the benchmark (stable baselines needs to be installed):

python industrial_benchmark_python/test_baselines.py

Documentation: The documentation is available online at: https://arxiv.org/abs/1709.09480

Source: D. Hein, S. Depeweg, M. Tokic, S. Udluft, A. Hentschel, T.A. Runkler, and V. Sterzing. 
	"A benchmark environment motivated by industrial control problems," in 2017 IEEE 
	Symposium Series on Computational Intelligence (SSCI), 2017, pp. 1-8. 

Citing Industrial Benchmark

To cite Industrial Benchmark, please reference:

D. Hein, S. Depeweg, M. Tokic, S. Udluft, A. Hentschel, T.A. Runkler, and V. Sterzing. "A 
	benchmark environment motivated by industrial control problems," in 2017 IEEE Symposium 
	Series on Computational Intelligence (SSCI), 2017, pp. 1-8. 

Additional references using Industrial Benchmark:

S. Depeweg, J. M. Hernández-Lobato, F. Doshi-Velez, and S. Udluft. "Learning and
	policy search in stochastic dynamical systems with Bayesian neural networks." arXiv
	preprint arXiv:1605.07127, 2016.

D. Hein, S. Udluft, M. Tokic, A. Hentschel, T.A. Runkler, and V. Sterzing. "Batch reinforcement 
	learning on the industrial benchmark: First experiences," in 2017 International Joint 
	Conference on Neural Networks (IJCNN), 2017, pp. 4214–4221.

S. Depeweg, J. M. Hernández-Lobato, F. Doshi-Velez, and S. Udluft. "Uncertainty decomposition 
	in Bayesian neural networks with latent variables." arXiv preprint arXiv:1605.07127, 
	2017.
	
D. Hein, A. Hentschel, T. A. Runkler, and S. Udluft. "Particle Swarm Optimization for Model 
	Predictive Control in Reinforcement Learning Environments," in Y. Shi (Ed.), Critical 
	Developments and Applications of Swarm Intelligence, IGI Global, Hershey, PA, USA, 
	2018, pp. 401–427.
	
S. Depeweg, J. M. Hernandez-Lobato, F. Doshi-Velez, and S. Udluft. "Decomposition of 
	Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning." 
	35th International Conference on Machine Learning, ICML 2018. Vol. 3. 2018.

D. Hein, S. Udluft, and T.A. Runkler. "Interpretable policies for reinforcement learning by 
	genetic programming." Engineering Applications of Artificial Intelligence, 76, 2018, 
	pp. 158-169.

D. Hein, S. Udluft, and T.A. Runkler. "Generating interpretable fuzzy controllers using 
	particle swarm optimization and genetic programming," in Proceedings of the Genetic 
	and Evolutionary Computation Conference Companion, ACM, 2018, pp. 1268-1275.

N. Di Palo, and H. Valpola. "Improving Model-Based Control and Active Exploration with 
	Reconstruction Uncertainty Optimization." arXiv preprint arXiv:1812.03955, 2018.

F. Linker. "Industrial Benchmark for Fuzzy Particle Swarm Reinforcement Learning." 
	http://felixlinker.de/doc/ib_fpsrl.pdf, 2019

H. Zhang, A. Zhou, and X. Lin. "Interpretable policy derivation for reinforcement learning 
	based on evolutionary feature synthesis." Complex & Intelligent Systems, 2020. 
	pp. 1-13.

P. Swazinna, S. Udluft, and T.A. Runkler. "Overcoming Model Bias for Robust Offline Deep 
	Reinforcement Learning." arXiv preprint arXiv:2008.05533, 2020.

A. Kumar. "Policy Optimization for Industrial Benchmark Using Deep Reinforcement Learning."
	PhD Thesis, Colorado State University, 2020.

R. Qin, S. Gao, X. Zhang, Z. Xu, S. Huang, Z. Li, Z. Weinan, and Y. Yu. "NeoRL: A Near 
	Real-World Benchmark for Offline Reinforcement Learning." arXiv preprint 
	arXiv:2102.00714, 2021.

P. Swazinna, S. Udluft, D. Hein, and T.A. Runkler. "Behavior Constraining in Weight 
	Space for Offline Reinforcement Learning." arXiv preprint arXiv:2107.05479, 2021.
	
V. Kurenkov, and S. Kolesnikov. "Showing Your Offline Reinforcement Learning Work: Online 
	Evaluation Budget Matters." arXiv preprint arXiv:2110.04156, 2021.

P. Swazinna, S. Udluft, and T.A. Runkler. "Measuring Data Quality for Dataset Selection in 
	Offline Reinforcement Learning," in 2021 IEEE Symposium Series on Computational 
	Intelligence (SSCI), 2021, pp. 1-8.

P. Swazinna, S. Udluft, D. Hein, and T.A. Runkler. "Comparing Model-free and Model-based 
	Algorithms for Offline Reinforcement Learning." arXiv preprint arXiv:2201.05433, 2022.

C. Feng, and G. Jinyan. "Reliable Offline Model-based Optimization for Industrial Process 
	Control." arXiv preprint arXiv:2205.07250, 2022.
	
P. Swazinna, S. Udluft, and T.A. Runkler. "User-Interactive Offline Reinforcement Learning." 
	arXiv preprint arXiv:2205.10629, 2022.

Additional references mentioning Industrial Benchmark:

Y. Li. "Deep reinforcement learning: An overview." arXiv preprint arXiv:1701.07274, 2017.

D. Ha, and J. Schmidhuber. "Recurrent world models facilitate policy evolution," in Advances 
	in Neural Information Processing Systems, 2018, pp. 2450-2462.

M. Schaarschmidt, A. Kuhnle, B. Ellis, K. Fricke, F. Gessert, and E. Yoneki. "Lift: 
	Reinforcement learning in computer systems by learning from demonstrations." arXiv 
	preprint arXiv:1808.07903, 2018.

M. Kaiser, C. Otte, T.A. Runkler, and C.H. Ek. "Data Association with Gaussian Processes." 
	arXiv preprint arXiv:1810.07158, 2018.

D. Lee, and J. McNair. "Deep reinforcement learning agent for playing 2D shooting games." Int. 
	J. Control Autom, 11, 2018, pp. 193-200.

D. Marino, and M. Manic. "Modeling and planning under uncertainty using deep neural networks." 
	IEEE Transactions on Industrial Informatics, 2019.

J. Fu, A. Kumar, O. Nachum, G. Tucker, and S. Levine. "Datasets for Data-Driven Reinforcement 
	Learning." arXiv preprint arXiv:2004.07219, 2020.

M. Schaarschmidt. "End-to-end deep reinforcement learning in computer systems." PhD Thesis, 
	University of Cambridge, 2020.

T. Gangwani, Y. Zhou, and J. Peng. "Learning Guidance Rewards with Trajectory-space Smoothing." 
	Advances in Neural Information Processing Systems 33, 2020.

A. Zubow, S. Rösler, P. Gawłowicz, F. Dressler. "GrGym: When GNU Radio goes to (AI) Gym," in 
	Proceedings of the 22nd International Workshop on Mobile Computing Systems and 
	Applications, 2021, pp. 8-14.
	
J. McLeod, H. Stojic, V. Adam, D. Kim, J. Grau-Moya, P. Vrancx, and F. Leibfried. "Bellman: 
	A Toolbox for Model-Based Reinforcement Learning in TensorFlow." arXiv preprint 
	arXiv:2103.14407, 2021.
	
T. Xu, and Y. Liang. "Provably Efficient Offline Reinforcement Learning with Trajectory-Wise 
	Reward." arXiv preprint arXiv:2206.06426, 2022.
	
F.M. Luo, T. Xu, H. Lai, X.H. Chen, W. Zhang, and Y. Yu. "A survey on model-based reinforcement 
	learning." arXiv preprint arXiv:2206.09328, 2022.

B. Han, Z. Ren, Z. Wu, Y. Zhou, and J. Peng. "Off-Policy Reinforcement Learning with 
	Delayed Rewards." arXiv preprint arXiv:2106.11854, 2021.
	
M. Kaiser. "Structured Models with Gaussian Processes." Doctoral dissertation, Technische 
	Universität München, 2021.
	
M. Videau. "Découverte de Politiques Interprétables pour l'Apprentissage par Renforcement 
	via la Programmation Génétique." Doctoral dissertation, Université Paris Dauphine-PSL,
	2021.

Z. Ren, R. Guo, Y. Zhou, and J. Peng. "Learning Long-Term Reward Redistribution via Randomized 
	Return Decomposition." arXiv preprint arXiv:2111.13485, 2021.

M. Videau, A. Leite, O. Teytaud, and M. Schoenauer. "Multi-objective Genetic Programming for 
	Explainable Reinforcement Learning," in European Conference on Genetic Programming 
	(Part of EvoStar), Springer, Cham, 2022, pp. 278-293.

X.-Y. Liu, Z. Xia, J. Rui, J. Gao, H. Yang, M. Zhu, C.D. Wang, Z. Wang, and J. Guo. 
	"FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial 
	Reinforcement Learning." arXiv preprint arXiv:2211.03107, 2022.
	
T. Xu. "Towards the Understanding of Sample Efficient Reinforcement Learning Algorithms." 
	Doctoral dissertation, The Ohio State University, 2022.

M. Schlappa, J. Hegemann, and S. Spinler. "Optimizing Control of Waste Incineration Plants 
	Using Reinforcement Learning and Digital Twins." IEEE Transactions on Engineering 
	Management, 2022.
	
F. Vignat, N. Béraud, and T.T.D. Montcel. "Toolpath Calculation Using Reinforcement Learning 
	in Machining," in International Joint Conference on Mechanics, Design Engineering & 
	Advanced Manufacturing, Springer, Cham, 2023, pp. 1149-1158.
	
F. Huang, J. Xu, D. Wu, Y. Cui, Z. Yan, W. Xing, and X.Zhang. "A general motion controller 
	based on deep reinforcement learning for an autonomous underwater vehicle with 
	unknown disturbances." Engineering Applications of Artificial Intelligence 117, 2023.

More Repositories

1

jailhouse

Linux-based partitioning hypervisor
C
1,720
star
2

ros-sharp

ROS# is a set of open source software libraries and tools in C# for communicating with ROS from .NET applications, in particular Unity3D
C#
956
star
3

kas

Setup tool for bitbake based projects
Python
358
star
4

edgeshark

Discover and capture container network traffic from your comfy desktop Wireshark, using a containerized service and a Wireshark plugin.
Markdown
244
star
5

confluence-dumper

Tool to export Confluence spaces and pages recursively via its API
Python
194
star
6

ix

Siemens Industrial Experience is a design system for designers and developers, to consistently create the perfect digital experience for industrial software products.
TypeScript
190
star
7

embb

Embedded Multicore Building Blocks (EMB²): Library for parallel programming of embedded systems. Star us on GitHub? +1
C++
179
star
8

meta-iot2050

SIMATIC IOT2050 Isar/Debian Board Support Package
Python
130
star
9

meta-iot2000

SIMATIC IOT2000 Yocto Board Support Package
C
127
star
10

efibootguard

Simple UEFI boot loader with support for safely switching between current and updated partition sets
C
100
star
11

fluffi

FLUFFI (Fully Localized Utility For Fuzzing Instantaneously) - A distributed evolutionary binary fuzzer for pentesters
C++
99
star
12

simatic-s7-webserver-api

A .NET API Client Library for the SIMATIC S7 PLC Webserver API
C#
97
star
13

powergym

A Gym-like environment for Volt-Var control in power distribution systems.
Python
70
star
14

policy_search_bb-alpha

Python
69
star
15

codeface

Codeface is a framework for analysing technical and social aspects of software development
R
67
star
16

jailhouse-images

Jailhouse demonstration and testing images
C
56
star
17

linux-entra-sso

Browser plugin for Linux to SSO on Microsoft Entra ID using a locally running microsoft identity broker (Intune).
JavaScript
49
star
18

freertos-cell

FreeRTOS for Jailhouse Cells
C
47
star
19

dtasm

Digital Twin Assembly - A portable and sandboxed package format for executable simulation modules based on WebAssembly
Rust
39
star
20

wfx

a lightweight, general-purpose workflow executor
Go
37
star
21

sourcegrid

sourcegrid
C#
36
star
22

mtda

Multi-Tenant Device Access
Python
35
star
23

cshargextcap

[Edgeshark] Wireshark extcap plugin for container traffic capture live streaming.
Go
30
star
24

pycontainerd

Python bindings for containerd API
Python
30
star
25

restapidoc

A RESTful API documentation plugin for the Grails web application framework
Groovy
28
star
26

ix-icons

Icon library for @siemens/ix
TypeScript
25
star
27

opc-ua-pubsub-dotnet

opc-ua-pubsub-dotnet is a library which implements OPC UA PubSub encoding and decoding in a simplified way.
C#
25
star
28

continuous-clearing

The Continuous Clearing Tool scans and collects the 3rd party OSS components used in a NPM/NuGet/Debian/Maven/Python/Conan/Aipine project and uploads it to SW360 and Fossology by accepting respective project ID for license clearing.
C#
23
star
29

libuta

Unified Trust Anchor Library
C
20
star
30

xenomai

Mirror of xenomai-forge for pull requests
C
20
star
31

siapp-sdk

Software Development Kit to build SIAPPs
C++
20
star
32

ix-blazor

Siemens IX library for Blazor
CSS
19
star
33

cvss-suite

CvssSuite - This Ruby gem helps you to process the vector of the Common Vulnerability Scoring System.
Ruby
18
star
34

GoScans

Collection of network scan modules for infrastructure discovery and information gathering.
Go
17
star
35

Large-Scale-Discovery

A network scanning solution for information gathering in large IT/OT network environments.
Go
16
star
36

u-boot

U-Boot
C
16
star
37

JMiniZinc

JMiniZinc
Java
15
star
38

CtrlppCheck

C++
14
star
39

simatic-ai-launcher

Get started with the toolbox of SIMATIC Artificial Intelligence Launcher.
Python
14
star
40

sparring

Network simulation for malware analysis.
Python
13
star
41

dfu-util

Device Firmware Upgrade Utilities
C
12
star
42

packetflix

[Edgeshark] container "binge watching" live packet streams for Wireshark(nado) nerds!
Go
11
star
43

pydcc

PyDCC is a software library for reading and automated processing of digital calibration certificates (DCCs).
Python
11
star
44

gencmpclient

generic CMP [RFC 4210, RFC 9483] client library and CLI, based on CMPforOpenSSL (https://github.com/mpeylo/cmpossl)
C
10
star
45

LightweightCmpRa

CLI-based Registration Authority application PoC
Java
10
star
46

ghostwire

[Edgeshark] Virtual communication discovery in container hosts
TypeScript
9
star
47

link-checker-service

The self-hosted Link Checker web service runs cached and otherwise optimized broken link checks
Go
8
star
48

meta-coral

ISAR Layer for Coral TPU
Smarty
7
star
49

OOASP

Jupyter Notebook
7
star
50

meta-efibootguard

Yocto layer for EFI Boot Guard
Python
7
star
51

evaluation-framework

Tools for specifying and evaluating many variants of a (Unity) simulation model and visualizing the results in a decision support system.
C#
7
star
52

isar-riscv

Isar layer to support RISC-V architecture for QEMU and also for upcoming NOEL-V hardware within SELENE EU project
Shell
6
star
53

tstomp

tStomp
Tcl
6
star
54

omniproperties

Think of java properties deluxe
Java
6
star
55

cyclonedx-property-taxonomy

CycloneDX Property Taxonomy for the 'siemens' namespace
6
star
56

spdx-licenselist

SPDX license list
5
star
57

libsecutils

Wrapper library for commonly used OpenSSL features, providing a simpler, more application-oriented API.
C
5
star
58

sicam-gridedge-configurationtemplates

this is a collection of templates which can be used in SICAM GridEdge product in order to collect data (measurements/asset information data) and publish the data to a cloud system.
5
star
59

cityhub-sdk

CityHub SDK
JavaScript
5
star
60

sinema-rc-tools

This repository contains tooling for the SINEMA-RC solution of Siemens.
Python
5
star
61

snap-to-bucket

A python tool to automate moving AWS snapshots into an AWS S3 bucket and restore backups from there as an AWS EBS volume.
Python
5
star
62

modular-application-creator-use-cases

HTML
5
star
63

tia-addin-build-package

4
star
64

csharg

[Edgeshark] client and CLI to packet capture streaming API
Go
4
star
65

turtlefinder

reusable container engine discovery, not only for Edgeshark
Go
4
star
66

embeddedCMP

proof-of-concept CMP client on the NXP FRDM-K64F platform
C
4
star
67

ix-starter

Siemens Industrial Experience is a design system for designers and developers, to consistently create the perfect digital experience for industrial software products.
TypeScript
4
star
68

androit_shmem

Androit: Real-time capable Android shared memory service
C++
3
star
69

cmp-ra-component

A CMP Registration Authority (RA)
Java
3
star
70

mkdocs-siemens

3
star
71

android-x86-siemens

Android-x86 build configuration for Siemens products, for upstream see http://git.android-x86.org/?p=device/siemens.git
Shell
3
star
72

k3-rti-wdt

RTI Watchdog Firmware for TI K3 SoCs
Assembly
3
star
73

vanilc

C++
2
star
74

.github

2
star
75

ProductConfigurationWithSHACL

Example code and models for a paper published at the Configuration Workshop 2018
Java
2
star
76

fact

fact
Python
2
star
77

lion

Code to reproduce experiments from "User-Interactive Offline Reinforcement Learning" (ICLR 2023)
Python
2
star
78

python-smime-email

Generate x509 SMIME signed emails with ease!
Python
2
star
79

simatic-s7-1500-tm-fast-vhdl-application-examples

VHDL Examples for the S7-1500 TM FAST Module from SIEMENS AG
VHDL
2
star
80

ASLanPPConnector

ASLan++ connector
Java
1
star
81

ix-docs

1
star
82

M-Fit

the M-Fit suite, a set of programs for testing and validating traffic controllers against the various ITE/AASHTO/NEMA ATC and Caltrans TEES 2009 standards
C
1
star
83

django-mantis-core

A wrapper around the Django Dingos app for the Mantis Cyber Threat Intelligence Mgmt. Framework.
Python
1
star
84

OOCSP

OOCSP
1
star
85

puppet-pam_pkcs11

A Puppet module to manage pam_pkcs11
Ruby
1
star
86

continuous-conformance

Azure DevOps extension to create automatically conformance work items.
TypeScript
1
star