SeFT - Set functions for Time Series
This is the main source code for the submission Set Functions for Time Series. It depends on two further packages
keras-transformer
(fork with support for sequences of different lengths) and
medical-ts-datasets
(containing the implementation of the datasets used).
Citing our work
@InProceedings{pmlr-v119-horn20a,
title = {Set Functions for Time Series},
author = {Horn, Max and Moor, Michael and Bock, Christian and Rieck, Bastian and Borgwardt, Karsten},
booktitle = {Proceedings of the 37th International Conference on Machine Learning},
pages = {4353--4363},
year = {2020},
editor = {Hal Daumé III and Aarti Singh},
volume = {119},
series = {Proceedings of Machine Learning Research},
month = {13--18 Jul},
publisher = {PMLR},
}Installation
The project requires python version 3.7 (newer versions of python are unfortunately
incompatible with tensorflow 1.15.x) and is set up to use poetry packaging utility.
The easiest way to get started is to create a virtual python environment and to install
the package inside this environment.
git clone https://github.com/BorgwardtLab/Set_Functions_for_Time_Series.git
cd Set_Functions_for_Time_Series
# Install poetry
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python
# Install the `SeFT` package with its dependencies
poetry installQuickstart
One can quickly test the models on one of the online available datasets. If
they are not yet downloaded, they will be automatically downloaded and stored
in the directory ~/tensorflow_datasets.
Example usage (in SeFT subdirectory):
$ poetry run seft_fit_model --dataset physionet2019 --balance --log_dir test_transformer TransformerModel
Recreate run using following command:
seft_fit_model --random_seed 982927477 --dataset physionet2019 --balance \
--max_epochs 300 --early_stopping 30 --log_dir test_transformer \
TransformerModel --learning_rate 0.001 --batch_size 64 --warmup_steps 1000 \
--n_dims 128 --n_heads 4 --n_layers 1 --dropout 0.0 --attn_dropout 0.0 \
--aggregation_fn mean --max_timescale 100.0
Train on 176 steps, validate on 101 steps
Epoch 1/300
5/176 [..............................] - ETA: 8:18 - loss: 0.1676 - acc: 0.1056 Usage
$ poetry run seft_fit_model --help
usage: seft_fit_model [-h] [--max_epochs MAX_EPOCHS]
[--random_seed RANDOM_SEED] [--debug]
[--log_dir LOG_DIR] [--early_stopping EARLY_STOPPING]
[--dataset {physionet2012,physionet2019,mimic3_mortality,mimic3_phenotyping}]
[--balance] [--hypersearch]
{config,GRUSimpleModel,PhasedLSTMModel,InterpolationPredictionModel,GRUDModel,TransformerModel,DeepSetAttentionModel}
...
positional arguments:
{config,GRUSimpleModel,PhasedLSTMModel,InterpolationPredictionModel,GRUDModel,TransformerModel,DeepSetAttentionModel}
optional arguments:
-h, --help show this help message and exit
--max_epochs MAX_EPOCHS
--random_seed RANDOM_SEED
--debug
--log_dir LOG_DIR Where to log results. If ends on backslash assume we
need to create a directory
--early_stopping EARLY_STOPPING
--dataset {physionet2012,physionet2019,mimic3_mortality,mimic3_phenotyping}
--balance Balance the dataset
--hypersearch Sample hyperparameters
For each model all hyperparameters can be set using the command line:
poetry run seft_fit_model GRUDModel --help
usage: seft_fit_model GRUDModel [-h] [--learning_rate LEARNING_RATE]
[--batch_size BATCH_SIZE]
[--warmup_steps WARMUP_STEPS]
[--n_units N_UNITS] [--dropout DROPOUT]
[--recurrent_dropout RECURRENT_DROPOUT]
optional arguments:
-h, --help show this help message and exit
--learning_rate LEARNING_RATE
--batch_size BATCH_SIZE
--warmup_steps WARMUP_STEPS
--n_units N_UNITS
--dropout DROPOUT
--recurrent_dropout RECURRENT_DROPOUTAvailable datasets
While the physionet2012 and physionet2019 datasets are publicly available
and thus can be automatically downloaded, this is not the case for the
mimic3_mortality dataset. Here, after fulfilling the requirements for access
the data needs to be manually downloaded and provided in the form of
a compressed file. We are planning to make this file available in the MIMIC-III
preprocessed data repository.
Adding new datasets
The datasets used in this code are implemented in a separate package medical_ts_dataset. In total you would need to run the following steps:
- Implement dataset in a fork of
medical_ts_datasetsor create you own package with the implementation. The easiest way is probably to adapt one of the readers to fit your format (see themedical_ts_datasetspackage). For some further information I recommend consulting the tfds documentation. In the end the following code should be able to run:import tensorflow_datasets as tfds import medical_ts_datasets # this registers your dataset or any other dataset with tensorflow datasets import my_package_with_dataset # alternatively if you decide to implement you datasets in a separate package tfds.load('<your_dataset_name>')
- Add a an entry to the directory here defining which type of endpoint you dataset provides.
- Optional, if you decide to implement the dataset in a separate package) add import statements to you package here
Then you should be able to run all models in this codebase on your data. On a final note as I assume you are working with medical data (which is usually not publically accessable on the internet) this section of the tfds documentation might come in handy.
Available models
The following models are supported:
GRUSimpleModel, PhasedLSTMModel, InterpolationPredictionModel,
GRUDModel, TransformerModel, DeepSetAttentionModel
It was brought to my attention, that the GRUD implementation is not entirely in line with the original paper. For further information please refer to this issue.
Copyright
This collective work is copyright (c) 2020 by Max Horn. Individual portions may be copyright by individual contributors, and are included in this collective work with permission of the copyright owners.