Kuma-san's Toolkit 2020
γ γ γβΌββΌ
γ γ β©οΌΏβοΌΏβ©
|γ γ½
/ β β |
| (_β_) γ < There is absolutely no warranty. >
彑｀ |βͺ| ο½€ο½οΌΌ
/ οΌΏοΌΏ γ½γ /Β΄> )
(οΌΏοΌΏοΌΏοΌ / (_οΌ
Usage
Copy all the files to your working directory via:
git clone https://github.com/analokmaus/kuma_utils.git
See tutorial notebooks below.
WIP
- Multi-node DDP
Environment
Create a new environment and:
pip install -r reqirements.txt
Optional requirements
xfeat
pip install -q https://github.com/pfnet-research/xfeat/archive/master.zip
Category Encoders
pip install category_encoders
PyTorch
For mixed precision training, you must install version >= 1.6.0 .
Follow official instructions.
Pytorch/XLA
Follow official instructions.
japanize-matplotlib
pip install japanize-matplotlib
Directory
β£ visualization
β β£ explore_data - Simple exploratory data analysis.
β
β£ preprocessing
β β£ xfeat - xfeat modifications.
β β β£ TargetEncoder
β β β£ Pipeline
β β£ DistTransformer - Distribution transformer for numerical features.
β β£ LGBMImputer - Regression imputer for missing values using LightGBM.
β
β£ training
β β£ Trainer - Amazing wrapper for scikit-learn API models.
β β£ CrossValidator - Amazing cross validation wrapper.
β β£ LGBMLogger - Logger callback for LightGBM/XGBoost/Optuna.
β β£ StratifiedGroupKFold - Stratified group k-fold split.
β β£ optuna - optuna modifications.
β β£ lightgbm - Optune lightgbm integration with modifiable n_trials.
β
β£ metrics - Universal metrics
β β£ SeAtFixedSp - Sensitivity at fixed specificity.
β β£ RMSE
β β£ AUC
β β£ Accuracy
β β£ QWK
β
β£ torch
β£ lr_scheduler
β β£ ManualScheduler
β β£ CyclicCosAnnealingLR
β β£ CyclicLinearLR
β
β£ optimizer
β β£ SAM
β
β£ modules
β β (activation)
β β£ Mish
β β (pooling)
β β£ AdaptiveConcatPool2d/3d
β β£ GeM
β β (attention)
β β£ CBAM2d
β β (normalization)
β β£ GroupNorm1d/2d/3d
β β£ convert_groupnorm - Convert all BatchNorm to GroupNorm.
β β£ etc...
β
β£ TorchTrainer - PyTorch Wrapper.
β£ EarlyStopping - Early stopping callback for TorchTrainer.
β£ SaveEveryEpoch - Save snapshot at the end of every epoch.
β£ SaveSnapshot - Checkpoint callback.
β£ SaveAverageSnapshot - Moving average snapshot callback.
β£ TorchLogger - Logger
β£ TensorBoardLogger - TensorBoard Logger
β£ SimpleHook - Simple train hook for almost any tasks (see tutorial).
β£ TemperatureScaler - Probability calibration for pytorch models.
Tutorial
- Exploratory data analysis
- Data preprocessing
- Train and validate scikit-learn API models
- Train CIFAR-10 (TorchTrainer tutorial)
License
The source code in this repository is released under the MIT license.