Awesome Online Machine Learning

Online machine learning is a subset of machine learning where data arrives sequentially. In contrast to the more traditional batch learning, online learning methods update themselves incrementally with one data point at a time.

• Courses and books
• Blog posts
• Software
  • Modelling
  • Deployment
• Papers
  • Linear models
  • Support vector machines
  • Neural networks
  • Decision trees
  • Unsupervised learning
  • Time series
  • Drift detection
  • Anomaly detection
  • Metric learning
  • Graph theory
  • Ensemble models
  • Expert learning
  • Active learning
  • Miscellaneous
  • Surveys
  • General-purpose algorithms
  • Hyperparameter tuning
  • Evaluation

Courses and books

• Machine Learning for Streaming Data with Python
• IE 498: Online Learning and Decision Making
• Introduction to Online Learning
• Machine Learning the Feature — Gives some insights into the inner workings of Vowpal Wabbit, especially the slides on online linear learning.
• Machine learning for data streams with practical examples in MOA
• Online Methods in Machine Learning (MIT)
• Streaming 101: The world beyond batch
• Prediction, Learning, and Games
• Introduction to Online Convex Optimization
• Reinforcement Learning and Stochastic Optimization: A unified framework for sequential decisions — The entire book builds upon Online Learning paradigm in applied learning/optimization problems, Chapter 3 Online learning being the reference.
• Big Data course at the CILVR lab at NYU — Focus on linear models and bandits. Some courses are given by John Langford, the creator of Vowpal Wabbit.
• Machine Learning for Personalization — Course from Columbia by Tony Jebara, covers bandits.

Blog posts

• Fennel AI blog posts about online recsys
• Anomaly Detection with Bytewax & Redpanda (Bytewax, 2022)
• The online machine learning predict/fit switcheroo (Max Halford, 2022)
• Real-time machine learning: challenges and solutions (Chip Huyen, 2022)
• Anomalies detection using River (Matias Aravena Gamboa, 2021)
• Introdução (não-extensiva) a Online Machine Learning (Saulo Mastelini, 2021)
• Machine learning is going real-time (Chip Huyen, 2020)
• The correct way to evaluate online machine learning models (Max Halford, 2020)
• What is online machine learning? (Max Pagels, 2018)
• What Is It and Who Needs It (Data Science Central, 2015)

Software

See more here.

Modelling

• River — A Python library for general purpose online machine learning.
• dask
• Jubatus
• Flink ML - Apache Flink machine learning library
• LIBFFM — A Library for Field-aware Factorization Machines
• LIBLINEAR — A Library for Large Linear Classification
• LIBOL — A collection of online linear models trained with first and second order gradient descent methods. Not maintained.
• MOA
• scikit-learn — Some of scikit-learn's estimators can handle incremental updates, although this is usually intended for mini-batch learning. See also the "Computing with scikit-learn" page.
• Spark Streaming — Doesn't do online learning per say, but instead mini-batches the data into fixed intervals of time.
• SofiaML
• StreamDM — A machine learning library on top of Spark Streaming.
• Tornado
• VFML
• Vowpal Wabbit

Deployment

• KappaML
• django-river-ml — a Django plugin for deploying River models
• chantilly — a prototype meant to be compatible with River (previously Creme)

Papers

Linear models

• Field-aware Factorization Machines for CTR Prediction (2016)
• Practical Lessons from Predicting Clicks on Ads at Facebook (2014)
• Ad Click Prediction: a View from the Trenches (2013)
• Normalized online learning (2013)
• Towards Optimal One Pass Large Scale Learning with Averaged Stochastic Gradient Descent (2011)
• Dual Averaging Methods for Regularized Stochastic Learning andOnline Optimization (2010)
• Adaptive Regularization of Weight Vectors (2009)
• Stochastic Gradient Descent Training forL1-regularized Log-linear Models with Cumulative Penalty (2009)
• Confidence-Weighted Linear Classification (2008)
• Exact Convex Confidence-Weighted Learning (2008)
• Online Passive-Aggressive Algorithms (2006)
• Logarithmic Regret Algorithms forOnline Convex Optimization (2007)
• A Second-Order Perceptron Algorithm (2005)
• Online Learning with Kernels (2004)
• Solving Large Scale Linear Prediction Problems Using Stochastic Gradient Descent Algorithms (2004)

Support vector machines

• Pegasos: Primal Estimated sub-GrAdient SOlver for SVM (2007)
• A New Approximate Maximal Margin Classification Algorithm (2001)
• The Relaxed Online Maximum Margin Algorithm (2000)

Neural networks

• Three scenarios for continual learning (2019)

Decision trees

• AMF: Aggregated Mondrian Forests for Online Learning (2019)
• Mondrian Forests: Efficient Online Random Forests (2014)
• Mining High-Speed Data Streams (2000)

Unsupervised learning

• Online Clustering: Algorithms, Evaluation, Metrics, Applications and Benchmarking (2022)
• Online hierarchical clustering approximations (2019)
• DeepWalk: Online Learning of Social Representations (2014)
• Online Learning with Random Representations (2014)
• Online Latent Dirichlet Allocation with Infinite Vocabulary (2013)
• Web-Scale K-Means Clustering (2010)
• Online Dictionary Learning For Sparse Coding (2009)
• Density-Based Clustering over an Evolving Data Stream with Noise (2006)
• Knowledge Acquisition Via Incremental Conceptual Clustering (2004)
• Online and Batch Learning of Pseudo-Metrics (2004)
• BIRCH: an efficient data clustering method for very large databases (1996)

Time series

• Online Learning for Time Series Prediction (2013)

Drift detection

• A Survey on Concept Drift Adaptation (2014)

Anomaly detection

• Leveraging the Christoffel-Darboux Kernel for Online Outlier Detection (2022)
• Interpretable Anomaly Detection with Mondrian Pólya Forests on Data Streams (2020)
• Fast Anomaly Detection for Streaming Data (2011)

Metric learning

• Online Metric Learning and Fast Similarity Search (2009)
• Information-Theoretic Metric Learning (2007)
• Online and Batch Learning of Pseudo-Metrics (2004)

Graph theory

• DeepWalk: Online Learning of Social Representations (2014)

Ensemble models

• Optimal and Adaptive Algorithms for Online Boosting (2015) — An implementation is available here
• Online Bagging and Boosting (2001)
• A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting (1997)

Expert learning

• On the optimality of the Hedge algorithm in the stochastic regime

Active learning

• A survey on online active learning (2023)

Miscellaneous

• Multi-Output Chain Models and their Application in Data Streams (2019)
• A Complete Recipe for Stochastic Gradient MCMC (2015)
• Online EM Algorithm for Latent Data Models (2007) — Source code is available here
• StreamAI: Dealing with Challenges of Continual Learning Systems for Serving AI in Production (2023)

Surveys

• Machine learning for streaming data: state of the art, challenges, and opportunities (2019)
• Online Learning: A Comprehensive Survey (2018)
• Online Machine Learning in Big Data Streams (2018)
• Incremental learning algorithms and applications (2016)
• Batch-Incremental versus Instance-Incremental Learning in Dynamic and Evolving Data
• Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey (2011)
• Online Learning and Stochastic Approximations (1998)

General-purpose algorithms

• Maintaining Sliding Window Skylines on Data Streams (2006)
• The Sliding DFT (2003) — An online variant of the Fourier Transform, a concise explanation is available here
• Sketching Algorithms for Big Data

Hyperparameter tuning

• ChaCha for Online AutoML (2021)

Evaluation

• Delayed labelling evaluation for data streams (2019)
• Efficient Online Evaluation of Big Data Stream Classifiers (2015)
• Issues in Evaluation of Stream Learning Algorithms (2009)