Researches and Practices in Autonomous Databases

Continuously update the autonomous database works based on our past tutorials.

Kindly let us know if we have missed any great papers. Thank you!

Conference deadlines: https://ccfddl.github.io/

(Note conference postponement may not be promptly synchronized, so just consider it as a reference.)

0. Survey and Tutorial (16)
1. Database Configuration
2. Query Optimization
3. Workload Scheduling (2)
4. Database Design
5. Database Monitoring (9)
6. Database Diagnosis
- 6.1 System Diagnosis (7)
- 6.2 Query Diagnosis (1)
7. General Techniques
8. Database Frameworks (16)
9. Demonstrations (13)
S1. Large Language Models Meet Database (10)
S2. AI Paper&Code List (4)
S3. Open Datasets And SQLs (3)

Great talks you should not miss >>

Make Your Database System Dream of Electric Sheep : Towards Self-Driving Operation. Andy Pavlo, Matthew Butrovich, Lin Ma, et al. [link]

Towards instance-optimized data systems. Tim Kraska. [link]

AI-Native Database. Guoliang Li. [link]

From BERT to GPT-3 Codex: Harnessing the Potential of Very Large Language Models for Data Management. Immanuel Trummer. [link]

Retrieval-based Language Models and Applications. Akari Asai, Sewon Min, Zexuan Zhong, Danqi Chen. [link]

0. Survey and Tutorial

Survey

Database meets deep learning: Challenges and opportunities.

Wei Wang, Meihui Zhang, Gang Chen, et al. SIGMOD Record, 2016. [paper]

Database Meets Artificial Intelligence: A Survey.

Xuanhe Zhou, Chengliang Chai, Guoliang Li, et al. TKDE, 2020. [paper]

A Survey on Advancing the DBMS Query Optimizer: Cardinality Estimation, Cost Model, and Plan Enumeration.

Hai Lan, Zhifeng Bao, Yuwei Peng. Data Science and Engineering, 2021. [paper]

A Survey on Deep Reinforcement Learning for Data Processing and Analytics.

Qingpeng Cai, Can Cui, Yiyuan Xiong, et al. TKDE, 2022. [paper]

Self-Driving Database Papers (CMU Spring Course). 2022.

https://15799.courses.cs.cmu.edu/spring2022/schedule.html

Automatic Database Knob Tuning: A Survey.

Xinyang Zhao, Xuanhe Zhou, Guoliang Li. TKDE, 2023. [paper] [code]

Tutorial

From auto-tuning one size fits all to self-designed and learned data-intensive systems.

Stratos Idreos, Tim Kraska. SIGMOD, 2019. [paper]

Speedup Your Analytics: Automatic Parameter Tuning for Databases and Big Data Systems.

Jiaheng Lu, Yuxing Chen, Herodotos Herodotou, Shivnath Babu. VLDB, 2019. [paper] [slides]

Tutorial: Adaptive Replication and Partitioning in Data Systems.

Brad Glasbergen, Michael Abebe, Khuzaima Daudjee. Middleware, 2018. [paper]

A Tutorial on Learned Multi-dimensional Indexes.

Abdullah Al-Mamun, Hao Wu, Walid G. Aref. SIGSPATIAL, 2020. [paper]

AI Meets Database: AI4DB and DB4AI.

Guoliang Li, Xuanhe Zhou, Lei Cao. SIGMOD, 2021. [paper] [slides]

Machine Learning for Databases.

Guoliang Li, Xuanhe Zhou, Lei Cao. VLDB, 2021. [paper][slides]

Machine Learning for Cloud Data Systems: the Promise, the Progress, and the Path Forward.

Alekh Jindal, Matteo Interlandi. VLDB, 2021. [paper]

Workload-Aware Performance Tuning for Autonomous DBMSs.

Zhengtong Yan, Jiaheng Lu, Naresh Chainani, et al. ICDE, 2021. [paper]

Learned Query Optimizer: At the Forefront of AI-Driven Databases.

Zhu, Rong, Ziniu Wu, Chengliang Chai, et al. EDBT, 2022. [paper]

From BERT to GPT-3 Codex: Harnessing the Potential of Very Large Language Models for Data Management.

Immanuel Trummer. VLDB, 2022. [paper]

1. Database Configuration

Knob Tuner

Heuristic

PGTune: https://pgtune.leopard.in.ua.

OpenTuner: An Extensible Framework for Program Autotuning

Ansel J, Kamil S, Veeramachaneni K, et al. PACT, 2014. [paper]

BestConfig: Tapping the Performance Potential of Systems via Automatic Configuration Tuning

Zhu Y, Liu J, Guo M, et al. SoCC, 2017. [paper]

BO-based

Tuning Database Conﬁguration Parameters with iTuned

Duan, S., Thummala, V., & Babu, S. VLDB, 2009. [paper]

Automatic database management system tuning through large-scale machine learning

Van Aken D, Pavlo A, Gordon G J, et al. SIGMOD, 2017. [paper]

Black or White? How to Develop an AutoTuner for Memory-based Analytics

Kunjir M, Babu S. SIGMOD, 2020. [paper]

ResTune: Resource Oriented Tuning Boosted by Meta-Learning for Cloud Databases

Zhang X, Wu H, Chang Z, et al. SIGMOD, 2021. [paper]

CGPTuner: a Contextual Gaussian Process Bandit Approach for the Automatic Tuning of IT Configurations Under Varying Workload Conditions

Cereda S, Valladares S, Cremonesi P, et al. VLDB, 2021. [paper]

Towards Dynamic and Safe Configuration Tuning for Cloud Databases

Zhang X, Wu H, Li Y, et al. SIGMOD, 2022. [paper]

LlamaTune: Sample-Efficient DBMS Configuration Tuning

Kanellis K, Ding C, Kroth B, et al. VLDB, 2022. [paper]

DL-based

iBTune: Individualized Buffer Tuning for Large-scale Cloud Databases

Jian Tan, Tieying Zhang, Feifei Li, et al. VLDB, 2019. [paper]

RL-based

An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning

Ji Zhang, Yu Liu, Ke Zhou, Guoliang Li, et al. SIGMOD, 2019. [paper]

QTune: A Query-Aware Database Tuning System with Deep Reinforcement Learning

Li G, Zhou X, Li S, et al. VLDB, 2019. [paper]

Watuning: A workload-aware tuning system with attention-based deep reinforcement learning

Ge J K, Chai Y F, Chai Y P. JCST, 2021. [paper]

The Case for NLP-Enhanced Database Tuning: Towards Tuning Tools that "Read the Manual"

Trummer I. VLDB, 2021. [paper]

DB-BERT: a Database Tuning Tool that “Reads the Manual”

Trummer I. SIGMOD, 2022. [paper]

HUNTER- An Online Cloud Database Hybrid Tuning System for Personalized Requirements

Cai B, Liu Y, Zhang C, et al. SIGMOD, 2022. [paper]

Knob Selection

SARD: A statistical approach for ranking database tuning parameters

Debnath B K, Lilja D J, Mokbel M F. ICDE Workshops 2008. [paper]

Too Many Knobs to Tune? Towards Faster Database Tuning by Pre-selecting Important Knobs

Kanellis K, Alagappan R, Venkataraman S. HotStorage 2020. [paper]

Benefit Estimation

IWEK: An Interpretable What-If Estimator for Database Knobs

Yu Yan, Hongzhi Wang, Jian Geng, et al. arXiv 2023. [paper]

Experiments

An inquiry into machine learning-based automatic configuration tuning services on real-world database management systems

Van Aken D, Yang D, Brillard S, et al. VLDB, 2021. [paper]

Facilitating Database Tuning with Hyper-Parameter Optimization- A Comprehensive Experimental Evaluation

Zhang X, Chang Z, Li Y, et al. VLDB, 2022. [paper]

View Advisor

Selecting subexpressions to materialize at datacenter scale

A. Jindal, K. Karanasos, S. Rao, and H. Patel. PVLDB, 11(7):800–812, 2018. [paper]

Automated generation of materialized views in Oracle

Ahmed, R., Bello, R., Witkowski, A., & Kumar, P. (2020). VLDB, 2020. [paper]

Computation reuse in analytics job service at microsoft

Jindal, A., Qiao, S., Patel, H., Yin, Z., Di, J., Bag, M., Friedman, M., Lin, Y., Karanasos, K. and Rao, S., SIGMOD, 2018 (pp. 191-203). [paper]

Automatic View Generation for Equivalent Subqueries with Deep Learning and Reinforcement Learning

Yuan, H., Sun, J., & Li, G. (2020). ICDE, 2020. [paper]

An Autonomous Materialized View Management System with Deep Reinforcement Learning

Han, Y., Li, G., Yuan, H., & Sun, J. ICDE, 2021. [paper]

AutoView: An Autonomous Materialized View Management System with Encoder-Reducer

Han, Y., Li, G., Yuan, H. and Sun, J., TKDE, 2022. [paper]

Dynamic Materialized View Management using Graph Neural Network

Yue Han, Chengliang Chai, Jiabin Liu, Guoliang Li, Chuangxian Wei, Chaoqun Zhan. ICDE 2023. [paper]

A novel coral reefs optimization algorithm for materialized view selection in data warehouse environments

Azgomi, H. and Sohrabi, M.K., Applied Intelligence, 2019, 49, pp.3965-3989. [paper]

Index Advisor

[EA & B] Jan Kossmann, Stefan Halfpap, Marcel Jankrift, Rainer Schlosser: Magic mirror in my hand, which is the best in the land? An Experimental Evaluation of Index Selection Algorithms. Proc. VLDB Endow. 13(11): 2382-2395 (2020) [paper]

[Industry, Microsoft Azure] Sudipto Das, Miroslav Grbic, Igor Ilic, Isidora Jovandic, Andrija Jovanovic, Vivek R. Narasayya, Miodrag Radulovic, Maja Stikic, Gaoxiang Xu, Surajit Chaudhuri: Automatically Indexing Millions of Databases in Microsoft Azure SQL Database. SIGMOD Conference 2019: 666-679 [paper]

[Industry, Meta] Ritwik Yadav, Satyanarayana R. Valluri, Mohamed Zait: AIM: A practical approach to automated index management for SQL databases. ICDE 2023 [paper]

[Heuristic-based, AutoAdmin] Surajit Chaudhuri, Vivek R. Narasayya: An Efficient Cost-Driven Index Selection Tool for Microsoft SQL Server. VLDB 1997: 146-155 [paper]

[Heuristic-based, DB2Advis] Gary Valentin, Michael Zuliani, Daniel C. Zilio, Guy M. Lohman, Alan Skelley: DB2 Advisor: An Optimizer Smart Enough to Recommend Its Own Indexes. ICDE 2000: 101-110 [paper]

[Heuristic-based, Relaxation] Nicolas Bruno, Surajit Chaudhuri: Automatic Physical Database Tuning: A Relaxation-based Approach. SIGMOD Conference 2005: 227-238 [paper]

[Heuristic-based, COLT] Karl Schnaitter, Serge Abiteboul, Tova Milo, Neoklis Polyzotis: On-Line Index Selection for Shifting Workloads. ICDE Workshops 2007: 459-468 [paper]

[Heuristic-based, Extend] Rainer Schlosser, Jan Kossmann, Martin Boissier: Efficient Scalable Multi-attribute Index Selection Using Recursive Strategies. ICDE 2019: 1238-1249 [paper]

[Learning-based, DQN] Hai Lan, Zhifeng Bao, Yuwei Peng: An Index Advisor Using Deep Reinforcement Learning. CIKM 2020: 2105-2108 [paper]

[Learning-based, DQN] Zahra Sadri, Le Gruenwald, Eleazar Leal: Online Index Selection Using Deep Reinforcement Learning for a Cluster Database. ICDE Workshops 2020: 158-161 [paper]

[Learning-based, DQN] Gabriel Paludo Licks, Júlia Mara Colleoni Couto, Priscilla de Fátima Miehe, Renata De Paris, Duncan Dubugras A. Ruiz, Felipe Meneguzzi: SmartIX: A Database Indexing Agent based on Reinforcement Learning. Appl. Intell. 50(8): 2575-2588 (2020) [paper]

[Learning-based, DQN] Vishal Sharma, Curtis E. Dyreson, Nicholas Flann: MANTIS: Multiple Type and Attribute Index Selection using Deep Reinforcement Learning. IDEAS 2021: 56-64 [paper]

[Learning-based, DQN] Yu Yan, Shun Yao, Hongzhi Wang, Meng Gao: Index selection for NoSQL database with deep reinforcement learning. Inf. Sci. 561: 20-30 (2021) [paper]

[Learning-based, DQN] Vishal Sharma, Curtis E. Dyreson: Indexer++: Workload-aware Online Index Tuning with Transformers and Reinforcement Learning. SAC 2022: 372-380 [paper]

[Learning-based, MAB] R. Malinga Perera, Bastian Oetomo, Benjamin I. P. Rubinstein, Renata Borovica-Gajic: DBA bandits: Self-driving index tuning under ad-hoc, analytical workloads with safety guarantees. ICDE 2021: 600-611 [paper]

[Learning-based, MAB] R. Malinga Perera, Bastian Oetomo, Benjamin I. P. Rubinstein, Renata Borovica-Gajic: HMAB: Self-Driving Hierarchy of Bandits for Integrated Physical Database Design Tuning. Proc. VLDB Endow. 16(2): 216-229 (2022) [paper]

[Learning-based, MCTS] Xuanhe Zhou, Luyang Liu, Wenbo Li, Lianyuan Jin, Shifu Li, Tianqing Wang, Jianhua Feng: AutoIndex: An Incremental Index Management System for Dynamic Workloads. ICDE 2022: 2196-2208 [paper]

[Learning-based, MCTS] Wentao Wu, Chi Wang, Tarique Siddiqui, Junxiong Wang, Vivek R. Narasayya, Surajit Chaudhuri, Philip A. Bernstein: Budget-aware Index Tuning with Reinforcement Learning. SIGMOD Conference 2022: 1528-1541 [paper]

[Optimization, Learned Cost] Bailu Ding, Sudipto Das, Ryan Marcus, Wentao Wu, Surajit Chaudhuri, Vivek R. Narasayya: AI Meets AI: Leveraging Query Executions to Improve Index Recommendations. SIGMOD Conference 2019: 1241-1258 [paper]

[Optimization, Learned Cost] Jiachen Shi, Gao Cong, Xiaoli Li: Learned Index Benefits: Machine Learning Based Index Performance Estimation. Proc. VLDB Endow. 15(13): 3950-3962 (2022) [paper]

[Optimization, Learned Cost] Jianling Gao, Nan Zhao, Ning Wang, Shuang Hao: SmartIndex: An Index Advisor with Learned Cost Estimator. CIKM 2022: 4853-4856 [paper]

[Optimization, Workload Summarization] Tarique Siddiqui, Saehan Jo, Wentao Wu, Chi Wang, Vivek R. Narasayya, Surajit Chaudhuri: ISUM: Efficiently Compressing Large and Complex Workloads for Scalable Index Tuning. SIGMOD Conference 2022: 660-673 [paper]

[Optimization, What-if Call] Tarique Siddiqui, Wentao Wu, Vivek R. Narasayya, Surajit Chaudhuri: DISTILL: Low-Overhead Data-Driven Techniques for Filtering and Costing Indexes for Scalable Index Tuning. Proc. VLDB Endow. 15(10): 2019-2031 (2022) [paper]

Partition Advisor

Automating physical database design in a parallel database.

Jun Rao, Chun Zhang, Nimrod Megiddo, Guy M. Lohman. SIGMOD, 2002. [paper]

Schism: a Workload-Driven Approach to Database Replication and Partitioning.

Carlo Curino, Yang Zhang, Evan P. C. Jones, Samuel Madden. PVLDB, 2010. [paper]

Locality-aware partitioning in parallel database systems.

Erfan Zamanian, Carsten Binnig, Abdallah Salama. SIGMOD, 2015. [paper]

Query centric partitioning and allocation for partially replicated database systems.

Tilmann Rabl, Hans-Arno Jacobsen. SIGMOD, 2017. [paper]

Workload-driven horizontal partitioning and pruning for large HTAP systems.

Martin Boissier, Kurzynski Daniel. ICDE Workshop, 2018. [paper]

Towards learning a partitioning advisor with deep reinforcement learning.

Benjamin Hilprecht, Carsten Binnig, Uwe Röhm. aiDM@SIGMOD, 2019. [paper]

Automated vertical partitioning with deep reinforcement learning.

Campero Durand G, Piriyev R, Pinnecke M, et al. ADBIS, 2019. [paper]

Fast and effective distribution-key recommendation for amazon redshift.

Panos Parchas, Yonatan Naamad, Peter Van Bouwel, et al. PVLDB, 2020. [paper]

Adaptive partitioning and indexing for in situ query processing.

Olma, M., Karpathiotakis, M., Alagiannis, I., Athanassoulis, et al. VLDB Journal. [paper]

Learning a Partitioning Advisor for Cloud Databases.

Benjamin Hilprecht, Carsten Binnig, Uwe Röhm. SIGMOD, 2020. [paper]

Grep: A Graph Learning Based Database Partitioning System.

Xuanhe Zhou, Guoliang Li, Jianhua Feng, et al. SIGMOD, 2023. [paper] [demo]

Hybrid Advisor

Universal Database Optimization using Reinforcement Learning

Wang J, Trummer I, Basu D. VLDB, 2021. [paper]

A Unified and Efficient Coordinating Framework for Autonomous DBMS Tuning

Xinyi Zhang, Zhuo Chang, HONG WU, et al. SIGMOD, 2023. [paper]

2. Query Optimization

Query Rewriter

(note other interesting problems like text2SQL are not within the scope)

Traditional

[Rewrite Rules] Béatrice Finance, Georges Gardarin. A Rule-Based Query Rewriter in an Extensible DBMS. ICDE 1991. [paper]

[Rewrite Rules] Hamid Pirahesh, Joseph M. Hellerstein, Waqar Hasan. Extensible/Rule Based Query Rewrite Optimization in Starburst. SIGMOD Conference 1992. [paper]

[Cost/Heuristic Rewrite] Rafi Ahmed, Allison W. Lee, Andrew Witkowski, et al. Cost-Based Query Transformation in Oracle. VLDB 2006: 1026-1036. [paper]

[Heuristic Rewrite] De Araújo, A. H. M., Monteiro, J. M., Antônio, J., De Macêdo, F., Tavares, J. A., Brayner, A., & Lifschitz, S. (2014). ARe-SQL: An Online, Automatic and Non-Intrusive Approach for Rewriting SQL Queries. JIDM, 2014. [paper]

[Semantic Equivalence] Shumo Chu, Konstantin Weitz, Alvin Cheung, Dan Suciu. HoTTSQL: proving query rewrites with univalent SQL semantics. PLDI 2017: 510-524. [paper]

[Optimization Engine] Begoli, E., Camacho-Rodríguez, J., Hyde, J., Mior, M. J., & Lemire, D. (2018). Apache calcite: A foundational framework for optimized query processing over heterogeneous data sources. SIGMOD, 2018. [paper]

[Map-Reduce Rewrite] Partho Sarthi, Kaushik Rajan, Akash Lal, Abhishek Modi, et al. Generalized Sub-Query Fusion for Eliminating Redundant I/O from Big-Data Queries. OSDI 2020: 209-224. [paper]

[Streaming] Wentao Wu, Philip A. Bernstein, Alex Raizman, Christina Pavlopoulou. Cost-based Query Rewriting Techniques for Optimizing Aggregates Over Correlated Windows. CoRR abs/2008.12379 (2020) [paper]

[Rewrite Rules] Zhaoguo Wang, Zhou Zhou, Yicun Yang, Haoran Ding, Gansen Hu, Ding Ding, Chuzhe Tang, Haibo Chen, Jinyang Li. WeTune: Automatic Discovery and Verification of Query Rewrite Rules. SIGMOD Conference 2022: 94-107. [paper]

[Rewrite Rules] Qiushi Bai, Sadeem Alsudais, Chen Li. QueryBooster: Improving SQL Performance Using Middleware Services for Human-Centered Query Rewriting. VLDB, 2023. [paper]

Learning-based

[Predicate Rewrite] Qi Zhou, Joy Arulraj, Shamkant B. Navathe, William Harris, Jinpeng Wu. Sia : Optimizing Queries using Learned Predicates. SIGMOD, 2021. [paper]

[Rewrite Strategy] Xuanhe Zhou, Guoliang Li, Chengliang Chai, Jianhua Feng. A Learned Query Rewrite System using Monte Carlo Tree Search. VLDB, 2022. [paper]

Cardinality Estimation

[Card, Query-based] Dutt, A., Wang, C., Nazi, A., Kandula, S., Narasayya, V., & Chaudhuri, S. (2018). Selectivity estimation for range predicates using lightweight models. Proceedings of the VLDB Endowment, 12(9), 1044–1057, 2018. [paper]

[Card, Query-based] Kipf A, Kipf T, Radke B, et al. Learned cardinalities: Estimating correlated joins with deep learning. CIDR, 2019. [paper]

[Card, Query-based] Woltmann L, Hartmann C, Thiele M, et al. Cardinality estimation with local deep learning models. aiDM, 2019. [paper]

[Card, Query-based] Hayek, R., & Shmueli, O. (2020). NN-based Transformation of Any SQL Cardinality Estimator for Handling DISTINCT, AND, OR and NOT. arXiv， 2020. [paper]

[Card, Query-based] Tzoumas K, Deshpande A, Jensen C S. Lightweight graphical models for selectivity estimation without independence assumptions[J]. Proceedings of the VLDB Endowment, 4(11): 852-863, 2011. [paper]

[Card, Query-based] Xiao Hu, Yuxi Liu, Haibo Xiu, Pankaj K. Agarwal, Debmalya Panigrahi, Sudeepa Roy, Jun Yang. Selectivity Functions of Range Queries are Learnable. SIGMOD, 2022. [paper]

[Card, Query-based, Adaptability] Beibin Li, Yao Lu, Srikanth Kandula: Warper: Efficiently Adapting Learned Cardinality Estimators to Data and Workload Drifts. SIGMOD Conference 2022: 1920-1933 [paper]

[Card, Query-based, Robust Encoding & Training] Negi, Parimarjan, Ziniu Wu, Andreas Kipf, Nesime Tatbul, Ryan Marcus, Sam Madden, Tim Kraska, and Mohammad Alizadeh: Robust Query Driven Cardinality Estimation under Changing Workloads. VLDB, 2023. [paper]

[Card, Data-based] Leis, V., Radke, B., Gubichev, A., Kemper, A., & Neumann, T. (2017). Cardinality estimation done right: Index-based join sampling. CIDR, 2017. [paper]

[Card, Data-based] Yang, Z., Liang, E., Kamsetty, A., Wu, C., Duan, Y., Chen, X., … Stoica, I. (2019). Deep Unsupervised Cardinality Estimation. VLDB, 2019. [paper]

[Card, Data-based] Yang, Z., Kamsetty, A., Luan, S., Liang, E., Duan, Y., Chen, X., & Stoica, I. (2020). Neurocard: One cardinality estimator for all tables. Proceedings of the VLDB Endowment, 14(1), 61–73, 2020. [paper]

[Card, Data-based] Zhu R, Wu Z, Han Y, et al. FLAT: Fast, Lightweight and Accurate Method for Cardinality Estimation[J]. arXiv preprint arXiv:2011.09022, 2020. [paper]

[Card, Data-based] Wu Z, Shaikhha A, Zhu R, et al. BayesCard: Revitilizing Bayesian Frameworks for Cardinality Estimation. arXiv preprint arXiv: 2012.14743, 2020. [paper]

[Card, Data-based] Hilprecht, B., Schmidt, A., Kulessa, M., Molina, A., Kersting, K., & Binnig, C. (2020). DeepDB: Learn from data, not from queries! VLDB, 13(7), 992–1005, 2020. [paper]

[Card, Data-based] Yongjoo Park, Shucheng Zhong, and Barzan Mozafari. Quicksel: Quick selectivity learning with mixture models. SIGMOD 2020. [paper]

[Card, Data-based] Lu Y, Kandula S, König A C, et al. Pre-training summarization models of structured datasets for cardinality estimation[J]. Proceedings of the VLDB Endowment, 2021. [paper]

[Card, Data-based] Zhu, R., Wu, Z., Han, Y., Zeng, K., Pfadler, A., Qian, Z., … Cui, B. (2020). FLAT: Fast, Lightweight and Accurate Method for Cardinality Estimation. VLDB, 2021. [paper]

[Card, Data-based] Jiayi Wang, Chengliang Chai, Jiabin Liu, Guoliang Li. FACE: A Normalizing Flow based Cardinality Estimator. VLDB 2022. [paper]

[Card, Data-based] Yao Lu, Srikanth Kandula, Arnd Christian König, Surajit Chaudhuri. Pre-training summarization models of structured datasets for cardinality estimation. VLDB 2022. [paper]

[Card, Query&Data-based] Wu P, Cong G. A Unified Deep Model of Learning from both Data and Queries for Cardinality Estimation. SIGMOD. 2021: 2009-2022. [paper]

[Card] Parimarjan Negi, Ryan C. Marcus, Andreas Kipf, Hongzi Mao, Nesime Tatbul, Tim Kraska, Mohammad Alizadeh. Flow-Loss: Learning Cardinality Estimates That Matter. VLDB Endow, 14(11): 2019-2032, 2021. [paper]

[Card, Model Selection] Jintao Zhang, Chao Zhang, Guoliang Li, Chengliang Chai. AutoCE: An Accurate and Efficient Model Advisor for Learned Cardinality Estimation. ICDE, 2023. [paper]

[Card] Xiaoye Miao, Yangyang Wu, Jiazhen Peng, et al. Efficient and Effective Cardinality Estimation for Skyline Family. SIGMOD, 2023. [paper]

[Card] Ziniu Wu, Parimarjan Negi, Mohammad Alizadeh, Tim Kraska, Samuel Madden. FactorJoin: A New Cardinality Estimation Framework for Join Queries. SIGMOD, 2023. [paper]

[Card, Query-based] Fang Wang, Xiao Yan, Man Lung Yiu, Shuai Li, Zunyao Mao, and Bo Tang. Speeding Up End-to-end Query Execution via Learning-based Progressive Cardinality Estimation. SIGMOD, 2023. [paper]

[ EA&B ] Wang, X., Qu, C., Wu, W., Wang, J., & Zhou, Q. (2021). Are We Ready For Learned Cardinality Estimation? Proc. VLDB Endow. 14(9): 1640-1654 (2021). [paper]

[ EA&B ] Sun, J., Zhang, J., Sun, Z., Li, G., & Tang, N. (n.d.). Learned Cardinality Estimation : A Design Space Exploration and a Comparative Evaluation [ EA & B ]. 14(1). VLDB, 2022. [paper]

[ EA&B ] Yuxing Han, Ziniu Wu, Peizhi Wu, et al. Cardinality Estimation in DBMS: A Comprehensive Benchmark Evaluation Yuxing. VLDB, 2022. [paper]

[ EA&B ] Kyoungmin Kim, Jisung Jung, In Seo, Wook-Shin Han, Kangwoo Choi, Jaehyok Chong: Learned Cardinality Estimation: An In-depth Study. SIGMOD Conference 2022: 1214-1227 [paper]

[ EA&B ] Harmouch, H., & Naumann, F. (2018). Cardinality Estimation: An Experimental Survey. Pvldb, 11(4), 4999–512, 2017. [paper]

Cost Estimation

[Cost] Marcus, R., & Papaemmanouil, O. (2019). Plan-Structured Deep Neural Network Models for Query Performance Prediction. 1733–1746. [paper]

[Cost] Sun, J., & Li, G. (n.d.). An End-to-End Learning-based Cost Estimator. VLDB, 2020. [paper]

[Cost] Benjamin Hilprecht, Carsten Binnig. Zero-Shot Cost Models for Out-of-the-box Learned Cost Prediction. VLDB, 2022. [paper]

Plan Optimization

Continuously Adaptive Query Processing

Ron Avnur, Joseph M. Hellerstein. Eddies. SIGMOD, 2000. [paper]

How Good Are Query Optimizers, Really?

Leis, V., Gubichev, A., Mirchev, A., Boncz, P., Kemper, A., & Neumann, T. Proceedings of the VLDB Endowment (2016), 9(3), 204–215. [paper]

Neo: A Learned query optimizer

Marcus, R., Negi, P., Mao, H., Zhang, C., Alizadeh, M., Kraska, T., … Tatbul, N. (2018). Proceedings of the VLDB Endowment, 12(11), 1705–1718, 2018. [paper]

Deep reinforcement learning for join order enumeration

Marcus, R., & Papaemmanouil, O. (2018). Proceedings of the 1st International Workshop on Exploiting Artificial Intelligence Techniques for Data Management, AiDM 2018, 0–3. [paper]

SkinnerDB : Regret-Bounded Query Evaluation via Reinforcement Learning

Trummer, I., Wang, J., Maram, D., Moseley, S., Jo, S., & Antonakakis, J. (n.d.). SIGMOD, 2019. [paper]

Progressive Join Algorithms Considering User Preference

Ding, M., Chen, S., & Manegold, S. (2021). CIDR, 2021. [paper]

Reinforcement Learning with Tree-LSTM for Join Order Selection

Yu, X., Li, G., Tang, N. (n.d.). ICDE, 2020. [paper]

Towards a Learning Optimizer for Shared Clouds

Chenggang Wu, Alekh Jindal, Saeed Amizadeh, Hiren Patel, Wangchao Le, Shi Qiao, Sriram Rao. Proc. VLDB Endow. 12(3): 210-222, 2018. [paper]

SQL Plan Observability through Hints in Oracle Autonomous Database

Pasupuleti, K., Park, M., & Valluri, S. (n.d.).

Bao: Making Learned Query Optimization Practical

Marcus, R., Negi, P., Mao, H., Tatbul, N., Alizadeh, M., & Kraska, T. (2020). SIGMOD, 2021. [paper]

Steering Query Optimizers: A Practical Take on Big Data Workloads

Parimarjan Negi, Matteo Interlandi, Ryan Marcus, Mohammad Alizadeh, Tim Kraska, Marc Friedman, Alekh Jindal. SIGMOD, 2021. [paper]

SkinnerMT: Parallelizing for Efficiency and Robustness in Adaptive Query Processing on Multicore Platforms

Ziyun Wei, Immanuel Trummer. PVLDB, 2022. [paper]

Learning a Query Optimizer Without Expert Demonstrations

Zongheng Yang, Wei-Lin Chiang, Sifei Luan, Gautam Mittal, Michael Luo, Ion Stoica. Balsa. SIGMOD, 2022 [paper]

Workload-driven, Lazy Discovery of Data Dependencies for Query Optimization

Jan Kossmann. CIDR, 2022 [paper]

LOGER: A Learned Optimizer towards Generating Efficient and Robust Query Execution Plans

Tianyi Chen, Jun Gao, Hedui Chen, and Yaofeng Tu. PVLDB, 2023. [paper]

BASE: Bridging the Gap between Cost and Latency for Query Optimization

Chen, Xu, Zhen Wang, Shuncheng Liu, et al. [paper]

COOOL: A Learning-To-Rank Approach for SQL Hint Recommendations

Xu, Xianghong, Zhibing Zhao, Tieying Zhang, et al. [paper]

Lero: A Learning-to-Rank Query Optimizer

Rong Zhu, Wei Chen, Bolin Ding, Xingguang Chen, Andreas Pfadler, Ziniu Wu, Jingren Zhou. VLDB 2023. [paper]

Cost-based or Learning-based? A Hybrid Query Optimizer for Query Plan Selection

Xiang Yu, Chengliang Chai, Guoliang Li, Jiabin Liu. VLDB 2023. [paper]

Leveraging Query Logs and Machine Learning for Parametric Query Optimization

Kapil Vaidya, Anshuman Dutt, Vivek Narasayya, Surajit Chaudhuri. VLDB 2022. [paper]

Kepler: Robust Learning for Parametric Query Optimization

Lyric Doshi, Vincent Zhuang, Gaurav Jain, Ryan C Marcus, Haoyu Huang, Deniz Altınbüken, Eugene Brevdo, Campbell Fraser. SIGMOD 2023.[paper]

LEON: A New Framework for ML-Aided Query Optimization

Xu Chen, Haitian Chen, Zibo Liang, Shuncheng Liu, Jinghong Wang, Kai Zeng, Han Su, and Kai Zheng.[paper]

3. Workload Scheduling

Ibrahim Sabek, Tenzin Samten Ukyab, Tim Kraska. LSched: A Workload-Aware Learned Query Scheduler for Analytical Database Systems. SIGMOD, 2022. [paper]

Chi Zhang, Ryan Marcus, and et al. Buffer Pool Aware Query Scheduling via Deep Reinforcement Learning. In VLDB, 2020. [paper]

4. Database Design

Index

One-dimensional Index

[1-D, Immutable] Kraska, T., Beutel, A., Chi, E. H., Dean, J., & Polyzotis, N. (2018). The case for learned index structures. SIGMOD, 2018. [paper] [code]

[1-D, Mutable] Galakatos, A., Markovitch, M., Binnig, C., Fonseca, R., & Kraska, T. (2019). Fiting-tree: A data-aware index structure. SIGMOD, 2019. [paper]

[1-D, Mutable, Secondary] Wu, Y., Yu, J., Tian, Y., Sidle, R., Barber, R. (2019). Designing succinct secondary indexing mechanism by exploiting column correlations. SIGMOD 2019. [paper]

[1-D, Mutable] Ferragina, P., & Vinciguerra, G. (2020). The PGM-index : a fully-dynamic compressed learned index with provable worst-case bounds. VLDB, 2020. [paper]

[1-D, Mutable] Ding, J., Minhas, U. F., Yu, J., Wang, C., Do, J., Li, Y., Zhang, H., Chandramouli, B., Gehrke, J., Kossmann, D., Lomet, D., & Kraska, T. (2020). ALEX: An Updatable Adaptive Learned Index. SIGMOD, 2020. [paper] [code]

[1-D, Mutable, Persistent] Lu, B., Ding, J., Lo, E., Minhas, U. F., & Wang, T. (2021). APEX: A High-Performance Learned Index on Persistent Memory. VLDB, 2021. [paper]

[1-D, Immutable, Auto-generated] Dittrich, J., Nix, J., & Schön, C. (2021). The next 50 Years in Database Indexing or: The Case for Automatically Generated Index Structures. VLDB, 2021. [paper] [code]

[1-D, Mutable, Concurrency] Li, P., Hua, Y., Jia, J., Zuo, P. (2021). FINEdex: A Fine-grained Learned Index Scheme for Scalable and Concurrent Memory Systems. VLDB, 2021. [paper]

[1-D, Mutable] Wu, J., Zhang, Y., Chen, S., Wang, J., Chen, Y., Xing, C. (2021). Updatable learned index with precise positions. VLDB, 2021. [paper]

[1-D, Mutable] Ma, C., Yu, X., Li, Y., Meng, X., & Maoliniyazi, A. (2022). FILM: A Fully Learned Index for Larger-Than-Memory Databases. VLDB, 2022. [paper]

[1-D, Mutable, Concurrency] Wang, Z., Chen, H., Wang, Y., & Tang, C. (2022). The Concurrent Learned Indexes for Multicore Data Storage. ACM Transactions on Storage, 18(1), 1-35. [paper] [code]

[1-D, Mutable] Jiaoyi Zhang, Yihan Gao. (2022). CARMI: A Cache-Aware Learned Index with a Cost-based Construction Algorithm. VLDB, 2022. [paper]

[1-D, Mutable] Shangyu Wu. (2022). NFL: Robust Learned Index via Distribution Transformation. VLDB, 2022. [paper]

[1-D, Mutable, Persistent] Zhang, Z., Chu, Z., Jin, P., Luo, Y., Xie, X., Wan, S., Luo, Y., Wu, X., Zou, P., Zheng, C., Wu, G., Rudoff. A. (2022). PLIN: A Persistent Learned Index for Non-Volatile Memory with High Performance and Instant Recovery. VLDB, 2022. [paper]

[1-D, Mutable] Li, Pengfei, Hua Lu, Rong Zhu, Bolin Ding, et al. (2023). DILI: A Distribution-Driven Learned Index. VLDB, 2023. [paper]

[1-D, Mutable, Persistent] Yulai Tong, Jiazhen Liu, Hua Wang, Ke Zhou, Rongfeng He, Qin Zhang, and Cheng Wang. (2023). Sieve: A Learned Data-Skipping Index for Data Analytics. VLDB, 2023. [paper]

Multi-dimensional Index

[Multi-D, Immutable] Nathan, V., Ding, J., Alizadeh, M., & Kraska, T. (2020). Learning multi-dimensional indexes. SIGMOD, 2020. [paper]

[Multi-D, Mutable, Persistent] Li, P., Lu, H., Zheng, Q., Yang, L., & Pan, G. (2020). LISA: A Learned Index Structure for Spatial Data. SIGMOD, 2020. [paper]

[Multi-D, Mutable, Persistent] Qi, J., Liu, G., Jensen, C.S., Kulik, L. (2020). Effectively learning spatial indices. VLDB, 2020. [paper]

[Multi-D, Immutable] Ding, J., Nathan, V., Alizadeh, M., & Kraska, T. (2020). Tsunami: A learned multi-dimensional index for correlated data and skewed workloads. VLDB, 2020. [paper]

[Multi-D, Mutable] Dong, H., Chai, C., Luo, Y., Liu, J., Feng, J., Zhan, C. (2022). RW-Tree: A Learned Workload-aware Framework for R-tree Construction. ICDE, 2022. [paper]

[Multi-D, Immutable] Gao, J., Cao, X., Yao, X., Zhang, G., & Wang, W. (2023). LMSFC: A Novel Multidimensional Index based on Learned Monotonic Space Filling Curves. VLDB, 2023. [paper]

Experiment and Analysis

[1-D, Immutable, Analysis] Ferragina, P., Lillo, F., & Vinciguerra, G. (2020). Why are learned indexes so effective?. ICML, 2020. [paper]

[1-D, Immutable, Experiment] Marcus, R., Stoian, M., Kipf, A., Misra, S., van Renen, A., Kemper, A., Neumann, T., & Kraska, T. (2020). Benchmarking learned indexes. VLDB, 2020. [paper] [code]

[1-D, Poisoning Attack] Evgenios M. Kornaropoulos, Silei Ren, Roberto Tamassia. (2022). The Price of Tailoring the Index to Your Data: Poisoning Attacks on Learned Index Structures. SIGMOD, 2022. [paper]

[1-D, Mutable, Experiment] Wongkham, C., Lu, B., Liu, C., Zhong, Z., Lo, E., Wang, T. (2022). Are Updatable Learned Indexes Ready?. VLDB, 2022. [paper]

[1-D, Immutable, Experiment] Maltry, M., Dittrich, J. (2022). A critical analysis of recursive model indexes. VLDB, 2022. [paper]

[1-D, Hash Index, Experiment] Sabek, I., Vaidya, K., Horn TUM, D., Kipf, A., Mitzenmacher, M., Kraska, T., Horn, D., Kraska Can, T. (2022) Can Learned Models Replace Hash Functions?. VLDB, 2022. [paper]

[1-D, Mutable, Experiment] Sun, Z., Zhou, X., Li, G. (2023). Learned Index: A Comprehensive Experimental Evaluation. VLDB, 2023. [paper] [code]

[1-D, Immutable, Experiment] Sabek, I., & Kraska, T. (2023). The Case for Learned In-Memory Joins. VLDB, 2023. [paper]

Layout

[Learned Layout] Liwen Sun, Michael J. Franklin, Sanjay Krishnan, et al. Fine-grained partitioning for aggressive data skipping. SIGMOD, 2014. [paper]

[Learned Layout] Yang, Z., Chandramouli, B., Wang, C., Gehrke, J., Li, Y., Minhas, U. F., … Acharya, R. (n.d.). Qd-tree: Learning Data Layouts for Big Data Analytics. SIGMOD, 2020. [paper]

[Learned Layout] Jialin Ding, Umar Farooq Minhas, Badrish Chandramouli, et al. Instance-Optimized Data Layouts for Cloud Analytics Workloads. SIGMOD, 2021. [paper]

[Learned Layout] Bandle, M., Giceva, J., & Neumann, T. (2021). To Partition, or Not to Partition, That is the Join Question in a Real System. SIGMOD, 2021. [paper]

[Data Container] Madden S, Ding J, Kraska T, Sudhir S, Cohen D, Mattson T, Tatbul N. Self-Organizing Data Containers. CIDR, 2022. [paper]

[Learned Layout] Teng Zhang, Jian Tan, Xin Cai, Jianying Wang, Feifei Li, Jianling Sun. SA-LSM : Optimize Data Layout for LSM-tree Based Storage using Survival Analysis. VLDB, 2022. [paper]

[Learned Layout] Michael Abebe. Tiresias: Enabling Predictive Autonomous Storage and Indexing. VLDB, 2022. [paper]

Query Execution

[CodeGen] Immanuel Trummer. CodexDB: Synthesizing Code for Qery Processing from Natural Language Instructions using GPT-3 Codex. VLDB, 2022. [paper]

Zhang, C., Marcus, R., Kleiman, A., & Papaemmanouil, O. (2020). Buffer Pool Aware Query Scheduling via Deep Reinforcement Learning. AIDB@VLDB, 2020. [paper]

5. Database Monitoring

[Trend Prediction] L. Ma, D. V. Aken, A. Hefny, G. Mezerhane, A. Pavlo, and G. J. Gordon, “Query-based Workload Forecasting for Self-driving Database Management Systems,” in SIGMOD, 2018. [paper]

[Performance Prediction] Dorn, J., Apel, S., & Siegmund, N. (n.d.). Mastering Uncertainty in Performance Estimations of Configurable Software Systems. (3).

[Performance Prediction] Marcus, R., & Papaemmanouil, O. (2019). Plan-structured deep neural network models for query performance prediction. Proceedings of the VLDB Endowment, 12(11), 1733–1746. [paper]

[Performance Prediction] Wu, W., Chi, Y., Hacig̈um̈uş, H., & Naughton, J. F. (2013). Towards predicting query execution time for concurrent and dynamic database workloads. Proceedings of the VLDB Endowment, 6(10), 925–936. [paper]

[Performance Prediction] Duggan, J., Papaemmanouil, O., Cetintemel, U., & Upfal, E. (2014). Contender: A resource modeling approach for concurrent query performance prediction. Advances in Database Technology - EDBT 2014: 17th International Conference on Extending Database Technology, Proceedings, 109–120. [paper]

[Performance Prediction] Wu, W., Chi, Y., Zhu, S., Tatemura, J., Hacigümüş, H., & Naughton, J. F. (2013). Predicting query execution time: Are optimizer cost models really unusable? Proceedings - International Conference on Data Engineering, (1), 1081–1092. [paper]

[Performance Prediction] Higginson, A. S., Dediu, M., Arsene, O., Paton, N. W., & Embury, S. M. (2020). Database Workload Capacity Planning using Time Series Analysis and Machine Learning. Proceedings of the ACM SIGMOD International Conference on Management of Data, 769–783. [paper]

[Performance Prediction] Unterbrunner, P., Giannikis, G., Alonso, G., Fauser, D., & Kossmann, D. (2009). Predictable performance for unpredictable workloads. Proceedings of the VLDB Endowment, 2(1), 706–717. [paper]

[Performance Prediction] Xuanhe Zhou, Ji Sun, Guoliang Li, Jianhua Feng. Query Performance Prediction for Concurrent Queries using Graph Embedding. [paper]

6. Database Diagnosis

System and Kernel Causes

Automatic Performance Diagnosis and Tuning in Oracle

Karl Dias, Mark Ramacher, Uri Shaft, et al. CIDR, 2005. [paper]

DBSherlock: A Performance Diagnostic Tool for Transactional Databases.

Yoon, D. Y., Niu, N., & Mozafari, B. SIGMOD, 2016. [paper]

iQCAR: inter-Query Contention Analyzer for Data Analytics Frameworks.

Kalmegh, P., Babu, S., & Roy, S. SIGMOD, 2019. [paper]

FluxInfer: Automatic Diagnosis of Performance Anomaly for Online Database System

Ping Liu, Shenglin Zhang, Yongqian Sun, et al. IPCCC, 2020. [paper]

Diagnosing Root Causes of Intermittent Slow Queries in Cloud Databases.

Minghua Ma, Zheng Yin, Shenglin Zhang, et al. VLDB, 2020. [paper]

Generic and Robust Performance Diagnosis via Causal Inference for OLTP Database Systems.

Xianglin Lu, Zhe Xie, Zeyan Li, et al. CCGrid, 2022. [paper]

DBPA: A Benchmark for Transactional Database Performance Anomalies.

Shiyue Huang,Ziwei Wang, Xinyi Zhang, et al. SIGMOD, 2023. [paper]

Bottleneck Queries

PinSQL: Pinpoint Root Cause SQLs to Resolve Performance Issues in Cloud Databases.

Xiaoze Liu, Zheng Yin, Chao Zhao, et al. ICDE, 2022. [paper]

7. General Techniques

Feature Engineering for DB

[PlanEncoding] Yue Zhao, Gao Cong, Jiachen Shi, Chunyan Miao. QueryFormer: A Tree Transformer Model for Query Plan Representation. VLDB, 2022. [paper]

[Plan2Feature] Debjyoti Paul, Jie Cao, Feifei Li, Vivek Srikumar. Database Workload Characterization with Query Plan Encoders. VLDB, 2022. [paper]

[Pretrained Representation] Xiu Tang, Sai Wu, Mingli Song, Shanshan Ying, Feifei Li, Gang Chen: PreQR: Pre-training Representation for SQL Understanding. SIGMOD Conference 2022: 204-216 [paper]

[WorkloadAsGraph] Sanjay Agrawal, Eric Chu, Vivek R. Narasayya. Automatic physical design tuning: workload as a sequence. SIGMOD, 2006. [paper]

[DataSummary] Brit Youngmann et al. Guided Exploration of Data Summaries. VLDB, 2022. [paper]

Jiang H, Liu C, Paparrizos J, et al. Good to the Last Bit: Data-Driven Encoding with CodecDB. SIGMOD 2021. [paper]

Feature Engineering for AI

Apache flink: Stream and batch processing in a single engine[J].

Carbone P, Katsifodimos A, Ewen S, et al. The Bulletin of the Technical Committee on Data Engineering, 2015, 38(4). [paper]

Feature selection in machine learning: A new perspective[J/OL].

CAI J, LUO J, WANG S et al. Neurocomputing (Amsterdam), 2018, 300: 70-79. DOI:10.1016/j.neucom.2017.11.077. [paper]

Optimizing in-memory database engine for AI-powered on-line decision augmentation using persistent memory.

Cheng Chen, Jun Yang, Mian Lu, Taize Wang, Zhao Zheng, Yuqiang Chen, Wenyuan Dai, Bingsheng He, Weng-Fai Wong, Guoan Wu, Yuping Zhao, and Andy Rudoff. 2021. Proc. VLDB Endow. 14, 5 (January 2021), 799–812. [paper]

Managing ML pipelines: feature stores and the coming wave of embedding ecosystems[J].

Orr L, Sanyal A, Ling X, et al. arXiv preprint arXiv:2108.05053, 2021. [paper]

A System for Time Series Feature Extraction in Federated Learning.

Siqi Wang, Jiashu Li, Mian Lu, Zhao Zheng, Yuqiang Chen, and Bingsheng He. 2022. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management (CIKM '22). Association for Computing Machinery, New York, NY, USA, 5024–5028. [paper]

FEBench: A Benchmark for Real-Time Relational Data Feature Extraction.

Xuanhe Zhou, Cheng Chen, Kunyi Li, Bingsheng He, Mian Lu, Qiaosheng Liu, Wei Huang, Guoliang Li, Zhao Zheng, Yuqqiang Chen. 2023. Proc. VLDB Endow. [paper]

Model Transfer

Meghdad Kurmanji, Peter Triantafillou. Detect, Distill and Update: Learned DB Systems Facing Out of Distribution Data. SIGMOD, 2023. [paper]

Query And Data Generation

Query Generation

L.Zhang, C.Chai, X.Zhou, and G.Li. Learned sqlgen: Constraint-aware sql generation using reinforcement learning. In SIGMOD, 2022. [paper]

Liu X, Kong X, Liu L, et al. TreeGAN: syntax-aware sequence generation with generative adversarial networks. In ICDM, 2018. [paper]

Data Generation

[DeepAR] Jingyi Yang, Peizhi Wu, Gao Cong, Tieying Zhang, Xiao He. SAM: Database Generation from Query Workloads with Supervised Autoregressive Models. SIGMOD, 2022. [paper]

Francesco Ventura, Zoi Kaoudi, Jorge-Arnulfo Quiané-Ruiz, Volker Markl. Expand your training limits! Generating training data for ML-based data management. SIGMOD, 2021 [paper]

Ju Fan, Tongyu Liu, Guoliang Li, Yuwei Shen, Xiaoyong Du. Relational Data Synthesis using Generative Adversarial Networks: A Design Space Exploration. VLDB 2020. [paper]

8. Database Frameworks

Self-Driving Database Management Systems.

Andrew Pavlo, Gustavo Angulo, Joy Arulraj, et al. CIDR, 2017. [paper]

Cloud native database systems at Alibaba: Opportunities and challenges.

Feifei Li. VLDB, 2018. [paper]

SageDB: A learned database system.

Tim Kraska, Mohammad Alizadeh, Alex Beutel, et al. CIDR, 2019. [paper]

MonetDBLite: An embedded analytical database.

Mark Raasveldt. SIGMOD, 2018. [paper]

XuanYuan: An AI-Native Database.

Guoliang Li, Xuanhe Zhou, Sihao Li. Data Eng., 2019 [paper]

DBMS Fitting: Why should we learn what we already know?

Benjamin Hilprecht, Tiemo Bang, Muhammad El-Hindi, et al. CIDR, 2020. [paper]

MB2 : Decomposed Behavior Modeling for Self-Driving Database Management Systems.

Lin Ma, William Zhang, Jie Jiao, et al. SIGMOD, 2021. [paper]

openGauss: An Autonomous Database System.

Guoliang Li, Xuanhe Zhou, Ji Sun, et al. VLDB, 2021. [paper]

From Natural Language Processing to Neural Databases.

James Thorne, Majid Yazdani, Marzieh Saeidi, et al. VLDB, 2021. [paper]

One Model to Rule them All: Towards Zero-Shot Learning for Databases.

Benjamin Hilprecht, Carsten Binnig. CIDR, 2022. [paper]

A Unified Transferable Model for ML-Enhanced DBMS.

Ziniu Wu, et al. CIDR, 2022. [paper]

PerfGuard: Deploying ML-for-Systems without Performance Regressions, Almost!

Remmelt Ammerlaan, Gilbert Antonius, Marc Friedman, et al. VLDB, 2022. [[paper]

Database Gyms.

Lim, Wan Shen, Matthew Butrovich, William Zhang, et al. CIDR, 2023. [paper]

mutable: A Modern DBMS for Research and Fast Prototyping.

Immanuel L Haffner, Jens Dittrich. CIDR, 2023. [paper]

SageDB: An Instance-Optimized Data Analytics System.

Jialin Ding, Ryan Marcus, Andreas Kipf, et al. VLDB, 2022. [paper]

Towards Building Autonomous Data Services on Azure.

Yiwen Zhu, Yuanyuan Tian, Joyce Cahoon, et al. SIGMOD, 2023. [paper]

9. Demonstrations

[DB Tuning] Bohan Zhang, Dana Van Aken, Justin Wang, Tao Dai, Shuli Jiang, Jacky Lao, Siyuan Sheng, Andrew Pavlo, Geoffrey J. Gordon. A Demonstration of the ottertune automatic database management system tuning service. VLDB, 2018. [paper]

[O&M Platform] Xuanhe Zhou, Lianyuan Jin, Ji Sun, Xinyang Zhao, Xiang Yu, Shifu Li, Tianqing Wang, Kun Li, luyang liu. DBMind: A Self-Driving Platform in openGauss. VLDB, 2021. [paper] [website]

[DB Tuning] Junxiong Wang, Immanuel Trummer, Debabrota Basu. Demonstrating UDO: A Unified Approach for Optimizing Transaction Code, Physical Design, and System Parameters via Reinforcement Learning. SIGMOD, 2021. [paper]

[DB Tuning] Immanuel Trummer. Demonstrating DB-BERT: A Database Tuning Tool that "Reads" the Manual. SIGMOD, 2022. [paper]

[DB Tuning] Luming Sun, Tao Ji, Cuiping Li, Hong Chen. DeepO: A Learned Query Optimizer. SIGMOD, 2022. [paper]

[DB Tuning] Xuanhe Zhou, Guoliang Li, Jianming Wu, Jiesi Liu, Zhaoyan Sun, Xinning Zhang. A Learned Query Rewrite System. VLDB, 2023. [paper] [website]

[DB Tuning] Wei Zhou, Chen Lin, Xuanhe Zhou, Guoliang Li, Tianqing Wang. Demonstration of ViTA: Visualizing, Testing and Analyzing Index Advisors. CIKM, 2023. [video]

[DB Tuning] Qiushi Bai, Sadeem Alsudais, Chen Li. Demo of QueryBooster: Supporting Middleware-based SQL Query Rewriting as a Service. VLDB, 2023. [paper]

[DB Tuning] Christoph Anneser, Mario Petruccelli, Nesime Tatbul, David Cohen, Zhenggang Xu, Prithviraj Pandian, Nikolay Laptev, Ryan Marcus, and Alfons Kemper. QO-Insight: Inspecting Steered Query Optimizers. VLDB, 2023. [paper]

[DB Tuning] Zilong Wang, Qixiong Zeng, Ning Wang, Haowen Lu, Yue Zhang. CEDA: Learned Cardinality Estimation with Domain Adaptation. VLDB, 2023. [paper]

[DB Tuning] Junxiong Wang, Immanuel Trummer, Ahmet Kara, Dan Olteanu. ADOPT: Adaptively Optimizing Attribute Orders for Worst-Case Optimal Join Algorithms via Reinforcement Learning. VLDB, 2023. [paper]

[DB Tuning] Immanuel Trummer. Demonstrating GPT-DB: Generating Query-Specifc and Customizable Code for SQL Processing with GPT-4. VLDB, 2023. [paper]

[DB Diagnosis] Xiu Tang, Sai Wu, Dongxiang Zhang, Ziyue Wang, Gongsheng Yuan, and Gang Chen. A Demonstration of DLBD: Database Logic Bug Detection System. VLDB, 2023. [paper]