Machine Learning for Software Engineering

This repository contains a curated list of papers, PhD theses, datasets, and tools that are devoted to research on Machine Learning for Software Engineering. The papers are organized into popular research areas so that researchers can find recent papers and state-of-the-art approaches easily.

Please feel free to send a pull request to add papers and relevant content that are not listed here.

Note: to quickly access this page, use ml4se.dev

Papers

Type Inference

TypeT5: Seq2seq Type Inference using Static Analysis (2023), ICLR'23, Wei, Jiayi, et al. [pdf]
Do Machine Learning Models Produce TypeScript Types that Type Check? (2023), arxiv, Yee, M., and Arjun G. [pdf]
Cross-Domain Evaluation of a Deep Learning-Based Type Inference System (2022), arxiv, Gruner, Bernd, et al. [pdf] [code]
Learning To Predict User-Defined Types (2022), TSE'22, Jesse, Keven, et al. [pdf]
Recovering Container Class Types in C++ Binaries (2022), CGO'22, Wang, Xudong, et al.
Finding the Dwarf: Recovering Precise Types from WebAssembly Binaries (2022), PLDI'22, Lehmann, Daniel and Pradel, Michael [pdf]
Type4Py: Practical Deep Similarity Learning-Based Type Inference for Python (2022), ICSE'22, Mir, Amir, et al. [pdf][code]
Static Inference Meets Deep Learning: A Hybrid Type Inference Approach for Python (2022), ICSE'22, Peng, Yun, et al. [pdf]
Type Inference as Optimization (2021), NeurIPS'21 AIPLANS, Pandi, Irene Vlassi, et al. [pdf]
SimTyper: Sound Type Inference for Ruby using Type Equality Prediction (2021), OOPSLA'21, Kazerounian, Milod, et al.
Learning type annotation: is big data enough? (2021), FSE 2021, Jesse, Kevin, et al. [pdf][code]
Cross-Lingual Adaptation for Type Inference (2021), arxiv 2021, Li, Zhiming, et al. [pdf]
PYInfer: Deep Learning Semantic Type Inference for Python Variables (2021), arxiv 2021, Cui, Siwei, et al. [pdf]
Advanced Graph-Based Deep Learning for Probabilistic Type Inference (2020), arxiv 2020, Ye, Fangke, et al. [pdf]
Typilus: Neural Type Hints (2020), PLDI 2020, Allamanis, Miltiadis, et al. [pdf][code]
LambdaNet: Probabilistic Type Inference using Graph Neural Networks (2020), arxiv 2020, Wei, Jiayi, et al. [pdf]
TypeWriter: Neural Type Prediction with Search-based Validation (2019), arxiv 2019, Pradel, Michael, et al. [pdf]
NL2Type: Inferring JavaScript Function Types from Natural Language Information (2019), ICSE 2019, Malik, Rabee S., et al. [pdf][code]
Deep Learning Type Inference (2018), ESEC/FSE 2018, Hellendoorn, Vincent J., et al. [pdf][code]
Python Probabilistic Type Inference with Natural Language Support (2016), FSE 2016, Xu, Zhaogui, et al.
Predicting Program Properties from “Big Code” (2015) ACM SIGPLAN 2015, Raychev, Veselin, et al. [pdf]

Code Completion

Enriching Source Code with Contextual Data for Code Completion Models: An Empirical Study (2023), MSR'23, van Dam, Tim, et al. [pdf]
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation (2023), arxiv, Zhang, Fengji, et al. [pdf]
COCOMIC: ✿✿✿✿ Code ✿✿✿✿ Completion By Jointly Modeling In-file and ✿✿Cross-file Context (2022), Ding, Yangruibo, et al. [pdf]
Boosting source code suggestion with self-supervised Transformer Gated Highway (2022), JSS, Hussain, Yasir, et al.
Syntax-Aware On-the-Fly Code Completion (2022), arxiv, Takerngsaksiri, W., et al. [pdf]
Learning to Prevent Profitless Neural Code Completion (2022), arxiv, Sun, Z., et al. [pdf]
All You Need Is Logs: Improving Code Completion by Learning from Anonymous IDE Usage Logs (2022), arxiv, Bibaev, Vitaliy, et al. [pdf]
CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequences (2022), ICSE'22, Izadi, Maliheh, et al. [pdf] [code]
Code Completion by Modeling Flattened Abstract Syntax Trees as Graphs (2021), AAAI'21, Wang, Yanlin, et al. [pdf]
Code Prediction by Feeding Trees to Transformers (2021), ICSE'21, Kim, Seohyun, et al. [pdf]
Fast and Memory-Efficient Neural Code Completion (2020), arxiv 2020, Svyatkovskoy, Alexey, et al. [pdf]
Pythia: AI-assisted Code Completion System (2019), KDD'19, Svyatkovskiy, Alexey, et al. [pdf]
Code Completion with Neural Attention and Pointer Networks (2018), arxiv 2018, Li, Jian, et al. [pdf]

Code Generation

Think Outside the Code: Brainstorming Boosts Large Language Models in Code Generation (2023), arxiv, Li, Xin-Ye, et al. [pdf]
Neural Machine Translation for Code Generation (2023), arxiv, KC, Dharma, and Clayton T. M. [pdf]
CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Evaluations on HumanEval-X (2023), arxiv, Zheng, Qinkai, et al. [pdf]
Towards Enhancing In-Context Learning for Code Generation (2023), arxiv, Li, Jia, et al. [pdf]
Knowledge Transfer for Pseudo-code Generation from Low Resource Programming Language (2023), arxiv, Sontakke, Ankita, et al. [pdf]
MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation (2023), TSE, Paul, Rishov, et al.
Self-collaboration Code Generation via ChatGPT (2023), arxiv, Dong, Yihong, et al. [pdf]
Greener yet Powerful: Taming Large Code Generation Models with Quantization (2023), arxiv, Wei, Xiaokai, et al. [pdf]
A Syntax-Guided Multi-Task Learning Approach for Turducken-Style Code Generation (2023), arxiv, Yang, Guang, et al. [pdf]
WikiCoder: Learning to Write Knowledge-Powered Code (2023), arxiv, Matricon, Théo, et al. [pdf]
Self-planning Code Generation with Large Language Model (2023), arxiv, Jiang, Xue, et al. [pdf]
Systematically Finding Security Vulnerabilities in Black-Box Code Generation Models. (2023), arxiv, Hajipour, Hossein, et al. [pdf]
Exploring Data Augmentation for Code Generation Tasks (2023), arxiv, C., Pinzhen, and G. Lampouras [pdf]
Controlling Large Language Models to Generate Secure and Vulnerable Code (2023), arxiv, He, J., and M. Vechev [pdf]
SKCODER: A Sketch-based Approach for Automatic Code Generation (2023), arxiv, Li, Jia, et al. [pdf]
LEVER: Learning to Verify Language-to-Code Generation with Execution (2023), arxiv, Ni, Ansong, et al. [pdf]
CodeScore: Evaluating Code Generation by Learning Code Execution (2023), arxiv, Dong, Yihong, et al. [pdf]
Program Generation from Diverse Video Demonstrations (2023), arxiv, Manchin, Anthony, et al. [pdf]
Execution-based Code Generation using Deep Reinforcement Learning (2023), arxiv, Shojaee, Parshin, et al. [pdf]
SantaCoder: don't reach for the stars! (2023), arxiv, Allal, Loubna Ben, et al. [pdf]
Exploring and Evaluating Personalized Models for Code Generation, FSE'22, Zlotchevski, Andrei, et al.
Natural Language to Code Generation in Interactive Data Science Notebooks (2022), arxiv, Yin, Pengcheng, et al. [pdf]
Asking Clarification Questions for Code Generation in General-Purpose Programming Language (2022), arxiv, Li, Haau-Sing, et al. [pdf]
ExploitGen: Template-augmented exploit code generation based on CodeBERT (2022), JSS journal, Yang, Guang, et al.
Explicit Knowledge Transfer for Weakly-Supervised Code Generation (2022), arxiv, Azerbayev, Zhangir, et al. [pdf]
Program Generation from Diverse Video Demonstrations (2022), Manchin123, Anthony, et al. [pdf]
Coder Reviewer Reranking for Code Generation (2022), arxiv, Zhang, Tianyi, et al. [pdf]
Execution-based Evaluation for Data Science Code Generation Models (2022), arxiv, Huang, Junjie, et al. [pdf]
Multi-lingual Evaluation of Code Generation Models (2022), arxiv, Athiwaratkun, Ben, et al. [pdf][code]
DocCoder: Generating Code by Retrieving and Reading Docs (2022), arxiv, Zhou, Shuyan, et al. [pdf]
Compilable Neural Code Generation with Compiler Feedback (2022), ACL'22, Wang, Xin, et al. [pdf]
T5QL: Taming language models for SQL generation (2022), arxiv, Arcadinho, S., et al. [pdf]
Incorporating Domain Knowledge through Task Augmentation for Front-End JavaScript Code Generation (2022), arxiv, Shen, Sijie, et al. [pdf]
Language Models Can Teach Themselves to Program Better (2022), arxiv, Haluptzok, Patrick, et al. [pdf]
DocCoder: Generating Code by Retrieving and Reading Docs (2022), arxiv, Zhou, Shuyan, et al. [pdf]
CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning (2022), arxiv, Le, Hung, et al. [pdf]
Repository-Level Prompt Generation for Large Language Models of Code (2022), arxiv, Shrivastava, Disha, et al. [pdf]
CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generation (2022), arxiv, Zan, Daoguang, et al. [pdf]
NatGen: Generative pre-training by “Naturalizing” source code (2022), FSE'22, Chakraborty, Saikat, et al. [pdf]
StructCoder: Structure-Aware Transformer for Code Generation (2022), arxiv, Tipirneni, Sindhu, et al. [pdf]
Compilable Neural Code Generation with Compiler Feedback (2022), arxiv 2022, Wang, Xin, et al. [pdf]
Predictive Synthesis of API-Centric Code (2022), arxiv 2022, Nam, Daye, et al. [pdf]
Code Prediction by Feeding Trees to Transformers (2020), arxiv 2020, Kim, Seohyun, et al. [pdf]
TreeGen: A Tree-Based Transformer Architecture for Code Generation (2019), arxiv 2019, Zhu, Qihao, et al. [pdf]
A Parallel Corpus of Python Functions and Documentation Strings for Automated Code Documentation and Code Generation (2017), arxiv 2017, Barone, Antonio V. M., et al. [pdf]

Code Summarization

Tram: A Token-level Retrieval-augmented Mechanism for Source Code Summarization (2023), arxiv, Ye, Tong, et al. [pdf]
Automatic Code Summarization via ChatGPT: How Far Are We? (2023), arxiv, Sun, Weisong, et al.
Function Call Graph Context Encoding for Neural Source Code Summarization (2023), TSE, Bansal, Aakash, et al.
Label Smoothing Improves Neural Source Code Summarization (2023), arxiv, Haque, Sakib, et al. [pdf]
Demystifying What Code Summarization Models Learned (2023), arxiv, Wang, Yu, and Ke Wang. [pdf]
CoSS: Leveraging Statement Semantics for Code Summarization (2023), TSE, Shi, Chaochen, et al.
An Extensive Study of the Structure Features in Transformer-based Code Semantic Summarization (2023), RG, Yang, Kang, et al. [pdf]
Interpretation-based Code Summarization (2023), arxiv, Geng, Mingyang, et al. [pdf]
Towards Retrieval-Based Neural Code Summarization: A Meta-Learning Approach (2023), TSE, Zhou, Ziyi, et al.
CLG-Trans: Contrastive Learning for Code Summarization via Graph Attention-based Transformer (2023), SCP journal, Zeng, Jianwei, et al.
ClassSum: a deep learning model for class-level code summarization (2022), Springer NCA, Li, Mingchen, et al. [code]
Boosting Code Summarization by Embedding Code Structures (2022), COLING'22, Son, Jikyoeng, et al. [pdf]
Low-Resources Project-Specific Code Summarization (2022), ASE'22, Xie, Rui, et al. [pdf]
Few-shot training LLMs for project-specific code-summarization (2022), arxiv, A., Toufique, and P. Devanbu. [pdf]
Are We Building on the Rock? On the Importance of Data Preprocessing for Code Summarization (2022), FSE'22, Shi, Lin, et al. [pdf]
Learning code summarization from a small and local dataset (2022), arxiv, Ahmed, Toufique, and Devanbu, P. [pdf]
Modeling Hierarchical Syntax Structure with Triplet Position for Source Code Summarization (2022), ACL'22, Guo, Juncai, et al. [pdf]
AST-Trans: Code Summarization with Efficient Tree-Structured Attention (2022), ICSE'22, Tang, Ze, et al. [pdf]
GypSum: Learning Hybrid Representations for Code Summarization (2022), ICPC'22, Wang, Yu, et al. [pdf]
M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarization (2022), ICPC'22, Gao, Yuexiu and Lyu, Chen [pdf]
Project-Level Encoding for Neural Source Code Summarization of Subroutines (2021), ICPC'21, Bansal, Aakash, et al. [pdf]
Code Structure Guided Transformer for Source Code Summarization (2021), arxiv 2021, Gao, Shuzheng, et al. [pdf]
Source Code Summarization Using Attention-Based Keyword Memory Networks (2020), IEEE BigComp 2020, Choi, YunSeok, et al.
A Transformer-based Approach for Source Code Summarization (2020), arxiv 2020, Ahmad, Wasi Uddin, et al. [pdf]
Learning to Represent Programs with Graphs (2018), ICLR'18, Allamanis, Miltiadis, et al. [pdf]
A Convolutional Attention Network for Extreme Summarization of Source Code (2016), ICML 2016, Allamanis, Miltiadis, et al. [pdf]

Code Embeddings/Representation

xASTNN: Improved Code Representations for Industrial Practice (2023), arxiv, Xu, Zhiwei, et al. [pdf]
Toward Interpretable Graph Tensor Convolution Neural Network for Code Semantics Embedding (2023), TOSEM, Yang, Jia, et al.
Practical Binary Code Similarity Detection with BERT-based Transferable Similarity Learning (2022), ACSAC'22, Ahn, Sunwoo, et al.
CLAWSAT: Towards Both Robust and Accurate Code Models (2022), arxiv, Jia, Jinghan, et al. [pdf]
sem2vec: Semantics-Aware Assembly Tracelet Embedding (2022), TSE, Wang, Huaijin, et al.
COMBO: Pre-Training Representations of Binary Code Using Contrastive Learning (2022), arxiv, Zhang, Yifan, et al. [pdf]
Soft-Labeled Contrastive Pre-training for Function-level Code Representation (2022), arxiv, Li, Xiaonan, et al. [pdf]
A Tree-structured Transformer for Program Representation Learning (2022), arxiv, Wang, Wenhan, et al. [pdf]
What does Transformer learn about source code? (2022), arxiv, Zhang, Kechi, et al. [pdf]
Diet Code is Healthy: Simplifying Programs for Pre-Trained Models of Code (2022), arxiv, Zhang, Zhaowei, et al. [pdf]
MetaTPTrans: A Meta Learning Approach for Multilingual Code Representation Learning (2022), arxiv, Pian, Weiguo, et al. [pdf]
Towards Learning (Dis)-Similarity of Source Code from Program Contrasts (2022), ACL'22, Ding, Yangruibo, et al. [pdf]
Towards Learning Generalizable Code Embeddings using Task-agnostic Graph Convolutional Networks (2022), TOSEM, Ding, Zishuo, et al.
Learning to Represent Programs with Code Hierarchies (2022), arxiv, Nguyen, Minh and Nghi DQ Bui, [pdf]
CV4Code: Sourcecode Understanding via Visual Code Representations (2022), arxiv, Shi, Ruibo, et al. [pdf]
Hyperbolic Representations of Source Code (2022), AAAI'22, Khan, Raiyan, et al. [pdf]
Unified Abstract Syntax Tree Representation Learning for Cross-Language Program Classification (2022), ICPC'22, Wang, Kesu, et al. [pdf]
Hierarchical Semantic-Aware Neural Code Representation (2022), JSS'22, Jiang, Yuan, et al.
CODE-MVP: Learning to Represent Source Code from Multiple Views with Contrastive Pre-Training (2022), arxiv 2022, Wang, Xin, et al. [pdf]
Hierarchical Heterogeneous Graph Attention Network for Syntax-Aware Summarization (2022), AAAI'22, Song, Z., and King, I., [pdf]
Unleashing the Power of Compiler Intermediate Representation to Enhance Neural Program Embeddings (2022), ICSE'22, Li, Zongjie, et al. [pdf]
XCode: Towards Cross-Language Code Representation with Large-Scale Pre-Training (2022), TOSEM'22, Lin, Zehao, et al.
Fold2Vec: Towards a Statement Based Representation of Code for Code Comprehension (2022), TOSEM'22, Bertolotti, Francesco and Cazzola, Walter
HELoC: Hierarchical Contrastive Learning of Source Code Representation (2022), ICPC'22, Wang, Xiao, et al. [pdf]
Multi-View Graph Representation for Programming Language Processing: An Investigation into Algorithm Detection (2022), AAAI'22, Long, Tin et al. [pdf]
UniXcoder: Unified Cross-Modal Pre-training for Code Representation (2022), arxiv 2022, Guo, Daya, et al. [pdf]
SPT-Code: Sequence-to-Sequence Pre-Training for Learning Source Code Representations (2022), ICSE'22, Niu, Changan, et al. [pdf]
GraphCode2Vec: Generic Code Embedding via Lexical and Program Dependence Analyses (2022), MSR'22, Ma, Wei, et al.
OSCAR: How could Neural Networks understand Programs? (2021), ICML'21, Peng, Dinglan, et al. [pdf]
PROGRAML: A Graph-based Program Representation for Data Flow Analysis and Compiler Optimizations (2021), ICML'21, Cummins, Chris, et al. [pdf]
CoTexT: Multi-task Learning with Code-Text Transformer (2021), arxiv, Phan, Long, et al. [pdf]
TreeCaps: Tree-Based Capsule Networks for Source Code Processing (2021), AAAI'21, Bui, Nghi DQ, et al. [pdf] [code]
Language-Agnostic Representation Learning of Source Code from Structure and Context (2021), ICLR'21, Zügner, Daniel, et al. [pdf]
IR2Vec: LLVM IR Based Scalable Program Embeddings (2020), TACO journal, VenkataKeerthy, S., et al.
Compiler-Based Graph Representations for Deep Learning Models of Code (2020), CC'20, Brauckmann, Alexander, et al.
Learning and Evaluating Contextual Embedding of Source Code (2020), ICML 2020, Kanade, Aditya, et al. [pdf]
Learning Semantic Program Embeddings with Graph Interval Neural Network (2020), OOPSLA'20, Wang, Yu, et al.
Contrastive Code Representation Learning (2020), arxiv 2020, Jain, Paras, et al. [pdf]
SCELMo: Source Code Embeddings from Language Models (2020), arxiv 2020, Karampatsis, Rafael-Michael, et al. [pdf]
code2vec: Learning Distributed Representations of Code (2019), ACM POPL 2019, Alon, Uri, et al. [pdf]
COSET: A Benchmark for Evaluating Neural Program Embeddings (2019), arxiv 2019, Wang, Ke, et al. [pdf]
A Literature Study of Embeddings on Source Code (2019), arxiv 2019, Chen, Zimin, et al. [pdf]
code2seq: Generating Sequences from Structured Representations of Code (2018), arxiv 2018, Alon, Uri, et al. [pdf]
Neural Code Comprehension: A Learnable Representation of Code Semantics (2018), NIPS 2018, Ben-Nun, Tal, et al. [pdf]
Convolutional Neural Networks over Tree Structures for Programming Language Processing (2016), AAAI'16, Mou, Lili, et al. [pdf]

Code Changes

CCT5: A Code-Change-Oriented Pre-Trained Model (2023), arxiv, Lin, Bo, et al. [pdf]
GrACE: Generation using Associated Code Edits (2023), arxiv, Gupta, Priyanshu, et al. [pdf]
Slice-Based Code Change Representation Learning (2023), arxiv, Zhang, Fengyi, et al. [pdf]
Towards Generating Functionally Correct Code Edits from Natural Language Issue Descriptions (2023), arxiv, Fakhoury, Sarah, et al. [pdf]
CCRep: Learning Code Change Representations via Pre-Trained Code Model and Query Back (2023), arxiv, Liu, Zhongxin, et al. [pdf]
CoditT5: Pretraining for Source Code and Natural Language Editing (2022), ASE 2022, Jiyang, Zhang, et al. [pdf]
Commit2Vec: Learning Distributed Representations of Code Changes (2021), SN Computer Science, Lozoya, Rocío Cabrera, et al.
CODIT: Code Editing with Tree-Based Neural Models (2020), TSE 2020, Chakraborty, Saikat, et al.
On learning meaningful code changes via neural machine translation (2019), ICSE 2019, Tufano, Michele, et al.

Code Comments

An Empirical Study on Using Large Language Models for Multi-Intent Comment Generation (2023), arxiv, Geng, Mingyang, et al. [pdf]
An Intra-Class Relation Guided Approach for Code Comment Generation (2023), EACL'23, Wang, Zhenni, et al. [pdf]
APIContext2Com: Code Comment Generation by Incorporating Pre-Defined API Documentation (2023), arxiv, Shahbazi, R., and Fard F. [pdf]
Developer-Intent Driven Code Comment Generation (2023), arxiv, Mu, Fangwen, et al. [pdf]
ALSI-Transformer: Transformer-Based Code Comment Generation With Aligned Lexical and Syntactic Information (2023), IEEE Access, Park, Youngmi, et al.

Bug/Vulnerability Detection

A New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verification (2023), arxiv, Charalambous, Yiannis, et al. [pdf]
An Unbiased Transformer Source Code Learning with Semantic Vulnerability Graph (2023), arxiv, Islam, Nafis Tanveer, et al. [pdf]
Large Language Models and Simple, Stupid Bugs (2023), arxiv, Jesse, Kevin, et al. [pdf]
Vulnerability Detection with Graph Simplification and Enhanced Graph Representation Learning (2023), arxiv, Wen, Xin-Cheng, et al. [pdf]
DeepVD: Toward Class-Separation Features for Neural Network Vulnerability Detection (2023), arxiv, Wang, Wenbo, et al. [pdf]
CSGVD: A deep learning approach combining sequence and graph embedding for source code vulnerability detection (2023), JSS journal, Tang, Wei, et al.
Fixing Hardware Security Bugs with Large Language Models (2023), arxiv, Ahmad, Baleegh, et al. [pdf]
VulEye: A Novel Graph Neural Network Vulnerability Detection Approach for PHP Application (2023), Applied Sciences journal, Lin, Chun, et al. [pdf]
VDGraph2Vec: Vulnerability Detection in Assembly Code using Message Passing Neural Networks (2022), ICMLA'22, Diwan, Ashita, et al. [pdf]
VulChecker: Graph-based Vulnerability Localization in Source Code (2022), Usenix, Mirsky, Yisroel, et al. [pdf]
DeepVulSeeker: A Novel Vulnerability Identification Framework via Code Graph Structure and Pre-training Mechanism (2022), arxiv, Wang, Jin, et al. [pdf]
Compact Abstract Graphs for Detecting Code Vulnerability with GNN Models (2022), ACSAC'22, Luo, Yu, et al.
An Empirical Study of Deep Learning Models for Vulnerability Detection (2022), arxiv, Steenhoek, Benjamin, et al. [pdf]
Variable-Based Fault Localization via Enhanced Decision Tree (2022), arxiv, Jiang, Jiajun, et al. [pdf]
SPVF: security property assisted vulnerability fixing via attention-based models (2022), EMSE, Zhou, Zhou, et al.
Modeling function-level interactions for file-level bug localization (2022), EMSE, Liang, H., et al.
Practical Automated Detection of Malicious npm Packages (2022), ICSE'22, Sejfia, A., and M. Schäfer [pdf]
Machine Learning for Source Code Vulnerability Detection: What Works and What Isn't There Yet (2022), IEEE Security & Privacy, Marjanov, Tina, et al.
Path-sensitive code embedding via contrastive learning for software vulnerability detection (2022), ISSTA'22, Cheng, Xiao, et al.
VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detection (2022), arxiv 2022, Hanif, H. and Maffeis, S. [pdf]
Katana: Dual Slicing-Based Context for Learning Bug Fixes (2022), arxiv 2022, Sintaha, Mifta, et al. [pdf]
LineVul: A Transformer-based Line-Level Vulnerability Prediction (2022), MSR'22, Fu, M., & Tantithamthavorn, C. [pdf][code]
Transformer-Based Language Models for Software Vulnerability Detection: Performance, Model's Security and Platforms (2022), arxiv 2022, Thapa, Chandra, et al. [pdf]
LineVD: Statement-level Vulnerability Detection using Graph Neural Networks (2022), MSR'22, Hin, David, et al. [pdf]
Nalin: Learning from Runtime Behavior to Find Name-Value Inconsistencies in Jupyter Notebooks (2022), ICSE'22, Patra, Jibesh, et al. [pdf]
Hoppity: Learning graph transformations to detect and fix bugs in programs (2020), ICLR 2020, Dinella, Elizabeth, et al. [pdf]
Deep Learning based Software Defect Prediction (2020), Neurocomputing, Qiao, Lei, et al.
Software Vulnerability Discovery via Learning Multi-domain Knowledge Bases (2019), IEEE TDSC, Lin, Guanjun, et al.
Neural Bug Finding: A Study of Opportunities and Challenges (2019), arxiv 2019, Habib, Andrew, et al. [pdf]
Automated Vulnerability Detection in Source Code Using Deep Representation Learning (2018), ICMLA 2018, Russell, Rebecca, et al.
DeepBugs: A Learning Approach to Name-based Bug Detection (2018), ACM PL 2018, Pradel, Michael, et al. [pdf]
Automatically Learning Semantic Features for Defect Prediction (2016), ICSE 2016, Wang, Song, et al.

Source Code Modeling

ContraBERT: Enhancing Code Pre-trained Models via Contrastive Learning (2023), arxiv, Liu, Shangqing, et al. [pdf]
ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages (2022), arxiv, Chai, Yekun, et al. [pdf]
Do Bugs Lead to Unnaturalness of Source Code? (2022), FSE'22, Jiang, Yanjie, et al.
On the Naturalness of Bytecode Instructions (2022), ASE'22, Choi, Y., and J. Nam. [pdf]
CodeBERT-nt: code naturalness via CodeBERT (2022), arxiv, Khanfir, Ahmed, et al. [pdf]
CommitBART: A Large Pre-trained Model for GitHub Commits (2022), arxiv, Liu, S., et al, [pdf]
Towards Learning (Dis)-Similarity of Source Code from Program Contrasts (2022), ACL'22, Ding, Yangruibo, et al. [pdf]
Multilingual training for Software Engineering (2022), ICSE'22, Ahmed, Toufique, et al. [pdf]
Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code (2020), ICSE'20, Karampatsis, Rafael-Michael, et al.
Maybe Deep Neural Networks are the Best Choice for Modeling Source Code (2019), arxiv 2019, Karampatsis, Rafael-Michael, et al. [pdf]
Are Deep Neural Networks the Best Choice for Modeling Source Code? (2017), FSE 2017, Hellendoorn, Vincent J., et al. [pdf]

Program Repair

Revisiting the Plastic Surgery Hypothesis via Large Language Models (2023), arxiv, Xia, Chunqiu Steven et al. [pdf]
A Survey on Automated Program Repair Techniques (2023), arxiv, Huang, Kai, et al. [pdf]
Keep the Conversation Going: Fixing 162 out of 337 bugs for $0.42 each using ChatGPT (2023), arxiv, Xia, C. S., and Lingming Z. [pdf]
MUFIN: Improving Neural Repair Models with Back-Translation (2023), arxiv, Silva, André, et al. [pdf]
Explainable Automated Debugging via Large Language Model-driven Scientific Debugging (2023), arxiv, Kang, Sungmin, et al. [pdf]
A study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repair (2023), arxiv, Cao, Jialun, et al. [pdf]
ITER: Iterative Neural Repair for Multi-Location Patches (2023), arxiv, Ye, He, and Martin M. [pdf]
TraceFixer: Execution Trace-Guided Program Repair (2023), arxiv, Bouzenia, Islem, et al. [pdf]
PatchZero: Zero-Shot Automatic Patch Correctness Assessment (2023), arxiv, Zhou, Xin, et al. [pdf]
Rete: Learning Namespace Representation for Program Repair (2023), ICSE'23, Parasaram, Nikhil et al. [pdf]
InferFix: End-to-End Program Repair with LLMs over Retrieval-Augmented Prompts (2023), arxiv, Jin, Matthew, et al. [pdf]
Automated Program Repair in the Era of Large Pre-trained Language Models (2023), arxiv, Xia, C. S. et al. [pdf]
KNOD: Domain Knowledge Distilled Tree Decoder for Automated Program Repair (2023), ICSE'23, Jiang, Nan, et al. [pdf]
Impact of Code Language Models on Automated Program Repair (2023), ICSE'23, Jiang, Nan, et al. [pdf]
Embedding Context as Code Dependencies for Neural Program Repair (2023), ICST'23, Nashid, Noor, et al. [pdf]
Tare: Type-Aware Neural Program Repair (2023), arxiv, Zhu, Qihao, et al. [pdf]
Boosting Automated Patch Correctness Prediction via Pre-trained Language Mode (2023), arxiv, Zhang, Quanjun, et al. [pdf]
Conversational Automated Program Repair (2023), arxiv, Xia, Chunqiu Steven et al. [pdf]
An Analysis of the Automatic Bug Fixing Performance of ChatGPT (2023), arxiv, Sobania, Dominik, et al. [pdf]
Improving Automated Program Repair with Domain Adaptation (2023), arxiv, Zirak, A., and Hemati, H. [pdf]
A Survey of Learning-based Automated Program Repair (2023), arxiv, Zhang, Quanjun, et al. [pdf]
Program Repair: Survey (2022), arxiv, Gao, Xiang, et al. [pdf]
SelfAPR: Self-supervised Program Repair with Test Execution Diagnostics (2022), ASE'22, He et al. [pdf]
Neural Program Repair using Execution-based Backpropagation (2022), ICSE'22, He et al. [pdf]
Practical Program Repair in the Era of Large Pre-trained Language Models (2022), arxiv, Xia, C. S. et al. [pdf]
SYNSHINE: improved fixing of Syntax Errors (2022), IEEE TSE, Ahmed, T. et al.
TransRepair: Context-aware Program Repair for Compilation Errors (2022), ASE'22, Li, Xueyang, et al. [pdf]
Repairing Bugs in Python Assignments Using Large Language Models (2022), arxiv, Zhang, Jialu, et al. [pdf]
Repair Is Nearly Generation: Multilingual Program Repair with LLMs (2022), arxiv, Joshi, Harshit, et al. [pdf]
VulRepair: A T5-Based Automated Software Vulnerability Repair (2022), FSE'22, Fu, Michael, et al. [pdf]
Less Training, More Repairing Please: Revisiting Automated Program Repair via Zero-shot Learning (2022), FSE'22, Xia, Chunqiu Steven, and Lingming Z. [pdf]
Can we learn from developer mistakes? Learning to localize and repair real bugs from real bug fixes (2022), arxiv, Richter, Cedric, and Heike W. [pdf]
AdaptivePaste: Code Adaptation through Learning Semantics-aware Variable Usage Representations (2022), arxiv 2022, Liu, Xiaoyu, et al. [pdf]
DEAR: A Novel Deep Learning-based Approach for Automated Program Repair (2022), ICSE'22, Li, Yi, et al. [pdf]
TFix: Learning to Fix Coding Errors with a Text-to-Text Transformer (2021), ICML'21, Berabi, Berkay, et al. [pdf]
Neural Transfer Learning for Repairing Security Vulnerabilities in C Code (2021), Chen, Zimin, et al. [pdf]
Generating Bug-Fixes Using Pretrained Transformers (2021), arxiv 2021, Drain, Dawn, et al. [pdf]
Global Relational Models of Source Code (2020), ICLR'20, Hellendoorn, Vincent J., et al. [pdf]
Neural Program Repair by Jointly Learning to Localize and Repair (2019), arxiv 2019, Vasic, Marko, et al. [pdf]

Program Translation

On ML-Based Program Translation: Perils and Promises (2023), arxiv, Malyala, Aniketh, et al. [pdf]
Boosting Neural Networks to Decompile Optimized Binaries (2022), ACSAC'22, Cao, Ying, et al.
The Effectiveness of Transformer Models for Analyzing Low-Level Programs (2022), MIT Primes, Zifan Guo [pdf]
Code Translation with Compiler Representations (2022), arxiv, Szafraniec, Marc, et al. [pdf]
BabelTower: Learning to Auto-parallelized Program Translation (2022), ICML'22, Wen, Yuanbo, et al. [pdf]
Multilingual Code Snippets Training for Program Translation (2022), AAAI'22, Zhu, Ming, et al. [pdf]
Semantics-Recovering Decompilation through Neural Machine Translation (2021), arxiv 2021, Liang, Ruigang, et al. [pdf]
Unsupervised Translation of Programming Languages (2020), arxiv 2020, Lachaux, Marie-Anne et al. [pdf]

Program Analysis

AutoPruner: Transformer-Based Call Graph Pruning (2022), FSE'22, Le-Cong, Thanh, et al. [pdf][code]
Striking a Balance: Pruning False-Positives from Static Call Graphs (2022), ICSE'22, Utture, Akshay, et al. [pdf][code]

Software Testing

LTM: Scalable and Black-box Similarity-based Test Suite Minimization based on Language Models (2023), arxiv, Pan, Rongqi, et al. [pdf]
ChatGPT and Software Testing Education: Promises & Perils (2023), arxiv, Jalil, Sajed, et al. [pdf]
Adaptive Test Generation Using a Large Language Model (2023), arxiv, Schäfer, Max, et al. [pdf]
CODAMOSA: Escaping Coverage Plateaus in Test Generation with Pre-trained Large Language Models (2023), ICSE'23, Lemieux, Caroline, et al. [pdf]
Learning Deep Semantics for Test Completion (2023), arxiv, Nie, Pengyu, et al. [pdf]
A3Test: Assertion-Augmented Automated Test Case Generation (2023), arxiv, Alagarsamy, Saranya, et al. [pdf]
Efficient Mutation Testing via Pre-Trained Language Models (2023), arxiv, Khanfir, Ahmed, et al. [pdf]
Test2Vec: An Execution Trace Embedding for Test Case Prioritization (2022), arxiv, Jabbar, Emad, et al. [pdf]
Generating Accurate Assert Statements for Unit Test Cases using Pretrained Transformers (2022), AST'22, Tufano, Michele, et al.
On Learning Meaningful Assert Statements for Unit Test Cases (2020), ICSE'20, Watson, Cody, et al.

Code Clone Detection

CCT-Code: Cross-Consistency Training for Multilingual Clone Detection and Code Search (2023), arxiv, Sorokin, Nikita, et al. [pdf]
Neuro-symbolic Zero-Shot Code Cloning with Cross-Language Intermediate Representation (2023), arxiv, Hasija, Krishnam, et al. [pdf]
Pathways to Leverage Transcompiler based Data Augmentation for Cross-Language Clone Detection (2023), arxiv, Pinku, Subroto Nag et al. [pdf]
Graph-based code semantics learning for efficient semantic code clone detection (2022), IST journal, Yu, Dongjin, et al.
Efficient transformer with code token learner for code clone detection (2022), arxiv, Zhang, Aiping, et al.
Evaluation of Contrastive Learning with Various Code Representations for Code Clone Detection (2022), arxiv, Zubkov, Maksim, et al. [pdf]
Cross-Language Source Code Clone Detection Using Deep Learning with InferCode (2022), arxiv 2022, Yahya, M., and Kim, D., [pdf]
funcGNN: A Graph Neural Network Approach to Program Similarity (2020), ESEM'20, Nair, Aravind, et al. [pdf]
Cross-Language Clone Detection by Learning Over Abstract Syntax Trees (2019), MSR'19, Perez, Daniel, et al.
The Adverse Effects of Code Duplication in Machine Learning Models of Code (2019), Onward! 2019, Allamanis, Miltiadis, [pdf]

Code Search

Improving Code Search with Multi-Modal Momentum Contrastive Learning (2023), ICPC'23, Shi, Zejian, et al. [pdf]
MulCS: Towards a Unified Deep Representation for Multilingual Code Search (2023), SANER'23, Ma, Yingwei, et al. [pdf]
A mutual embedded self-attention network model for code search (2023), JSS, Hu, Haize, et al.
You See What I Want You to See: Poisoning Vulnerabilities in Neural Code Search (2022), FSE'22, Wan, Yao, et al.
How to Better Utilize Code Graphs in Semantic Code Search? (2022), FSE'22, Shi, Yucen, et al.
Exploring Representation-Level Augmentation for Code Search (2022), EMNLP'22, Li, Haochen, et al. [pdf][code]
A code search engine for software ecosystems (2022), CEUR, Pfaff, Chris, et al. [pdf]
Cross-Domain Deep Code Search with Meta Learning (2022), ICSE'22, Chai, Yitian, et al. [pdf]

Code Language Models

Gorilla: Large Language Model Connected with Massive APIs (2023), arxiv, Patil, Shishir G., et al. [pdf]
CodeT5+: Open Code Large Language Models for Code Understanding and Generation (2023), arxiv, Wang, Yue, et al. [pdf]
Better Language Models of Code through Self-Improvement (2023), arxiv, To, Hung Quoc, et al. [pdf]
A Systematic Evaluation of Large Language Models of Code (2022), arxiv 2022, Xu, Frank F., et al. [pdf][code]
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation (2021), EMNLP'21, Wang, Yue, et al. [pdf]
JavaBERT: Training a Transformer-Based Model for the Java Programming Language (2021), ASEW'21, De Sousa, N. T., and W. Hasselbring
TreeBERT: A Tree-Based Pre-Trained Model for Programming Language (2021), UAI'21, Jiang, Xue, et al. [pdf]
PLBART: Unified Pre-training for Program Understanding and Generation (2021), NAACL'21, Ahmad, Wasi Uddin, et al. [pdf]
Evaluating Large Language Models Trained on Code (2021), arxiv 2021, Chen, Mark, et al. [pdf] [code]
GraphCodeBERT: Pre-training Code Representations with Data Flow (2021), arxiv, Guo, Daya, et al. [pdf]
C-BERT: Exploring Software Naturalness through Neural Language Models (2020), arxiv, Buratti, Luca, et al. [pdf]
Codebert: A Pre-trained Model for Programming and Natural Languages (2020), arxiv 2020, Feng, Zhangyin, et al. [pdf]

Empirical Studies

Can Large Language Models Reason About Program Invariants? (2023), ICML'23, Sutton, Charles, et al.
The Scope of ChatGPT in Software Engineering: A Thorough Investigation (2023), arxiv, Ma, Wei, et al. [pdf]
Evaluating AIGC Detectors on Code Content (2023), arxiv, Wang, Jian, et al. [pdf]
“What It Wants Me To Say”: Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models (2023), CHI'23, Liu, Michael Xieyang, et al. [pdf]
Constructing Effective In-Context Demonstration for Code Intelligence Tasks: An Empirical Study (2023), arxiv, Gao, Shuzheng, et al. [pdf]
Automated Program Repair Based on Code Review: How do Pre-trained Transformer Models Perform? (2023), arxiv, Paul, Rishov, et al. [pdf]
Investigating Code Generation Performance of ChatGPT with Crowdsourcing Social Data (2023), COMPSAC'23, Feng, Yunhe, et al. [pdf]
Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT (2023), arxiv, Yetiştiren, Burak, et al. [pdf]
Is ChatGPT the Ultimate Programming Assistant - How far is it? (2023), arxiv, Tian, Haoye, et al. [pdf]
Study of Distractors in Neural Models of Code (2023), InteNSE'23, Rabin, Md Rafiqul Islam, et al. [pdf]
Judging Adam: Studying the Performance of Optimization Methods on ML4SE Tasks (2023), arxiv, Pasechnyuk, Dmitry, et al. [pdf]
Boosting Source Code Learning with Data Augmentation: An Empirical Study (2023), arxiv, Dong, Zeming, et al. [pdf]
Source Code Recommender Systems: The Practitioners’ Perspective (2023), arxiv, Ciniselli, Matteo, et al. [pdf]
An Empirical Comparison of Pre-Trained Models of Source Code (2023), arxiv, Niu, Changan, et al. [pdf]
On the Reliability and Explainability of Automated Code Generation Approaches (2023), arxiv, Liu, Yue, et al. [pdf]
On the Robustness of Code Generation Techniques: An Empirical Study on GitHub Copilot (2023), arxiv, Mastropaolo, Antonio, et al. [pdf]
Practitioners’ Expectations on Code Completion (2023), arxiv, Wang, Chaozheng, et al. [pdf]
Is Self-Attention Powerful to Learn Code Syntax and Semantics? (2022), arxiv, Ma, Wei, et al. [pdf]
Piloting Copilot and Codex: Hot Temperature, Cold Prompts, or Black Magic? (2022), arxiv, Döderlein et al. [pdf]
Explainable AI for Pre-Trained Code Models: What Do They Learn? When They Do Not Work? (2022), arxiv, Mohammadkhani, Ahmad Haji, et al. [pdf]
How Important are Good Method Names in Neural Code Generation? A Model Robustness Perspective (2022), arxiv, Yang, Guang, et al. [pdf]
“It would work for me too”: How Online Communities Shape Software Developers’ Trust in AI-Powered Code Generation Tools (2022), arxiv, Cheng, Ruijia, et al. [pdf]
Are Neural Bug Detectors Comparable to Software Developers on Variable Misuse Bugs? (2022), ASE'22, Richter, Cedric, et al. [pdf]
Do Pre-trained Language Models Indeed Understand Software Engineering Tasks? (2022), arxiv, Li, Yao, et al. [pdf]
A large-scale empirical study of commit message generation: models, datasets and evaluation (2022), EMSE, Tao, Wei, et al.
Examining Zero-Shot Vulnerability Repair with Large Language Models (2022), IEEE SP, Pearce, H., et al.
Extracting Meaningful Attention on Source Code: An Empirical Study of Developer and Neural Model Code Exploration (2022), arxiv, Paltenghi, M., et al. [pdf]
SimSCOOD: Systematic Analysis of Out-of-Distribution Behavior of Source Code Models (2022), arxiv, Hajipour, H., et al. [pdf]
Are Neural Bug Detectors Comparable to Software Developers on Variable Misuse Bugs? (2022), ASE'22, Richter, Cedric, et al. [pdf]
Open Science in Software Engineering: A Study on Deep Learning-Based Vulnerability Detection (2022), TSE, Nong, Yu, et al. [pdf]
A controlled experiment of different code representations for learning-based program repair (2022), EMSE, Namavar, M., et al.
What is it like to program with artificial intelligence? (2022), arxiv, Sarkar, Advait, et al. [pdf]
Security Implications of Large Language Model Code Assistants: A User Study (2022), arxiv, Sandoval, Gustavo, et al. [pdf]
An Empirical Study of Code Smells in Transformer-based Code Generation Techniques (2022), arxiv, Siddiq, M. L. et al. [pdf]
No More Fine-Tuning? An Experimental Evaluation of Prompt Tuning in Code Intelligence (2022), FSE'22, Wang, Chaozheng, et al. [pdf]
Generating Realistic Vulnerabilities via Neural Code Editing: An Empirical Study (2022), FSE'22, Nong, Yu, et al. [pdf]
GitHub Copilot AI pair programmer: Asset or Liability? (2022), arxiv, Dakhel, Arghavan Moradi, et al. [pdf]
Evaluating the Impact of Source Code Parsers on ML4SE Models (2022), arxiv, Utkin, Ilya, et al [pdf]
An extensive study on pre-trained models for program understanding and generation (2022), ISSTA'22, Zeng, Zhengran, et al.
Code Generation Tools (Almost) for Free? A Study of Few-Shot, Pre-Trained Language Models on Code (2022), arxiv, Bareiß, Patrick, et al. [pdf]
Assessing Project-Level Fine-Tuning of ML4SE Models (2022), arxiv, Bogomolov, Egor, et al. [pdf]
On the Transferability of Pre-trained Language Models for Low-Resource Programming Languages (2022), ICPC'22, Chen, Fuxiang, et al. [pdf]
Learning Program Semantics with Code Representations: An Empirical Study (2022), SANER'22, Siow, Jing Kai, et al. [pdf][code]
Assessing Generalizability of CodeBERT (2021), ICSME'21, Zhou, Xin, et al.
Thinking Like a Developer? Comparing the Attention of Humans with Neural Models of Code (2021), ASE'21, Paltenghi, M. & Pradel, M.
An Empirical Study of Transformers for Source Code (2021), FSE'21, Chirkova, N., & Troshin, S.
An Empirical Study on the Usage of Transformer Models for Code Completion (2021), MSR'21, Ciniselli, Matteo, et al.

Surveys

When Neural Model Meets NL2Code: A Survey (2022), arxiv 2022, Zan, Daoguang, et al. [pdf]
Deep Learning Meets Software Engineering: A Survey on Pre-Trained Models of Source Code (2022), arxiv 2022, Niu, Changan, et al. [pdf]
A Survey of Deep Learning Models for Structural Code Understanding (2022), arxiv 2022, Wu, Ruoting, et al. [pdf]
A Survey on Machine Learning Techniques for Source Code Analysis (2021), arxiv 2021, Sharma, Tushar, et al. [pdf]
Deep Learning & Software Engineering: State of Research and Future Directions (2020), arxiv 2020, Devanbu, Prem, et al. [pdf]
A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research (2020), arxiv 2020, Watson, Cody, et al. [pdf]
Machine Learning for Software Engineering: A Systematic Mapping (2020), arxiv 2020, Shafiq, Saad, et al. [pdf]
Synergy between Machine/Deep Learning and Software Engineering: How Far Are We? (2020), arxiv 2020, Wang, Simin, et al. [pdf]
Software Engineering Meets Deep Learning: A Literature Review (2020), arxiv 2020, Ferreira, Fabio, et al. [pdf]
Software Vulnerability Detection Using Deep Neural Networks: A Survey (2020), Proceedings of the IEEE, Lin, Guanjun, et al.
Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges (2020), arxiv 2020, Le, Triet HM, et al. [pdf]
A Survey of Machine Learning for Big Code and Naturalness (2018), ACM Computing Surveys, Allamanis, Miltiadis, et al. [pdf]

Misc

Towards Tracing Code Provenance with Code Watermarking (2023), arxiv, Li, Wei, et al. [pdf]
SLaDe: A Portable Small Language Model Decompiler for Optimized Assembler (2023), arxiv, Armengol-Estapé, Jordi, et al. [pdf]
Text-to-SQL Error Correction with Language Models of Code (2023), arxiv, Chen, Ziru, et al. [pdf]
Improving API Knowledge Discovery with ML: A Case Study of Comparable API Methods (2023), ICSE'23, Nam, Daye, et al. [pdf]
Beryllium: Neural Search for Algorithm Implementations (2023), arxiv, Kulkarni, Adithya, et al. [pdf]
Zero-shot Prompting for Code Complexity Prediction Using GitHub Copilot (2023), arxiv, Siddiq, Mohammed Latif, et al. [pdf]
One Adapter for All Programming Languages? Adapter Tuning for Code Search and Summarization (2023), arxiv, Wang, Deze, et al. [pdf]
GraphBinMatch: Graph-based Similarity Learning for Cross-Language Binary and Source Code Matching (2023), arxiv, TehraniJamsaz, Ali, et al. [pdf]
Teaching Large Language Models to Self-Debug (2023), arxiv, Chen, Xinyun, et al. [pdf]
Improving Few-shot Prompts with Relevant Static Analysis Products (2023), arxiv, Ahmed, Toufique, et al. [pdf]
Self-Supervised Learning to Prove Equivalence Between Straight-Line Programs via Rewrite Rules (2023), TSE, Kommrusch, Steve, et al.
XCODEEVAL: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval (2023), arxiv, Khan, Mohammad Abdullah Matin, et al. [pdf]
BenchDirect: A Directed Language Model for Compiler Benchmarks (2023), arxiv, Tsimpourlas, Foivos, et al. [pdf]
Creating CREATE queries with multi-task deep neural networks (2023), KBS journal, Diker, S. N., and C. Okan Sakar
Representation Learning for Stack Overflow Posts: How Far are We? (2023), arxiv, He, Junda, et al. [pdf]
Generation-based Code Review Automation: How Far Are We? (2023), arxiv, Zhou, Xin, et al. [pdf]
Model-Agnostic Syntactical Information for Pre-Trained Programming Language Models (2023), arxiv, Saberi, I., and Fatemeh F. [pdf]
Automating Method Naming with Context-Aware Prompt-Tuning (2023), arxiv, Zhu, Jie, et al. [pdf]
Knowledge Transfer for Pseudo-code Generation from Low Resource Programming Language (2023), arxiv, Sontakke, Ankita, et al. [pdf]
LExecutor: Learning-Guided Execution (2023), arxiv, Souza, B., and M. Pradel [pdf]
Keeping Pace with Ever-Increasing Data: Towards Continual Learning of Code Intelligence Models (2023), arxiv, Gao, Shuzheng, et al. [pdf]
CrossCodeBench: Benchmarking Cross-Task Generalization of Source Code Models (2023), arxiv, Niu, Changan, et al. [pdf]
On the Applicability of Language Models to Block-Based Programs (2023), arxiv, Niu, Changan, et al. [pdf]
AttSum: A Deep Attention-Based Summarization Model for Bug Report Title Generation (2023), IEEE TOR, Ma, Xiaoxue, et al.
CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code (2023), arxiv, Zhou, Shuyan, et al. [pdf]
VULGEN: Realistic Vulnerability Generation Via Pattern Mining and Deep Learning (2023), ICSE'23, Nong, Yu, et al. [pdf]
When to Say What: Learning to Find Condition-Message Inconsistencies (2023), ICSE'23, B., Islem, and M. Pradel. [pdf]
Automated Summarization of Stack Overflow Posts (2023), ICSE'23, Kou, Bonan, et al. [pdf]
Learning Graph-based Code Representations for Source-level Functional Similarity Detection (2023), arxiv, Liu, Jiahao, et al. [pdf]
Retrieval-Based Prompt Selection for Code-Related Few-Shot Learning (2023), ICSE'23, Nashid, Noor, et al. [pdf]
D-ACT: Towards Diff-Aware Code Transformation for Code Review Under a Time-Wise Evaluation (2023), arxiv, Pornprasit, Chanathip, et al. [pdf]
API Entity and Relation Joint Extraction from Text via Dynamic Prompt-tuned Language Model (2023), arxiv, Huang, Qing, et al [pdf]
FLAME: A small language model for spreadsheet formulas (2023), arxiv, Joshi, Harshit, et al. [pdf]
Callee: Recovering Call Graphs for Binaries with Transfer and Contrastive Learning (2023), IEEE SP, Zhu, Wenyu, et al.
Asteria-Pro: Enhancing Deep-Learning Based Binary Code Similarity Detection by Incorporating Domain Knowledge (2023), arxiv, Yang, Shouguo, et al. [pdf]
Fuzzing Deep-Learning Libraries via Large Language Models (2022), arxiv, Deng, Yinlin, et al. [pdf]
Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries (2023), SANER23, Al-Kaswan, Ali, et al. [pdf]
CFG2VEC: Hierarchical Graph Neural Network for Cross-Architectural Software Reverse Engineering (2023), arxiv, Yu, Shih-Yuan, et al. [pdf]
Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models (2023), ICSE'23, Ahmed, Toufique, et al. [pdf]
Detect-Localize-Repair: A Unified Framework for Learning to Debug with CodeT5 (2022), arxiv, Bui, Nghi DQ, et al. [pdf]
Unleashing the power of pseudo-code for binary code similarity analysis (2022), Cybersecurity journal, Zhang, Weiwei, et al.
Reinforcement Learning assisted Loop Distribution for Locality and Vectorization (2022), Jain, Shalini, et al. [pdf]
Learning to Parallelize Source Code via OpenMP with Transformers (2022), Harel, Re’em, et al. [pdf]
Codex Hacks HackerRank: Memorization Issues and a Framework for Code Synthesis Evaluation (2022), arxiv, Karmakar, Anjan, et al. [pdf]
BCGen: a comment generation method for bytecode (2022), ASE, Huang, Yuan, et al.
Explaining Software Bugs Leveraging Code Structures in Neural Machine Translation (2022), arxiv, Mahbub, Parvez, et al. [pdf]
Neural Language Models for Code Quality Identification (2022), arxiv, Sengamedu, S., et al.
Detecting Security Patches in Java Projects Using NLP Technology (2022), ICNLSP'22, Stefanoni, Andrea, et al. [pdf]
Program Merge Conflict Resolution via Neural Transformers (2022), FSE'22, Svyatkovskiy, Alexey, et al.
Automating code review activities by large-scale pre-training (2022), FSE'22, Li, Zhiyu, et al. [[pdf]]
Teaching Algorithmic Reasoning via In-context Learning (2022), arxiv, Zhou, Hattie, et al [pdf]
Improved Evaluation of Automatic Source Code Summarisation (2022), arxiv, Phillips, Jesse, et al. [pdf]
Towards Generalizable and Robust Text-to-SQL Parsing (2022), arxiv, Gao, Chang, et al. [pdf]
CodeEditor: Learning to Edit Source Code with Pre-trained Models (2022), arxiv, Li, Jia, et al. [pdf]
Poison Attack and Defense on Deep Source Code Processing Models (2022), arxiv, Li, Jia, et al. [pdf]
NEUDEP: Neural Binary Memory Dependence Analysis (2022), FSE'22, Pei, Kexin, et al. [pdf]
Novice Type Error Diagnosis with Natural Language Models (2022), arxiv, Geng, Chuqin, et al. [pdf]
CAT-probing: A Metric-based Approach to Interpret How Pre-trained Models for Programming Language Attend Code Structure (2022), arxiv, Chen, Nuo, et al. [pdf]
Using Large Language Models to Enhance Programming Error Messages (2022), SIGCSE'22, Leinonen, J., et al. [pdf]
Automatic Code Documentation Generation Using GPT-3 (2022), ASE'22, Khan, J. Y., and G. Uddin. [pdf]
So Much in So Little: Creating Lightweight Embeddings of Python Libraries (2022), arxiv, Golubev, Yaroslav, et al. [pdf]
Code Compliance Assessment as a Learning Problem (2022), arxiv, Sawant, N., and S. H. Sengamedu [pdf]
Learning-based Identification of Coding Best Practices from Software Documentation (2022), ICSME'22, Sawant, N., and S. H. Sengamedu [pdf]
Learning to Answer Semantic Queries over Code (2022), arxiv, Sahu, Surya Prakash, et al. [pdf]
XFL: Naming Functions in Binaries with Extreme Multi-label Learning (2022), arxiv, Patrick-Evans, J., et al. [pdf]
SymLM: Predicting Function Names in Stripped Binaries via Context-Sensitive Execution-Aware Code Embeddings (2022), Jin, Xin, et al. [pdf]
Out of the BLEU: how should we assess quality of the Code Generation models? (2022), arxiv, Evtikhiev, Mikhail, et al. [pdf]
Compressing Pre-trained Models of Code into 3 MB (2022), arxiv, Shi, Jieke, et al. [pdf]
A Scalable and Extensible Approach to Benchmarking NL2Code for 18 Programming Languages (2022), arxiv, Cassano, Federico, et al. [pdf]
AUGER: Automatically Generating Review Comments with Pre-training Models (2022), FSE'22, Li, Lingwei, et al. [pdf]
Overwatch: Learning Patterns in Code Edit Sequences (2022), arxiv, Zhang, Yuhao, et al. [pdf]
Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing (2022), KDD'22, Wang, Lihan, et al. [pdf]
DIRE and its Data: Neural Decompiled Variable Renamings with Respect to Software Class (2022), TOSEM, Dramko, Luke, et al.
Making Python Code Idiomatic by Automatic Refactoring Non-Idiomatic Python Code with Pythonic Idioms (2022), arxiv, Zhang, Zejun, et al. [pdf]
DeepPERF: A Deep Learning-Based Approach For Improving Software Performance (2022), arxiv, Garg, Spandan, et al. [pdf]
CrystalBLEU: Precisely and Efficiently Measuring the Similarity of Code (2022), ICSE ’22 Companion, Eghbali, Aryaz, and Michael, P. [pdf]
Impact of Evaluation Methodologies on Code Summarization (2022), ACL, Nie, Pengyu, et al. [pdf]

PhD Theses

Improving Programming Productivity with Statistical Models (2022), Tam Nguyen [pdf]
Learning to Find Bugs in Programs and their Documentation (2021), Andrew Habib [pdf]
Machine Learning and the Science of Software Engineering (2020), Vincent Hellendoorn
Deep learning for compilers (2020), Christopher E. Cummins [pdf]
Deep Learning in Software Engineering (2020), Cody Watson [pdf]
Learning Code Transformations via Neural Machine Translation (2019), Michele Tufano [pdf]
Improving the Usability of Static Analysis Tools Using Machine Learning (2019), Ugur Koc [pdf]
Learning Natural Coding Conventions (2016), Miltiadis Allamanis [pdf]

Talks

Machine Learning for Software Engineering: AMA, MSR 2020 [video]
Understanding Source Code with Deep Learning, FOSDEM 2019 [video]

Datasets

DiverseVul - A Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection
RunBugRun - An Executable Dataset for Automated Program Repair
ODEX - An open-domain execution-based natural language (NL) to code generation dataset
PI-Link - A Ground-Truth Dataset of Links Between Pull-Requests and Issues in GitHub
ml-Codesmell - A code smell prediction dataset for machine learning approaches
JEMMA - An Extensible Java Dataset for ML4Code Applications
CS1QA (2022) - A Dataset for Assisting Code-based Question Answering in an Introductory Programming Course
XLCoST (2022) - A Benchmark Dataset for Cross-lingual Code Intelligence
CodeS (2022) - CodeS: A Distribution Shift Benchmark Dataset for Source Code Learning
methods2test (2022) - A supervised dataset consisting of Test Cases and their corresponding Focal Methods from a set of Java repositories
ManyTypes4TypeScript (2022) - Type prediction dataset for TypeScript
HumanEval - Program synthesis from code comments
GitHub Code (2022) - 115M LoC in 32 programming languages
D2A (2021) - A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysis
CodeXGLUE (2021)
ogbg-code2 (2021)
ManyTypes4Py (2021) - Type prediction dataset for Python
CodeSearchNet (2020)
ManySStuBs4J (2019)
150k Python Dataset (2016)
150k Javascript Dataset (2016)
GitHub Java Corpus (2013)

Tools

Source Code Analysis & Processing

LibSA4Py - LibSA4Py: Light-weight static analysis for extracting type hints and features
LibCST - A concrete syntax tree parser library for Python
python-graphs - A static analysis library for computing graph representations of Python programs suitable for use with graph neural networks.
Semantic - Parsing, analyzing, and comparing source code across many languages
GraphGen4Code - A toolkit for creating code knowledge graphs based on WALA code analysis and extraction of documentation
Joern - Code analysis platform for C/C++/Java/Binary/Javascript/Python/Kotlin based on code property graphs
NaturalCC - An Open-Source Toolkit for Code Intelligence
Scalpel - The Python Static Analysis Framework
WALA - T.J. Watson Libraries for Analysis, with frontends for Java, Android, and JavaScript
CodeGen - General toolkit to apply machine learning to code, from dataset creation to model training and evaluation (from Facebook AI Research)

Machine Learning

SentencePiece - Unsupervised text tokenizer for Neural Network-based text generation
Hugging Face - Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Code de-duplication

CD4Py - Code De-Duplication for Python
Near-duplicate Source Code Detector

Misc

Utilities by the DPU team of Microsoft
A set of tools to work with Big Code - Fetching GitHub repos, tokenizers, embeddings and etc
cloc - Counts blank lines, comment lines, and physical lines of source code in many programming languages.

Research Groups

Software Engineering Research Group (SERG), Delft University of Technology
Secure, Reliable, and Intelligent Systems Lab (SRI), ETH Zurich
Software Lab (SOLA), University of Stuttgart
Machine Learning for the Analysis of Source Code Text (MAST), Edinburgh University
Deep Program Understanding, Microsoft Research
DECAL (Davis Excellent/Eclectic/Extreme Computational Analytics Lab), UC Davis
JetBrains Research
SMart software Analysis and Trustworthy computing Lab (SMAT), Monash University

Venues

ICSE, the International Conference on Software Engineering
FSE, Symposium on the Foundations of Software Engineering
ASE, the International Conference on Automated Software Engineering
MSR, the Mining Software Repositories conference
ICPC, the International Conference on Program Comprehension
ICLR, the International Conference on Learning Representations
ICML, the International Conference on Machine Learning
ICMLA, the International Conference on Machine Learning and Applications
AAAI, the Association for the Advancement of Artificial Intelligence
ACL, the Association for Computational Linguistics
OOPSLA, the ACM Conference on Systems, Programming, Languages, and Applications
TSE, the IEEE Transactions on Software Engineering
TOSEM, ACM Transactions on Software Engineering and Methodology
JSS, Journal of Systems and Software

saltudelft/ml4se

saltudelft

Reviews

Repository Details

Machine Learning for Software Engineering

Content

Papers

Type Inference

Code Completion

Code Generation

Code Summarization

Code Embeddings/Representation

Code Changes

Code Comments

Bug/Vulnerability Detection

Source Code Modeling

Program Repair

Program Translation

Program Analysis

Software Testing

Code Clone Detection

Code Search

Code Language Models

Empirical Studies

Surveys

Misc

PhD Theses

Talks

Datasets

Tools

Source Code Analysis & Processing

Machine Learning

Code de-duplication

Misc

Research Groups

Venues

More Repositories