• Stars
    star
    3,241
  • Rank 13,436 (Top 0.3 %)
  • Language
    Python
  • License
    MIT License
  • Created about 4 years ago
  • Updated 17 days ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A unified, comprehensive and efficient recommendation library

RecBole Logo


RecBole (伯乐)

“世有伯乐,然后有千里马。千里马常有,而伯乐不常有。”——韩愈《马说》

PyPi Latest Release Conda Latest Release License arXiv

HomePage | Docs | Datasets | Paper | Blogs | Models | 中文版

RecBole is developed based on Python and PyTorch for reproducing and developing recommendation algorithms in a unified, comprehensive and efficient framework for research purpose. Our library includes 78 recommendation algorithms, covering four major categories:

  • General Recommendation
  • Sequential Recommendation
  • Context-aware Recommendation
  • Knowledge-based Recommendation

We design a unified and flexible data file format, and provide the support for 28 benchmark recommendation datasets. A user can apply the provided script to process the original data copy, or simply download the processed datasets by our team.

RecBole v0.1 architecture
Figure: RecBole Overall Architecture

In order to support the study of recent advances in recommender systems, we construct an extended recommendation library RecBole2.0 consisting of 8 packages for up-to-date topics and architectures (e.g., debiased, fairness and GNNs).

Feature

  • General and extensible data structure. We design general and extensible data structures to unify the formatting and usage of various recommendation datasets.

  • Comprehensive benchmark models and datasets. We implement 78 commonly used recommendation algorithms, and provide the formatted copies of 28 recommendation datasets.

  • Efficient GPU-accelerated execution. We optimize the efficiency of our library with a number of improved techniques oriented to the GPU environment.

  • Extensive and standard evaluation protocols. We support a series of widely adopted evaluation protocols or settings for testing and comparing recommendation algorithms.

RecBole News

new 11/06/2022: We release the optimal hyperparameters of the model and their tuning ranges.

new 10/05/2022: We release RecBole v1.1.1.

new 06/28/2022: We release RecBole2.0 with 8 packages consisting of 65 newly implement models.

02/25/2022: We release RecBole v1.0.1.

09/17/2021: We release RecBole v1.0.0.

03/22/2021: We release RecBole v0.2.1.

01/15/2021: We release RecBole v0.2.0.

12/10/2020: 我们发布了RecBole小白入门系列中文博客(持续更新中)

12/06/2020: We release RecBole v0.1.2.

11/29/2020: We constructed preliminary experiments to test the time and memory cost on three different-sized datasets and provided the test result for reference.

11/03/2020: We release the first version of RecBole v0.1.1.

Latest Update for SIGIR 2023 Submission

To better meet the user requirements and contribute to the research community, we present a significant update of RecBole in the latest version, making it more user-friendly and easy-to-use as a comprehensive benchmark library for recommendation. We summarize these updates in "Towards a More User-Friendly and Easy-to-Use Benchmark Library for Recommender Systems" and submit the paper to SIGIR 2023. The main contribution in this update is introduced below.

Our extensions are made in three major aspects, namely the models/datasets, the framework, and the configurations. Furthermore, we provide more comprehensive documentation and well-organized FAQ for the usage of our library, which largely improves the user experience. More specifically, the highlights of this update are summarized as:

  1. We introduce more operations and settings to help benchmarking the recommendation domain.

  2. We improve the user friendliness of our library by providing more detailed documentation and well-organized frequently asked questions.

  3. We point out several development guidelines for the open-source library developers.

These extensions make it much easier to reproduce the benchmark results and stay up-to-date with the recent advances on recommender systems. The datailed comparison between this update and previous versions is listed below.

Aspect RecBole 1.0 RecBole 2.0 This update
Recommendation tasks 4 categories 3 topics and 5 packages 4 categories
Models and datasets 73 models and 28 datasets 65 models and 8 new datasets 86 models and 41 datasets
Data structure Implemented Dataset and Dataloader Task-oriented Compatible data module inherited from PyTorch
Continuous features Field embedding Field embedding Field embedding and discretization
GPU-accelerated execution Single-GPU utilization Single-GPU utilization Multi-GPU and mixed precision training
Hyper-parameter tuning Serial gradient search Serial gradient search Three search methods in both serial and parallel
Significance test - - Available interface
Benchmark results - Partially public (GNN and CDR) Benchmark configurations on 82 models
Friendly usage Documentation Documentation Improved documentation and FAQ page

Installation

RecBole works with the following operating systems:

  • Linux
  • Windows 10
  • macOS X

RecBole requires Python version 3.7 or later.

RecBole requires torch version 1.7.0 or later. If you want to use RecBole with GPU, please ensure that CUDA or cudatoolkit version is 9.2 or later. This requires NVIDIA driver version >= 396.26 (for Linux) or >= 397.44 (for Windows10).

Install from conda

conda install -c aibox recbole

Install from pip

pip install recbole

Install from source

git clone https://github.com/RUCAIBox/RecBole.git && cd RecBole
pip install -e . --verbose

Quick-Start

With the source code, you can use the provided script for initial usage of our library:

python run_recbole.py

This script will run the BPR model on the ml-100k dataset.

Typically, this example takes less than one minute. We will obtain some output like:

INFO ml-100k
The number of users: 944
Average actions of users: 106.04453870625663
The number of items: 1683
Average actions of items: 59.45303210463734
The number of inters: 100000
The sparsity of the dataset: 93.70575143257098%
INFO Evaluation Settings:
Group by user_id
Ordering: {'strategy': 'shuffle'}
Splitting: {'strategy': 'by_ratio', 'ratios': [0.8, 0.1, 0.1]}
Negative Sampling: {'strategy': 'full', 'distribution': 'uniform'}
INFO BPRMF(
    (user_embedding): Embedding(944, 64)
    (item_embedding): Embedding(1683, 64)
    (loss): BPRLoss()
)
Trainable parameters: 168128
INFO epoch 0 training [time: 0.27s, train loss: 27.7231]
INFO epoch 0 evaluating [time: 0.12s, valid_score: 0.021900]
INFO valid result:
recall@10: 0.0073  mrr@10: 0.0219  ndcg@10: 0.0093  hit@10: 0.0795  precision@10: 0.0088
...
INFO epoch 63 training [time: 0.19s, train loss: 4.7660]
INFO epoch 63 evaluating [time: 0.08s, valid_score: 0.394500]
INFO valid result:
recall@10: 0.2156  mrr@10: 0.3945  ndcg@10: 0.2332  hit@10: 0.7593  precision@10: 0.1591
INFO Finished training, best eval result in epoch 52
INFO Loading model structure and parameters from saved/***.pth
INFO best valid result:
recall@10: 0.2169  mrr@10: 0.4005  ndcg@10: 0.235  hit@10: 0.7582  precision@10: 0.1598
INFO test result:
recall@10: 0.2368  mrr@10: 0.4519  ndcg@10: 0.2768  hit@10: 0.7614  precision@10: 0.1901

If you want to change the parameters, such as learning_rate, embedding_size, just set the additional command parameters as you need:

python run_recbole.py --learning_rate=0.0001 --embedding_size=128

If you want to change the models, just run the script by setting additional command parameters:

python run_recbole.py --model=[model_name]

Auto-tuning Hyperparameter

Open RecBole/hyper.test and set several hyperparameters to auto-searching in parameter list. The following has two ways to search best hyperparameter:

  • loguniform: indicates that the parameters obey the uniform distribution, randomly taking values from e^{-8} to e^{0}.
  • choice: indicates that the parameter takes discrete values from the setting list.

Here is an example for hyper.test:

learning_rate loguniform -8, 0
embedding_size choice [64, 96 , 128]
train_batch_size choice [512, 1024, 2048]
mlp_hidden_size choice ['[64, 64, 64]','[128, 128]']

Set training command parameters as you need to run:

python run_hyper.py --model=[model_name] --dataset=[data_name] --config_files=xxxx.yaml --params_file=hyper.test
e.g.
python run_hyper.py --model=BPR --dataset=ml-100k --config_files=test.yaml --params_file=hyper.test

Note that --config_files=test.yaml is optional, if you don't have any customize config settings, this parameter can be empty.

This processing maybe take a long time to output best hyperparameter and result:

running parameters:                                                                                                                    
{'embedding_size': 64, 'learning_rate': 0.005947474154838498, 'mlp_hidden_size': '[64,64,64]', 'train_batch_size': 512}                
  0%|                                                                                           | 0/18 [00:00<?, ?trial/s, best loss=?]

More information about parameter tuning can be found in our docs.

Time and Memory Costs

We constructed preliminary experiments to test the time and memory cost on three different-sized datasets (small, medium and large). For detailed information, you can click the following links.

NOTE: Our test results only gave the approximate time and memory cost of our implementations in the RecBole library (based on our machine server). Any feedback or suggestions about the implementations and test are welcome. We will keep improving our implementations, and update these test results.

RecBole Major Releases

Releases Date
v1.1.1 10/05/2022
v1.0.0 09/17/2021
v0.2.0 01/15/2021
v0.1.1 11/03/2020

Open Source Contributions

As a one-stop framework from data processing, model development, algorithm training to scientific evaluation, RecBole has a total of 11 related GitHub projects including

In the following table, we summarize the open source contributions of GitHub projects based on RecBole.

Projects Stars Forks Issues Pull requests
RecBole Stars Forks Issues Pull requests
RecBole2.0 Stars Forks Issues Pull requests
RecBole-DA Stars Forks Issues Pull requests
RecBole-MetaRec Stars Forks Issues Pull requests
RecBole-Debias Stars Forks Issues Pull requests
RecBole-FairRec Stars Forks Issues Pull requests
RecBole-CDR Stars Forks Issues Pull requests
RecBole-GNN Stars Forks Issues Pull requests
RecBole-TRM Stars Forks Issues Pull requests
RecBole-PJF Stars Forks Issues Pull requests
RecSysDatasets Stars Forks Issues Pull requests

Contributing

Please let us know if you encounter a bug or have any suggestions by filing an issue.

We welcome all contributions from bug fixes to new features and extensions.

We expect all contributions discussed in the issue tracker and going through PRs.

We thank the insightful suggestions from @tszumowski, @rowedenny, @deklanw et.al.

We thank the nice contributions through PRs from @rowedenny@deklanw et.al.

Cite

If you find RecBole useful for your research or development, please cite the following papers: RecBole[1.0], RecBole[2.0] and RecBole[1.1.1].

@inproceedings{recbole[1.0],
  author    = {Wayne Xin Zhao and Shanlei Mu and Yupeng Hou and Zihan Lin and Yushuo Chen and Xingyu Pan and Kaiyuan Li and Yujie Lu and Hui Wang and Changxin Tian and Yingqian Min and Zhichao Feng and Xinyan Fan and Xu Chen and Pengfei Wang and Wendi Ji and Yaliang Li and Xiaoling Wang and Ji{-}Rong Wen},
  title     = {RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms},
  booktitle = {{CIKM}},
  pages     = {4653--4664},
  publisher = {{ACM}},
  year      = {2021}
}
@inproceedings{recbole[2.0],
  title={RecBole 2.0: Towards a More Up-to-Date Recommendation Library},
  author={Zhao, Wayne Xin and Hou, Yupeng and Pan, Xingyu and Yang, Chen and Zhang, Zeyu and Lin, Zihan and Zhang, Jingsen and Bian, Shuqing and Tang, Jiakai and Sun, Wenqi and others},
  booktitle={Proceedings of the 31st ACM International Conference on Information \& Knowledge Management},
  pages={4722--4726},
  year={2022}
}
@misc{recbole[1.1.1],
  author = {Xu, Lanling and Tian, Zhen and Zhang, Gaowei and Wang, Lei and Zhang, Junjie and Zheng, Bowen and Li, Yifan and Hou, Yupeng and Pan, Xingyu and Chen, Yushuo and Zhao, Wayne Xin and Chen, Xu and Wen, Ji-Rong},
  title = {Recent Advances in RecBole: Extensions with more Practical Considerations},
  journal   = {arXiv preprint arXiv:2211.15148},
  year = {2022}
}

The Team

RecBole is developed by RUC, BUPT, ECNU, and maintained by RUC.

Here is the list of our lead developers in each development phase. They are the souls of RecBole and have made outstanding contributions.

Time Version Lead Developers Paper
June 2020
~
Nov. 2020
v0.1.1 Shanlei Mu (@ShanleiMu), Yupeng Hou (@hyp1231),
Zihan Lin (@linzihan-backforward), Kaiyuan Li (@tsotfsk)
PDF
Nov. 2020
~
Oct. 2022
v0.1.2 ~ v1.0.1 Yushuo Chen (@chenyushuo), Xingyu Pan (@2017pxy) PDF
Oct. 2022
~
now
v1.1.0 ~ v1.1.1 Lanling Xu (@Sherry-XLL), Zhen Tian (@chenyuwuxin), Gaowei Zhang (@Wicknight), Lei Wang (@Paitesanshi), Junjie Zhang (@leoleojie) PDF

License

RecBole uses MIT License. All data and code in this project can only be used for academic purposes.

More Repositories

1

LLMSurvey

The official GitHub page for the survey paper "A Survey of Large Language Models".
Python
8,693
star
2

TextBox

TextBox 2.0 is a text generation library with pre-trained language models
Python
1,055
star
3

Awesome-RSPapers

Recommender System Papers
902
star
4

RecSysDatasets

This is a repository of public data sources for Recommender Systems (RS).
Python
731
star
5

CRSLab

CRSLab is an open-source toolkit for building Conversational Recommender System (CRS).
Python
474
star
6

Top-conference-paper-list

A collection of classified and organized top conference paper list.
362
star
7

HaluEval

This is the repository of HaluEval, a large-scale hallucination evaluation benchmark for Large Language Models.
Python
298
star
8

LLMRank

[ECIR'24] Implementation of "Large Language Models are Zero-Shot Rankers for Recommender Systems"
Python
182
star
9

Negative-Sampling-Paper

This repository collects 100 papers related to negative sampling methods.
173
star
10

DenseRetrieval

170
star
11

RecBole2.0

An up-to-date, comprehensive and flexible recommendation library
167
star
12

UniSRec

[KDD'22] Official PyTorch implementation for "Towards Universal Sequence Representation Learning for Recommender Systems".
Python
158
star
13

RecBole-GNN

Efficient and extensible GNNs enhanced recommender library based on RecBole.
Python
154
star
14

LLMBox

Python
117
star
15

NCL

[WWW'22] Official PyTorch implementation for "Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning".
Python
113
star
16

RSPapers

Must-read papers on Recommender System. 推荐系统相关论文整理(内含40篇论文,并持续更新中)
89
star
17

RecBole-CDR

This is a library built upon RecBole for cross-domain recommendation algorithms
Python
78
star
18

UniCRS

[KDD22] Official PyTorch implementation for "Towards Unified Conversational Recommender Systems via Knowledge-Enhanced Prompt Learning".
Python
68
star
19

MVP

This repository is the official implementation of our paper MVP: Multi-task Supervised Pre-training for Natural Language Generation.
67
star
20

TG-ReDial

the dataset TG-ReDial
59
star
21

VQ-Rec

[WWW'23] PyTorch implementation for "Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders".
Python
51
star
22

RecBole-PJF

Python
46
star
23

ChatCoT

The official repository of "ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models"
Python
41
star
24

CORE

[SIGIR'22] Official PyTorch implementation for "CORE: Simple and Effective Session-based Recommendation within Consistent Representation Space".
Python
37
star
25

Multi-View-Co-Teaching

Code for our CIKM 2020 paper "Learning to Match Jobs with Resumes from Sparse Interaction Data using Multi-View Co-Teaching Network"
Python
29
star
26

JiuZhang

Our code will be public soon .
Python
25
star
27

ELMER

This repository is the official implementation of our EMNLP 2022 paper ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation
Python
24
star
28

BAMBOO

Python
23
star
29

Language-Specific-Neurons

Python
17
star
30

RecBole-DA

Python
17
star
31

CARP

Python
16
star
32

SAFE

The pytorch implementation of the SAFE model presented in NAACL-Findings-2022
Python
16
star
33

RecBole-TRM

Python
13
star
34

Erya

12
star
35

MML

Python
12
star
36

CFCRS

[KDD23] Official PyTorch implementation for "Improving Conversational Recommendation Systems via Counterfactual Data Simulation".
Python
11
star
37

Context-Tuning

This is the repository for COLING 2022 paper "Context-Tuning: Learning Contextualized Prompts for Natural Language Generation".
11
star
38

UniWeb

The official repository for our ACL 2023 Findings paper: The Web Can Be Your Oyster for Improving Language Models
9
star
39

PPGM

[ICDM'22] PyTorch implementation for "Privacy-Preserved Neural Graph Similarity Learning".
Python
6
star
40

LIVE

The official repository our ACL 2023 paper: "Learning to Imagine: Visually-Augmented Natural Language Generation"."
Python
5
star
41

Social-Datasets

A collection of social datasets for RecBole-GNN.
5
star
42

M3SRec

4
star
43

FIGA

Python
3
star
44

Contrastive-Curriculum-Learning

Python
3
star
45

Data-CUBE

3
star
46

Div-Ref

The official repository of "Not All Metrics Are Guilty: Improving NLG Evaluation Diversifying References".
Python
2
star
47

GenRec

Python
1
star
48

ETRec

Python
1
star