Dataset in LibCity

This repository is used to introduce the dataset in LibCity.

Dataset Conversion Tools

The dataset used in LibCity is stored in a unified data storage format named atomic files. In order to directly use the datasets we collected in LibCity, we have converted all datasets into the format of atomic files, and provide the conversion tools in this repository.

All conversion tools take the original dataset in the ./input/ directory as input, and output the converted atomic files to the ./output/ directory. In addition, we provide a link to obtain the original dataset in the first line of each conversion tool. You can download the original dataset through this link and place it in the ./input/ directory. Imitating our conversion tools, you can easily convert your own traffic dataset to adapt it to LibCity.

Besides, you can simply download the datasets we have processed, the data link is BaiduDisk with code 1231 or Google Drive.

Dataset Statistics Infomation

Here we present the statistics of the datasets we have processed.

• Traffic State Datasets: Point-based Flow or Speed or Occupancy
• Traffic State Datasets: Grid-based In-Flow and Out-Flow
• Traffic State Datasets: OD-based Flow
• Traffic State Datasets: Grid-OD-based Flow
• Traffic State Datasets: Risk
• GPS Point Trajectory Datasets
• Road Segment-based Trajectory Datasets
• POI-based Trajectory Datasets
• Road Network Datasets

Traffic State Datasets-Point-based Flow or Speed or Occupancy

Collected from sensors or Pre-processed from trajectory data.

Traffic State Datasets-Grid-based In-Flow and Out-Flow

Pre-processed from trajectory data.

Traffic State Datasets-OD-based Flow

Traffic State Datasets-Grid-OD-based Flow

Traffic State Datasets-Risk

GPS Point Trajectory Datasets

Road Segment-based Trajectory Datasets

POI-based Trajectory Datasets

Road Network Datasets

Note：

• NYCTAXI_DYNA is a dataset that counts the inflow and outflow of the region with an irregular area division method.
• NYCTAXI_OD is a dataset that counts the origin-destination flow between regions with an irregular area division method.
• NYCTAXI_GRID is a dataset that counts the inflow and outflow of the region with a grid-base division method.
• NYC_TOD is a dataset that counts the origin-destination flow between regions with a grid-base division method.

Cite

Our paper is accepted by ACM SIGSPATIAL 2021. If you find LibCity useful for your research or development, please cite our paper.

@inproceedings{10.1145/3474717.3483923,
  author = {Wang, Jingyuan and Jiang, Jiawei and Jiang, Wenjun and Li, Chao and Zhao, Wayne Xin},
  title = {LibCity: An Open Library for Traffic Prediction},
  year = {2021},
  isbn = {9781450386647},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3474717.3483923},
  doi = {10.1145/3474717.3483923},
  booktitle = {Proceedings of the 29th International Conference on Advances in Geographic Information Systems},
  pages = {145–148},
  numpages = {4},
  keywords = {Spatial-temporal System, Reproducibility, Traffic Prediction},
  location = {Beijing, China},
  series = {SIGSPATIAL '21}
}

Jingyuan Wang, Jiawei Jiang, Wenjun Jiang, Chao Li, and Wayne Xin Zhao. 2021. LibCity: An Open Library for Traffic Prediction. In Proceedings of the 29th International Conference on Advances in Geographic Information Systems (SIGSPATIAL '21). Association for Computing Machinery, New York, NY, USA, 145–148. DOI:https://doi.org/10.1145/3474717.3483923

04/27/2023 Update: We published a long paper on LibCity, including (1) classification and base units of urban spatial-temporal data and proposed a unified storage format, i.e., atomic files, (2) a detailed review of urban spatial-temporal prediction field (including macro-group prediction, micro-individual prediction, and fundamental tasks), (3) proposed LibCity, an open source library for urban spatial-temporal prediction, detailing each module and use cases, and providing a web-based experiment management and visualization platform, (4) selected more than 20 models and datasets for comparison experiments based on LibCity, obtained model performance rankings and summarized promising future research directions. Please check this link for more details.

For the long paper, please cite it as follows:

@article{libcitylong,
  title={Towards Efficient and Comprehensive Urban Spatial-Temporal Prediction: A Unified Library and Performance Benchmark}, 
  author={Jingyuan Wang and Jiawei Jiang and Wenjun Jiang and Chengkai Han and Wayne Xin Zhao},
  journal={arXiv preprint arXiv:2304.14343},
  year={2023}
}