PrimeKG

Lab Website | Nature Publication | Harvard Dataverse

TL;DR

Precision Medicine Knowledge Graph (PrimeKG) presents a holistic view of diseases. PrimeKG integrates 20 high-quality biomedical resources to describe 17,080 diseases with 4,050,249 relationships representing ten major biological scales. We accompany PrimeKG’s graph structure with text descriptions of clinical guidelines for drugs and diseases to enable multimodal analyses. Download this csv file to get started!

Updates

• [July 2023] PrimeKG construction scripts are updated to include primary source data releases up to July 2023. Note that the files published on Harvard DataVerse remain unchanged; however, we provide new scripts and updated links should users wish to build their own current version of PrimeKG. For more details, please see the relevant section below.
• [Feb 2023] PrimeKG is published in Nature Scientific Data.
• [Jun 2022] PrimeKG crosses 5,000 downloads on Harvard Dataverse!
• [Apr 2022] PrimeKG is live on bioRxiv and Harvard Dataverse!

Unique Features of PrimeKG

• Diverse coverage of diseases: PrimeKG contains over 17,000 diseases including rare dieases. Disease nodes in PrimeKG are densely connected to other nodes in the graph and have been optimized for clinical relevance in downstream precision medicine tasks.
• Heterogeneous knowledge graph: PrimeKG contains over 100,000 nodes distributed over various biological scales as depicted below. PrimeKG also contains over 4 million relationships between these nodes distributed over 29 types of edges.
• Multimodal integration of clinical knowledge: Disease and drug nodes in PrimeKG are augmented with clinical descriptors that come from medical authorities such as Mayo Clinic, Orphanet, Drug Bank, and so forth.
• Ready-to-use datasets: PrimeKG is minimally dependent on external packages. Our knowledge graph can be retrieved in a ready-to-use format from Harvard Dataverse.
• Data functions: PrimeKG provides extensive data functions, including processors for primary resources and scripts to build an updated knowledge graph.

Environment setup

Using pip

To install the dependencies required to run the PrimeKG code, use pip:

pip install -r requirements.txt

Or use conda

conda env create --name PrimeKG --file=environments.yml

Using PrimeKG

For a quick start in Python, you can download the raw data files in .csv format directly from Harvard Dataverse or load PrimeKG using the following community dataloaders.

Getting started in Python

Download PrimeKG from Harvard Dataverse using the following bash command. You can replace kg.csv with any file path.

wget -O kg.csv https://dataverse.harvard.edu/api/access/datafile/6180620

You can use the following code to load PrimeKG and visualize its data.

import pandas as pd
primekg = pd.read_csv('kg.csv', low_memory=False)
primekg.query('y_type=="disease"|x_type=="disease"')

Dataloader: Therapeutics Data Commons

website | docs

pip install PyTDC

from tdc.resource import PrimeKG
data = PrimeKG(path = './data')
drug_feature = data.get_features(feature_type = 'drug')
data.to_nx()
data.get_node_list(type = 'disease')

Dataloader: PyKEEN

website | docs

pip install pykeen

import pykeen.datasets
pykeen.datasets.has_dataset('primekg')

Building an updated PrimeKG

Downloading primary data resources

All persistent identifiers and weblinks to download the 20 primary data resources used to build PrimeKG are systematically provided in the Data Records section of our article. We have also mentioned the exact filenames that were downloaded from each resource for easy corroboration.

Curating primary data resources

We provide the scripts used to process all primary data resources and the names of the resulting output files generated by those scripts. We would be happy to share the intermediate processing datasets that were used to create PrimeKG on request.

Harmonizing datasets into PrimeKG

The code to harmonize datasets and construct PrimeKG is available at build_graph.ipynb. Simply run this jupyter notebook in order to construct the knowledge graph from the outputs of the processing files mentioned above. This jupyter notebook produces all three versions of PrimeKG, kg_raw.csv, kg_giant.csv, and the complete version kg.csv.

Feature extraction

The code required to engineer features can be found at engineer_features.ipynb and mapping_mayo.ipynb.

July 2023 update

In July 2023, this repository was updated to rebuild PrimeKG and update the knowledge graph to include database releases up to July 2023.Note that the files published on Harvard DataVerse remain unchanged; however, we provide new scripts and updated links should users wish to build their own current version of PrimeKG. For more details, see this pull request.

17 scripts datasets/processing_scripts/ are re-run or updated to build a new version of PrimeKG, while datasets/feature_construction/ scripts may remain out-of-date. Re-run or updated primary data sources include Bgee, Comparative Toxicogenomics Database, DisGeNET, DrugBank, DrugCentral, NCBI Gene, Gene Ontology, Human Phenotype Ontology, MONDO, Reactome, SIDER, UBERON, and UMLS.

For more information, see datasets/primary_data_resources.sh. Changes include the following:

General

Created script to automatically create directory structure, pull data, and run all necessary processing and feature extraction steps.

• Fixed broken environment construction script.
• Script automatically creates required directories.
• Added commands to retrieve gene names, details, and NCBI ID to UniProt ID mapping from www.genenames.org, then output to vocab/gene_names.csv and vocab/gene_map.csv.

Bgee

• 58405/5257181 gold quality calls with expression rank < 25000 now specify cell type in a particular tissue (e.g., UBERON:0000473 ∩ CL:0000089, which denotes germ line stem cell in testis).
• These rows are dropped in bgee.py.
• URL updated to here.

Comparative Toxicogenomics Database

• URL updated to here.

DisGeNET

• No changes needed.

DrugBank

• Fixed paths in parsexml_drugbank.py. Output to new /parsed subdirectory. Removed extraneous lines in Parsed_feature.ipynb.
• ✅ Successfully ran drugbank_drug_drug.py and drugbank_drug_protein.py.
• ⚠️ parsexml_drugbank.py and Parsed_feature.ipynb may need updates.

DrugCentral

• Modified drugcentral_queries.txt to work on O2, the Harvard Medical School high-performance computing cluster.
• ⚠️ drugcentral_feature.Rmd may need updates.

NCBI Gene

• No changes needed.

Gene Ontology

• Used -L flag to follow redirects. No other changes needed.

Human Phenotype Ontology

• Used -L flag to follow redirects. No other changes needed to hpo.py.
• Updated hpoa.py to replace old column names with new column names.

MONDO

• Added check for NoneType values in external references (line 29).

Reactome

• No changes needed.

SIDER

• No changes needed.

UBERON

• Checked for NA values, dropped two obsolete terms (UBERON:0039300 and UBERON:0039302) not marked as obsolete in the source file.

UMLS

• UMLS data pulled and paths updated for 2023 data.
• ⚠️ umls.ipynb may need updates.

Cite Us

If you find PrimeKG useful, cite our work:

@article{chandak2022building,
  title={Building a knowledge graph to enable precision medicine},
  author={Chandak, Payal and Huang, Kexin and Zitnik, Marinka},
  journal={Nature Scientific Data},
  doi={https://doi.org/10.1038/s41597-023-01960-3},
  URL={https://www.nature.com/articles/s41597-023-01960-3},
  year={2023}
}

Data Server

PrimeKG is hosted on Harvard Dataverse with the following persistent identifier https://doi.org/10.7910/DVN/IXA7BM. When Dataverse is under maintenance, PrimeKG datasets cannot be retrieved. That happens rarely; please check the status on the Dataverse website.

License

PrimeKG codebase is under MIT license. For individual dataset usage, please refer to the dataset license found in the website.