• Stars
    star
    158
  • Rank 237,131 (Top 5 %)
  • Language
    C
  • Created over 1 year ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Constructing a pangenome gene graph

Getting Started

# Check prebuilt graphs at https://pangene.bioinweb.org

# Install pangene
git clone https://github.com/lh3/pangene
cd pangene && make

# Alternatively, download the precompiled binaries for arm-mac and x64-linux
curl -L https://github.com/lh3/pangene/releases/download/v1.1/pangene-1.1-bin.tar.bz2|tar jxf -

# The C4 example with provided alignment
./pangene test/C4/*.paf.gz > C4.gfa        # generate the graph
k8 pangene.js call C4.gfa > C4.bubble.txt  # identify bubbles

# Deploy the GFA server on the C4 example; require pangene-1.1-bin
cd pangene-1.1-bin                         # run gfa-server in this directory
bin_arm64-mac/gfa-server /path/to/C4.gfa   # or use bin_linux-x64 on x64-linux
# open "http://127.0.0.1:8000/view?gene=C4A,C4B" in a web browser

# Deploy the GFA server on the human graph
bin_arm64-mac/gfa-server -d html data/*.gfa.gz 2> server.log &
# open "http://127.0.0.1:8000" in a web browser

# Align proteins to each genome (general use cases; no example data)
miniprot --outs=0.97 --no-cs -Iut16 genome1.fna proteins.faa > genome1.paf
miniprot --outs=0.97 --no-cs -Iut16 genome2.fna proteins.faa > genome2.paf

# Construct a pangene graph
pangene genome1.paf genome2.paf > graph.gfa

# Check manpage
man ./pangene.1

Table of Contents

Introduction

Pangene is a command-line tool to construct a pangenome gene graph. In this graph, a node repsents a marker gene and an edge between two genes indicates their genomic adjaceny on input genomes. Pangene takes the miniprot alignment between a protein set and multiple genomes and produces a graph in the GFA format. It attempts to reduce the redundancy in the input proteins and filter spurious alignments while preserving close but non-identical paralogs. The output graph can be visualized in generic GFA viewers such as BandageNG or via a web interface. Users can explore local human subgraphs at a public server. Prebuilt pangene graphs can be found at DOI:10.5281/zenodo.8118576.

Graph Construction

Pangene takes a list of protein-to-genome alignment as input. To generate these alignments, you need to align the same set of proteins to multiple genomes. How to choose the protein set can be tricky.

Preparing a protein set

For constructing a human pangene graph, the simplest choice is to use annotated genes in GRCh38. It is highly recommended to name a protein sequence like RGPD6:ENSP00000512633.1 where RGPD6 is the gene name and ENSP00000512633.1 is the unique protein identifier. In the output GFA, nodes are named after genes, so you would want to use human-readable gene names for visualization later. You may use the following command line to extract protein sequences from Ensembl or GenCode annotation:

k8 pangene.js getaa gene-anno.gtf protein-seq.faa > proteins.faa

With pangene, different isoforms or diverged alleles of the same gene can be present in the protein set, though in practice, we find selecting the canonical isoform per gene tends to give a cleaner graph probably possibly due to annotation errors among rare isoforms. For the GenCode annotation, use getaa -c to extract canonical isoforms only.

For a new species without good gene annotation, you may use protein annotations from a closely related species. You may pool proteins from multiple closely related species as well. Pangene aims to work with such input but this use case has not been thoroughly carefully evaluated. Given a bacteria pangenome of the same species, you may predict genes with existing tools, cluster them with CD-HIT or MMseqs2 and feed the representative protein in each cluster to pangene.

Aligning proteins to genomes

Pangene currently only works with miniprot's PAF output. We usually use the following command line:

miniprot --outs=0.97 --no-cs -Iut16 genomeX.fna proteins.faa > genomeX.paf

For bacterial data, add -S to disable splicing.

Constructing a pangene graph

The following command-line constructs a pangene graph

pangene *.paf > graph.gfa

If the output graph is cluttered in the Bandage viewer, you may add option -a2 to filter out edges supported by a single genome. By default, pangene filters out genes occurring in less than 5% of the genomes after deredundancy. If you want to retain low-frequency genes, add -p0 to disable the filter.

Analyzing a graph

The GFA file is the master output. You can extract various information from this file. You may find local gene-level variations with

k8 pangene.js call graph.gfa > bubble.txt

or get the presence/absence of each gene with

k8 pangene.js gfa2matrix graph.gfa > gene_presence_absence.Rtab

Graph Visualization

You can look at the entire graph in the Bandage GFA viewer. If you are interested in a particular gene, it is best to set up gfa-server which is part of gfatools. Here is a public server for human genes. You can deploy this server on your machine with

curl -L https://github.com/lh3/pangene/releases/download/v1.1/pangene-1.1-bin.tar.bz2|tar jxf -
cd pangene-1.1-bin
bin_arm64-mac/gfa-server -d html data/*.gfa.gz 2> server.log # for Mac

Then you can open link http://127.0.0.1:8000/ in your browser, type gene names and visualize a local subgraph around the desired genes.

Citation

If you use pangene in your work, please consider to cite:

H Li, M Marin, MR Farhat (2024) Exploring gene content with pangenome gene graphs, arXiv:2402.16185

Limitations

  • Pangene only works with miniprot's PAF output.

  • In the output graph, arcs on W-lines may be absent from L-lines.

More Repositories

1

minimap2

A versatile pairwise aligner for genomic and spliced nucleotide sequences
C
1,754
star
2

bwa

Burrow-Wheeler Aligner for short-read alignment (see minimap2 for long-read alignment)
C
1,504
star
3

seqtk

Toolkit for processing sequences in FASTA/Q formats
C
1,363
star
4

bioawk

BWK awk modified for biological data
C
590
star
5

minigraph

Sequence-to-graph mapper and graph generator
C
402
star
6

miniprot

Align proteins to genomes with splicing and frameshift
C
316
star
7

miniasm

Ultrafast de novo assembly for long noisy reads (though having no consensus step)
TeX
302
star
8

wgsim

Reads simulator
C
258
star
9

gfatools

Tools for manipulating sequence graphs in the GFA and rGFA formats
C
208
star
10

biofast

Benchmarking programming languages/implementations for common tasks in Bioinformatics
C
175
star
11

readfq

Fast multi-line FASTA/Q reader in several programming languages
C
170
star
12

cgranges

A C/C++ library for fast interval overlap queries (with a "bedtools coverage" example)
C
165
star
13

kmer-cnt

Code examples of fast and simple k-mer counters for tutorial purposes
C++
162
star
14

psmc

Implementation of the Pairwise Sequentially Markovian Coalescent (PSMC) model
C
147
star
15

bedtk

A simple toolset for BED files (warning: CLI may change before bedtk becomes stable)
C
134
star
16

ksw2

Global alignment and alignment extension
C
127
star
17

yak

Yet another k-mer analyzer
C
111
star
18

fermikit

De novo assembly based variant calling pipeline for Illumina short reads
TeX
108
star
19

minimap

This repo is DEPRECATED. Please use minimap2, the successor of minimap.
C
106
star
20

hickit

TAD calling, phase imputation, 3D modeling and more for diploid single-cell Hi-C (Dip-C) and general Hi-C
C
105
star
21

bgt

Flexible genotype query among 30,000+ samples whole-genome
C
96
star
22

dipcall

Reference-based variant calling pipeline for a pair of phased haplotype assemblies
JavaScript
92
star
23

srf

SRF: Satellite Repeat Finder
TeX
86
star
24

unimap

A EXPERIMENTAL fork of minimap2 optimized for assembly-to-reference alignment
C
85
star
25

minipileup

Simple pileup-based variant caller
C
79
star
26

fermi

A WGS de novo assembler based on the FMD-index for large genomes
C
74
star
27

dna-nn

Model and predict short DNA sequence features with neural networks
C
72
star
28

fermi-lite

Standalone C library for assembling Illumina short reads in small regions
C
72
star
29

bfc

High-performance error correction for Illumina resequencing data
TeX
68
star
30

ropebwt2

Incremental construction of FM-index for DNA sequences
TeX
67
star
31

tabtk

Toolkit for processing TAB-delimited format
C
59
star
32

gwfa

Proof-of-concept implementation of GWFA for sequence-to-graph alignment
C
56
star
33

CHM-eval

TeX
49
star
34

miniwfa

A reimplementation of the WaveFront Alignment algorithm at low memory
C
49
star
35

jstreeview

Interactive phylogenetic tree viewer/editor
JavaScript
46
star
36

samtools

This is *NOT* the official repository of samtools.
C
46
star
37

etrf

Exact Tandem Repeat Finder (not a TRF replacement)
C
45
star
38

ref-gen

Human reference genome analysis sets
Makefile
44
star
39

bioseq-js

For live demo, see http://lh3lh3.users.sourceforge.net/bioseq.shtml
HTML
37
star
40

lv89

C implementation of the Landau-Vishkin algorithm
C++
35
star
41

partig

An experimental tool to estimate the similarity between all pairs of contigs
C
35
star
42

asub

A unified array job submitter for LSF, SGE/UGE and Slurm
Perl
32
star
43

klib.nim

Experimental getopt, gzip reader, FASTA/Q parser and interval queries in nim-lang
Nim
32
star
44

calN50

Compute N50/NG50 and auN/auNG
JavaScript
31
star
45

sdust

Symmetric DUST for finding low-complexity regions in DNA sequences
C
31
star
46

gffio

C
31
star
47

pre-pe

Preprocessing paired-end reads produced with experiment-specific protocols
C
31
star
48

hapdip

The CHM1-NA12878 benchmark for single-sample SNP/INDEL calling from WGS Illumina data
JavaScript
30
star
49

fermi2

C
26
star
50

misc

Useful small programs
C
26
star
51

varcmp

The first CHM1 paper (Li, 2014)
TeX
25
star
52

minisv

Lightweight mosaic/somatic SV caller for long reads (WIP)
JavaScript
24
star
53

lianti

Tools to process LIANTI sequence data
C
23
star
54

rtgeval

Wrapper for RTG's vcfeval; DEPRECATED!
Shell
21
star
55

nasw

Dynamic programming for aa-to-nt alignment with affine gap, splicing and frameshift
C
18
star
56

sgdp-fermi

FermiKit small variant calls for public SGDP samples
17
star
57

gfa1

This repo is deprecated. Please use gfatools instead.
C
16
star
58

pubLRasm

16
star
59

PortableCrystal

Portable Crystal binary distributions for Linux on x86_64
15
star
60

foreign

Modified or extracted from other programs
C
15
star
61

trimadap

Fast but inaccurate adapter trimmer for Illumina reads
C
14
star
62

lh3-snippets

C
14
star
63

ropebwt3

Construction and utility of BWT for DNA string sets
C
13
star
64

treebest

TreeBeST: Tree Building guided by Species Tree
C
13
star
65

unicall

A wrapper for calling small variants from human germline high-coverage single-sample Illumina data
Perl
12
star
66

fastARG

Fast heuristic ARG construction
C
12
star
67

proot-wrapper

Demonstrating the PRoot program
Perl
12
star
68

rmaxcut

An experimental tool to find approximate max-cuts in a large graph
C
11
star
69

bwa-docker

Minimal docker image for bwa. Not developed any more.
11
star
70

sdg

EXPERIMENTAL implementation of side graph
C
10
star
71

naivepca

Naive PCA for genotype data
C
10
star
72

mdust

mdust from DFCI Gene Indices Software Tools (archived for a historical record only)
C
10
star
73

editdist-U85

Fast implementation of Ukkenon's O(ND) algorithm for computing edit distance
C
9
star
74

lh3.github.com

TeX
9
star
75

libdivsufsort

Automatically exported from code.google.com/p/libdivsufsort
C
8
star
76

mem-paper

Manuscript for BWA-MEM
6
star
77

bcf2

Experimental bcftools port to support BCF2; DEPRECATED by htslib and htsbox
C
6
star
78

thesis

PhD thesis
TeX
5
star
79

ropebwt

C
4
star
80

fermi-paper

The first fermi paper (Li, 2012)
3
star
81

crlf

Concise Run-Length Format for small alphabets; DEPRECATED
C
3
star
82

psnw

prototype
C
2
star
83

centos5-vm

Instructions on how to deploy CentOS 5 virtual machines
2
star
84

mag2gfa

DEPRECATED. Code has been moved to lh3/gfa1/misc
C
2
star
85

ibsget

Download files from Illumina BaseSpace (*OUTDATED* as BaseSpace has changed APIs)
C
2
star
86

smtl-paper

Samtools statistics paper (Li, 2011)
Lua
1
star
87

mssa-bench

Evaluating the performance of multi-string SA construction
C
1
star
88

samtools-legacy

For testing only. DON'T USE!
C
1
star