• Stars
    star
    158
  • Rank 237,131 (Top 5 %)
  • Language
    Python
  • License
    Mozilla Public Li...
  • Created over 9 years ago
  • Updated 5 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

GTFS ORM using SQLAlchemy

GTFSDB

Join the chat at https://gitter.im/OpenTransitTools/gtfsdb

Supported Databases

  • PostgreSQL (PostGIS for Geo tables) - preferred
  • Oracle - tested
  • MySQL - tested
  • SQLite - tested

GTFS (General Transit Feed Specification) Database

Python code that will load GTFS data into a relational database, and SQLAlchemy ORM bindings to the GTFS tables in the gtfsdb. The gtfsdb project's focus is on making GTFS data available in a programmatic context for software developers. The need for the gtfsdb project comes from the fact that a lot of developers start out a GTFS-related effort by first building some amount of code to read GTFS data (whether that's an in-memory loader, a database loader, etc...); GTFSDB can hopefully reduce the need for such drudgery, and give developers a starting point beyond the first step of dealing with GTFS in .csv file format.

Available on pypi: https://pypi.python.org/pypi/gtfsdb

Install from source via github (if you want the latest code) :

  1. Install Python 3.x https://www.python.org/downloads/ (code also runs on 2.7 if you are stuck on that version)

  2. pip install zc.buildout - https://pypi.org/project/zc.buildout

  3. (optinal step for postgres users: 'pip install psycopg2-binary')

  4. git clone https://github.com/OpenTransitTools/gtfsdb.git

  5. cd gtfsdb

  6. buildout install prod -- NOTE: if you're using postgres, do a 'buildout install prod postgresql'

  7. bin/gtfsdb-load --database_url <db url> <gtfs file | url>

  8. examples:

    • bin/gtfsdb-load --database_url sqlite:///gtfs.db gtfsdb/tests/large-sample-feed.zip

    • bin/gtfsdb-load --database_url sqlite:///gtfs.db http://developer.trimet.org/schedule/gtfs.zip

    • bin/gtfsdb-load --database_url postgresql://postgres@localhost:5432 --is_geospatial http://developer.trimet.org/schedule/gtfs.zip

      Note

      adding the is_geospatial cmdline flag, when paired with a spatial-database ala PostGIS (e.g., is_spatial is meaningless with sqllite), will take longer to load...but will create geometry columns for both rendering and calculating nearest distances, etc...

  9. view db ( example: https://sqliteonline.com )

The best way to get gtfsbd up and running is via the 'zc.buildout' tool. Highly recommended to first install buildout (e.g., pip install zc.buildout) before doing much of anything else.

Postgres users, gtfsdb requires the psycopg2-binary database driver. Installing that via pip install psychopg2-binary will relieve gtfsdb from re-installing locally as part of the build. And if after the fact, you see exceptions mentioning

Note

if you get the message "ImportError: No module named psycopg2", then 'pip install psychopg2-binary' should fix things. (Assumes you have postgres also installed on the machine you're trying to use the pg driver).

Usage with Docker:

  1. Build the image with docker build -t gtfsdb .
  2. Run it with:
docker run gtfsdb --database_url <db url>  <gtfs file | url>

Note

The entrypoint command is bin/gtfsdb-load so the arguments will be passed to it.

Example Queries:

  • get first stop time of each trip for route_id 1

    select *
    from trips t, stop_times st
    where t.route_id = '1'
    and t.trip_id = st.trip_id
    and st.stop_sequence = 1
  • get agency name and number of routes

    select a.agency_name, a.agency_id, count(r.route_id)
    from routes r, agency a
    where r.agency_id = a.agency_id
    group by a.agency_id, a.agency_name
    order by 3 desc

More Repositories

1

ttpub

HTML
26
star
2

gtfsdb_realtime

Real-time GTFS database loader and ORM library.
Python
11
star
3

transitcast

Collect gtfs vehicle movement data for ML model training.
Go
11
star
4

loader

deploy
Python
9
star
5

view

Pyramid front-end web view(s) for OTT
Mako
5
star
6

data

load gtfs & gtfs-rt into a database ... using python, gtfsdb, GTFSrDB, etc...
Python
4
star
7

otp_client_py

OpenTripPlanner API wrapper for python
Python
4
star
8

carshare

Carshare database loader and json services in Python/Pyramid
Python
3
star
9

trafficdb

Traffic database conflates transit (gtfs), traffic, road speed data and osm street data.
Python
3
star
10

utils

utility objects ... share them across other apps...
Python
2
star
11

omt_server

OpenMapTiles server for styles, etc...
Shell
2
star
12

view_header_footer

CSS
1
star
13

osm

Python
1
star
14

services

web services for transit ... in python & pyramid
Python
1
star
15

OpenTransitTools.github.io

website
HTML
1
star
16

otp_client_java

Java client for OTP
Java
1
star
17

geocoder

SOLR GeoCoder / Database Load
Shell
1
star
18

pelias.adapter

Give Pelais a SOLR veneer (in order swap in Pelias on clients written against SOLR)
Python
1
star
19

boundary

Python
1
star
20

integration-tests

scripts to help run web app tests via the command-line using selenium web driver
Python
1
star
21

map_server

PostGIS + GeoServer + gtfsdb + Docker = ott.map_server
Python
1
star
22

geo

GeoServer and PostGIS docker config, etc....
Shell
1
star
23

docs

Python
1
star
24

cache

Web File Cache
Python
1
star
25

convert

convert files to different formats, ala yaml -> json, csv -> json, etc...
Python
1
star
26

geoserver_data_dir

empty geoserver data directory
Scheme
1
star
27

log_parser

Parse server log files -- and OTP trip plan requests therein
Python
1
star
28

gbfsdb

General Bikeshare Feed Spec Database
Python
1
star
29

icons

Python
1
star
30

web_snippets

small simple standalone pieces of code for use as demo apps in other projects, etc...
JavaScript
1
star
31

gtfsdb-seg

gtfsdb add-on for creating route segments between stops
Python
1
star
32

pytransitcast

Python components that are part of transitcast system
Python
1
star