• Stars
    star
    110
  • Rank 316,770 (Top 7 %)
  • Language
    Python
  • License
    MIT License
  • Created over 9 years ago
  • Updated about 6 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A command line utility for generating topojson from various data sources for fast maps.

mapturner

A command line utility for generating consolidated TopoJSON from various data sources. Used for making fast vector maps with D3.

Important links:

Install

You will need to have the following non-Python dependencies installed:

mapturner itself can be installed with pip:

pip install mapturner

Note: Since mapturner relies on topojson 1.x (old version of topojson), we recommend installing this version of topojson inside your project root folder by running:

npm install -g [email protected] --prefix node_modules

To install [email protected] globally instead run:

npm install -g [email protected]

mapturner will search first for the topojson binaries installed within the project and fallback to searching the topojson binary on your $PATH.

Developer install process:

git clone git://github.com/nprapps/mapturner.git
cd mapturner
mkvirtualenv mapturner

pip install -r requirements.txt

python setup.py develop

Usage

Define a YAML configuration file, such as the following example. The complete list of valid options is further on in this documentation.

bbox: '77.25 24.28 91.45 31.5'
layers:
    countries:
        type: 'shp'
        path: 'http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_admin_0_countries.zip'
        id-property: 'NAME'
        properties:
            - 'country=NAME'

    cities:
        type: 'shp'
        path: 'http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_populated_places_simple.zip'
        id-property: 'name'
        properties:
            - 'featurecla'
            - 'city=name'
        where: adm0name = 'Nepal' AND scalerank < 8

    quakes:
        type: 'csv'
        path: 'examples/nepal.csv'
        all-properties: True

Then run it!

mapturner input.yaml output.json

The following layer types are currently supported:

  • ESRI Shapefile (shp)
  • GeoJSON or TopoJSON (json)
  • CSV (csv)

How it works

For each layer defined in the configuration file:

  • If path is a URL the file will be downloaded and cached locally. (It will not be redownloaded on subsequent runs.)
  • If path is to a zipped file it will be unzipped.
  • All layers will be clipped to the specified bounding box (using ogr2ogr).
  • For each layer, if a where attribute is specified, the layer data will be filtered by that clause.
  • For each layer, all fields in the layer not specified in the properties array will be removed (to reduce file size), unless all-properties is specified, in which case all will be kept.
  • For each layer, if an id-property is set, data from that property will be set as the identifier for the features in the layer.
  • The layer will be converted to TopoJSON.

After each layer has been processed all of them will be concatenated into a single TopoJSON file. Each layer's key name will be used to identify it in the output.

Complete list of configuration options

For all layer types:

  • type: The type of layer. Valid types are shp, json (GeoJSON or TopoJSON), and csv. (Required)
  • path: The path (relative or absolute) to the layer data file. (Required)
  • id-property: A property from the data file to use as the unique identifier for features. See also, the TopoJSON command-line documentation.
  • properties: A list of properties from the data to be kept in the output. All other properties are dropped.
  • all-properties: If true, then all properties are kept for this layer.
  • where: A SQL-like query predicate that will filter the feature data. This This uses exactly the same query syntax as ogr2ogr.

CSV layers only:

  • latitude: The name of a column in the data containing the latitude of the point/feature.
  • longitude: The name of a column in the data containing the longitude of the point/feature.

Cached data

Cached shapefiles are stored in ~/.mapturner. You may wish to clear this folder periodically to free up space and ensure updated shapefiles are redownloaded.

More Repositories

1

app-template

The NPR visuals team's opinionated project template for client-side apps.
JavaScript
1,536
star
2

bestpractices

Best practices and coding conventions for the NPR Visuals team.
286
star
3

dailygraphics

NPR Visuals' rig for deploying daily graphics projects in responsive iframes.
JavaScript
285
star
4

copytext

A library for accessing a spreadsheet as a native Python object suitable for templating.
Python
225
star
5

lunchbox

Image tools for social media sharing
JavaScript
173
star
6

tshirt

Planet Money Makes A T-Shirt
JavaScript
151
star
7

quotable

REPO DEPRECATED; see the current version in Lunchbox http://github.com/nprapps/lunchbox
JavaScript
93
star
8

nprapps.github.com

The NPR visuals team's blog.
HTML
77
star
9

interactive-template

A Node-based template for starting news apps and interactive pages
JavaScript
55
star
10

newscast.js

A library to radically simplify Chromecast web app development.
JavaScript
54
star
11

dailygraphics-next

NPR's daily graphics rig, 2.0
JavaScript
43
star
12

heat-income

Analysis of heat and income in U.S. cities
Python
34
star
13

book-concierge

A concierge for every year
Less
33
star
14

armslist-scraper

A simple scraper for armslist.com listings
Python
32
star
15

waterbug

REPO DEPRECATED; see the current version in Lunchbox http://github.com/nprapps/lunchbox
JavaScript
29
star
16

sidechain

Modern responsive iframes
JavaScript
29
star
17

roku-tinydesk

Tiny Desk Concerts Roku app
Brightscript
28
star
18

books13

NPR's Book Concierge: Our Guide To 2013's Great Reads
JavaScript
27
star
19

totebot2

Everything is better in the new building, even the totebot.
CoffeeScript
25
star
20

django-starter-kit

Opinionated template for Django projects on Python 3 and PostgreSQL
Python
24
star
21

lookatthis

Stories about how you see the world.
JavaScript
22
star
22

betty

An unambiguous dialect of ArchieML
JavaScript
20
star
23

trump-tweet-analysis

Data and sentiment analysis of Trump's tweets
Jupyter Notebook
20
star
24

anno-docs

Live transcription rig
JavaScript
19
star
25

graphics-archive

Archived graphics published using our dailygraphics rig
JavaScript
16
star
26

worldvalues

World Values Survey data analysis
Python
15
star
27

elections16

App for 2016 primary elections
JavaScript
15
star
28

carebot

NPR Visual's Carebot (deprecated, now in: https://github.com/thecarebot/carebot)
Python
15
star
29

walmart

Mapping the growth of Wal-Mart in urban areas.
Shell
14
star
30

ucr-clearance-parser

parse uniform crime reporting clearance data
Python
13
star
31

envivo

A live-blogging application.
Python
13
star
32

bestsongs14

Songs We Love 2014
JavaScript
13
star
33

electris

Elections 2012
PLpgSQL
13
star
34

copydoc

Like copytext, but for docs
HTML
12
star
35

papertrail

Rig for deploying DocumentCloud viewers to S3.
JavaScript
12
star
36

tumble

A rig for handling static tumblr themes in a reasonable fashion.
CSS
12
star
37

playgrounds2

A community-edited guide to accessible playgrounds in the United States.
JavaScript
11
star
38

elections14

We're having an election party.
JavaScript
11
star
39

barkedu

The world is starting to forget about Ebola. The village of Barkedu can't.
JavaScript
10
star
40

dailygraphics-templates

Graphic templates for the dailygraphics-next rig
JavaScript
9
star
41

stl-lobbying

Lobbying in Missouri project with SLPR
JavaScript
9
star
42

bernard

Python
9
star
43

servers

Server setup scripts for NPR Apps servers.
Shell
9
star
44

hollerith

Publish Sheets to S3 as JSON
JavaScript
8
star
45

leso

Processing scripts for Defense Logistics Agency LESO data
Shell
8
star
46

books14

NPR's Book Concierge app
JavaScript
8
star
47

photo-finder

An internal-facing tool for searching instagram.
JavaScript
8
star
48

arrested-development

The one about Arrested Development.
JavaScript
8
star
49

us-wildfires

Fire-forecast data for the United States.
JavaScript
8
star
50

ap-election-loader

basic AP election loader
Shell
8
star
51

elections20-interactive

Front-end graphics for the 2020 general election
JavaScript
8
star
52

mental-health

A Silent Epidemic: The Mental Health Crisis In Our Schools
JavaScript
8
star
53

elections20-primaries

Primary results for 2020
JavaScript
7
star
54

borders-map

Borderland collaboration with CIR
JavaScript
7
star
55

cron-starter-kit

This is a simple starter kit for deploying and maintaining cron jobs on EC2 servers.
Python
7
star
56

austin

The Austin 100
JavaScript
7
star
57

shelf-life

A Tumblr rig for pantry raids.
JavaScript
7
star
58

school-choice

Data analysis for education's school choice in Indiana project
Jupyter Notebook
6
star
59

elections22

Data pipeline and results graphics for the 2022 general election. Building off elections20-interactive.
JavaScript
6
star
60

nprchat

Experimental chat using Firebase.
JavaScript
6
star
61

armslist-analysis

Analysis of the armslist dataset
Python
6
star
62

liveblog-standalone

NPR's liveblog rig 2.0
JavaScript
6
star
63

inauguration

NPR Inauguration 2013 live chat (built on Scribble Live) and "Dear Mr. President" listener call-out (built on Tumblr)
JavaScript
6
star
64

wolves

Big beautiful Gilkey photos of wolves & Nate Rott's audio opus
JavaScript
6
star
65

civilrights

Behind The Civil Rights Act: How it was made and what it means today—commentary on the landmark Civil Rights Act of 1964.
JavaScript
6
star
66

congress-bot

Bot to track legislation and other activity by members of Congress from a specific state
Python
5
star
67

graeae

JavaScript
5
star
68

books18

Best books of 2018
JavaScript
5
star
69

books16

book concierge 2016 edition
JavaScript
5
star
70

elections18-graphics

2018 midterm election front-end; iteration upon 2016 GE work
JavaScript
5
star
71

oscars

Oscar Night 2013 live chat / liveblog (built on Scribble Live), and Best Picture cheat sheet
JavaScript
5
star
72

anno-lambda-authorizer

AWS lambda function that serves as a custom authorizer for AWS API Gateway
Python
5
star
73

geocode-nominatim

Geocode structured & unstructured addresses using Nominatim service
Python
4
star
74

cypher

A Vagrant config for running NPR's news apps
4
star
75

clerk

A cron job --> Slack webhook that posts new action from the House floor
Python
4
star
76

pym-particle

Active development moved to Sidechain
JavaScript
4
star
77

ahca

Python
4
star
78

rockymountain

JavaScript
4
star
79

in-memoriam-2013

NPR Music remembers the singers, instrumentalists, songwriters and personalities who died in 2013.
JavaScript
4
star
80

disability

HTML breakout story page for the Planet Money / This American Life / NPR project "Unfit For Work."
JavaScript
4
star
81

executive-orders

Cron job that posts to Slack incoming webhook when new executive orders (or actions or memos) are published to whitehouse.gov
Python
4
star
82

nola

The End Of Neighborhood Schools: New Orleans charter schools
JavaScript
3
star
83

autocomplete-input

JavaScript
3
star
84

sanctuary-cities

Python
3
star
85

bestsongs15-midyear

best songz, 2015 midyear edition
JavaScript
3
star
86

commencement

The Best Commencement Speeches, Ever
JavaScript
3
star
87

sea-level-scroll

Visual narrative: Who Will Pay To Protect Tech Giants From Rising Seas?
JavaScript
3
star
88

okkervil

An audio-guided tour through a Leaflet map
JavaScript
3
star
89

syria

With Syria engulfed in civil war, here are four stories of families struggling to stay together.
JavaScript
3
star
90

elections18-general

2018 midterm election back-end: Associated Press data ETL, database, admin panel, and JSON output; iteration upon 2016 GE work
JavaScript
3
star
91

science-of-joy

It's a joy generator
JavaScript
3
star
92

factcheck-db

Python
3
star
93

sotu

Application for aggregating NPR's State of the Union coverage (built on Scribble Live)
JavaScript
3
star
94

elections16-general

You already know what it is.
JavaScript
2
star
95

musicgame

JavaScript
2
star
96

elections16graphics

Front-end for 2016 elections
JavaScript
2
star
97

wildfire-scroll

Visual narrative: United States of Wildfire
JavaScript
2
star
98

us-drought

JavaScript
2
star
99

liveblog-headlines

RSS widget for liveblogs or NPR.org API feeds
JavaScript
2
star
100

familymeal

An NPR project (built on Tumblr) centered around user-submitted photos of their dinners.
JavaScript
2
star