pbf2json creates a JSON stream of openstreetmap data from any PBF extract, you can pick-and-choose only the bits of the file you want, the library will take care of de-normalizing the relational data (nodes/ways/relations) so you can put it straight in to your favourite document-store, inverted index or graph database.
Run from pre-built binary
You don't need to have Go
installed on your system to use one of the precompiled binaries in ./build
:
# AMD 64-bit linux/macOS/windows
$ ./build/pbf2json.linux-x64
$ ./build/pbf2json.darwin-x64
$ ./build/pbf2json.win32-x64
# ARM 64-bit linux/macOS
$ ./build/pbf2json.linux-arm64
$ ./build/pbf2json.darwin-arm64
Usage
To control which tags are output you must pass the -tags=
flag to pbf2json
and the PBF filepath:
$ ./build/pbf2json.linux-x64 -tags="amenity" /tmp/wellington_new-zealand.osm.pbf
{"id":170603342,"type":"node","lat":-41.289843000000005,"lon":174.7944402,"tags":{"amenity":"fountain","created_by":"Potlatch 0.5d","name":"Oriental Bay Fountain","source":"knowledge"},"timestamp":"0001-01-01T00:00:00Z"}
{"id":170605346,"type":"node","lat":-41.2861039,"lon":174.7711539,"tags":{"amenity":"fountain","created_by":"Potlatch 0.10c","source":"knowledge"},"timestamp":"0001-01-01T00:00:00Z"}
Advanced Usage
Multiple tags can be specified with commas, records will be returned if they match one OR
the other:
# all buildings and shops
-tags="building,shop"
Tags can also be grouped with the +
symbol, records will only be returned if they match one AND
the other:
# only records with BOTH housenumber and street specified
-tags="addr:housenumber+addr:street"
You can also combine the above 2 delimiters to get even more control over what get's returned:
# only highways and waterways which have a name
-tags="highway+name,waterway+name"
If you need to target only specific values for a tag you can specify exactly which values you wish to extract using the ~
symbol:
# only extract cuisine tags which have the value of vegetarian or vegan
-tags="cuisine~vegetarian,cuisine~vegan"
Denormalization
When processing the ways, the node refs are looked up for you and the lat/lon values are added to each way.
Since version 3.0
centroids are also computed for each way, since version 5.0
bounds are now also computed.
Output of the nodes
array (as seen below) is optional, this was disabled by default in version 5.0
but can be enabled with the flag --waynodes=true
.
{
"id": 301435061,
"type": "way",
"tags": {
"addr:housenumber": "33",
"addr:postcode": "N5 1TH",
"addr:street": "Highbury Park",
"building": "residential"
},
"centroid": {
"lat": "51.554679",
"lon": "-0.098485"
},
"bounds": {
"e": "-0.0983673",
"n": "51.5547179",
"s": "51.5546574",
"w": "-0.0985915"
},
"nodes": [
{
"lat": "51.554663",
"lon": "-0.098369"
},
{
"lat": "51.554657",
"lon": "-0.098529"
},
{
"lat": "51.554656",
"lon": "-0.098592"
},
{
"lat": "51.554676",
"lon": "-0.098590"
},
{
"lat": "51.554680",
"lon": "-0.098529"
},
{
"lat": "51.554715",
"lon": "-0.098529"
},
{
"lat": "51.554720",
"lon": "-0.098369"
},
{
"lat": "51.554663",
"lon": "-0.098369"
}
]
}
Relations
Since version 6.0
centroids and bounding boxes are also computed for relations, the calulations are based off the largest member way by area.
Note: if a relation
does not contain at least one way
then it will not be output.
Leveldb
This library uses leveldb
to store the lat/lon info about nodes so that it can denormalize the ways for you.
By default the leveldb path is set to /tmp
, you can change where it stores the data with a flag:
$ ./build/pbf2json.linux-x64 -leveldb="/tmp/somewhere"
Batched writes
Since version 3.0
writing of node info to leveldb is done in batches to improve performance.
By default the batch size is 50000
, you can change this with the following flag:
$ ./build/pbf2json.linux-x64 -batch="1000"
NPM module
var pbf2json = require('pbf2json'),
through = require('through2');
var config = {
file: '/tmp/wellington_new-zealand.osm.pbf',
tags: [
'addr:housenumber+addr:street'
],
leveldb: '/tmp'
};
pbf2json.createReadStream( config )
.pipe( through.obj( function( item, e, next ){
console.log( item );
next();
}));
Run the go code from source
Make sure Go
is installed and configured on your system, see: https://gist.github.com/missinglink/4212a81a7d9c125b68d9
Note: You should install the latest version of Golang, at least 1.5+
, last tested on 1.6.2
sudo apt-get install mercurial;
go get;
go run pbf2json.go;
Compile source for all supported architecture
If you are doing a release and would like to compile for all supported architectures:
note if this is your first time doing this please read the notes in './compile.sh' to set it all up on your machine.
bash compile.sh;
Compile source for a new architecture
If you would like to compile a version of this lib for an architecture which isn't currently supported you can:
go get;
go build;
chmod +x pbf2json;
mv pbf2json build/pbf2json.{platform}-{arch};
Note you will need to change the variables {platform} and {arch} to match those returned by nodejs
for your system:
$ node
> var os=require('os')
> os.platform()
'linux'
> os.arch()
'x64'
Then submit a pull request, you are awesome ;)