• Stars
    star
    425
  • Rank 102,094 (Top 3 %)
  • Language
    JavaScript
  • License
    Other
  • Created almost 12 years ago
  • Updated about 6 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Designed and built for Humanitarian OpenStreetMap Team collaborative emergency/disaster mapping, the OSM Tasking Manager 2.0 divides an area into individual squares that can be rapidly mapped by thousands of volunteers.

OpenStreetMap Tasking Manager v2

Build Status Coverage Status

About

OpenStreetMap Tasking Manager enables collaborative work on specific areas in OpenStreetMap by defining clear workflows to be achieved and by breaking tasks down into pieces.

This is version 2.0 of the Tasking Manager. Most development work is now taking place on version 3.0

V2 Tasking Manager still powers many Tasking Manager installations. It is written in Python using the Pyramid framework.

Installation

First clone the git repository:

git clone --recursive git://github.com/hotosm/osm-tasking-manager2.git

Installing OSMTM in a Virtual Python environment is recommended.

To create a virtual Python environment:

cd osm-tasking-manager2
sudo easy_install virtualenv
virtualenv --no-site-packages env
./env/bin/pip install -r requirements.txt

Tip: if you encounter problems installing psycopg2 especially on Mac, it is recommended to follow advice proposed here.

Database

OSMTM requires a PostgreSQL/PostGIS database. Version 2.3 or higher of PostGIS is required.

First create a database user/role named www-data:

sudo -u postgres createuser -SDRP www-data

Then create a database named osmtm:

sudo -u postgres createdb -T template0 osmtm -E UTF8 -O www-data
sudo -u postgres psql -d osmtm -c "CREATE EXTENSION postgis;"

Local settings

You certainly will need some local specific settings, like the db user or password. For this, you can create a local.ini file in the project root, where you can then override every needed setting. For example:

[app:main]
sqlalchemy.url = postgresql://www-data:PASSWORD@localhost/osmtm
default_comment_prefix = #yourinstancename-project
check_expiration_interval = 60

Note: you can also put your local settings file anywhere else on your file system, and then create a LOCAL_SETTINGS_PATH environment variable to make the project aware of this.

Currently, these are the settings you can over-ride:

  • sqlalchemy.url: Postgres URL to use for database connection
  • default_comment_prefix: Default prefix to use for changeset comments, defaults to #hotosm-project
  • check_expiration_interval: The interval at which the database should be checked for expired tasks, in seconds. Defaults to 5 seconds.

Populate the database

You're now ready to do the initial population of the database. An initialize_osmtm_db script is available in the virtual env for that:

./env/bin/initialize_osmtm_db

Launch the application

./env/bin/pserve --reload development.ini

You will see messages, hopefully including a line like serving on http://0.0.0.0:6543. Visit that address in your web browser - you should see your local Tasking Manager!

Running the application behind proxy server

You need to make the following changes to the osmtm/views/osmauth.py file.

# Add the below lines in the starting
import httplib2
httplib2.debuglevel = 4
PROXY = httplib2.ProxyInfo(httplib2.socks.PROXY_TYPE_HTTP_NO_TUNNEL, 'PROXY-SERVER', PROXY-PORT)

NOTE: Replace the PROXY-SERVER with your proxy server address and PROXY-PORT with the port number on which your proxy is established.

# then add "proxy_info=PROXY" for every line in oauth.Client.
client = oauth.Client(consumer, proxy_info=PROXY)

client = oauth.Client(consumer, token, proxy_info=PROXY)

Replace the host address in the development.ini file with your IP address of the system.

host='SYSTEM-IP-ADDRESS'

Styles

The CSS stylesheet are compiled using less. Launch the following command as soon as you change the css:

lessc -ru osmtm/static/css/main.less > osmtm/static/css/main.css

Launch the application

env/bin/pserve --reload development.ini

Tests

The tests use a separate database. Create that database first:

sudo -u postgres createdb -O www-data osmtm_tests
sudo -u postgres psql -d osmtm_tests -c "CREATE EXTENSION postgis;"

Create a local.test.inifile in the project root, where you will add the settings for the database connection. For example:

[app:main]
sqlalchemy.url = postgresql://www-data:www-data@localhost/osmtm_tests

To run the tests, use the following command:

./env/bin/nosetests

Application deployment

  1. pull latest updates from the repository: git pull origin
  2. update the submodules: git submodule update --init
  3. update/install python modules: ./env/bin/pip install -r requirements.txt
  4. create database dump: pg_dump -Fc -f osmtm2_latest.dmp database_name
  5. run database migrations: ./env/bin/alembic upgrade head
  6. compile messages: ./env/bin/python setup.py compile_catalog
  7. restart application server

Installation as a mod_wsgi Application

an example Apache configuration file

WSGIDaemonProcess OSMTM_process user=ubuntu group=ubuntu processes=1 \
        threads=4 \
        python-path=/home/ubuntu/osm-tasking-manager2:/home/ubuntu/osm-tasking-manager2/env/lib/python2.7/site-packages
WSGIRestrictStdin Off

<VirtualHost *:80>
        ServerName example.org
        # Use only 1 Python sub-interpreter.  Multiple sub-interpreters
        # play badly with C extensions.
        WSGIPassAuthorization On
        WSGIScriptAlias /osmtm /home/ubuntu/osm-tasking-manager2/env/OSMTM.wsgi

        <Location />
            WSGIProcessGroup OSMTM_process
            WSGIApplicationGroup %{GLOBAL}
        </Location>

        <Directory /home/ubuntu/osm-tasking-manager2/env>
            <Files OSMTM.wsgi>
                Require all granted
            </Files>
            Order allow,deny
            Allow from all
        </Directory>

        LogLevel warn

        CustomLog /var/log/apache2/osmtm-access.log combined
        ErrorLog /var/log/apache2/osmtm-error.log
</VirtualHost>

Customization

In case you install your own instance you may want to customize its look and feel. You can do so by modifying the following files: osmtm/templates/custom.mako, osmtm/static/css/custom.less & osmtm/static/img/favicon.ico

Localization

OSMTM is localized on Transifex service.

It's possible to create translations for two resources: current and master. Current resource represents currently deployed instance of the OSMTM http://tasks.hotosm.org. Master resource represents actively developed code that will become current once it gets deployed.

Initializing translation files

In general managing translation files involves:

  • generate pot file: ./env/bin/python setup.py extract_messages
  • initialize a message catalogue file (english): ./env/bin/python setup.py init_catalog -l en
    • if the catalogue is already created use: ./env/bin/python setup.py update_catalog
  • eventually compile messages: ./env/bin/python setup.py compile_catalog
  • append new language to the available_languages configuration variable in production.ini file, for example available_languages = en fr

Using Transifex service

  • in the project top level directory, initialize transifex service (after installing transifex-client): tx init
    • the init process will ask for service URL and username/password, which will be saved to ~/.transifexrc file
  • if the project has already been initialized, but you are missing ~/.transifexrc, create the file and modify it's access privileges chmod 600 ~/.transifexrc

Example .transifexrc file:

[https://www.transifex.com]
hostname = https://www.transifex.com
password = my_super_password
token =
username = my_transifex_username
  • after creating the project on the Transifex service: osm-tasking-manager2, generate the pot file, and add it as a master resource on the project, full resource name, in this case, is osm-tasking-manager2.master

Setting up resources

  • add initial source file, in this case English:
    • tx set --source -r osm-tasking-manager2.master -l en osmtm/locale/en/LC_MESSAGES/osmtm.po
  • add existing source files, in this case French:
    • tx set -r osm-tasking-manager2.master -l fr osmtm/locale/fr/LC_MESSAGES/osmtm.po
  • push resources on the transifex service (this will overwrite any existing resources on the service)
    • tx push -s -t
    • -s - pushes source files (English)
    • -t - pushes translation files (French)

Pulling changes

  • to pull latest changes from Transifex service execute: tx pull
    • this will pull every available language from the Transifex service, even the languages that are not yet mapped
    • if the language is not mapped, translated language file will be saved to the local .tx directory, which is not what we want so we need to define the mapping
    • for example, if we want to correctly map Croaian language you need to execute: tx set -r osm-tasking-manager2.master -l hr osmtm/locale/hr/LC_MESSAGES/osmtm.po
      • there is no need to create the actual po file or the directory structure, Transifex client will manage that for us
  • there is also a possibility to pull specific languages: tx pull -l hr
  • or pull only languages that have a certain completeness percentage: tx pull --minimum-perc=90

Transifex workflow

Updating source files, locally and on the service

  • update pot and source po file

    • ./env/bin/python setup.py extract_messages
    • ./env/bin/python setup.py init_catalog -l en
  • push the source file to Transifex service

    • tx push -s

Pull latest changes from the service

  • when adding a new language:
    • we need to configure local mapping: tx set -r osm-tasking-manager2.master -l hr osmtm/locale/hr/LC_MESSAGES/osmtm.po
    • append the new language to the available_languages configuration variable in production.ini file: available_languages = en fr hr
  • after there are some translation updates, pull latest changes for mapped resources
    • tx pull -l fr,hr
  • compile language files
    • ./env/bin/python setup.py compile_catalog

API

The tasking manager exposes some of its functionality via a RESTful API. It is documented on the following page: https://github.com/hotosm/osm-tasking-manager2/wiki/API.

More Repositories

1

tasking-manager

Tasking Manager - The tool to team up for mapping in OpenStreetMap
JavaScript
507
star
2

learnosm

LearnOSM.org content, Jekyll layouts & issue tracking. This repository is dedicated to helping people learn how to map in OpenStreetMap (OSM) and use many of the software and tools in the OSM community.
CSS
247
star
3

OpenAerialMap

OpenAerialMap is an open service to provide access to a commons of openly licensed imagery and map layer services.
242
star
4

osm-export-tool

Web service to download customised OSM data in various file formats
JavaScript
145
star
5

HDM-CartoCSS

CartoCSS project focused on the Humanitarian Data Model
CartoCSS
136
star
6

osm-analytics

OSM Analytics lets you interactively analyze how specific OpenStreetMap features are mapped in a specific region.
JavaScript
103
star
7

osm-tasking-manager

Deprecated! - See osm-tasking-manager2
CSS
84
star
8

fAIr

AI Assisted Mapping
Jupyter Notebook
79
star
9

tech

Resources and issue tracking for Technical Working Group and all things Tech within HOT. Start here to get more information about how to get involved in HOT.
68
star
10

hotosm-website

The website of the Humanitarian OpenStreetMap Team. See instructions below for writing blog posts or updating your profile.
HTML
66
star
11

fmtm

Field Mapping Tasking Manager - coordinated field mapping.
TypeScript
46
star
12

osm-export-tool-python

command line tool + Python library for exporting OSM in various file formats.
Python
44
star
13

oam-browser

OAM Imagery browser
JavaScript
42
star
14

ml-enabler

A service that integrates ML models to mapping applications.
Python
40
star
15

MapCampaigner

Campaign management tool
JavaScript
38
star
16

oam-dynamic-tiler

Dynamic tiling of raster data for OpenAerialMap + others
Python
35
star
17

underpass

A customizable data engine for processing mapping data
C++
31
star
18

oam-api

A catalog for OpenAerialMap imagery
JavaScript
24
star
19

visualize-change

A toolkit to visualize changes in OSM, part of the OSM Analytics ecosystem. Work in Progress.
JavaScript
23
star
20

iso-countries-languages

Country names translated in 89 languages
Clean
23
star
21

raw-data-api

Raw Data API is a set of high-performant APIs for transforming and exporting OpenStreetMap (OSM) data in different GIS file formats
Python
23
star
22

Geo-Data-Collect

This is mobile data collection based on ODK-Collect and OSM-Tracker. This app have both form collection capabilities and gps tracker capabilities
Java
19
star
23

toolbox

Documentation and training materials for mapping and data collection
JavaScript
17
star
24

presets

JOSM presets
16
star
25

old-export-tool

Version 1 of the Export Tool is suspended - please see V3 https://github.com/hotosm/osm-export-tool
Ruby
16
star
26

osm-fieldwork

Processing field data from ODK to OpenStreetMap format, and other field data collection utils.
Python
16
star
27

osm-analytics-cruncher

Backend code for osm-analytics
JavaScript
15
star
28

galaxy-api

Backend to fetch data from Underpass
Python
14
star
29

osma-health

HOT Analytics for Health
CSS
12
star
30

oam-server-tiler

DEPRECATED: OAM Server Tiler. See marblecutter-openaerialmap instead
Scala
12
star
31

HDM-JOSM-style

Humanitarian Data Model targeted JOSM style
12
star
32

drone-tm

Drone Tasking Manager - community-driven drone imagery collection.
TypeScript
12
star
33

hugo-book

HTML
12
star
34

fAIr-utilities

Utilities for AI-Assisted Mapping fAIr
Jupyter Notebook
11
star
35

oam-uploader

DEPRECIATED - The web frontend to the OAM Uploader API
JavaScript
10
star
36

oam-design-system

OAM Design System
CSS
10
star
37

tilemill-projects-windows

Sample TileMill projects that work on Windows. Get up and running with SQLite and PostGIS.
10
star
38

oam-qgis-plugin

QGIS plugin to access and upload to OAM
Python
10
star
39

installer

Windows installer for custom elements of software stack used in field by HOTOSM
Shell
9
star
40

tracing-guides

CSS
8
star
41

imagery-coordination

Coordination of Imagery Requests for HOT Activations and other humanitarian mapping
JavaScript
8
star
42

hotosm-project-ideas

A place to forge new ideas for HOT projects. Started as a repository for the GSoC and Outreachy programs.
8
star
43

ui

Shared UI components with HOT theming
TypeScript
7
star
44

datm-research

Research for the Drone Aerial Tasking Manager
Python
7
star
45

data_protection_project

Policies, guidance, tools, and resources for Data Protection in humanitarian mapping
6
star
46

openaerialmap.org

SCSS
6
star
47

qgis-templates-and-symbology-plugin

QGIS Map Templates and Symbology Plugin
Scheme
6
star
48

hot-design-system

CSS
6
star
49

drone-flightplan

Drone Flight Plan generator
Python
5
star
50

galaxy-ui

This is the frontend interface for the OSM Galaxy project built using React.
JavaScript
5
star
51

fmtm-splitter

A utility for splitting an AOI into multiple tasks.
Python
5
star
52

docs

Documentation index across HOT's tools.
5
star
53

scripts

HOTOSM Scripts
Python
5
star
54

overture-to-tiles

Provides handful scripts for converting overture data to vector tiles
Jupyter Notebook
5
star
55

osm-rawdata

A python module for accessing OSM data in a postgres database.
Python
4
star
56

ml-enabler-cli

Command line interface for the ml-enabler
Python
4
star
57

styles

Map styles used for HOTOSM
4
star
58

oam-server

DEPRECATED: OpenAerialMap processing and tile server
JavaScript
4
star
59

RamaniHuria

Assorted assets created during the "Dar Ramani Huria" project.
CartoCSS
4
star
60

visualtags

HOT Visual Tag Chooser
CSS
3
star
61

Portal-ID

Portal for OpenStreetMap.or.id
PHP
3
star
62

overture-data-viewer

View overture data releases in Map
3
star
63

techdoc

A repository of documentation processes
3
star
64

osm-merge

Merge data into existing OSM data.
Python
3
star
65

uav-mapping-guidelines

Technical Guidelines for Mapping with Small Unmanned Aerial Vehicles (UAV)
Python
3
star
66

oeg-auto-reports

Test version of automated OEG reports
3
star
67

oam-uploader-api

DEPRECIATED - The OAM Uploader API server
JavaScript
3
star
68

design-strategy

Tracking HOT's brand, design, and UI/UX strategy
2
star
69

fairpredictor

Standalone Module for Model Predictions
Jupyter Notebook
2
star
70

hot-qa-tiles

QA Tiles Generation
JavaScript
2
star
71

TM-Extractor

Raw Data Extractor for Tasking Manager Projects
Python
2
star
72

oam-docs

OAM Documentation
JavaScript
2
star
73

hot-uganda

Assorted assets created during the "Uganda Refugee Mapping" project.
2
star
74

osm-login-python

Package to manage OAuth 2.0 login for OSM in Python.
Python
2
star
75

ux-review

Collecting Contributions to a HOT product UX review
2
star
76

oam-uploader-admin

CSS
2
star
77

tech-feedback-review

Collecting contributions for techncial feedback review
2
star
78

underpass-ui

An User Interface for Underpass
JavaScript
2
star
79

hot-indo

Place to log tickets for the HOT Indonesia Team
2
star
80

hotosm-website-commonknowledge

Python
2
star
81

tasking-manager-terraform-modules

Terraform modules to get Tasking Manager up and running
HCL
2
star
82

piwik-archive

An archiving tool for piwik.hotosm.org
Python
2
star
83

gh-workflows

Github workflows that can be shared across all HOT projects.
Dockerfile
2
star
84

website

Python
2
star
85

dar-drainage-style

Mapbox and QGIS styles for making drainage maps
CartoCSS
2
star
86

community

Issue tracker for the HOT Community Working Group
1
star
87

hot-summit-2019

CSS
1
star
88

Facebook-ID

Ruby
1
star
89

field-campaigner-data

1
star
90

oeg-reporter

Organized editing guidelines reporter
Python
1
star
91

hot-summit-2017

CSS
1
star
92

oam-status

A simple status dashboard for oam-catalog
CSS
1
star
93

oam-server-deployment

DEPRECATED: Amazon Web Services deployment tooling OAM Server.
Python
1
star
94

GDAL_scripts

Various utility scripts/snippets for managing geographical data
Shell
1
star
95

omdena-hot-2024

Jupyter Notebook
1
star
96

tm-admin

Administrative modules for Tasking Manager style projects
Python
1
star
97

projects

An experimental container for HOT project management
1
star
98

hotdocs

Hugo theme for HOT Documentation sites
SCSS
1
star
99

osm-analytics-entrypoint

An index of OSM Analytics tools from different organizations
CSS
1
star
100

fmtm-installer

Easy install script for Field Tasking Mapping Manager (FMTM)
1
star