• Stars
    star
    267
  • Rank 153,621 (Top 4 %)
  • Language
    Ruby
  • License
    GNU General Publi...
  • Created over 11 years ago
  • Updated 5 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Upload your own aerial images, position (rubbersheet) them in a web interface over existing map data, and share via web or composite and export for print.

MapKnitter

Code of Conduct Build Status codecov Join the chat at https://publiclab.org/chat first-timers-only-friendly View performance data on Skylight

Use Public Lab's open source MapKnitter to upload your own aerial photographs (for example, those from balloon or kite mapping: https://publiclab.org/wiki/balloon-mapping) and combine them into:

  • Web "slippy maps" like Google Maps
  • GeoTiff
  • TMS
  • High resolution JPEG

demo

Table of Contents

  1. Architecture
  2. Installation
  3. Logging in when running locally
  4. Running tests
  5. Bugs and support
  6. Developers
  7. Staging infrastructure and testing
  8. License
  9. MapKnitter in depth

Architecture

MapKnitter is broken into three major components:

  1. Map user interface
  2. Application
  3. Exporting system

Component 1 has been broken out into a new Leaflet plugin, Leaflet.DistortableImage, which allows for client-side, CSS3-based distortion of images over a Leaflet base map.

Component 2 is a Ruby on Rails application which is the core of what you've looked at. It stores images, image corner locations, annotations, map details, and user accounts.

Component 3 is a set of calls to an external application, MapKnitter Exporter, which performs the distortions and geolocations and produces export products like GeoTiff, TMS, JPG, etc. The external exporter service is built in a small Sinatra app at mapknitter-exporter-sinatra using the mapknitter-exporter gem. Requests for exports are sent and progress is tracked using the API.

Installation

Please consider which installation method you prefer. Cloud Installation requires fewer steps and is platform agnostic, but you may value working from your terminal, for familiarity, more.


Cloud Installation

Cloud installation is possible with GitPod using this link and is a fully-automated process:

https://gitpod.io/#https://github.com/publiclab/mapknitter


Standard Installation


Prerequisites for Standard Installation

Make sure you have the 3 prerequisites below installed before moving forward with the Installation Steps.

Instructions are for an Ubuntu/Debian system. Varies slightly for mac/fedora/etc.

macOS 10.14 users may need this: https://silvae86.github.io/2018/07/05/fixing-missing-headers-for-homebrew-in-mac-osx-mojave/

  1. MySQL - Database
  2. RVM - Ruby version manager
  3. Yarn - Package manager

MySQL

macOS and Linux users, please reference MYSQL.md instead.

  1. Install MySQL:
$ sudo apt-get install mysql-server
  1. Install application-specific dependencies:
$ sudo apt-get install bundler libmysqlclient-dev imagemagick ruby-rmagick libfreeimage3 libfreeimage-dev ruby-dev libmagickcore-dev libmagickwand-dev
  1. (Optional): For exporting, you'll need GDAL >=1.7.x (gdal.org) as well as curl and zip-- but these are not needed for much of development unless you're working on the exporting features.
$ sudo apt-get install gdal-bin python-gdal curl libcurl4-openssl-dev libssl-dev zip

==================

RVM

This is for RVM, but the alternative, rbenv, also works (instructions not listed here). Don't install RVM if you already have rbenv!

  1. Install RVM: (http://rvm.io)
$ curl -L https://get.rvm.io | bash -s stable
  1. At this point during the process, you may want to log out and log back in or open a new terminal window. RVM will then properly load in your environment.

    • Ubuntu users only: you may need to enable Run command as a login shell in Ubuntu's Terminal under Profile Preferences > Title and Command. Then close the terminal and reopen it.
  2. Use RVM to install version 2.4.6 of Ruby:

    • macOS Big Sur users only: You may encounter a Error running '__rvm_make -j8' compilation error while installing Ruby with RVM. Run export warnflags=-Wno-error=implicit-function-declaration and then retry the installation command below. if you still encounter issues, this thread provides very helpful suggestions.
$ rvm install 2.4.6

==================

Yarn

We use Yarn as our package manager, and it is available through npm.

  1. Install npm:
$ sudo apt-get install npm
  1. Ubuntu users only: you may need to also install the nodejs-legacy package, as due to a naming collision, some versions of Ubuntu already have an unrelated package called node. To do this, run:
$ sudo apt-get install nodejs-legacy
  1. Once npm is installed, you should be able to use it to install Yarn:
$ npm install -g yarn

NOTE: Refer to this Stack Overflow question in case npm throws permission errors.

==================

Installation Steps

You'll need Ruby v2.4.6 (use your local ruby version management system - RVM or rbenv - to install and set up locally).

  1. Fork the repo from https://github.com/publiclab/mapknitter
  2. Clone the forked repo into any fresh, temporary folder with git clone https://github.com/your_username/mapknitter.git
  3. Change your working directory into the root folder you just cloned locally with cd <folder-name>
  4. Add upstream with git remote add upstream https://github.com/publiclab/mapknitter.git
  5. Install gems with bundle install. You may need to run bundle update if you have older gems in your environment.
  6. Copy and configure config/database.yml from config/database.yml.example using a new empty database you've created.
  7. Copy and configure config/config.yml from config/config.yml.example. For now, this is used for adding the Google Maps API Key configurations and a path for logging in when running locally. Both are optional.
  8. Initialize database with bundle exec rails db:setup
  9. Enter ReCaptcha public and private keys in config/initializers/recaptcha.rb (copied from config/initializers/recaptcha.rb.example). To get keys, visit https://www.google.com/recaptcha/admin/create
  10. Install static assets (like external javascript libraries and fonts) with yarn install
  11. Start rails with bundle exec rails s and open http://localhost:3000 in a web browser. For some, just rails s will work; adding bundle exec ensures you're using the version of passenger you just installed with Bundler.

==================

Installation video

For a run-through of the Prerequisites and Installation Steps listed above, you can watch the installation video at:

http://youtu.be/iGYGpS8rZMY (it may be slightly out-of-date but gives an helpful overview of the entire installation process.)


Windows Installation

We recommend either working in a virtual environment or on a dual-booted system to avoid dependency issues. In addition, the Unix operating system works smoother with Ruby on Rails. Using a Linux or macOS will not only benefit you now with the Mapknitter project, but will also make your life easier while working on other Ruby projects in the future.

  1. Dual Booting
  2. Setting Up a Linux Virtual Environment

Logging in when running locally

Because MapKnitter uses a remote, OpenID login system that depends on PublicLab.org, it can be hard to log in when running it locally. To get around this, we've created a local login route that requires no password:

You can log in locally at the path http://localhost:3000/local/USERNAME where USERNAME is the login name of a user saved on your database.

For this to work:

  • You will need to have copied and configured config/config.yml from config/config.yml.example.

  • The user has to be an existing record. For your convenience, we have added two user accounts in seeds.rb to make their corresponding paths available in development after installation:

# basic account path - http://localhost:3000/local/harry
# created from:
User.create({login: 'harry', name: 'harry potter', email: '[email protected]'})

# admin account path - http://localhost:3000/local/albus
# created from:
u_admin = User.create({login: 'albus', name: 'albus dumbledore', email: '[email protected]', role: 'admin'})

Running tests

When you try to run tests in MapKnitter, you can run the default Rake tasks, such as:

rails test:unit rails test:controllers rails test:integration

or simply:

rails test

Running tests of a specific file:

rails test test/unit/some_file.rb

Running a single test from the test suite:

rails test test/functional/some_file.rb:[line number of the test]

Bugs and support

To report bugs and request features, please use the GitHub issue tracker provided at https://github.com/publiclab/mapknitter/issues

For additional support, join the Public Lab mailing list at http://publiclab.org/lists or for urgent requests, email [email protected]

For questions related to the use of this software and balloon or kite mapping, the mailing-list page above links to the "grassrootsmapping" discussion group.

Code of Conduct

Please read and abide by our Code of Conduct. Our community aspires to be a respectful place both during online and in-person interactions.

Developers

Help improve Public Lab software!

Staging infrastructure and testing

In addition to automatic testing with Travis CI, we have a branch (unstable) that is set to auto-build and deploy to a staging instance. This instance includes a copy of the production database and is intended for experimenting or debugging purposes in a production-like environment. We also have a stable build at https://stable.mapknitter.org/ which builds off of our main branch. Any commits or PRs merged to the main branch will trigger the stable server to rebuild, and you can monitor the progress at https://jenkins.laboratoriopublico.org/


License

MapKnitter is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

MapKnitter is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with MapKnitter. If not, see http://www.gnu.org/licenses/.


MapKnitter in depth

MapKnitter is a free and open source software created and run by Public Lab. MapKnitter is hosted through a donation of server space by Rackspace.

MapKnitter can make maps from any image source, but it particularly lends itself to making maps with balloons and kites. The manual process of making maps with MapKnitter differs greatly from automated aerial imaging systems. In those systems, the imaging is of higher precision and processed with spatial and telemetry data collected along with the imagery, typically at higher altitudes and with consistent image overlap in the flight path sequence.

With MapKnitter, the cartographer dynamically places each image and selects which images to include in the mosaic. However, the approaches are similar in that they use some type of additional information (usually pre-existing imagery of a lower resolution) as a reference, and that they are bound to specific cartographic elements such as map scale and map projection.

More Repositories

1

plots2

a collaborative knowledge-exchange platform in Rails; we welcome first-time contributors! 🎈
Ruby
958
star
2

Leaflet.DistortableImage

A Leaflet extension to distort or "rubber sheet" images
JavaScript
270
star
3

community-toolbox

Tools to understand and welcome people into a contributor community
JavaScript
141
star
4

webjack

a JavaScript library that uses an audio software modem to communicate with an Arduino via a headphone jack
JavaScript
136
star
5

spectral-workbench

Web-based tools for collecting, analyzing, and sharing data from a DIY spectrometer
Ruby
126
star
6

leaflet-environmental-layers

Collection of different environmental map layers in an easy to use Leaflet library, similar to https://github.com/leaflet-extras/leaflet-providers#leaflet-providers
JavaScript
99
star
7

PublicLab.Editor

A general purpose, JS/Bootstrap UI framework for rich text posting. An author-friendly, minimal, mobile/desktop interface for creating blog-like content, designed for PublicLab.org
JavaScript
79
star
8

spectral-workbench.js

The JavaScript heart of Spectral Workbench; a Public Lab project to record, manipulate, and analyze spectrometric data.
JavaScript
46
star
9

infragram

A minimal core of the Infragram.org project in JavaScript
HTML
44
star
10

simple-data-grapher

Turns CSVs into graphs in a few simple steps; embeds onto other websites!
JavaScript
38
star
11

leaflet-blurred-location

A Leaflet-based interface for selecting a "blurred" or low-resolution location, to preserve privacy
JavaScript
35
star
12

inline-markdown-editor

An inline wysiwyg markdown document editor based on replacing string subsections. WYSIWYG possible via woofmark.
JavaScript
30
star
13

pi-builder

Developing a reproducible build script system for making Raspberry Pi .img files
Ruby
22
star
14

leaflet-multispectral

Multispectral color manipulation and processing (NDVI etc) for Leaflet image overlays with pure JavaScript in Image Sequencer
HTML
22
star
15

plotsbot

A bot for Public Lab
JavaScript
17
star
16

papercraft-spectrometer

An introductory papercraft spectrometry kit
13
star
17

sky-camera

Public Lab's Android Aerial Acquisition App
Java
12
star
18

spectrometer3

Public Lab's open source Desktop Spectrometry Kit
11
star
19

leaflet-blurred-location-display

A library to display points which have been "blurred" for privacy with leaflet-blurred-location
JavaScript
11
star
20

SmART-Form

A community science effort to measure formaldehyde in the home environment
Java
9
star
21

webjack-firmata

A wrapper for WebJack to use as transport layer for firmata.js, for firmata access to an Arduino from a web browser.
JavaScript
9
star
22

mapmill

A Hot-or-Not styled crowdsourcing engine for sorting raw map imagery
HTML
8
star
23

colorimetry

A web-based colorimetry toolkit, using Image Sequencer
JavaScript
8
star
24

infragram-ps

Contents of the SD card to be used on the Infragram Point and Shoot - an open hardware multispectral camera developed by Public Lab: http://infragram.org
7
star
25

image-sequencer-app

An Image Processing server based on image-sequencer
JavaScript
7
star
26

community-microscope

Community Microscope design files
G-code
6
star
27

leaflet-tile-filter

Multispectral color manipulation and processing (NDVI etc) for Leaflet tile layers with pure JavaScript in Image Sequencer
HTML
6
star
28

mapknitter-exporter

The GDAL/ImageMagick-based exporter system from MapKnitter
Ruby
5
star
29

blurred-location

A JavaScript library to help manage variable location privacy through a "blurred location" model
JavaScript
4
star
30

mapknitter-exporter-sinatra

A minimal Sinatra app to run MapKnitter exports in the cloud
Ruby
3
star
31

plots-leaflet-viewer

A leaflet-based TMS viewer which accepts a TMS URL as a GET parameter. Good for fullscreen map viewing and embedding.
HTML
2
star
32

wherewebreathe

JavaScript
2
star
33

spectralchallenge

a website for Public Lab's Spectral Challenge, at http://spectralchallenge.org
JavaScript
1
star
34

lego-spectrometer

Design files for the Public Lab Lego Spectrometer
1
star
35

thermal-flashlight

Files for the Thermal Flashlight project
C++
1
star
36

multibot

A general-purpose Node.js bot library for writing interactive bot behaviors on various platforms
JavaScript
1
star
37

unearthing-pvd

A platform for historical data-driven analyses of industrial land use, story-mapping, and community organizing. (dataset)
1
star
38

publiclab-kits

Tracker for manufacturing issues for Public Lab kits
1
star
39

spectral-workbench-mobile

PhoneGap-based mobile app version of SpectralWorkbench.org, focus on iOS
JavaScript
1
star
40

SWBSearchModule

The spectral search module for the Spectral WorkBench
Python
1
star
41

leaflet-archive

A Leaflet map that displays geocoded images from Archive.org
JavaScript
1
star