okfn/reconcile-csv

Stars
113
Rank 310,115 (Top 7 %)
Language
Clojure
License
BSD 2-Clause "Sim...
Created almost 11 years ago
Updated over 9 years ago

okfn/reconcile-csv

okfn

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

A simple OpenRefine reconciliation service that runs on top of a CSV file

reconcile-csv

A OpenRefine reconciliation service that works from CSV files.

Introduction

Someone just handed you some datasets they claim are related and you should be able to mesh them up easily - no problem you're the data wizard. Except there is no clear unique identifier you could use to join the dataset. All you know is that if several columns are the same, it should be the same. Enter typos and misspellings and here you go: A long night ahead.

Reconcile-CSV aims to ease your pain: It acts as a OpenRefine reconciliation service and performs fuzzy matching to identify related entries.

Usage

Create a column with Unique-ID's you will use to match.

Pre-compiled:

java -Xmx2g -jar reconcile-csv-0.1.2.jar <file> <primary search column> <column with id's>

With Leiningen:

lein run <file> <primary search column> <column with id's>

Then add http://localhost:8000/reconcile as a reconciliation service to refine. You can add more columns through the reconcile-interface in Refine.

Reconcile away!

Then use:

cell.recon.match.id

to get the ID from the match.

License

Copyright © 2013 Michael Bauer, Open Knowledge Foundation

Distributed under the BSD-2 Clause license. See LICENSE for details

messytables

Tools for parsing messy tabular data. This is now superseded by https://github.com/frictionlessdata/tabulator-py

facetview

FacetView is a pure javascript frontend for ElasticSearch.

timemapper

Create and share elegant timelines and timemaps fast

opendatahandbook

Open Data Handbook v2

handbook

Guides and introductions for participating in Labs and some of its projects.

opendataeditor

The Open Data Editor (ODE) is a no-code application to explore, validate and publish data in a simple way. Forever free and open source project powered by the Frictionless Framework.

bubbletree

Radial Bubble Tree Visualization

textus

A toolbox and web application for working with and presenting textual material from Shakespeare to Schopenhauer, and letters to literature.

dataexplorer

View, visualize, clean and process data in the browser.

opendatasurvey

The Open Data Survey application

bibserver

BibServer is open-source software what makes it easy to publish, manage and find bibliographies. BibServer is RESTful and web-friendly.

labs-opd

dataportals.org

Open Data Portals and Sites around the world

opendefinition

Open Definition source

website

The code behind okfn.org

dataconverters

Python library and command line tool for converting data from one format to another

timeliner

[DEPRECATED] See new TimeMapper repo https://github.com/okfn/timemapper

opendataday

Open Data Day website

docker-ckan

Docker images and Docker Compose setup for CKAN

okfn.github.com

Open Knowledge Labs website (and general issue tracker).

Jupyter Notebook

elasticsearch.js

A simple javascript library for working with ElasticSearch

wikipediajs

A small Javascript library for accessing information in Wikipedia articles such as dates, places, abstracts and more

dataproxy

Web application (targeted at appengine) to proxy data from certain data types into a JSON-P data type so that users can create mashups against remote data sets.

opendatahandbook-old

Open Data Handbook

infra

Repository for OKF infrastructure and systems administration tools

opd-product-browser-web

A repository of code, documentation and database scripts for the Open Product Data project.

licenses

Open source and open knowledge (data and content) licenses together with API and web service.

ckan.js

Javascript client library for CKAN with support for Node and the browser. It also provides a Recline compatible backend.

vdm

Versioned domain model. Python library for revisioning/versioning of databases.

webstore

webstore is a web-api enabled datastore backed onto sql databases especially sqlite. It supports the RESTful JSON APIs standard to nosql dbs like CouchDB, MongoDB, Riak but with the power, reliability and familiarity of SQL and RDBMS.

opendataindex

Open Data Index website

opd-product-browser-android

Android application for the Open Product Data project

coding-standards

Open Knowledge coding standards and style guide.

ofs

A simple python-based abstraction library for the various blob storage out there including s3, google storage and local disk.

listify

Take a Google Doc spreadsheet and turn it into a beautiful, searchable listing on your website - embed in seconds into Wordpress, simple HTML or ...

dashboard

Dashboard for communities: what's up, who's here and what are they working on. Designed for use at the Open Knowledge Foundation.

sqlaload

REPLACED BY dataset

ckanclient-deprecated

DEPRECATED - please see https://github.com/ckan/ckanapi. [Python client library for CKAN]

docker-rt

A full installation of RT in a docker image. To be used as a base for customisation.

bad-data

Examples of bad data, especially from government.

ddjbook

Data Driven Journalism Handbook

data-catalog-spec

Data Catalog Specification (Schema and Protocol)

ckanjs-deprecated

DEPRECATED - see https://github.com/okfn/ckan.js [Javascript client library and app for CKAN data hub software]

gobetween

A simple app for proxying requests with CORS support. Map any domain to any URI as a base path, or, use a dedicated endpoint for proxying any URI.

helmut

A generic Google Refine Reconciliation API implementation

oed

Extraction and Interface for Oxford English Dictionary (OED) 1st Edition

datautil

Utilities for working with data.

shakespeare-material

An open (free to use, reuse and redistribute) set of Shakespeare material including plays, poems, images and scripts.

bibjson

BibJSON spec and website

measure

Measure is scripts and conventions to build KPI dashboards for projects.

textus-viewer

A sophisticated HTML-based viewer for texts

opendatacommons.org

Open Data Commons licenses and website

handbook-theme

Open Knowledge standard "nice" jekyll theme - used for OpenDataHandbook etc

datadigitizer

Crowd-sourcing digitization of data

pdcalc

Public Domain Calculators - determine what is public domain and what's not.

recline.backend.gdocs

Javascript library for accessing Google Docs Spreadsheet - and also functions as a Recline backend!

rtei

Right to Education Index website

ckanext-s3filestore

Use Amazon S3 as a filestore for CKAN

yourtopia

Web app for crowdsourcing preferences about index weighting such as the Human Development Index. Powers the YourTopia site - Global Development beyond GDP.

ckanext-wordpresser

Wordpress integration for CKAN

sphinx-theme-okfn

Standard Open Knowledge Foundation sphinx theme resembling that on http://okfn.org/

iatitools

Tools with working with IATI data from http://iatiregistry.org/

a

A is for assets. This repository holds OKI web assets, pushed using CI to the S3 bucket behind:

ckanext-datahub

Plugin for datahub

webstore-client

Python client library for webstore

opd-brand-manager

A web interface for managing a directory of brands for the Open Product Data working group.

ckanext-example

CKAN Extension that demonstrates some common patterns for customising a CKAN instance.

gutenizer

Get texts from Project Gutenberg, extract and format

moinmoin2mediawiki

Convert moinmoin data to mediawiki

ckan-drupal

Everything to do with ckan drupal integration.

docker-fiware-ckan

Docker and Docker compose configuration for the FIWARE CKAN instance

ckanext-pdeu

Custom CKAN Extension for PublicData.eu

beagle

Watch web resources for changes and email assigned users when changes happen to sites they're assigned to watch.

recline.backend.couchdb

CouchDB backend for Recline

opendatasites

Dataset of open data websites (portals and catalogs) from around the world based on http://datacatalogs.org/

ibp-explorer

[ARCHIVED] Data Explorer for the Open Budget Survey, built in collaboration with the International Budget Partnership.

opd-product-browser-ios

iOS browser client for the Open Product Data project

dataexplorer-2011

This repo is deprecated and has been replaced by http://github.com/okfn/recline. Pure JS app for viewing and visualizing data

crime

Experimental crime dashboards

ckanext-lacounts

CKAN extension for the LA Counts project

okfn.org

Issue tracker for http://okfn.org/ and other Open Knowledge Foundation websites

wordpress-theme

OKFN corporate wordpress theme

oki-py

This is an example Python repository implementing OKI coding standards.

tellme

TellMe is a toolkit to create *user-facing reports* from things happening in code.

shakespeare

Open Shakespeare platform. Repository for managing and code and service for http://openshakespeare.org/ and related websites.

ckan-google-docs

Google docs app script.

network

Main repository for the Open Knowledge Network.

opendatacommons

Legacy Open Data Commons website. Moved to https://github.com/okfn/opendatacommons.org

ckanext-vegaview

VegaJS' view extension for CKAN

ckanext-rdf

[DEPRECATED] - Please use https://github.com/ckan/ckanext-dcat

recline.backend.solr

A simple javascript library that wraps the SOLR JSON API and is usable by Recline

elastic-proxy

elastic search on the web

ckanext-inspire

data.okfn.org-new

Simple data package based data portal (and original site for frictionless data effort)

datapackage_pipelines_od4tj

Open Data for Tax Justice scraping pipelines and processors

ckanext-mx

Theme for datos.gob.mx

climate-request

A new project at Open Knowledge Foundation allowing users to request climate data.

cop21

Negotiating texts for COP21, the 21st Conference of Parties (COP21) to the UNFCCC in Paris December 2015.

githubActivity.js

A simple javascript widget to provide a nice HTML list Github activity in a set of repositories

vendorjs

This is a common repo for storing javascript (and some css) vendor libraries for use as a submodule across a variety of Open Knowledge Foundation (or other) projects.