• Stars
    star
    207
  • Rank 189,769 (Top 4 %)
  • Language
    Python
  • License
    MIT License
  • Created about 7 years ago
  • Updated 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Data model and processing tools for investigative entity data

Follow the Money

ftm-build

This repository contains a pragmatic data model for the entities most commonly used in investigative reporting: people, companies, assets, payments, court cases, etc.

The purpose of this is not to model reality in an ideal data model, but rather to have a working data structure for researchers.

followthemoney also contains code used to validate and normalize many of the elements of data, and to map tabular data into the model.

Documentation

For a general introduction to followthemoney, check the high-level introduction:

Part of this package is a command-line tool that can be used to process and transform data in various ways. You can find a tutorial here:

Besides the introductions, there is also a full reference documentation for the library and the contained ontology:

There's also a number of viewers for the RDF schema definitions generated from FollowTheMoney, e.g.:

Development environment

For local development with a virtualenv:

python3 -mvenv .env
source .env/bin/activate
pip install -e ".[dev]"

Now you can run the tests with

make test

Releasing

We release a lot of version of followthemoney because even small changes to the code base require a pypi release to begin being used in aleph. To this end, here's the steps for making a release:

git pull --rebase
make build
make test
git add . && git commit -m "Updating translation files"
bumpversion patch
git push --atomic origin main $(git describe --tags --abbrev=0)

This will create a new patch release and upload a distribution of it. If the changes are more significant, you can run bumpversion with the minor or major arguments.

When the schema is updated, please update the docs, ideally including the diagrams. For the RDF namespace and JavaScript version of the model, run make generate.

More Repositories

1

aleph

Search and browse documents and data; find the people and companies you look for.
JavaScript
1,996
star
2

memorious

Lightweight web scraping toolkit for documents and structured data.
Python
311
star
3

fingerprints

Make it easier to compare and cross-reference the names of companies and people by applying strong normalisation.
Python
142
star
4

cronodump

A Cronos database converter
Python
70
star
5

countrynames

Utility library to turn country names into ISO two-letter codes
Python
65
star
6

ingest-file

Ingestors extract the contents of mixed unstructured documents into structured (followthemoney) data.
Python
54
star
7

datadesktop

DEPRECATED. Desktop graph visualization application
TypeScript
50
star
8

pdflib

Binary Python bindings for poppler utils for content extraction
Python
42
star
9

synonames

Trying to generate name synonyms from wikidata
Python
33
star
10

alephclient

API client for Aleph, supports bulk entity and document upload.
Python
27
star
11

pantomime

Python library for MIME type parsing, normalisation and grouping.
Python
12
star
12

offshoreleaks

Converter for ICIJ Offshore Leaks data into FollowTheMoney format
Python
12
star
13

followthemoney-store

Fragment storage/database layer for FollowTheMoney entities
Python
10
star
14

react-ftm

React UI component library for aleph/followthemoney
TypeScript
10
star
15

languagecodes

A Python helper library to convert between ISO 639 two- and three-letter codes.
Python
10
star
16

countrytagger

Extract names of places from text and determine which country they may refer to
Python
8
star
17

servicelayer

Common interface definitions for aleph toolkit services and applications
Python
7
star
18

followthemoney-ocds

Import data formatted as OpenContracting Data Standard (OCDS) objects into FollowTheMoney
Python
7
star
19

panama

Parser for a 2008 scrape of the Panama companies registry
Python
6
star
20

docs

GitHub mirror of the GitBook documentation
6
star
21

followthemoney-predict

Experiments with FtM record linkage
Jupyter Notebook
5
star
22

alephr

R package wrapper for Aleph API
R
4
star
23

translate-service

Demo: document processing service for automated translation
Python
4
star
24

example-personadeinteres

Example how to load mixed document/entity graphs to Aleph
Python
4
star
25

aleph-elasticsearch

Custom ElasticSearch configuration for Aleph
Shell
3
star
26

followthemoney-typepredict

Predict ftm types for string input data
Python
3
star
27

followthemoney-compare

followthemoney-compare
Jupyter Notebook
2
star
28

document-categorization

DSSG document categorization repository
Jupyter Notebook
1
star
29

followthemoney-graph

Python
1
star