DigitalPebble/ansible-storm

Stars
7
Rank 2,235,480 (Top 46 %)
Language
Created almost 7 years ago
Updated 6 months ago

DigitalPebble/ansible-storm

DigitalPebble

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Ansible playbook for deploying a Storm cluster

storm-crawler

A scalable, mature and versatile web crawler based on Apache Storm

behemoth

Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.

TextClassification

A Text Classification API in Java originally developed by DigitalPebble Ltd. The API is independent from the ML implementations used and can be used as a front end to various ML algorithms. libSVM and liblinear are currently embedded.

textclassification-examples

Use cases for DigitalPebble's TextClassification API

stormcrawlerfight

Crawl configurations for benchmarking / testing StormCrawler

stormcrawler-docker

Resources for running StormCrawler with Docker services

TextClassificationPlugin

GATE Processing Resource wrapping DigitalPebble's TextClassification API

ngrams-api

Java API for querying a N-Grams corpus. Uses Lucene for searching and indexing from the Google Web-1T format

behemoth-commoncrawl

Support for old (pre 2013) CommonCrawl dataset in Behemoth

NutchFight

Resources for comparison between 1.8 and 2.x of Apache Nutch

tescobank

Setup for crawling tescobank with SC

sc-warc

WARC resources for StormCrawler

behemoth-textclassification

Module for classifying Behemoth documents with a model from our Text Classification API

crawlurlfrontier

Crawl config used to test URL Frontier on a large scale and produce WARCs for CommonCrawl.

behemoth-elasticsearch

ElasticSearch module for Behemoth

urlfrontier-client

URLFrontier client written in Rust (mostly as a way of learning Rust)