tokenmill/crawling-framework-example tokenmill/crawling-framework-example

  • Stars
    star
    3
  • Rank 3,963,521 (Top 79 %)
  • Language
    Java
  • License
    Other
  • Created about 7 years ago
  • Updated about 5 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
tokenmill/crawling-framework-example
github

tokenmill

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Demonstration on how to use the Crawling Framework to setup a simple science news crawler and store results in ElasticSearch. Use this configuration to set up your own crawler.

More Repositories

1

accelerated-text

Accelerated Text is a no-code natural language generation platform. It will help you construct document plans which define how your data is converted to textual descriptions varying in wording and structure.
JavaScript
384
star
2

beagle

Beagle helps you identify keywords, phrases, regexes, and complex search queries of interest in streams of text documents.
Clojure
52
star
3

clojure-graalvm-aws-lambda-template

Leiningen template for AWS Lambda custom runtime with GraalVM native image compiled Clojure projects.
Clojure
45
star
4

timewords

Multilingual library to easily parse date strings to java.util.Date objects.
Clojure
30
star
5

crawling-framework

Easily crawl news portals or blog sites using Storm Crawler.
Java
20
star
6

fast-url-access-checker

Easily run HTTP GET requests against a list of URLs to check their HTTP status.
Clojure
12
star
7

docx-utils

Easily work with .docx files from Clojure (a wrapper on Apache POI library).
Clojure
12
star
8

reaction-acc-text-demo

Integration between Reaction ECommerce and Accelerated Text to provide product descriptions for an e-shop.
JavaScript
10
star
9

dictionary-annotator

Fast and configurable UIMA dictionary annotator.
Java
7
star
10

snowball

Snowball version of the Porter stemmer for the Lithuanian language.
7
star
11

common-crawl-utils

Various Common Crawl utilities in Clojure.
Clojure
6
star
12

accelerated-text-project-template

Text generation project template to be used with Accelerated Text
Makefile
6
star
13

docker-images

Docker configurations, images, and examples of Dockerfiles for various TokenMill products and projects.Official source for Docker configurations, images, and examples of Dockerfiles for TokenMill products and projects
Dockerfile
5
star
14

reaction-acc-text-import

Plugin connecting ReactionCommerce and AcceleratedText by allowing to import products and generate their descriptions
JavaScript
2
star
15

beagle-performance-benchmarks

Performance benchmarks for the Beagle library, and comparisons with other stored-query solutions.
Clojure
1
star
16

es-utils

Clojure helper functions for Elasticsearch.
Clojure
1
star