trandoshan-io/scheduler

This repository has been archived on 10/Aug/2020
Stars
6
Rank 2,539,965 (Top 51 %)
Language
Go
License
GNU General Publi...
Created over 5 years ago
Updated almost 5 years ago

trandoshan-io/scheduler

trandoshan-io

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Go orchestrator process used to schedule website parsing

crawler

Go process used to crawl websites

k8s

Kubernetes configuration file to boostrap trandoshan cluster

trandoshan-parent

Parent repository used to centralize tools to run the application, manage source code, ...

dashboard

Web UI to manage a Trandoshan instance

persister

Go process used to abstract database interaction. It consumes messages from queue and put result in database allowing asynchronous saving operation

api

REST API used by other processes to gather data

feeder

Oneshot process used to put an URL in the todo queue. Useful to start crawling process