• Stars
    star
    3,408
  • Rank 13,122 (Top 0.3 %)
  • Language
    Go
  • License
    Other
  • Created about 12 years ago
  • Updated over 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

DEPRECATED: Data collection and processing made easy.

This project is deprecated. Please see this email for more details.

Heka

Data Acquisition and Processing Made Easy

Heka is a tool for collecting and collating data from a number of different sources, performing "in-flight" processing of collected data, and delivering the results to any number of destinations for further analysis.

Heka is written in Go, but Heka plugins can be written in either Go or Lua. The easiest way to compile Heka is by sourcing (see below) the build script in the root directory of the project, which will set up a Go environment, verify the prerequisites, and install all required dependencies. The build process also provides a mechanism for easily integrating external plug-in packages into the generated hekad. For more details and additional installation options see Installing.

WARNING: YOU MUST SOURCE THE BUILD SCRIPT (i.e. source build.sh) TO BUILD HEKA. Setting up the Go build environment requires changes to the shell environment, if you simply execute the script (i.e. ./build.sh) these changes will not be made.

Resources:

More Repositories

1

syncserver

Run-Your-Own Firefox Sync Server
Python
1,794
star
2

syncstorage-rs

Sync Storage server in Rust
Rust
964
star
3

hindsight

Hindsight - light weight data processing skeleton
C
670
star
4

screenshots

Firefox Screenshots: the best way to take screenshots on the web.
FreeMarker
620
star
5

socorro

Socorro is the Mozilla crash ingestion pipeline. It accepts and processes Breakpad-style crash reports. It provides analysis tools.
Python
584
star
6

lua_sandbox

Generic Lua sandbox for dynamic data analysis
C
226
star
7

ios-sync-client

A standalone iOS client for Firefox Sync
C
219
star
8

autopush

Python Web Push Server used by Mozilla
Python
218
star
9

autopush-rs

Push Server in Rust
Rust
196
star
10

Dockerflow

Cloud Services Dockerflow specification
JavaScript
192
star
11

google-cloud-rust

Google Cloud Client Library for Rust
Rust
176
star
12

autograph

Mozilla's digital signature service
Go
141
star
13

shavar-prod-lists

Shavar/tracking protection lists used in prod
Python
139
star
14

GitHub-Audit

INACTIVE - Collection of Tools & Procedures for double checking GitHub configurations
Python
138
star
15

loads

SUPERSEDED BY https://github.com/loads
Go
107
star
16

powerhose

Runs workers via zmq to perform any kind of task
Python
98
star
17

server-syncstorage

The SyncServer server software, as used by Firefox Sync
Python
87
star
18

websec-check

web security checklist for Firefox Services
72
star
19

lua_sandbox_extensions

Extension packages (sandboxes and modules) for the lua_sandbox project
Lua
72
star
20

cliquet

CLIQUET IS NOW DEPRECATED use kinto.core instead
Python
65
star
21

loop-server

The mozilla loop server
JavaScript
61
star
22

tokenserver

The Mozilla Token Server
Python
61
star
23

iprepd

Centralized IP reputation daemon
Go
56
star
24

axe-selenium-python

aXe Selenium Integration python package
Python
56
star
25

android-sync

An outdated mirror of services and related code for Firefox for Android. See gecko-dev/mozilla-central.
Java
52
star
26

tokenlib

generic support library for signed-token-based auth schemes
Python
50
star
27

firefox-send-tab-to-device

OBSOLETE. A Firefox add-on that uses Sync to send tabs to remote devices.
JavaScript
46
star
28

mozilla-pipeline-schemas

Schemas for Mozilla's data ingestion pipeline and data lake outputs
Python
46
star
29

queuey

Mozilla Message Queue
Python
42
star
30

syncclient

Python client for Firefox Sync
Python
42
star
31

pyramid_multiauth

stacked authentication policies for pyramid
Python
41
star
32

guardian-vpn-windows-deprecated

Mozilla VPN for Windows
C#
41
star
33

userplex

Propagate users from Mozilla's Person API to third party systems.
Go
40
star
34

fernet-rs

Fernet implementation in Rust
Rust
40
star
35

go-cose

go library for CBOR Object Signing and Encryption (COSE)
Go
40
star
36

requests-hawk

Hawk authentication strategy for the requests python library.
Python
39
star
37

megaphone

Firefox Global Broadcast API
Rust
38
star
38

python-dockerflow

A Python package to implement tools and helpers for Mozilla Dockerflow
Python
38
star
39

metlog-py

Python library for Services metrics logging
Python
37
star
40

readinglist

Reading List Server
Python
34
star
41

konfig

Yet another configuration object
Python
34
star
42

mozservices

Various utilities for Pyramid-based Mozilla applications
Python
34
star
43

reaper

Reaper culls leftover AWS resources
Go
34
star
44

go-syncstorage

INACTIVE - SyncStorage Server with more golang and less indexes!
Go
32
star
45

merino

Web service for Firefox Suggest
Rust
31
star
46

tecken

Mozilla Symbol Server
Python
31
star
47

data-pipeline

Mozilla Services Data Pipeline
Lua
30
star
48

heka-build

Heka build environment
Python
29
star
49

heka-py

DEPRECATED - Heka Python Library - DEPRECATED
Python
29
star
50

heka-mozsvc-plugins

Set of heka plugins in use by Mozilla Services
Go
28
star
51

aws-signing-proxy

signs http requests using AWS V4 signer
Go
27
star
52

foxsec-pipeline

Log analysis pipeline utilizing Apache Beam
Java
25
star
53

services-central-legacy

Sync working branch. `master` in this repository is stable, and tracks https://hg.mozilla.org/services/services-central. Other branches are owned by developers, and are subject to change.
C++
24
star
54

minidump-stackwalk

Socorro breakpad minidump stackwalker
C++
21
star
55

telescope

A dumb auditing service
Python
21
star
56

antenna

Antenna is the collector for the Socorro crash ingestion pipeline
Python
20
star
57

shavar

Tracking Protection update service for Firefox based on Safe Browsing protocol
Python
20
star
58

zktools

Zookeeper Tools
Python
19
star
59

contile

This is the back-end server for the Mozilla Tile Service (MTS)
Rust
19
star
60

tuxedo

An improved version of Mozilla's download load balancer Bouncer, with a user interface written in Django/Python.
Python
18
star
61

shavar-plugin-blocklist

โŒ Firefox plugin blocklist
XSLT
18
star
62

demoapp

An empty app for the next-gen Services app
Python
17
star
63

marteau

client-server on the top of Funkload
Python
16
star
64

msisdn-gateway

An MSISDN based Authentication Server.
JavaScript
16
star
65

docs

Documentation for Mozilla Services Services
Makefile
16
star
66

tigerblood

Deprecated, use https://github.com/mozilla-services/iprepd
Go
15
star
67

readinglist-client

Readinglist client
JavaScript
14
star
68

ldappool

A Pool for python-ldap
14
star
69

buildhub

DEPRECATED: Mozilla Build Metadata Service
Python
13
star
70

skeleton

A Skeleton actix app
Rust
13
star
71

heka-node

DEPRECATED - This repository is no longer maintained. Please go over to : https://github.com/disqus/heka-node
JavaScript
13
star
72

deepspeech-server

Rust
13
star
73

go-bouncer

A Go version of the redirector portion of bouncer.
Go
13
star
74

zipalign

DEPRECATED - Golang implementation of Android's ZipAlign tool
Go
13
star
75

kinto-dist

Kinto Distribution for Mozilla Services
Python
12
star
76

server-full2

Experimental Run-Your-Own Sync2.0 Server
Python
12
star
77

shavar-list-creation

Script to transform the Disconnect block-list into Safebrowsing v2 format for Firefox Tracking Protection
Python
12
star
78

go-mozlogrus

DEPRECATED - A logging library which conforms to Mozilla's logging standard for logrus users.
Go
12
star
79

FindMyDevice

Find My Device - ๐Ÿšจ๐ŸšจThis server is obsolete and unsupported.๐Ÿšจ๐Ÿšจ
Go
11
star
80

hindsight_admin

Hindsight Administration User Interface
JavaScript
11
star
81

walint

script to validate web apps
Python
11
star
82

pyramid_ipauth

a pyramid authentication policy based on remote ip address
Python
11
star
83

push-service

Top-level repository for the Push Service
11
star
84

macauthlib

low-level library for implementing MAC Access Authentication
Python
11
star
85

pyramid_whoauth

a pyramid authentication policy using repoze.who
Python
10
star
86

merino-py

Web Service for Firefox Suggest
Python
9
star
87

go-mozlog

A logging library which conforms to Mozilla's logging standard.
Go
9
star
88

oidc-gateway

A Docker container and Kubernetes Helm chart to gate access to upstream services.
Lua
9
star
89

redbarrel

JavaScript
9
star
90

iprepd-nginx

Openresty nginx module for integrating with iprepd
Python
9
star
91

services-test

Tools and test scripts used by the Mozilla Cloud Services team
JavaScript
9
star
92

nginx_moz_ingest

HTTP Data Pipeline Ingestion
C
9
star
93

logstash-metlog

Extensions to logstash for metlog
Ruby
9
star
94

shavar-list-creation-config

contains config files needed to run the jenkins task that builds the shavar lists
9
star
95

crashstats-tools

Command line tools and library for interacting with Crash Stats (https://crash-stats.mozilla.org/)
Python
9
star
96

topsites-proxy

Proxy server to track Top Sites default tile campaign attribution
JavaScript
8
star
97

hoverpad

DEPRECATED - Playing around with an Addons to sync some informations.
Elm
8
star
98

absearch

Python
8
star
99

updatebot

Automation for updating third party libraries for Firefox
Python
8
star
100

hawkauthlib

INACTIVE - low-level library for implementing MAC Access Authentication
Python
8
star