Discover m-lab/etl-schema Open Source project by M-Lab (@m-lab)

An exporter for converting BigQuery results into Prometheus metrics

prometheus-bigquery-exporter

ndt7-client-go

ndt7 reference client implementation in Go

tcp-info

Fast tcp-info collector in Go

prometheus-nagios-exporter

Prometheus exporter publishes Nagios service status read from the mk-livestatus plugin

Source code of the Measurement Lab website built using Jekyll

website

Jupyter Notebook

ndt7-client-ios

Swift

telescope

[INACTIVE] Python framework that was designed to extract raw measurements from M-Lab to allow the easy production of research.

Prometheus configuration for M-Lab running on GKE

etl

M-Lab ingestion pipeline

prometheus-support

A sidecar service which runs traceroute after a connection closes

traceroute-caller

ndt7-js

Contains code that processes M-Lab data and provides it in various formats for other use.

stats-pipeline

piecewise

A customizable online survey tool for running NDT tests.

Detailed analysis for data collected from M-Lab.

analysis

Jupyter Notebook

mlab-ns

M-Lab name server (load balancer for M-Lab servers)

ndt7 Android-native client

ndt7-client-android

Kotlin

ooni-support

Support scripts for OONI on M-Lab

Code for the M-Lab Measure App

measure-app

Gardener provides services for maintaining and reprocessing mlab data.

etl-gardener

mlab-vis-client

Measurement Lab Visualizations - Analyzing internet speeds worldwide.

Prometheus service discovery for targets running in App Engine Flex, Kubernetes, and generic HTTP(s) targets.

gcp-service-discovery

k8s-support

Setup for the kubernetes systems to control and run all the MLab nodes around the world

M-Lab Locate Service, a load balancer providing consistent “expected measurement quality” using access control

locate

packet-headers

A daemon for sniffing and archiving packet headers of all flows for a system.

signal-searcher

Find interesting and important patterns in M-Lab data

General purpose libraries / APIs for use in mlab code.

operator

M-Lab tools, configuration and notes to support operators of the platform.

Scrape experiment data off of MLab nodes and upload it to Google Cloud Storage

scraper

Measurements Swiss Army Knife

data-support

Jupyter Notebook

msak

libraries

A set of libraries to simplify network measurement tool creation.

M-Lab Public Site Information Automation

siteinfo

Jsonnet

jostler

An agent on M-Lab nodes to upload JSON data to GCS in JSONL format for autoloading to BigQuery.

annotation-service

Annotation integration service for M-Lab data

bigquery-maptiles

A service that manages the conversion of M-Lab BigQuery results into static world map tiles

HTML

epoxy-images

Support for building linux kernels, rootfs images, and ROMs for ePoxy

Implementation of the OONI probe tests (ooni.nu) as a Chrome packaged app

chrooniprobe

Dart

epoxy

A system for safe boot management over the Internet, based on iPXE.

collectd-mlab

Collectd plugin and configuration for Measurement Lab servers.

Cleansheet ndt server in go, using tcp-info library.

ndt-server-go

web100_userland-debian

Debian package for web100 userland library

ndt-javascript

M-Lab Visualization Dataflow pipelines for transforming ndt.all into the needed aggregation tables in bigtable.

mlab-vis-pipeline

Java

measure-location

A system for periodically saving all hardware, software, and config info for M-Lab nodes.

nodeinfo

mping

A CNI IPAM plugin for k8s on the M-Lab platform

moire

Dart

index2ip

pusher

Push data from nodes to cloud storage

ndt5-client-go

uuid

A system for allowing sockets to be universally and uniquely identified in M-Lab data.

package

Upload service for the M-Lab Measure Chrome Extension

measure-saver

ndt-support

Support scripts for NDT on M-Lab

API component of New and Exciting visualization collaboration with Measurement Lab

mlab-vis-api

Service to download meta data sources for archive.

downloader

gcs-exporter

A Prometheus exporter that reports on recent archive file stats from M-Lab GCS buckets

neubot-support

Support scripts for Neubot on M-Lab

A visualization service of Murakami tests using Koa and React.

murakami-viz

Experimental ndt7 client engine written in C++11

ndt7-client-cc

A support library for adding deployment automation in travis.

travis

Configuration and tools for configuring GCP for M-Lab

ndt-debian

Debian package for NDT

gcp-config

tcpinfo_lib

Wrapper library for collecting data through netlink.

Git hooks to help your project be better

git-hooks

mlab-browser-ext

MLab Browser Extension

Java NDT7 client for Android OS

ndt7-client-android-java

Java

archive-repacker

Process and repackage existing archives

mbm

Standalone and GAE pipeline endpoints for m-lab

pipeline

bigsanity

M-Lab’s Big Data Sanity Checker

This repo contains style guidelines and conventions to be used in creating public M-Lab assets like websites, presentations, documents, etc.

styleguide

clinic2019

Repo for HMC clinic

For recording human-curated annotations that have known impacts on M-Lab test data.

data-annotations

disco

DIScard COllector. Collects metrics from switches, writes them to disk and exports them.

measure-app-old

HTML

net-next-web10g

net-next from kernel.org with web10g patched in

ansible-style-guide

This style guide is designed to ensure consistent style and practices across M-Lab's Ansible playbooks and associated Ansible files.

scraper-sync

Synchronize the cloud datastore used by the scraper with the spreadsheet

autoloader

echo

simple echo server

Libraries and services for access control to the M-Lab platform

access

mlab-ops

Automatically exported from code.google.com/p/m-lab.ops

A tool that takes output from paris-traceroute and generates a dot file

eiffel

Dart

internet_deserts

The repository with all of the code for the internet deserts story

ndt-e2e-clientworker

Code for the client worker of the NDT end-to-end test framework