dsaidgovsg/spark-custom-addons

Stars
2
Language
Dockerfile
License
MIT License
Created over 5 years ago
Updated over 3 years ago

dsaidgovsg/spark-custom-addons

dsaidgovsg

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Dockerfile Set-up to add dependencies into `spark-custom` images

airflow-pipeline

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

terraform-modules

Reusable Terraform modules

multimodal-learning-hands-on-tutorial

Jupyter Notebook

k-shortest-path

Implements K shortest path algorithms for networkx

vigilantgantry

Face segmentation algorithm that the VigilantGantry uses to identify potential missed out causes by current thermal systems (due to occlusion from fringe, cap, head-gear).

python-spark

Docker image for a Python installation with Spark, Hadoop and Sqoop binaries

TDBSCAN

TDBSCAN with Move Ability: Spatiotemporal Density Clustering

spark-geo-privacy

Geospatial privacy functions for Apache Spark

yarn-reverse-proxy

Reverse proxy for the status pages of a YARN cluster

nomad-parametric-autoscaler

A customizable Nomad/EC2 auto-scaling service

folium-resource-server

Simple server to just host folium JS and CSS resources

stack-2022-differential-privacy-workshop

Jupyter Notebook

spark-k8s-addons

Dockerfile setup to install cloud related utilities onto the standard Spark K8s Docker images

registrywatcher

zeppelin

Zeppelin Dockerfile set-up with a wrapping dynamic GitHub releases JAR loader

spark-k8s

CI set-up to generate Spark with Kubernetes Docker images

sg-tileserver-gl

Repackaged repository to build Docker image for Singapore tiles only

python-node

spark-base

Dockerfile setup for Spark set-up, imbued with varying degree of Python data science packages

spark-jupyterhub

Experimental and opinionated set-up to conveniently get going JupyterHub

zeppelin-jar-loader

Provides a simple JAR loader for dynamic loading of JAR files at the start for Zeppelin

data-privacy-workshop

Jupyter Notebook

spark-custom

Dockerfile set-up for building Spark releases from source code

ljumphost

Proper jumphost set-up and instructions repository

benchmarking-differential-privacy-tools

Jupyter Notebook

spark-k8s-addons-ds

kms-reencrypt

Python boto3 script to recursively KMS re-encrypt objects

datavis-examples

Repo with sample dataset and codes to show some simple visualization.

Jupyter Notebook

ilytics

gunicorn, flask, and darknet model

folium-override-server

Webserver to re-generate uploaded folium map to use custom URLs

ura-subzones

Simple library for dealing with URA subzones