Discover cerndb/SparkTraining Open Source project

Stars
12
Rank 1,597,372 (Top 32 %)
Language
Jupyter Notebook
License
Creative Commons ...
Created about 2 years ago
Updated 6 months ago

cerndb

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Material for the course "Introduction to Apache Spark APIs for Data Processing" https://sparktraining.web.cern.ch/

dist-keras

Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.

Python

624

spark-dashboard

Spark-Dashboard is a solution for monitoring Apache Spark jobs. This repository provides the tooling and configuration for deploying an Apache Spark Performance Dashboard using containers technology.

Dockerfile

112

SparkPlugins

Code and examples of how to write and deploy Apache Spark Plugins. Spark plugins allow runnig custom code on the executors as they are initialized. This also allows extending the Spark metrics systems with user-provided monitoring probes.

Scala

hdfs-metadata

Tool for gathering blocks and replicas meta data from HDFS. It also builds a heat map showing how replicas are distributed along disks and nodes.

Java

SparkDLTrigger

Code and links to the data for the article "Machine Learning Pipelines with Modern Big DataTools for High Energy Physics"

Jupyter Notebook

grafana-mimir-cardinality-dashboards

Grafana Mimir dashboards used for cardinality exploration

Hadoop-Profiler

Hadoop Profiler, or hprofiler, is a tool which is able to analyze on- and off-CPU workloads on distributed computing environments.

Shell

sparkMeasure

This is a mirror of https://github.com/LucaCanali/sparkMeasure - sparkMeasure is a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task metrics.

Scala

cern-sso-python

Python Re-implementation of the cern-get-sso-cookie functionality

Python

flume-ng-audit-db

Apache Flume JDBC source, drop duplicated events interceptor, utility to infer Avro schema from table and much more!

Java

linux-firewall-tool

Linux iptables automation tool. It manages the firewall on CERN 's DB Servers.

Python

tf-spawner

TF-Spawner is an experimental tool for running TensorFlow distributed training on Kubernetes clusters.

Python

wls-cern-sso

Oracle Weblogic CERN SSO integration packages

Java

storage-api

Unified RESTful interface for managing CERNs data storage back-ends

Python

zkpolicy

Zookeeper Policy Audit Tool (aka zkPolicy) for checking and enforcing ACLs on ZNodes.

Java

SparkExecutorPlugins2.4

Spark Executor Plugins Examples for Spark 2.4

Java

hadoop-xrootd

Mirror of CERN db/hadoop-xrootd. Hadoop-XRootD Filesystem Connector

Java

hadoop-metrics-http-sink

Hadoop Metrics 2 plugin to push metrics to a HTTP end point (e.g. Elastic, Flume).

Java

dbod-core

DB On Demand management infrastructure core library

Perl

dbod-web

Future DB On Demand Web Interface implementation

TypeScript

dbod-api

DB On Demand API

Python

CERN-Hadoop-tutorials-ML-with-Apache-Spark

Tutorial materials for Analytics with Apache Spark and MLlib at CERN. https://indico.cern.ch/event/546003/

hloader

dbod-infra

dbod-webapp

netapp-api-python

A re-implementation of (parts of) NetApp's ZAPI in idiomatic Python using Requests

wls-cli

elastalert

nile-webapp

Nile Service Web Interface

TypeScript

tomcat-sso-integration-components

Set of valves classes that helps CERN applications with the integration in the CERN Authentication

Java

oracle-weblogic-1221-domain-ords-autodeploy

Oracle WebLogic docker image with ORDS deployed

PLSQL

coding-standards

Miscelenaeous coding standards or support files to use in IT-DB projects

Shell

hadoop-tutorials

Repository for hadoop tutorials code and guides

NotebooksExamples

This repository contains Jupyter notebook examples, intended to be linked with the SWAN Gallery

Jupyter Notebook

cern-openlab-oracle-hackzurich-2018

CERN openlab Oracle HackZurich 2018 collaboration

2qpgconf2017

CERN DB On Demand 2Q PGConf 2017 slides

cerndb/SparkTraining

cerndb

Reviews

Repository Details

More Repositories