CODAIT/spark-bench

Stars
238
Rank 168,834 (Top 4 %)
Language
Scala
License
Apache License 2.0
Created about 9 years ago
Updated over 1 year ago

CODAIT/spark-bench

CODAIT

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Benchmark Suite for Apache Spark

spark-bench

Benchmark Suite for Apache Spark

READ OUR DOCS

The documentation for Spark-Bench is all in our shiny new docs site: https://codait.github.io/spark-bench/

Versions And Compatibility

Spark Version

Spark-Bench is currently compiled against the Spark 2.1.1 jars and should work with Spark 2.x. If you experience compatibility issues between Spark-Bench and any 2.x version of Spark, please let us know!

Scala Version

Spark-Bench is written using Scala 2.11.8. It is incompatible with Spark versions running Scala 2.10.x

Installation

Follow the Quickstart guide from our docs site. For more details, see the Installation page.

Legacy Version

spark-bench has recently gone through an extensive rewrite. While we think you'll like the new capabilities, it is not quite feature complete with the previous version of spark-bench. Many of the workloads that were available in the legacy have not yet been ported over, but they will be!

In the meantime, if you would like to see the old version of spark-bench, it's preserved in the legacy branch.

You can also grab the last official release of the legacy version from here.

text-extensions-for-pandas

Natural language processing support for Pandas dataframes.

Jupyter Notebook

deep-histopath

A deep learning approach to predicting breast tumor proliferation scores for the TUPAC16 challenge

Jupyter Notebook

stocator

Stocator is high performing connector to object storage for Apache Spark, achieving performance by leveraging object storage semantics.

covid-notebooks

Jupyter notebooks that analyze COVID-19 time series data

Jupyter Notebook

max-central-repo

Central Repository of Model Asset Exchange project. This repository contains information about the available models, current project status, contribution guidelines and supporting assets.

aardpfark

A library for exporting Spark ML models and pipelines to PFA

presentations

Talks & Workshops by the CODAIT team

Jupyter Notebook

r4ml

Scalable R for Machine Learning

spark-ref-architecture

Reference Architectures for Apache Spark

graph_def_editor

GraphDef Editor: A port of the TensorFlow contrib.graph_editor package that operates over serialized graphs

magicat

🧙😺 magicat - Deep learning magic.. with the convenience of cat!

node-red-contrib-model-asset-exchange

Node-RED nodes for the Model Asset Exchange on IBM Developer

max-tfjs-models

Pre-trained TensorFlow.js models for the Model Asset Exchange

pardata

nlp-editor

Visual Editor for Natural Language Processing pipelines

flight-delay-notebooks

Analyzing flight delay and weather data using Elyra, IBM Data Asset Exchange, Kubeflow Pipelines and KFServing

Jupyter Notebook

spark-db2

DB2/DashDB Connector for Apache Spark

redrock

RedRock - Mobile Application prototype using Apache Spark, Twitter and Elasticsearch

spark-netezza

Netezza Connector for Apache Spark

Identifying-Incorrect-Labels-In-CoNLL-2003

Research into identifying and correcting incorrect labels in the CoNLL-2003 corpus.

Jupyter Notebook

max-vis

Image annotation library and command-line utility for MAX image models

fae-tfjs

WELCOME-TO-CODAIT

Welcome to the Center for Open-Source Data & AI Technologies (CODAIT) organization on GitHub! Learn more about our projects ...

spark-tracing

A flexible instrumentation package for visualizing the internal operation of Apache Spark and related tools

redrock-v2

RedRock v2 Repository

Jupyter Notebook

max-node-red-docker-image

Demo Docker image for the Model Asset Exchange Node-RED module

max-workshop-oscon-2019

notebook-exporter

One Click deployment of Notebooks - Bringing Notebooks to Production

redrock-ios

RedRock - Mobile Application prototype

max-base

This repo has been moved

max-status

Current status of the Model Asset Exchange ecosystem

project-codenet-notebooks

Jupyter Notebook

MAX-Web-App-skeleton

A fully functioning skeleton for MAX model web apps

development-guidelines

Development Guidelines and related resources for IBM Spark Technology Center

codait.github.io

CODAIT Homepage

dax-schemata

redrock-v2-ios

RedRock v2 iPad Application

max-pytorch-mnist

Jupyter Notebook

teach-nao-robot-a-new-skill

Teach your NAO robot a new skill using deep learning microservices

max-fashion-mnist-tutorial-app

MAX-cloud-deployment-cheatsheets

Work in progress

ddc-data-and-ai-2021-automate-using-open-source

Jupyter Notebook

exchange-metadata-converter

Basic conversion utility for YAML-based metadata descriptors

streaming-integration-sample

covid-trusted-ai-pipeline

Jupyter Notebook