mlcommons/ck_mlperf_results

Stars
1
Language
Python
License
Apache License 2.0
Created over 1 year ago
Updated about 1 year ago

mlcommons/ck_mlperf_results

mlcommons

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Aggregated benchmarking results from MLPerf Inference, Tiny and Training in the MLCommons CM format for the Collective Knowledge Playground. Our goal is to make it easier for the community to visualize, compare and reproduce MLPerf results and add derived metrics such as Performance/Watt or Performance/$

training

Reference implementations of MLPerf™ training benchmarks

inference

Reference implementations of MLPerf™ inference benchmarks

ck

Collective Knowledge (CK) is an educational community project to learn how to run AI, ML and other emerging workloads in the most efficient and cost-effective way across diverse models, data sets, software and hardware using MLCommons CM (Collective Mind workflow automation framework)

tiny

MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers

algorithmic-efficiency

MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.

GaNDLF

A generalizable application framework for segmentation, regression, and classification using PyTorch

mlcube

MLCube® is a project that reduces friction for machine learning by ensuring that models are easily portable and reproducible.

medperf

An open benchmarking platform for medical artificial intelligence using Federated Evaluation.

peoples-speech

The People’s Speech Dataset

Jupyter Notebook

training_policies

Issues related to MLPerf™ training policies, including rules and suggested changes

training_results_v0.7

This repository contains the results and code for the MLPerf™ Training v0.7 benchmark.

inference_results_v0.5

This repository contains the results and code for the MLPerf™ Inference v0.5 benchmark.

modelbench

Run safety benchmarks against AI models and view detailed reports showing how well they performed.

inference_policies

Issues related to MLPerf™ Inference policies, including rules and suggested changes

training_results_v0.6

This repository contains the results and code for the MLPerf™ Training v0.6 benchmark.

croissant

Croissant is a high-level format for machine learning datasets that brings together four rich layers.

Jupyter Notebook

training_results_v0.5

This repository contains the results and code for the MLPerf™ Training v0.5 benchmark.

training_results_v1.0

This repository contains the results and code for the MLPerf™ Training v1.0 benchmark.

hpc

Reference implementations of MLPerf™ HPC training benchmarks

Jupyter Notebook

storage

MLPerf™ Storage Benchmark Suite

inference_results_v1.0

This repository contains the results and code for the MLPerf™ Inference v1.0 benchmark.

mlcube_examples

MLCube® examples

chakra

Repository for MLCommons Chakra schema and tools

mobile_app_open

Mobile App Open

training_results_v2.0

This repository contains the results and code for the MLPerf™ Training v2.0 benchmark.

modelgauge

Make it easy to automatically and uniformly measure the behavior of many AI Systems.

policies

General policies for MLPerf™ including submission rules, coding standards, etc.

training_results_v1.1

This repository contains the results and code for the MLPerf™ Training v1.1 benchmark.

mobile_models

MLPerf™ Mobile models

logging

MLPerf™ logging library

inference_results_v2.1

This repository contains the results and code for the MLPerf™ Inference v2.1 benchmark.

ck-mlops

A collection of portable workflows, automation recipes and components for MLOps in a unified CK format. Note that this repository is outdated - please check the 2nd generation of the CK workflow automation meta-framework with portable MLOps and DevOps components here:

inference_results_v0.7

This repository contains the results and code for the MLPerf™ Inference v0.7 benchmark.

inference_results_v3.0

This repository contains the results and code for the MLPerf™ Inference v3.0 benchmark.

training_results_v2.1

This repository contains the results and code for the MLPerf™ Training v2.1 benchmark.

power-dev

Dev repo for power measurement for the MLPerf™ benchmarks

medical

Medical ML Benchmark

dynabench

training_results_v3.0

This repository contains the results and code for the MLPerf™ Training v3.0 benchmark.

tiny_results_v0.7

This repository contains the results and code for the MLPerf™ Tiny Inference v0.7 benchmark.

inference_results_v1.1

This repository contains the results and code for the MLPerf™ Inference v1.1 benchmark.

inference_results_v4.0

This repository contains the results and code for the MLPerf™ Inference v4.0 benchmark.

dataperf

Data Benchmarking

inference_results_v2.0

This repository contains the results and code for the MLPerf™ Inference v2.0 benchmark.

mobile_open

MLPerf Mobile benchmarks

science

https://mlcommons.org/en/groups/research-science/

Jupyter Notebook

tiny_results_v0.5

This repository contains the results and code for the MLPerf™ Tiny Inference v0.5 benchmark.

inference_results_v3.1

This repository contains the results and code for the MLPerf™ Inference v3.1 benchmark.

tiny_results_v1.0

This repository contains the results and code for the MLPerf™ Tiny Inference v1.0 benchmark.

hpc_results_v0.7

This repository contains the results and code for the MLPerf™ HPC Training v0.7 benchmark.

hpc_results_v2.0

This repository contains the results and code for the MLPerf™ HPC Training v2.0 benchmark.

hpc_results_v1.0

This repository contains the results and code for the MLPerf™ HPC Training v1.0 benchmark.

ck-venv

CK automation for virtual environments

cm-mlops

datasets_infra

training_results_v3.1

This repository contains the results and code for the MLPerf™ Training v3.1 benchmark.

research

tiny_results_v1.1

This repository contains the results and code for the MLPerf™ Tiny Inference v1.1 benchmark.

medperf-website

mobile_results_v2.1

This repository contains the results and code for the MLPerf™ Mobile Inference v2.1 benchmark.

hpc_results_v3.0

This repository contains the results and code for the MLPerf™ HPC Training v3.0 benchmark.