Discover mlcommons/research Open Source project by @mlcommons

Reference implementations of MLPerf™ inference benchmarks

1,495

inference

Collective Knowledge (CK) is an educational community project to learn how to run AI, ML and other emerging workloads in the most efficient and cost-effective way across diverse models, data sets, software and hardware using MLCommons CM (Collective Mind workflow automation framework)

966

MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers

605

tiny

MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.

293

algorithmic-efficiency

A generalizable application framework for segmentation, regression, and classification using PyTorch

210

GaNDLF

MLCube® is a project that reduces friction for machine learning by ensuring that models are easily portable and reproducible.

154

mlcube

An open benchmarking platform for medical artificial intelligence using Federated Evaluation.

149

medperf

The People’s Speech Dataset

144

peoples-speech

training_policies

This repository contains the results and code for the MLPerf™ Training v0.7 benchmark.

training_results_v0.7

This repository contains the results and code for the MLPerf™ Inference v0.5 benchmark.

inference_results_v0.5

Run safety benchmarks against AI models and view detailed reports showing how well they performed.

modelbench

This repository contains the results and code for the MLPerf™ Training v0.6 benchmark.

inference_policies

training_results_v0.6

Croissant is a high-level format for machine learning datasets that brings together four rich layers.

croissant

This repository contains the results and code for the MLPerf™ Training v0.5 benchmark.

training_results_v0.5

This repository contains the results and code for the MLPerf™ Training v1.0 benchmark.

training_results_v1.0

Reference implementations of MLPerf™ HPC training benchmarks

hpc

MLPerf™ Storage Benchmark Suite

storage

Shell

inference_results_v1.0

This repository contains the results and code for the MLPerf™ Inference v1.0 benchmark.

mlcube_examples

MLCube® examples

Repository for MLCommons Chakra schema and tools

chakra

mobile_app_open

Mobile App Open

This repository contains the results and code for the MLPerf™ Training v2.0 benchmark.

training_results_v2.0

Make it easy to automatically and uniformly measure the behavior of many AI Systems.

modelgauge

General policies for MLPerf™ including submission rules, coding standards, etc.

policies

This repository contains the results and code for the MLPerf™ Training v1.1 benchmark.

training_results_v1.1

mobile_models

MLPerf™ Mobile models

logging

MLPerf™ logging library

This repository contains the results and code for the MLPerf™ Inference v2.1 benchmark.

inference_results_v2.1

ck-mlops

A collection of portable workflows, automation recipes and components for MLOps in a unified CK format. Note that this repository is outdated - please check the 2nd generation of the CK workflow automation meta-framework with portable MLOps and DevOps components here:

This repository contains the results and code for the MLPerf™ Inference v0.7 benchmark.

inference_results_v0.7

This repository contains the results and code for the MLPerf™ Inference v3.0 benchmark.

inference_results_v3.0

training_results_v2.1

This repository contains the results and code for the MLPerf™ Training v2.1 benchmark.

Dev repo for power measurement for the MLPerf™ benchmarks

power-dev

medical

dynabench

training_results_v3.0

This repository contains the results and code for the MLPerf™ Training v3.0 benchmark.

This repository contains the results and code for the MLPerf™ Tiny Inference v0.7 benchmark.

tiny_results_v0.7

inference_results_v1.1

This repository contains the results and code for the MLPerf™ Inference v1.1 benchmark.

This repository contains the results and code for the MLPerf™ Inference v4.0 benchmark.

inference_results_v4.0

dataperf

Data Benchmarking

inference_results_v2.0

This repository contains the results and code for the MLPerf™ Inference v2.0 benchmark.

mobile_open

MLPerf Mobile benchmarks

https://mlcommons.org/en/groups/research-science/

science

This repository contains the results and code for the MLPerf™ Tiny Inference v0.5 benchmark.

tiny_results_v0.5

This repository contains the results and code for the MLPerf™ Inference v3.1 benchmark.

inference_results_v3.1

tiny_results_v1.0

This repository contains the results and code for the MLPerf™ Tiny Inference v1.0 benchmark.

hpc_results_v0.7

This repository contains the results and code for the MLPerf™ HPC Training v0.7 benchmark.

This repository contains the results and code for the MLPerf™ HPC Training v2.0 benchmark.

hpc_results_v2.0

This repository contains the results and code for the MLPerf™ HPC Training v1.0 benchmark.

hpc_results_v1.0

CK automation for virtual environments

ck-venv

cm-mlops

This repository contains the results and code for the MLPerf™ Training v3.1 benchmark.

datasets_infra

training_results_v3.1

This repository contains the results and code for the MLPerf™ Tiny Inference v1.1 benchmark.

tiny_results_v1.1

medperf-website

JavaScript

mobile_results_v2.1

This repository contains the results and code for the MLPerf™ Mobile Inference v2.1 benchmark.

hpc_results_v3.0

This repository contains the results and code for the MLPerf™ HPC Training v3.0 benchmark.

Aggregated benchmarking results from MLPerf Inference, Tiny and Training in the MLCommons CM format for the Collective Knowledge Playground. Our goal is to make it easier for the community to visualize, compare and reproduce MLPerf results and add derived metrics such as Performance/Watt or Performance/$

ck_mlperf_results