Discover NERSC/dayabay-data-conversion Open Source project

350

jupyterlab-slurm

slurm-magic

IPython magic for SLURM.

slurm-ray-cluster

Official TensorFlow 2.0 tutorial notebooks for the Deep Learning for Science School at LBNL

dl4sci-tf-tutorials

Material for the SC22 Deep Learning at Scale Tutorial

sc22-dl-tutorial

SC23 Deep Learning at Scale Tutorial Material

sc23-dl-tutorial

podman-hpc

PyTorch examples for NERSC systems

pytorch-examples

Collection of tutorials for using Shifter to bring containers to HPC

Shifter-Tutorial

sc21-dl-tutorial

Material for the SC21 Deep Learning at Scale Tutorial

Spawn JupyterHub single-user servers with ssh

sshspawner

[READ ONLY] Refer to gitlab repo for updated version - Total Knowledge of I/O Reference Implementation. Please see wiki for contribution guidelines.

data-seminars

pytokio

sc20-dl-tutorial

CosmoFlow

timemory-tutorials

Tutorials for Timemory

Includes Python bindings to instrumentation and tracing technology (ITT) APIs for VTune

itt-python

sc19-dl-tutorial

Hands-on material for the SC19 tutorial, Deep Learning at Scale

A JupyterLab extension that tracks recent files and directories.

jupyterlab-recents

OpenMP Training Series, May to October 2024

openmp-series-2024

jupyterlab-favorites

Add the ability to save favorite folders to JupyterLab for quicker browsing

intro-HPC-bootcamp-2023

single-GPU to multi-GPU training of PyTorch apps at NERSC

nersc-dl-multigpu

Docker Deployment of NERSC Jupyterhub (including auth and spawner modules)

jupyterhub-deploy

Keras tutorial code for the SC18 tutorial on Deep Learning at Scale

crash-course-supercomputing

sc18-dl-tutorial

sfapi_client

Python client for SF API

Tutorial contents for SC16 on November 14, 2016

2016-11-14-sc16-Container-Tutorial

dl4sci-scaling-tutorial

Deep Learning Scaling tutorial material for the Deep Learning for Science School at Berkeley Lab

spack-infrastructure

Scripts/Benchmarks for Running Tensorflow Distributed on Cori

cori-tf-distributed-examples

MiniDFT is a minimalist version of Quantum ESPRESSO that performs only LDA total energy and gradient calculations. The purpose of miniDFT is to explore new parallelization schemes, programming languages, programming models and computer architectures, and evaluate their suitability for plane-wave DFT calculations. These types of experiments will be more easily done with MiniDFT than with QE, because the current version of MiniDFT is significantly smaller than QE.

MiniDFT

Fortran

nersc-dl-wandb

Guidelines on using Weights and Biases logging for deep learning applications on NERSC machines

buildtest-nersc

ML Perlmutter User Training

ml-pm-training-2022

hpcpp

resources for user training exercises

train

Scalable Data Pipeline and Parallel IO in Deep Learning Framework

DL-Parallel-IO

Evaluating object stores with HPC science applications

object-store

c2d

c2d utility to take Conda environments and Jupyter notebooks from Cori to a local Docker image.

sshproxy

NERSC SSH Proxy Service

gsiauthenticator

Hacking Competition Code for NERSC's GPU for Science Day, July 2019

gpu-for-science-day-july-2019

Repository for building the NCCL OFI plugin from AWS and NVIDIA

nccl-ofi-plugin

isc19-dl-tutorial

Setup and simulation scripts for CompactCori

CompactCori

TensorFlow Benchmark for the HEP Deep Learning CNN Model

hep_cnn_benchmark

Demo code for the NERSC Data Day 2022 training event.

data-day-2022

Performance analysis tests for performance analysis tools

instrumentation-benchmark

CMake

ecp-container-tutorial

Material for the ECP Tutorial on Containers

jupyterlab-cpustatus

Show CPU usage in JupyterLab statusbar

base-images

Base Docker Images

spin-docker-compose-example

An extension to NBViewer for cloning notebooks to a local directory

clonenotebooks

Data and analysis demonstrating how to quantitatively design a Lustre file system

lustre-design-analysis

Some notebook demos from the new user training

new-user-training-notebooks

Contains Dockefiles for NERSC ML images

nersc-ml-images

Dockerfile

sshapiauthenticator

Custom Jupyter Authenticator that works against NERSC's SSH Auth API service

This repository contains scripts calling kernels for TensorFlow along with profiling scripts for Cori-GPU.

tf-perf-kernels

Understanding the IO Performance on KNL and Haswell

heterogeneous-IO

Tools for applying gitflow to Cray systems management

smwflow

Repository of Spin Recipes

spin-recipes

Dockerfile

chos

CHOS: The CHroot OS that lets users choose their OS.

slurm-helpers

handy bash and python snippets for interacting with Slurm at NERSC

JupyterHub Entrypoint Service. Registry that maintains custom entrypoint settings through a REST API.

jupyterhub-entrypoint

Tips and utilities for using Cmake in a project

cmake-helpers

Perlmutter_Training_Jan2022

Training material for Perlmutter User Training 2022

Cuda

mlperf-prof

Common set of utilities for MLPerf-HPC profiling

Inspect and report Python packages of interest

customs

General example notebooks for Jupyter at NERSC

example-jupyter-notebooks

Migrate-to-Perlmutter

Scripts to help run tensorboard at NERSC

nersc-tensorboard-helper

A suite of unit tests for the Burst Buffer

BB-unit-tests

Give and Take Command at NERSC

giveandtake

NERSC / NVIDIA AI for Science Bootcamp

QENESAP

Fortran

nersc-nvidia-ai4sci

PAM module to enable opt-in MFA

pam_mfa

Hadoop-on-Demand

Scripts for running Hadoop on demand on NERSC systems.

sc22-canopie-hpc-benchmarks

variable-time-job

This repo hosts the Data Analytics Tutorial for CUG 2019

cug19-da-tutorial

Learning to Extract Features from the Daya Bay Reactor Neutrino Experiment

dayabay-learn

A reference architecture for monitoring of Cray supercomputers

SMWG-reference-arch

swin_v2_weather

This repository contains the SwinV2_Weather model, developed for the "Analyzing and Exploring Training Recipes for Large-Scale Transformer-Based Weather Prediction" paper. The repo includes training scripts, pre-processing utilities, and model configuration files.

JupyterLab Extension that fetches and displays announcements to the user from an external API.

nersc-refresh-announcements

Predicting queue wait times for batch jobs on NERSC systems

qpredict

NERSC-themed LaTeX beamer presentation template

nersc-latex-presentation

TeX

jupyterlab_resuse

Proof of concept for NERSC

Tools and methods to analyze TOKIO-ABC results

tokio-abcutils

A collection of scripts for Shifter @ NERSC

shifter-tools

nersc-tf-tests

Repository of examples for the NERSC-CRAY Big Data Center collaboration

cray-bdc-deeplearning

inference_benchmarks

community-software

An ipywidget for choosing a path (file or directory) interactively

ipypathchooser

A third party FLAC filter for HDF5

FLAC-HDF5-plugin

rstudio-deploy

NERSC's R Studio docker deployment

Nginx

100

ecp19-dl-tutorial

Material for the Deep Learning tutorial at the 2019 ECP All-Hands Meeting