• Stars
    star
    374
  • Rank 114,346 (Top 3 %)
  • Language
    Jupyter Notebook
  • License
    Apache License 2.0
  • Created over 1 year ago
  • Updated 25 days ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

T81-558: PyTorch - Applications of Deep Neural Networks @Washington University in St. Louis

T81 558:Applications of Deep Neural Networks

Washington University in St. Louis

Instructor: Jeff Heaton

  • Section 1. Spring 2024, Monday, 2:30 PM, Location: Eads/216
  • Section 2. Spring 2024, Online

Course Description

Deep learning is a group of exciting new technologies for neural networks. Through a combination of advanced training techniques and neural network architectural components, it is now possible to create neural networks that can handle tabular data, images, text, and audio as both input and output. Deep learning allows a neural network to learn hierarchies of information in a way that is like the function of the human brain. This course will introduce the student to classic neural network structures, Convolution Neural Networks (CNN), Long Short-Term Memory (LSTM), Gated Recurrent Neural Networks (GRU), General Adversarial Networks (GAN) and reinforcement learning. Application of these architectures to computer vision, time series, security, natural language processing (NLP), and data generation will be covered. High Performance Computing (HPC) aspects will demonstrate how deep learning can be leveraged both on graphical processing units (GPUs), as well as grids. Focus is primarily upon the application of deep learning to problems, with some introduction to mathematical foundations. Students will use the Python programming language to implement deep learning using PyTorch. It is not necessary to know Python prior to this course; however, familiarity of at least one programming language is assumed. This course will be delivered in a hybrid format that includes both classroom and online instruction.

Objectives

  1. Explain how neural networks (deep and otherwise) compare to other machine learning models.
  2. Determine when a deep neural network would be a good choice for a particular problem.
  3. Demonstrate your understanding of the material through a final project uploaded to GitHub.

Syllabus

This syllabus presents the expected class schedule, due dates, and reading assignments. Download current syllabus.

Module Content
Module 1
Meet on 01/22/2024
Module 1: Python Preliminaries
  • Part 1.1: Course Overview
  • Part 1.2: Introduction to Python
  • Part 1.3: Python Lists, Dictionaries, Sets & JSON
  • Part 1.4: File Handling
  • Part 1.5: Functions, Lambdas, and Map/ReducePython Preliminaries
  • We will meet on campus this week! (first meeting)
Module 2
Week of 01/29/2024
Module 2: Python for Machine Learning
  • Part 2.1: Introduction to Pandas for Deep Learning
  • Part 2.2: Encoding Categorical Values in Pandas
  • Part 2.3: Grouping, Sorting, and Shuffling
  • Part 2.4: Using Apply and Map in Pandas
  • Part 2.5: Feature Engineering in Padas
  • Module 1 Program due: 01/30/2024
  • Icebreaker due: 01/30/2024
Module 3
Week of 02/05/2024
Module 3: PyTorch for Neural Networks
  • Part 3.1: Deep Learning and Neural Network Introduction
  • Part 3.2: Introduction to PyTorch
  • Part 3.3: Encoding a Feature Vector for PyTorch Deep Learning
  • Part 3.4: Early Stopping and Network Persistence
  • Part 3.5: Sequences vs Classes in PyTorch
  • Module 2: Program due: 02/06/2024
Module 4
Meet on 02/12/2024
Module 4: Training for Tabular Data
  • Part 4.1: Using K-Fold Cross-validation with PyTorch
  • Part 4.2: Training Schedules for PyTorch
  • Part 4.3: Dropout Regularization
  • Part 4.4: Batch Normalization
  • Part 4.5: RAPIDS for Tabular Data
  • Module 3 Program due: 02/13/2024
  • We will meet on campus this week! (second meeting)
Module 5
Week of 02/19/2024
Module 5: CNN and Computer Vision
  • 5.1 Image Processing in Python
  • 5.2 Using Convolutional Neural Networks
  • 5.3 Using Pretrained Neural Networks
  • 5.4 Looking at Generators and Image Augmentation
  • 5.5 Recognizing Multiple Images with YOLO
  • Module 4 Program due: 02/20/2024
Module 6
Week of 02/26/2024
Module 6: ChatGPT and Large Language Models
  • 6.1: Introduction to Transformers
  • 6.2: Accessing the ChatGPT API
  • 6.3: LLM Memory
  • 6.4: Introduction to Embeddings
  • 6.5: Prompt Engineering
  • Module 5 Program due: 02/27/2024
Module 7
Week of 03/04/2024
Module 7: Image Generative Models
  • 7.1: Introduction to Generative AI
  • 7.2: Generating Faces with StyleGAN3
  • 7.3: GANS to Enhance Old Photographs Deoldify
  • 7.4: Text to Images with StableDiffusion
  • 7.5: Finetuning with Dreambooth
  • Module 6 Assignment due: 03/05/2024
Module 8
Week of 03/18/2024
Module 8: Kaggle
  • 8.1 Introduction to Kaggle
  • 8.2 Building Ensembles with Scikit-Learn and PyTorch
  • 8.3 How Should you Architect Your PyTorch Neural Network: Hyperparameters
  • 8.4 Bayesian Hyperparameter Optimization for PyTorch
  • 8.5 Current Semester's Kaggle
  • Module 7 Assignment due: 03/19/2024
Module 9
Meet on 03/25/2024
Module 9: Facial Recognition
  • 9.1 Detecting Faces in an Image
  • 9.2 Detecting Facial Features
  • 9.3 Image Augmentation
  • 9.4 Application: Emotion Detection
  • 9.5 Application: Blink Efficiency
  • Module 8 Assignment due: 03/26/2024
  • We will meet on campus this week! (third meeting)
Module 10
Week of 04/01/2024
Module 10: Time Series in PyTorch
  • Time Series Data Encoding for Deep Learning, PyTorch
  • Seasonality and Trend
  • LSTM-Based Time Series with PyTorch
  • CNN-Based Time Series with PyTorch
  • Predicting with Meta Prophet
  • Module 9 Assignment due: 04/02/2024
Module 11
Week of 04/08/2024
Module 11: Natural Language Processing
  • 11.1 Introduction to Natural Language Processing
  • 11.2 Hugging Face Introduction
  • 11.3 Hugging Face Tokenizers
  • 11.4 Hugging Face Data Sets
  • 11.5 Training a Model in Hugging Face
  • Module 10 Assignment due: 04/09/2024
Module 12
Week of 04/15/2024
Module 12: Reinforcement Learning
  • Kaggle Assignment due: 04/15/2024 (approx 4-6PM, due to Kaggle GMT timezone)
  • Introduction to Gymnasium
  • Introduction to Q-Learning
  • Stable Baselines Q-Learning
  • Atari Games with Stable Baselines Neural Networks
  • Future of Reinforcement Learning
Module 13
Meet on 04/22/2024
Module 13: Deployment and Monitoring
  • Part 13.1: Using Denoising AutoEncoders
  • Part 13.2: Anomaly Detection
  • Part 13.3: Model Drift and Retraining
  • Part 13.4: Tensor Processing Units (TPUs)
  • Part 13.5: Future Directions in Artificial Intelligence
  • We will meet on campus this week! (fourth meeting)
  • Final project due: 04/29/2024

Datasets

More Repositories

1

t81_558_deep_learning

T81-558: Keras - Applications of Deep Neural Networks @Washington University in St. Louis
Jupyter Notebook
5,720
star
2

aifh

Artificial Intelligence for Humans
Java
927
star
3

encog-java-core

Java
742
star
4

encog-dotnet-core

C#
429
star
5

jh-kaggle-util

Jeff Heaton's Kaggle Utilities
Python
278
star
6

encog-javascript

Encog for Javascript.
JavaScript
196
star
7

present

Code from Jeff Heaton's YouTube videos, articles, and conference presentations.
Assembly
177
star
8

encog-java-examples

Java
162
star
9

encog-c

The Encog project for C/C++
C
112
star
10

pyimgdata

Python
110
star
11

jeffheaton-book-code

Source code from my older (pre Artificial Intelligence for Humans books) books. I am no longer updating these older editions.
Java
90
star
12

app_generative_ai

T81-559: Applications of Generative Artificial Intelligence
Jupyter Notebook
85
star
13

mergelife

Evolve complex cellular automata with a genetic algorithm.
Python
75
star
14

encog-java-workbench

Java
60
star
15

pretrained-gan-70s-scifi

Pretrained model 1024x1024 trained on 1970s scifi art
Jupyter Notebook
51
star
16

pretrained-gan-minecraft

Minecraft GAN
Jupyter Notebook
44
star
17

encog-dotnet-more-examples

This project will contain additional examples for Encog, beyond the console examples provided with Encog. These examples are primarily Winforms GUI applications, and may make use of third party libraries other than Encog.
C#
43
star
18

docker-stylegan2-ada

My Docker image for running Stylegan2 ADA with GPU
Dockerfile
39
star
19

papers

Repository to hold source code related to academic papers I've published.
Python
30
star
20

encog-sample-csharp

A sample application for Encog C#.
C#
28
star
21

pretrained-gan-fish

Pretrained model for fish, 256x256
Jupyter Notebook
25
star
22

article-code

Python
23
star
23

stylegan2-toys

Various projects that I've worked on for special effects in StyleGAN2.
Jupyter Notebook
18
star
24

encog-sample-java

Sample stand-alone Encog project
Java
15
star
25

libsvm-java

This is simply a repository to hold the latest libsvm Java, and allow me to track changes. This is not my own project. See homepage URL for source.
14
star
26

pretrained-merry-gan-mas

Jupyter Notebook
13
star
27

docker-jupyter-python-r

Jupyter notebook with Python and R.
Jupyter Notebook
12
star
28

ios_video_classify

A simple IOS application that uses mobilenet to classify 1000 different images from an IOS device's video camera.
Swift
11
star
29

proben1

A copy of the datasets for PROBEN1 from the paper "Proben1: A Set of Neural Network Benchmark Problems and Benchmarking Rules", Lutz Prechelt
Perl
11
star
30

phd-dissertation

Dissertation (Jeff Heaton)
Java
10
star
31

jeffheaton.github.io

The AIFH website.
HTML
10
star
32

cabi_genai_automation

Introduction to Automation with LangChain, Generative AI, and Python
Jupyter Notebook
10
star
33

apollo64

ËœApollo64 BBS: This is an old Commodore 64 based BBS that I created on back in the late 80's. Basic/6510 assembler.
Assembly
10
star
34

pretrained-gan-tech

Technology GAN
Jupyter Notebook
9
star
35

mergelife-experiments

Data for several runs of MergeLife
9
star
36

jeffheaton

8
star
37

ga-csharp

A simple C# Genetic Algorithm
C#
8
star
38

pysamppackage

A sample package for Python to learn how to structure a package.
8
star
39

tf-intro

Jupyter Notebook
7
star
40

encog-silverlight-core

Encog for Silverlight is no longer supported or maintained. Version 3.0 was the last released version.
6
star
41

jlatexmath-example

A simple example of using JLatexMath
Java
6
star
42

stylegan2-cloud

Utilities to run StyleGAN2 ADA Pytorch in the cloud
5
star
43

docker-stylegan3

Docker image to run StyleGAN3 with GPU
Dockerfile
4
star
44

jna-example

Simple Java JNA example with a Maven build script.
Java
3
star
45

jeffheaton-bookcode

Source code from books published by Jeff Heaton
3
star
46

data-mirror

A mirror of some of the files at data.heatonresearch.com
HTML
2
star
47

data

Some common datasets with headers added and properly setup for Pandas/others
2
star
48

jheaton_images

Images that I use for various sites, like Kaggle
2
star
49

baseenv-jupyter

Basic Docker environment for Jupyter and Python
Dockerfile
2
star
50

docker-jupyterhub

My Jupyterhub docker image
2
star
51

dynaface

Dynaface is both a computer application and Python library to measure facial symmetry utilizing advanced AI for both symmetric and asymmetric faces.
Python
1
star
52

kaggle-otto-group

Jeff Heaton's Entry for Kaggle Otto Group Product Classification Challenge
1
star
53

jheaton-ds2

Some of my datasets (public)
1
star
54

docker-mergelife

Docker image for MergeLife
Dockerfile
1
star