• Stars
    star
    291
  • Rank 139,596 (Top 3 %)
  • Language
    Jupyter Notebook
  • License
    Apache License 2.0
  • Created about 1 year ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

T81-558: PyTorch - Applications of Deep Neural Networks @Washington University in St. Louis

T81 558:Applications of Deep Neural Networks

Washington University in St. Louis

Instructor: Jeff Heaton

  • Section 1. Spring 2024, Monday, 2:30 PM, Location: Eads/216
  • Section 2. Spring 2024, Online

Course Description

Deep learning is a group of exciting new technologies for neural networks. Through a combination of advanced training techniques and neural network architectural components, it is now possible to create neural networks that can handle tabular data, images, text, and audio as both input and output. Deep learning allows a neural network to learn hierarchies of information in a way that is like the function of the human brain. This course will introduce the student to classic neural network structures, Convolution Neural Networks (CNN), Long Short-Term Memory (LSTM), Gated Recurrent Neural Networks (GRU), General Adversarial Networks (GAN) and reinforcement learning. Application of these architectures to computer vision, time series, security, natural language processing (NLP), and data generation will be covered. High Performance Computing (HPC) aspects will demonstrate how deep learning can be leveraged both on graphical processing units (GPUs), as well as grids. Focus is primarily upon the application of deep learning to problems, with some introduction to mathematical foundations. Students will use the Python programming language to implement deep learning using PyTorch. It is not necessary to know Python prior to this course; however, familiarity of at least one programming language is assumed. This course will be delivered in a hybrid format that includes both classroom and online instruction.

Objectives

  1. Explain how neural networks (deep and otherwise) compare to other machine learning models.
  2. Determine when a deep neural network would be a good choice for a particular problem.
  3. Demonstrate your understanding of the material through a final project uploaded to GitHub.

Syllabus

This syllabus presents the expected class schedule, due dates, and reading assignments. Download current syllabus.

Module Content
Module 1
Meet on 01/22/2024
Module 1: Python Preliminaries
  • Part 1.1: Course Overview
  • Part 1.2: Introduction to Python
  • Part 1.3: Python Lists, Dictionaries, Sets & JSON
  • Part 1.4: File Handling
  • Part 1.5: Functions, Lambdas, and Map/ReducePython Preliminaries
  • We will meet on campus this week! (first meeting)
Module 2
Week of 01/29/2024
Module 2: Python for Machine Learning
  • Part 2.1: Introduction to Pandas for Deep Learning
  • Part 2.2: Encoding Categorical Values in Pandas
  • Part 2.3: Grouping, Sorting, and Shuffling
  • Part 2.4: Using Apply and Map in Pandas
  • Part 2.5: Feature Engineering in Padas
  • Module 1 Program due: 01/30/2024
  • Icebreaker due: 01/30/2024
Module 3
Week of 02/05/2024
Module 3: PyTorch for Neural Networks
  • Part 3.1: Deep Learning and Neural Network Introduction
  • Part 3.2: Introduction to PyTorch
  • Part 3.3: Encoding a Feature Vector for PyTorch Deep Learning
  • Part 3.4: Early Stopping and Network Persistence
  • Part 3.5: Sequences vs Classes in PyTorch
  • Module 2: Program due: 02/06/2024
Module 4
Meet on 02/12/2024
Module 4: Training for Tabular Data
  • Part 4.1: Using K-Fold Cross-validation with PyTorch
  • Part 4.2: Training Schedules for PyTorch
  • Part 4.3: Dropout Regularization
  • Part 4.4: Batch Normalization
  • Part 4.5: RAPIDS for Tabular Data
  • Module 3 Program due: 02/13/2024
  • We will meet on campus this week! (second meeting)
Module 5
Week of 02/19/2024
Module 5: CNN and Computer Vision
  • 5.1 Image Processing in Python
  • 5.2 Using Convolutional Neural Networks
  • 5.3 Using Pretrained Neural Networks
  • 5.4 Looking at Generators and Image Augmentation
  • 5.5 Recognizing Multiple Images with YOLO
  • Module 4 Program due: 02/20/2024
Module 6
Week of 02/26/2024
Module 6: ChatGPT and Large Language Models
  • 6.1: Introduction to Transformers
  • 6.2: Accessing the ChatGPT API
  • 6.3: LLM Memory
  • 6.4: Introduction to Embeddings
  • 6.5: Prompt Engineering
  • Module 5 Program due: 02/27/2024
Module 7
Week of 03/04/2024
Module 7: Image Generative Models
  • 7.1: Introduction to Generative AI
  • 7.2: Generating Faces with StyleGAN3
  • 7.3: GANS to Enhance Old Photographs Deoldify
  • 7.4: Text to Images with StableDiffusion
  • 7.5: Finetuning with Dreambooth
  • Module 6 Assignment due: 03/05/2024
Module 8
Week of 03/18/2024
Module 8: Kaggle
  • 8.1 Introduction to Kaggle
  • 8.2 Building Ensembles with Scikit-Learn and PyTorch
  • 8.3 How Should you Architect Your PyTorch Neural Network: Hyperparameters
  • 8.4 Bayesian Hyperparameter Optimization for PyTorch
  • 8.5 Current Semester's Kaggle
  • Module 7 Assignment due: 03/19/2024
Module 9
Meet on 03/25/2024
Module 9: Facial Recognition
  • 9.1 Detecting Faces in an Image
  • 9.2 Detecting Facial Features
  • 9.3 Image Augmentation
  • 9.4 Application: Emotion Detection
  • 9.5 Application: Blink Efficiency
  • Module 8 Assignment due: 03/26/2024
  • We will meet on campus this week! (third meeting)
Module 10
Week of 04/01/2024
Module 10: Time Series in PyTorch
  • Time Series Data Encoding for Deep Learning, PyTorch
  • Seasonality and Trend
  • LSTM-Based Time Series with PyTorch
  • CNN-Based Time Series with PyTorch
  • Predicting with Meta Prophet
  • Module 9 Assignment due: 04/02/2024
Module 11
Week of 04/08/2024
Module 11: Natural Language Processing
  • 11.1 Introduction to Natural Language Processing
  • 11.2 Hugging Face Introduction
  • 11.3 Hugging Face Tokenizers
  • 11.4 Hugging Face Data Sets
  • 11.5 Training a Model in Hugging Face
  • Module 10 Assignment due: 04/09/2024
Module 12
Week of 04/15/2024
Module 12: Reinforcement Learning
  • Kaggle Assignment due: 04/15/2024 (approx 4-6PM, due to Kaggle GMT timezone)
  • Introduction to Gymnasium
  • Introduction to Q-Learning
  • Stable Baselines Q-Learning
  • Atari Games with Stable Baselines Neural Networks
  • Future of Reinforcement Learning
Module 13
Meet on 04/22/2024
Module 13: Deployment and Monitoring
  • Part 13.1: Using Denoising AutoEncoders
  • Part 13.2: Anomaly Detection
  • Part 13.3: Model Drift and Retraining
  • Part 13.4: Tensor Processing Units (TPUs)
  • Part 13.5: Future Directions in Artificial Intelligence
  • We will meet on campus this week! (fourth meeting)
  • Final project due: 04/29/2024

Datasets

More Repositories

1

t81_558_deep_learning

T81-558: Keras - Applications of Deep Neural Networks @Washington University in St. Louis
Jupyter Notebook
5,671
star
2

aifh

Artificial Intelligence for Humans
Java
909
star
3

encog-java-core

Java
744
star
4

encog-dotnet-core

C#
430
star
5

jh-kaggle-util

Jeff Heaton's Kaggle Utilities
Python
279
star
6

encog-javascript

Encog for Javascript.
JavaScript
197
star
7

present

Code from Jeff Heaton's YouTube videos, articles, and conference presentations.
Assembly
173
star
8

encog-java-examples

Java
163
star
9

encog-c

The Encog project for C/C++
C
112
star
10

pyimgdata

Python
107
star
11

jeffheaton-book-code

Source code from my older (pre Artificial Intelligence for Humans books) books. I am no longer updating these older editions.
Java
88
star
12

mergelife

Evolve complex cellular automata with a genetic algorithm.
Python
73
star
13

encog-java-workbench

Java
60
star
14

pretrained-gan-70s-scifi

Pretrained model 1024x1024 trained on 1970s scifi art
Jupyter Notebook
51
star
15

pretrained-gan-minecraft

Minecraft GAN
Jupyter Notebook
44
star
16

encog-dotnet-more-examples

This project will contain additional examples for Encog, beyond the console examples provided with Encog. These examples are primarily Winforms GUI applications, and may make use of third party libraries other than Encog.
C#
43
star
17

docker-stylegan2-ada

My Docker image for running Stylegan2 ADA with GPU
Dockerfile
39
star
18

papers

Repository to hold source code related to academic papers I've published.
Python
30
star
19

encog-sample-csharp

A sample application for Encog C#.
C#
28
star
20

app_generative_ai

T81-559: Applications of Generative Artificial Intelligence
Jupyter Notebook
27
star
21

pretrained-gan-fish

Pretrained model for fish, 256x256
Jupyter Notebook
24
star
22

article-code

Python
23
star
23

stylegan2-toys

Various projects that I've worked on for special effects in StyleGAN2.
Jupyter Notebook
18
star
24

encog-sample-java

Sample stand-alone Encog project
Java
15
star
25

libsvm-java

This is simply a repository to hold the latest libsvm Java, and allow me to track changes. This is not my own project. See homepage URL for source.
14
star
26

pretrained-merry-gan-mas

Jupyter Notebook
13
star
27

docker-jupyter-python-r

Jupyter notebook with Python and R.
Jupyter Notebook
12
star
28

ios_video_classify

A simple IOS application that uses mobilenet to classify 1000 different images from an IOS device's video camera.
Swift
11
star
29

phd-dissertation

Dissertation (Jeff Heaton)
Java
10
star
30

apollo64

˜Apollo64 BBS: This is an old Commodore 64 based BBS that I created on back in the late 80's. Basic/6510 assembler.
Assembly
10
star
31

jeffheaton.github.io

The AIFH website.
HTML
10
star
32

proben1

A copy of the datasets for PROBEN1 from the paper "Proben1: A Set of Neural Network Benchmark Problems and Benchmarking Rules", Lutz Prechelt
Perl
10
star
33

mergelife-experiments

Data for several runs of MergeLife
9
star
34

pretrained-gan-tech

Technology GAN
Jupyter Notebook
9
star
35

jeffheaton

8
star
36

pysamppackage

A sample package for Python to learn how to structure a package.
8
star
37

tf-intro

Jupyter Notebook
7
star
38

ga-csharp

A simple C# Genetic Algorithm
C#
7
star
39

encog-silverlight-core

Encog for Silverlight is no longer supported or maintained. Version 3.0 was the last released version.
6
star
40

jlatexmath-example

A simple example of using JLatexMath
Java
6
star
41

stylegan2-cloud

Utilities to run StyleGAN2 ADA Pytorch in the cloud
5
star
42

docker-stylegan3

Docker image to run StyleGAN3 with GPU
Dockerfile
3
star
43

jna-example

Simple Java JNA example with a Maven build script.
Java
3
star
44

jeffheaton-bookcode

Source code from books published by Jeff Heaton
3
star
45

data-mirror

A mirror of some of the files at data.heatonresearch.com
HTML
2
star
46

data

Some common datasets with headers added and properly setup for Pandas/others
2
star
47

jheaton_images

Images that I use for various sites, like Kaggle
2
star
48

baseenv-jupyter

Basic Docker environment for Jupyter and Python
Dockerfile
2
star
49

docker-jupyterhub

My Jupyterhub docker image
2
star
50

cabi_genai_automation

Introduction to Automation with LangChain, Generative AI, and Python
Jupyter Notebook
2
star
51

kaggle-otto-group

Jeff Heaton's Entry for Kaggle Otto Group Product Classification Challenge
1
star
52

jheaton-ds2

Some of my datasets (public)
1
star
53

docker-mergelife

Docker image for MergeLife
Dockerfile
1
star