dmarx/anthology-of-modern-ml dmarx/anthology-of-modern-ml

  • Stars
    star
    125
  • Rank 284,672 (Top 6 %)
  • Language
    Jupyter Notebook
  • License
    MIT License
  • Created about 3 years ago
  • Updated over 1 year ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
dmarx/anthology-of-modern-ml
github

dmarx

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Collection of important articles to be treated as a textbook

Anthology of Modern Machine Learning

A curated collection of significant/impactful articles to be treated as a textbook, because sometimes it's just best to go straight to the source. My hope is to provide a reference for understanding important developments in their historical context, so the techniques can be understood in the historical context that motivated them, the problems the authors were attempting to solve, and what particular features of the discovery were considered especially novel or impressive when it was first published.

I plan to have this organized a couple of ways:

  • broad-brush topics (textbook-ish sections)
  • publication date
  • parent-child research developments (co-citations?)

This multiple-organization idea might be more amenable to a wiki structure, in which case I could even add paper summaries and abridged versions.

"Classic" ML

Network Graphs / combinatorial optimization

Geometric Deep Learning and ML applications of group theory/representation theory

Misc optimization and numerical methods

Neural optimizers

Neural activations

Neural initializations

Neural layers

RL

Hyperparameter tuning / Architecture Search

Implicit Representation

Specific architectures/achievements, and other misc milestones

Computer Vision / representation learning

NLP

Representation Learning

Misc

Learning theory / Deep learning theory / model compression / interpretability

Information theory

  • Entropy
  • Fisher information
  • KL divergence

Causal Modeling / experimentation

  • Double machine learning
  • Doubly robust inference
  • Pearl's do calculus and graphical modeling / structural equation modeling
  • Rubin's potential outcomes model
  • model identification
  • d-separation
  • propensity scoring/matching
  • item-response model and adaptive testing
  • bandit learning for on-line experimentation
  • belief propagation

Time series forecasting

Misc Generative Art milestones and techniques

Ethics in ML

  • Data Privacy
    • See Netflix Prize
  • Differential Privacy
  • k-anonymity
  • Dataset bias - gendered words, differential treatment of skin color, race and zipcode in legal applications
  • YOLO author's resignation (blog post + reddit thread)
  • CV techniques used to subjugate minorities in SE Asia and China
  • Ethical issues surrounding classification of behavioral health and interventions
  • Metadata deanonymization and leaks of US domestic data collection programs with corporate participation
  • "fairness" algorithms
  • gerrymandering and algorithmic redistricting
  • Facebook's influence on elections and live-testing to influence people's emotions and behaviors w/o consent

Analytic Process

Misc important papers for generative models/art

  • Classifier-free Guidance (CFG)
  • VQGAN
  • SDEdit
  • Stable Diffusion
  • Cold Diffusion
  • Karras Diffusion Design Space
  • NeRF
  • NeRF + Triplanes
  • Cross Attention guidance
  • Controlnet/T2I adaptors

LLM Stuff

More Repositories

1

psaw

Python Pushshift.io API Wrapper (for comment/submission search)
Python
349
star
2

video-killed-the-radio-star

Notebook and tools for end-to-end automation of music video production with generative AI
Jupyter Notebook
196
star
3

ComfyUI-Keyframed

ComfyUI nodes to facilitate prompt/parameter keyframing
Python
82
star
4

Topological-Anomaly-Detection

Topological Anomaly Detection (TAD) per Gartley and Basener 2009
Python
66
star
5

bench-warmers

DigThatData's Public Brainstorming space
Python
59
star
6

Multi-Modal-Comparators

Unified API to facilitate usage of pre-trained "perceptor" models, a la CLIP
Jupyter Notebook
39
star
7

keyframed

Simple, expressive, pythonic datatypes for manipulating curves parameterized by keyframes and interpolators.
Python
34
star
8

zero-shot-intent-classifier

Minimal zero-shot intent classifier for arbitrary intent slot filling, via LLM prompting w LangChain.
Python
32
star
9

VideoLinkBot

Reddit bot that posts a comment with all of the video links in a submission. Currently only supports YouTube.
Python
28
star
10

notebooks

misc notebooks i wanted to put in tracking
Jupyter Notebook
17
star
11

digthatdata-comfyui-workflows

Sharing some of my secret sauce ;)
14
star
12

Scrape-Subreddit-Users

Utilities for identifying users subscribed to s subreddit, downloading their comment histories, and analyzing their commenting behavior
Python
13
star
13

not-a-package-manager

utilities to facilitate working with codebases that don't ascribe to normal package management paradigms, e.g. ML research code that can be cloned but not installed.
Python
12
star
14

cka_pytorch

Reproducing Raghu et al 2021 - Do Vision Transformers See Like Convolutional Neural Networks?
Jupyter Notebook
10
star
15

the-rest-of-the-fucking-owl

Trigger an LLM in your CI/CD to auto-complete your work
Python
10
star
16

ComfyUI-AudioReactive

porting audioreactivity pipeline from vktrs to comfyui.
Python
10
star
17

TextSummarization

Simple text LexRank text summarization in python
Python
8
star
18

reddit_comment_scraper

Simple python script to download reddit user comments.
Python
8
star
19

sd-lazy-wildcards

sd-webui wildcards filled by LLM, no wordlist files required
Python
6
star
20

awesome-llm-utilities

Tooling and frameworks to support building tools that utilize LLM agents
6
star
21

split_csv

This tool takes a CSV file and splits it into separate files based on the contents of one or several columns. Accompanying scripts add line counts to filenames and convert the output CSV's into formatted .xlsx files (specially formatted for my specific needs, but the code is easily modifiable).
Python
6
star
22

ai-ethics-in-the-wild

some stuff that's worth talking about
6
star
23

anthology-of-ml-for-ai-art

like dmarx/anthology-of-machine-learning, but focused on important papers in AI art
5
star
24

congressional-rollcalls

Database of congressional roll call votes scraped from THOMAS and accompanying code used to build database
Python
5
star
25

inverse-heat-dissipation

[WIP] Implementation of Rissanen et al, "Generative Modelling With Inverse Heat Dissipation" - https://arxiv.org/pdf/2206.13397.pdf
Python
5
star
26

workbench

A simple brainstorming space, powered by the github front-end + github actions.
Python
5
star
27

textbooks-dataset

Curated dataset of open source textbooks
4
star
28

llm-tools

unopinioated misc utilites and templates for building things with LLMs
Python
4
star
29

keyframed_chatgpt

An experiment pair-programming with ChatGPT. Reference the commit-to-commit diffs to see how this developed each step of the way.
HTML
3
star
30

jax-docker

containerized jax environment
Dockerfile
3
star
31

mines

Minesweeper clone in python. Mainly a toy for playing with probability and AI.
Python
3
star
32

ComfyUI-TSP

Animation smoothing via frame permutation using Traveling Salesman optimization
Python
3
star
33

make_for_datascience

Demonstration of how to use Gnu Make effectively in a robust data analytics pipeline
HTML
2
star
34

d3-mines

"High-Dimensional Minesweeper" game in D3
JavaScript
2
star
35

auto-tagger

use an LLM (chatgpt) to automatically categorize documents
Python
2
star
36

TravelingSalesmanMCMC

Playing with a simple traveling salesman problem as an excuse to play with some optimization algorithms I haven't had an opportunity to play with in a while.
Python
2
star
37

Reddit_response_to_Trump

Quantifying the Reddit community's response to US political turmoil following the election of Donald Trump
Jupyter Notebook
2
star
38

discord-comfybot

Simple and powerful Discord bot that uses ComfyUI as a backend
Python
2
star
39

ComfyUI-VKTRS

WIP Suite of interoperating node packs inspired by the Video Killed The Radio Star *Diffusion* notebook
2
star
40

fast-and-simple-dense-retrieval

yet another vector database
Python
1
star
41

owl-test3

Python
1
star
42

star-feed

github keeps killing the "following" feed. time to take matters into my own hands.
Shell
1
star
43

SuicideWatch

Experiment to build a model that detects suicidality in social media activity.
Jupyter Notebook
1
star
44

owl-test

won't it be fun if this works? https://github.com/dmarx/the-rest-of-the-fucking-owl
Python
1
star
45

Target-Shuffling

Demo of target shuffling, aka permutation testing
HTML
1
star
46

enl-supply

Python
1
star
47

whats-in-a-name

[WIP] probing identity and bias in text to image models
Jupyter Notebook
1
star
48

not-an-nft

git is the only blockchain that matters. https://dmarx.github.io/not-an-nft/images.html
TeX
1
star
49

RedditUtilities

A collection of useful tools for scraping reddit
Python
1
star
50

quorum

playing with the idea described [here](https://github.com/dmarx/bench-warmers/blob/main/quorum.md) and [here](https://github.com/dmarx/bench-warmers/blob/main/product_thinking_ai.md)
Python
1
star
51

twitterMonitor

Simple twitter event detection tool
Python
1
star
52

clip-auto-organizer

fuck it, why not - https://github.com/dmarx/bench-warmers/blob/main/clip_auto_organizer.md
Python
1
star
53

siren_unofficial

implementation of Sitzmann et al "Implicit Neural Representations with Periodic Activation Functions" - https://arxiv.org/pdf/2006.09661.pdf
Jupyter Notebook
1
star