Top Rating
- Top Contributors
  Discover the Top Open Source contributors by country or by language
- Interviews
  Discover real stories from Open Source developers
Discover

Discover your Favorite Language
Discover the top trending repositories and projects on Github. Explore the latest trends in your preferred languages.

HTML

PHP

F#

Go

Shell

Python

C++

Scala

More Languages
Awesome

Awesome repositories
Discover the most awesome repositories and projects of your favorite languages. Inspired by the Awesome-* lists trend in GitHub.

Kotlin

C++

Go

Elm

Erlang

Ruby

Clojure

C#

More Languages
By Country

Rankings by Country
Discover the community of talented open source contributors in each country.

🇪🇨 Ecuador

🇬🇷 Greece

🇿🇼 Zimbabwe

🇨🇳 China

🇦🇪 United Arab Emirates

🇦🇹 Austria

🇩🇰 Denmark

🇦🇿 Azerbaijan

All Countries Compare Countries

catid/dora

Stars
276
Rank 149,319 (Top 3 %)
Language
Python
License
MIT License
Created 9 months ago
Updated 5 months ago

catid/dora

catid

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Implementation of DoRA

DoRA

Implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation" (Liu et al, 2024) https://arxiv.org/pdf/2402.09353.pdf

Demo

Install conda: https://docs.conda.io/projects/miniconda/en/latest/index.html

git clone https://github.com/catid/dora.git
cd dora

conda create -n dora python=3.10 -y && conda activate dora

pip install -U -r requirements.txt

python dora.py

Output

(dora) ➜  dora git:(main) python dora.py
Total Parameters: 11
Trainable Parameters: 11
Final Evaluation Loss: 0.13415579497814178
Total Parameters: 65
Trainable Parameters: 54
Continuing training with DoRA layers...
Final (DoRA) Evaluation Loss: 0.06080308556556702

supercharger

Supercharge Open-Source AI Models

Zpng

Better lossless compression than PNG with a simpler algorithm

wirehair

Wirehair : O(N) Fountain Code for Large Data

self-discover

Implementation of Google's SELF-DISCOVER

WLANOptimizer

Single-header C library that fixes WiFi performance issues for online gaming and other low-latency real-time network traffic.

longhair

Longhair : O(N^2) Cauchy Reed-Solomon Block Erasure Code for Small Data

leopard

Leopard-RS : O(N Log N) MDS Reed-Solomon Block Erasure Code for Large Data

shorthair

Shorthair : Generational Block Streaming Erasure Codes

TimeSync

TimeSync: Time Synchronization Library in Portable C++

cm256

Fast GF(256) Cauchy MDS Block Erasure Codec in C

tonk

Tonk : Reliable UDP (rUDP) Network Library and Infinite Window Erasure Code

Zdepth

Zdepth :: Streaming Depth Compressor in C++ for Azure Kinect DK

xrcap

Azure Kinect multi-camera secure network capture/record/replay

snowshoe

Snowshoe - Portable, Secure, Fast Elliptic Curve Math Library in C

XRmonitors

XRmonitors : User-Friendly Virtual Multi-Monitors for the Workplace

tabby

Tabby - Strong, Fast, and Portable Cryptographic Signatures, Handshakes, and Password Authentication

gf256

GF256 - Fast 8-bit Galois Field Math in C

kvm

Low-Bandwidth IP KVM using Raspberry Pi 4

siamese

Siamese : Infinite-Window Streaming Erasure Code (HARQ)

bitnet_cpu

Experiments with BitNet inference on CPU

ZdepthLossy

Lossy version of Zdepth using video encoders

fecal

FEC-AL : O(N^2) Fountain Code for Small Data

CauchyCaterpillar

Cauchy Caterpillar : O(N^2) Short-Window Streaming Erasure Code

aiwebcam2

Second attempt at AI webcam, this time with OpenAI API

upsampling

Image Upsampling with PyTorch

calico

Calico - Strong, Fast, and Portable Authenticated Encryption

cymric

Cymric - Portable secure random number generator

loraftp

File transfer between two Raspberry Pis using the LoRa Pi HAT from Waveshare

mau

Network simulator for reliable UDP testing in C++

oaillama3

Simple setup to self-host LLaMA3-70B model with an OpenAI API

lllm

Latent Large Language Models

PacketAllocator

C++ Memory allocator for packet queues that free() in roughly the same order that they alloc().

spectral_ssm

Implementation of Spectral State Space Models

libcat

Common code library

AutoAudiobook

Automatically create an audiobook using OpenAI

minigpt4

MiniGPT-4 :: Updated to Torch 2.0, simple setup, easier API, cut out training code

sdxl

SDXL GPU cluster scripts

dataloader

High-performance tokenized language data-loader for Python C++ extension

fp61

Experiment: Fast finite field Fp=2^61-1 in C++

cuda_float_compress

Python package for compressing floating-point PyTorch tensors

phind

Locally hosted: 60% HumanEval

rtmp_receiver

Simple unidirectional RTMP video stream receiver

counter

C++ wrapper for counters that can roll-over (e.g. timestamps/ack-ids)

z16

16-bit monochrome image compressor based on Zstd

AQLM

llamanal.cpp

Static code analysis for C++ projects using llama.cpp and the best LLM you can run offline without an expensive GPU.

unfiltered-diffusers

Simple fork that disables NSFW filter

hloc

boss-balloon

libcatid

Automatically exported from code.google.com/p/libcatid

voron

Voron 3D Printer files

halide-test

Test v14/v15 performance regression

logger

Feature-rich portable C++ logging subsystem in 650 lines of code

cifar10deepspeed

Using DeepSpeed and Nvidia DALI to train various models to solve CIFAR-10

chainlit-anthropic

Chainlit AI UI with Anthropic Backend

fastest_gf_matrix_mult

A fairly hacked together piece of code that can quickly search for the optimal GF polynomials and Cauchy matrices for XOR-based GF matrix multiplication for erasure code encoders. May be useful if the matrices are of fixed size.

recfilter-2020-fail

This is a failed attempt to port Recfilter to latest Halide from 2020. Maybe someone else can figure this out?

bentpipe

Simple UDP rebroadcaster

whisper3

Testing out Whisper 3

sphynx

Sphynx - High Performance Network Transport Layer

textworld_llm_benchmark

TextWorld LLM Benchmark

pixel-perfect-sfm

pixel-perfect-sfm with some minor fixes

train_ticket

Gated PRNGs are all you need? Spoilers: No.

CatsChoice

PRNG Parameter Generation

CRC16Recovery

Optimized CRC16 with error recovery in C

Splane

Archived old code - /ban_ids/ is kind of interesting

rust_webgl_demo

Hello World : Rust Web Assembly

blog2022

swe_agent_playground

swe_agent_playground

Exapunks_Solutions

Exapunks Walkthrough Solutions

audio_prediction

Simple audio prediction example with RNN

never_forget

Implementation of Overcoming Catastrophic Forgetting

DependencyInjected

DependencyInjected : Light-weight and powerful Dependency Injection pattern for C++

quicsend

quicsend :: Super-fast Internet-ready file transfer right from Python