Algomancer/Low-Rank-Extension-Token-Linear Algomancer/Low-Rank-Extension-Token-Linear

  • Stars
    star
    3
  • Rank 3,963,521 (Top 79 %)
  • Language
    Python
  • Created 25 days ago
  • Updated 25 days ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
Algomancer/Low-Rank-Extension-Token-Linear
github

Algomancer

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

This linear layer allows dynamic reallocation of parameters post training (tokenformer). Additionally, it creates a low rank transformation effectively doubling the amount of parameter tokens you can attend to.

More Repositories

1

Bayesian-Flow-Networks

A simple implimentation of Bayesian Flow Networks (BFN)
Jupyter Notebook
234
star
2

Full-Stack-Scala-Starter

Play 2.5, ScalaJS, Binding.scala starter project.
Scala
197
star
3

The-Daily-Train

Training Models Daily
Python
16
star
4

Code-Network-Starter

Seed Projects for Hackathons, Resources for rapid prototyping, tutorials
16
star
5

VCReg

Minimal Implimentation of VCRec (2024) for collapse provention.
Python
3
star
6

Hyperspherical-Discrete-Bayesian-Flow-Like-Vibe-But-also-kinda-tangental-discrete-diffusion

It's a hyperspherical thingy for discrete diffusionishness.
Python
2
star
7

Stablizing-Transformers

Stablizing transformer training by preventing attention entropy collapse
Python
2
star
8

Energy-Sliced-Wasserstein

Computes the Energy Sliced Wasserstein Loss between two distributions. An optimal-transporty-energyish vibe distribution matching loss/regulariser.
Python
2
star