brandontrabucco/scaled_dot_product_attention brandontrabucco/scaled_dot_product_attention

  • Stars
    star
    3
  • Rank 3,963,521 (Top 79 %)
  • Language
    Python
  • License
    MIT License
  • Created over 5 years ago
  • Updated over 5 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
brandontrabucco/scaled_dot_product_attention
github

brandontrabucco

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Implements the scaled dot product attention mechamism from the transformer. Vaswani, A. et al. https://arxiv.org/pdf/1706.03762.pdf

More Repositories

1

da-fusion

Effective Data Augmentation With Diffusion Models
Python
193
star
2

design-bench

Benchmarks for Model-Based Optimization
Python
80
star
3

design-baselines

Baselines for Model-Based Optimization
Python
49
star
4

lstm-cuda

This is a c++ implementation of an LSTM Neural Network parallelized for a GPU using CUDA
Cuda
22
star
5

vq-vae-2

Vector Quantized Latent Variable Models In TF 2.0
Python
9
star
6

nerf

Neural Radiance Fields in PyTorch
Python
8
star
7

program-gan

We propose an architecture using the recent Generative Adversarial Networks and Hyper Networks to generate syntactically correct and behaviorally useful python source code from erroneous existing code.
Python
7
star
8

mass

A Simple Approach For Visual Room Rearrangement: 3D Mapping & Semantic Search (ICLR 2023)
Python
5
star
9

transformer_capsule_layer

Implements a convolutional capsule layer with a scaled dot product attention mechanism.
Python
4
star
10

detailed_captioning

Implements state-of-the-art image captioning algorithms, including the bottom-up top-down attention paper.
Python
4
star
11

pixelcnn

Conditional Gated Pixel CNN in TensorFlow 2
Python
4
star
12

lstm-python

This is an implementation of the LSTM Neural Network algorithm using Python 3.6.1 and NumPy.
Python
3
star
13

mineral

A minimalist reinforcement learning package for TensorFlow 2.0
Python
3
star
14

bvn

Birkhoff-von Neumann Decomposition with a Greedy Birkhoff Heuristic
Python
3
star
15

constrained_beam_search

A tensorflow implementation of constrained beam search that supports arbitrary constraint definitions.
Python
3
star
16

image_caption_machine

A simple ros package for captioning images streamed from a camera.
Python
3
star
17

cider

A pure python cider scorer using numpy, see: https://arxiv.org/abs/1411.5726.
Python
2
star
18

morphing-agents

Collection Of Dynamic Morphology Agents For MuJoCo
Python
2
star
19

playground

Deep Reinforcement Learning
Python
2
star
20

glove

An implementation of a loader for the GloVe word embeddings.
Python
2
star
21

spork

CLI For Experiments On Slurm / Singularity
Python
2
star
22

deepfashion_dataset

TensorFlow build script for the deep fashion dataset, depends on glove embeddings.
Python
2
star
23

hypercomplex-nn

A neural network implemented using a hypercomplex utility library for tensorflow.
Python
2
star
24

off-policy

Off-Policy Reinforcement Learning Algorithms
Python
2
star
25

adversarial_attack

This is an adversarial attack against the inception v3 network on an ImageNet example.
Jupyter Notebook
2
star
26

efficient-hrl

This is a fork of tensorflow/models/research/efficient-hrl with some changes.
Python
2
star
27

hypercomplex-ops

Utility functions for TensorFlow that enable working with differentiable hypercomplex numbers.
C++
2
star
28

best_first

Non-Sequential Decoding Strategies for Image Captioning
Python
1
star
29

autodiff

A simple package for automatically calculating high order jacobian tensors.
Python
1
star
30

skip_cell

A tensorflow implementation of the RNN cell I invented during my internship at Teuscher Lab at Portland State University in 2016.
Python
1
star
31

batch_attention_test

This is a prototype for the attention mechanism used in my research with image captioning.
Jupyter Notebook
1
star
32

gesture

A Java based gesture recognition app that uses a Convolutional LSTM Neural Network
Java
1
star
33

probabilistic_vq

Implements probabilistic vector quantization for training discrete VAEs
Jupyter Notebook
1
star
34

separable_attention

Implements a multi head attention mechanism for images.
Python
1
star
35

im2txt_match

This is an augmentation of the Show and Tell model trained with an image caption pairwise discriminator.
Python
1
star
36

up_down_cell

Implements the bottom-up top-down attention mechanism from Anderson, Peter, et al. https://arxiv.org/abs/1707.07998
Python
1
star
37

wikipedia_dataset

This is a repository using the Wiki Extractor to build and prepare WIKIPEDIA for use in tensorflow.
Python
1
star
38

webgen

A collection of tools for generating synthetic web data.
Python
1
star