tanaymeh/fim-pretraining tanaymeh/fim-pretraining

  • Stars
    star
    4
  • Rank 3,304,323 (Top 66 %)
  • Language
    Python
  • License
    MIT License
  • Created 12 months ago
  • Updated 12 months ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
tanaymeh/fim-pretraining
github

tanaymeh

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Fill-in-the-Middle Pre-training of Large Language Models in pure PyTorch!

More Repositories

1

mamba-train

A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM
Python
46
star
2

nadl

A small framework that can perform automatic differentiation to calculate first-order gradients of numpy arrays.
Python
18
star
3

alien-signal-detection

Detecting Extra-terrestrial signals with the help of Patch-based Image Classification Deep Learning models.
Python
16
star
4

gpt-search

Use GPT-3 and GPT-3.5 to query your documents!
Python
9
star
5

leaf-disease-detection

Deep Learning Model I made using PyTorch for Cassava Leaf Disease Classification
Python
7
star
6

gpt-pre-training-scratch

Basic GPT trained on Harry Potter books
Python
6
star
7

text-complexity-identification

Predicting complexity of a given text excerpt using PyTorch and NLP
Python
4
star
8

cryzig-blog

Blog engine made in Python using FastAPI and SQL Alchemy as db.
CSS
3
star
9

credit-fraud-scratch-tf

In this Mini-Project I am attempting to solve one of the most classic problems of Credit-Card Fraud Detection using Core Tensorflow (Basic Math and Optimization functions, no Higher Level APIs). Do give it a star if you find it helpful!
Jupyter Notebook
3
star
10

convnext-semantic-segmentation

Setting up and running Semantic Segmentation for ConvNext model
Python
2
star
11

math_mamba

Scripts to fine-tune the Mamba LLM on OpenMathInstruct-1 dataset by NVIDIA to improve maths and reasoning skills.
Python
2
star
12

llm-transpiler

Transpile code from language A to B using LLM agents powered by Langgraph
Python
2
star
13

implementations

This repository houses my implementations of different papers and interesting architectures.
Python
1
star
14

enigma-torch

A small framework written in pure PyTorch that can simplify the process of training Image and Text based models.
Python
1
star
15

depression-sentiment-prediction

Depression Sentiment Prediction Project
Jupyter Notebook
1
star
16

miniGPT

My PyTorch Implementation of a Mini-GPT that would be able to generate text on itself.
Python
1
star
17

jet-data-compression

This repository contains the Stacked Auto Encoder made in order to compress Hadron Jet Data from 4->3 Variables. Dataset provided by CERN
Jupyter Notebook
1
star
18

catheter-tube-detection

My work on classifying the presence and correct placement of catheter tubes on chest x-rays to save lives.
Python
1
star
19

flax-vision

Computer Vision Models from PyTorch, Tensorflow now for Flax!
Python
1
star
20

sentiment-analysis-ucity

This is my final Solution for the First Project in Udacity's MLE Nanodegree program - Sentiment Analysis model. (Only for my Submission purpose)
HTML
1
star
21

meta-kaggle-api

Python
1
star
22

melanoma-cancer-detection

Detecting Melanoma Cancer using Deep Learning with largely imbalanced 108 GB data!
Jupyter Notebook
1
star
23

nadl-new

New NADL Version built while keeping Accelerated Computing and Advanced Type-Checking in Focus.
Python
1
star
24

lancedb-haystack

An integration of LanceDB vector database backend with Haystack
Python
1
star