RulinShao/LightSeq

Stars
163
Rank 229,870 (Top 5 %)
Language
Python
Created 12 months ago
Updated 3 months ago

RulinShao/LightSeq

RulinShao

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers

on-the-adversarial-robustness-of-visual-transformer

Code for the paper "On the Adversarial Robustness of Visual Transformers"

FastCkpt

Python package for rematerialization-aware gradient checkpointing

VL-Instruct

Codes for vision-language instruction tuning. Currently support BLIP2-t5 and BLIP2-vicuna.

Digital-Image-Processing

Llava-doremi

Automatic task-balancing for vision-language instruction tuning using group distributionally robust optimization (Group DRO, the technique used in Doremi)