CodeTrans

CodeTrans is providing state of the art pre-trained models for source code. CodeTrans was trained on several Nvidia RTX 8000 GPUs and couple of Google TPUs using various State of the Art Transformers Models.

Take a look into our paper CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing for more information about our work.

This repository will be updated regulary with new pre-trained models for source code as part of supporting software engineering community in general, and Source Code for Covid-19 research specifically.

Table of Contents

• ⌛️  Models Availability

• 🚀  Usage

  • 🤖  Feature Extraction (FE)
  • 💥  Fine Tuning (FT)
  • 🧠  Prediction
  • ⚗️  Source Code Generation
  • 🧐  Visualization
  • 📈  Benchmark

• 📊  Expected Results

  • 💻  Function Documentation Generation
  • 💻  Source Code Summarization
  • 💻  Code Comment Generation
  • 💻  Commit Message Generation
  • 💻  API Sequence Recommendation
  • 💻  Programming Language and Synthesis

• ❤️  Community and Contributions

• 📫  Have a question?

• 🤝  Found a bug?

• ✅  Requirements

• 🤵  Team

• 💰  Sponsors

• 📘  License

• ✏️  Citation

⌛️  Models Availability

All CodeTrans original Tensorflow checkpoints are downloadable from this dropbox folder and the pytorch checkpoints in the Hugging Face model hub.

You can download all the datasets used in this research from dropbox folder.

🚀  Usage

How to use CodeTrans:

• 🤖  Feature Extraction (FE):
  coming soon.

• 💥  Fine Tuning (FT):
  coming soon.

• 🧠  Prediction:
  Please check: Prediction Section. More information coming soon.

• ⚗️  Code Sequences Generation:
  coming soon.

• 🧐  Visualization:
  coming soon.

• 📈  Benchmark:
  coming soon.

📊  Expected Results

• 💻  Function Documentation Generation (Bleu):
  

• 💻  Source Code Summarization (Bleu):
  

• 💻  Code Comment Generation (Bleu):
  

• 💻  Commit Message Generation (Bleu):
  

• 💻  API Sequence Recommendation (Bleu):
  

• 💻  Programming Language and Synthesis (Accuracy):
  

❤️  Community and Contributions

The CodeTrans project is a open source project supported by various partner companies and research institutions. We are committed to share all our pre-trained models and knowledge. We are more than happy if you could help us on sharing new ptrained models, fixing bugs, proposing new feature, improving our documentation, spreading the word, or support our project.

📫  Have a question?

We are happy to hear your question in our issues page CodeTrans! Obviously if you have a private question or want to cooperate with us, you can always reach out to us directly via our RostLab email

🤝  Found a bug?

Feel free to file a new issue with a respective title and description on the the CodeTrans repository. If you already found a solution to your problem, we would love to review your pull request!.

✅  Requirements

For prediction, Text to Text libraray is needed. For source code feature extraction or fine-tuning our pre-trained models, Pytorch and Transformers library from huggingface is needed. For model visualization, you need to install BertViz library.

🤵  Team

• Technical University of Munich:
  

Ahmed Elnaggar	Wei Ding	Florian Matthes	Burkhard Rost

• Google:
  

Llion Jones

• Nvidia:
  

Tom Gibbs	Tamas Feher	Christoph Angerer

💰  Sponsors

Google	Google	Nvidia	Software Campus

📘  License

The CodeTrans pretrained models are released under the under terms of the MIT License.

✏️  Citation

If you use this code or our pretrained models for your publication, please cite the original paper:

@misc{elnaggar2021codetrans,
      title={CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing}, 
      author={Ahmed Elnaggar and Wei Ding and Llion Jones and Tom Gibbs and Tamas Feher and Christoph Angerer and Silvia Severini and Florian Matthes and Burkhard Rost},
      year={2021},
      eprint={2104.02443},
      archivePrefix={arXiv},
      primaryClass={cs.SE}
}