Discover src-d/swivel-spark-prep Open Source project

Stars
5
Rank 2,861,937 (Top 57 %)
Language
Scala
License
Apache License 2.0
Created over 7 years ago
Updated over 7 years ago

src-d

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Distributed equivalent of prep.py and fastprep from Swivel using Apache Spark.

awesome-machine-learning-on-source-code

Cool links & research papers related to Machine Learning applied to source code (MLonCode)

6,247

go-git

Project has been moved to: https://github.com/go-git/go-git

4,904

hercules

Gaining advanced insights from Git repository history.

2,613

gitbase

SQL interface to git repositories, written in Go. https://docs.sourced.tech/gitbase

2,063

go-mysql-server

An extensible MySQL server implementation in Go.

1,040

go-kallax

Kallax is a PostgreSQL typesafe ORM for the Go language.

858

kmcuda

Large scale K-means and K-nn implementation on NVIDIA GPU / CUDA

Jupyter Notebook

800

proteus

Generate .proto files from Go source code.

734

wmd-relax

Calculates Word Mover's Distance Insanely Fast

Python

461

enry

A faster file programming language detector

460

datasets

source{d} datasets ("big code") for source code analysis and machine learning on source code

Jupyter Notebook

323

guide

Aiming to be a fully transparent company. All information about source{d} and what it's like to work here.

JavaScript

294

lapjv

Linear Assignmment Problem solver using Jonker-Volgenant algorithm - Python 3 native module.

C++

252

go-license-detector

Reliable project licenses detector.

237

engine-deprecated

[DISCONTINUED] Go to https://github.com/src-d/sourced-ce/

217

go-billy

The missing interface filesystem abstraction for Go

199

sourced-ce

source{d} Community Edition (CE)

188

beanstool

Dependency free beanstalkd admin tool

151

lookout

Assisted code review, running custom code analyzers on pull requests

149

sourced.ml is a library and command line tools to build and apply machine learning models on top of Universal Abstract Syntax Trees

Python

141

reading-club

Paper reading club at source{d}

115

minhashcuda

Weighted MinHash implementation on CUDA (multi-gpu).

C++

114

go-siva

siva - seekable indexed verifiable archiver

jgit-spark-connector

jgit-spark-connector is a library for running scalable data retrieval pipelines that process any number of Git repositories for source code analysis.

Scala

gitbase-web

gitbase web client; source{d} CE comes with a new UI, check it at https://docs.sourced.tech/community-edition/

gemini

Scala

apollo

Python

borges

borges collects and stores Git repositories.

okrs

Objectives & Key Results repository for the source{d} team

go-queue

Queue is a generic interface to abstract the details of implementation of queue systems.

vecino

Vecino is a command line application to discover Git repositories which are similar to the one that the user provides.

Python

jgscm

Jupyter support for Google Cloud Storage

Python

code2vec

MLonCode community effort to implement Learning Distributed Representations of Code (https://arxiv.org/pdf/1803.09473.pdf)

Python

coreos-nvidia

Yet another NVIDIA driver container for Container Linux (aka CoreOS)

Makefile

style-analyzer

Lookout Style Analyzer: fixing code formatting and typos during code reviews

Jupyter Notebook

code-annotation

🐈 Code Annotation Tool

JavaScript

flamingo

Flamingo is a very thin and simple platform-agnostic chat bot framework

blog

source{d} blog

HTML

sparkpickle

Pure Python implementation of reading SequenceFile-s with pickles written by Spark's saveAsPickleFile()

Python

go-errors

Yet another errors package, implementing error handling primitives.

homebrew

Real homebrew!

infrastructure-dockerfiles

Dockerfile-s to build the images which power source{d}'s computing infrastructure.

Dockerfile

conferences

Tracking events, CfPs, abstracts, slides, and all other even related things

tmsc

Python

models

Machine learning models for MLonCode trained using the source{d} stack

terraform-provider-online

Terraform provider for Online.net

modelforge

Python library to share machine learning models easily and reliably.

Python

identity-matching

source{d} extension to match Git signatures to real people.

tensorflow-swivel

C++

seriate

Optimal ordering of elements in a set given their distance matrix.

Python

gitcollector

go-vitess

An automatic filter-branch of Go libraries from the great Vitess project.

rovers

Rovers is a service to retrieve repository URLs from multiple repository hosting providers.

HTML

go-parse-utils

ml-core

source{d} MLonCode foundation - core algorithms and models.

Python

charts

Applications for Kubernetes

Smarty

role2vec

TeX

snippet-ranger

Jupyter Notebook

fsbench

a small tool for benchmarking filesystems

dev-similarity

Jupyter Notebook

go-log

Log is a generic logging library based on logrus

tab-vs-spaces

Jupyter Notebook

ghsync

GitHub API v3 > PostgreSQL

diffcuda

Accelerated bulk diff on GPU

ml-mining

Python

go-billy-siva

A limited go-billy filesystem implementation based on siva.

go-compose-installer

A toolkit to create installers based on docker compose.

github-reminder

A GitHub application to handle deadline reminders in a GitHub idiomatic way.

go-git-fixtures

several git fixtures to run go-git tests

docsrv

docsrv is an app to serve versioned documentation for GitHub projects on demand

go-cli

CLI scaffolding for Go

shell-complete

Python

kubernetes-local-pv-provisioner

Helping you setting up local persistent volumes

engine-analyses

Analyses of open source projects with source{d} Engine

Jupyter Notebook

sourced-ui

source{d} UI

JavaScript

gypogit

[UNMAINTAINED] go-git wrapper for Python

Python

go-borges

treediff

Python

engine-tour

Temporary storage for useful guides for the source{d} engine

Jupyter Notebook

jupyter-spark-docker

Dockerfile with jupyter and scala installed

Dockerfile

imports

git-validate

k8s-pod-headless-service-operator

landing

landing for source{d}

HTML

lookout-terraform-analyzer

This is a lookout analyzer that checks if your PR has been Terraform fmt'ed when submitting it.

Make-based build system for Go projects at source{d}

Shell

framework

[DEPRECATED]

platform-starter

Starter and basic configuration for platform frontend projects.

metadata-retrieval

lookout-sdk

SDK for lookout analyzers

Python

code-completion

autocompletion prototype

Python

siva-java

siva format implemented in Java

Java

design

All things design at source{d}: branding, guidelines, UI assets, media & co.

berserker

Large scale UAST extractor [DEPRECATED]

Shell

combustion

tm-experiments

Topic Modeling Experiments on Source Code

Python

go-YouTokenToMe

lookout-sdk-ml

SDK for ML based Lookout analyzers

Python

go-asdf

Advanced Scientific Data Format reader library in pure Go.

100

google-cloud-dns-healthcheck

src-d/swivel-spark-prep

src-d

Reviews

Repository Details

More Repositories