• Stars
    star
    763
  • Rank 58,070 (Top 2 %)
  • Language
    Scala
  • License
    Apache License 2.0
  • Created over 7 years ago
  • Updated about 1 month ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A simple library for creating complex neural networks

DeepLearning.scala ThoughtWorks

Join the chat at https://gitter.im/ThoughtWorksInc/DeepLearning.scala Build Status Latest version Scaladoc

DeepLearning.scala is a simple library for creating complex neural networks from object-oriented and functional programming constructs.

  • DeepLearning.scala runs on JVM, can be used either in standalone JVM applications or a Jupyter Notebooks.
  • DeepLearning.scala is expressive. Various types of neural network layers can be created by composing map, reduce or other higher order functions.
  • DeepLearning.scala supports plugins. There are various plugins providing algorithms, models, hyperparameters or other features.
  • All the above features are statically type checked.

Features

Differentiable programming

Like other deep learning toolkits, DeepLearning.scala allows you to build neural networks from mathematical formulas. It supports floats, doubles, GPU-accelerated N-dimensional arrays, and calculates derivatives of the weights in the formulas.

Dynamic neural networks

Unlike some other deep learning toolkits, the structure of neural networks in DeepLearning.scala is dynamically determined during running. Our neural networks are programs. All Scala features, including functions, expressions and control flows, are available in neural networks.

For example:

def ordinaryScalaFunction(a: INDArray): Boolean = {
  a.signnum.sumT > math.random
}

def myDynamicNeuralNetwork(input: INDArray) = INDArrayLayer(monadic[Do] {
  val outputOfLayer1 = layer1(input).forward.each
  if (ordinaryScalaFunction(outputOfLayer1.data)) {
    dynamicallySelectedLayer2(outputOfLayer1).forward.each
  } else {
    dynamicallySelectedLayer3(outputOfLayer1).forward.each
  }
})

The above neural network will go into different subnetworks according to an ordinary Scala function.

With the ability of creating dynamic neural networks, regular programmers are able to build complex neural networks from simple code. You write code almost as usual, the only difference being that code based on DeepLearning.scala is differentiable, which enables such code to evolve by modifying its parameters continuously.

Functional programming

DeepLearning.scala 2.0 is based on Monads, which are composable, thus a complex layer can be built from primitive operators or higher order functions like map/reduce. Along with the Monad, we provide an Applicative type class, to perform multiple calculations in parallel.

For example, the previous example can be rewritten in higher-order function style as following:

def myDynamicNeuralNetwork(input: INDArray) = INDArrayLayer {
  layer1(input).forward.flatMap { outputOfLayer1 =>
    if (ordinaryScalaFunction(outputOfLayer1.data)) {
      dynamicallySelectedLayer2(outputOfLayer1).forward
    } else {
      dynamicallySelectedLayer3(outputOfLayer1).forward
    }
  }
}

The key construct in DeepLearning.scala 2.0 is the dependent type class DeepLearning, which witnesses a differentiable expression. In other words, given the DeepLearning type class instance, you can activate the deep learning ability of any type.

Object-oriented programming

The code base of DeepLearning.scala 2.0 is organized according to Dependent Object Type calculus (DOT). All features are provided as mixin-able plugins. A plugin is able to change APIs and behaviors of all DeepLearning.scala types. This approach not only resolves expression problem, but also gives plugins the additional ability of virtually depending on other plugins.

For example, when a plugin author is creating the Adagrad optimizer plugin, he does not have to explicitly call functions related to learning rate. However, once a plugin user enables both the Adagrad plugin and the FixedLearningRate plugin, then computation in FixedLearningRate will get called eventually when the Adagrad optimization is executed.

Roadmap

v2.0

Version 2.0 is the current version with all of the above features.

v3.0

  • Support element-wise map/reduce and other higher-order functions on GPU.
  • Support distributed models and distributed training on Spark.

Links

Acknowledgements

DeepLearning.scala is sponsored by ThoughtWorks.

DeepLearning.scala is heavily inspired by my colleague @MarisaKirisame. Originally, we worked together on a prototype of a deep learning framework, and eventually split our work into this project and DeepDarkFantasy. Other contributors can be found at here.

Related projects

  • Shapeless provides a solid foundation for type-level programming used in DeepLearning.scala.
  • Scalaz and Algebra provides type classes used in DeepLearning.scala.
  • ThoughtWorks Each provides async/await-like syntax. You may want to use it to control your training process in an imperative style.
  • nd4j provides numerical computing used in DeepLearning.scala 2.0.
  • Compute.scala provides numerical computing used in DeepLearning.scala 3.0.
  • RAII.scala, future.scala and tryt.scala provides monadic asynchronous resource management used in DeepLearning.scala.
  • Plugins of DeepLearning.scala are based on Factory and other dependent type classes in feature.scala.
  • Import.scala is a Scala compiler plugin that enables magic imports. You may need it in those sbt project use DeepLearning.scala plugin hosted on Github Gist.
  • DeepLearning.scala can run in Jupyter Scala or Ammonite.
  • The unit tests of DeepLearning.scala are written in ScalaTest and example.scala syntax.
  • Some type classes in DeepLearning.scala are created by simulacrum's @typeclass annotation.

More Repositories

1

Binding.scala

Reactive data-binding for Scala
Scala
1,579
star
2

DeepDarkFantasy

A Programming Language for Deep Learning
Haskell
465
star
3

cd4ml-workshop

Repository with sample code and instructions for "Continuous Intelligence" and "Continuous Delivery for Machine Learning: CD4ML" workshops
Jupyter Notebook
316
star
4

Dsl.scala

A framework to create embedded Domain-Specific Languages in Scala
Scala
255
star
5

each

A macro library that converts native imperative syntax to scalaz's monadic expressions
Scala
253
star
6

guia-de-desenvolvimento-tecnico

JavaScript
206
star
7

Compute.scala

Scientific computing with N-dimensional arrays
Scala
199
star
8

CD4ML-Scenarios

Repository with sample code and instructions for "Continuous Intelligence" and "Continuous Delivery for Machine Learning: CD4ML" workshops
Python
138
star
9

microbuilder

A toolset that helps you build system across multiple micro-services and multiple languages.
HTML
93
star
10

sbt-api-mappings

An Sbt plugin that fills apiMappings for common Scala libraries.
Scala
88
star
11

enableIf.scala

A library that toggles Scala code at compile-time, like #if in C/C++
Scala
65
star
12

todo

Binding.scala • TodoMVC
Scala
60
star
13

sinais

🔣 Desenvolvimento passo a passo do exemplo `sinais` em Go.
Go
60
star
14

sbt-best-practice

Configure common build settings for a Scala project
Scala
56
star
15

TWU101-TDDIntro

Java
46
star
16

template.scala

C++ Flavored Template Metaprogramming in Scala
Scala
40
star
17

future.scala

Stack-safe asynchronous programming
Scala
39
star
18

ml-app-template

An ML project template with sensible defaults
Python
37
star
19

sbt-scala-js-map

A Sbt plugin that configures source mapping for Scala.js projects hosted on Github
Scala
36
star
20

aws_role_credentials

Generates AWS credentials for roles using STS
Python
34
star
21

transervicos

Ruby
33
star
22

RAII.scala

Resource Acquisition Is Initialization
Scala
32
star
23

sbt-example

Run Scaladoc as unit tests
Scala
31
star
24

feature.scala

Access Scala language features on the type-level
Scala
31
star
25

sbt-ammonite-classpath

Export the classpath for Ammonite and Almond
Scala
27
star
26

Import.scala

A Scala compiler plugin for magic imports
Scala
26
star
27

JS-Monthly-Chengdu

CSS
23
star
28

infra-problem

resources for the infrastructure as code practical assessment
Clojure
23
star
29

bindable.scala

User-friendly Binding.scala components
Scala
23
star
30

ml-cd-starter-kit

Set up cross-cutting services (e.g. CI server, monitoring) for ML projects using kubernetes and helm
Smarty
23
star
31

implicit-dependent-type

Scala
22
star
32

Extractor.scala

Make PartialFunction and extractors composable
Scala
22
star
33

objective8

For the most up to date version of this project, see https://github.com/d-cent/objective8
Clojure
21
star
34

oktaauth

Module and CLI client to handle Okta authentication
Python
20
star
35

js-test-project

JavaScript
18
star
36

tryt.scala

Monad transformers for exception handling
Scala
18
star
37

lein-s3-static-deploy

Lein task to deploy static website to s3 bucket.
Clojure
17
star
38

DesignPattern.scala

Functional Programming Design Patterns
Scala
17
star
39

TWTraining

Open source ThoughtWorks training materials
HTML
15
star
40

dsl-domains-cats

Scala
12
star
41

Q.scala

Convert any value to code
Scala
12
star
42

ZeroCost.scala

Zero-cost Abstractions in Scala
Scala
10
star
43

Constructor.scala

Mixin classes and traits dynamically
Scala
10
star
44

dataclouds

Blog for dataclouds@thoughtworks.
CSS
10
star
45

tf-image-interpreter

Object detection and text spotting from images of any size. Based on TensorFlow.
Python
10
star
46

WorkingEffectivelyWithLegacyCode

Java
10
star
47

Binding.scala-website

Scala
9
star
48

voter-service

The Voter Spring Boot RESTful Web Service, backed by MongoDB, is used for DevOps-related training and testing.
Java
9
star
49

OpenStack-EC2-Driver

OpenStack-EC2-Driver
9
star
50

stonecutter

[Main repo found at https://github.com/d-cent/stonecutter] A D-CENT project: an easily deployable oauth server for small organisations.
Clojure
9
star
51

streaming-data-pipeline

Streaming pipeline repo for data engineering training program
Scala
9
star
52

JavaBootcamp

Java
8
star
53

java-test-project

Java
7
star
54

infra-code-workshop

TechRadar Academy em PoA - Cloud
7
star
55

twseleniumworkshop

Workshop Selenium Belo Horizonte - Setembro 2014
Java
7
star
56

DeepLearning.scala-website

The website of DeepLearning.scala
CSS
7
star
57

AS101-4-workshop

Python
7
star
58

skadoosh

Here we have the building blocks of a virtual entity in the making (in crude words, a chat bot - but don't call it that. It gets offended).
Python
7
star
59

json-stream-core

Universal Serialization Framework for JSON
Haxe
6
star
60

ScaleWorks_YUMChina

Ruby
6
star
61

lein-filegen

A leiningen plugin to generate files
Clojure
6
star
62

microbuilder-core

Haxe
6
star
63

clj-http-s3

Middleware to allow cli-http to authenticate with s3
Clojure
6
star
64

expend-rs

Internal application to submit certain expenses to ThoughtWorks' system
Rust
6
star
65

LatestEvent.scala

bidirectional data-binding and routing for Scala.js
Scala
6
star
66

sbt-jdeps

an sbt plugin to run JDeps
Scala
5
star
67

modularizer

Scala
5
star
68

twu-toolkit

Calendar generator for TWU
Ruby
5
star
69

twcss

CSS Coding Guidelines
5
star
70

loans-lah-tdd-workshop

JavaScript
5
star
71

Binding.scala-play-template

Scala
5
star
72

TypeOf.scala

Create types from expressions
Scala
4
star
73

SG-ObjectBootcamp

Java
4
star
74

sbt-delombok

an sbt plug-in to delombok Java sources files that contain Lombok annotations
Scala
4
star
75

zeratul

a wrapper for JPA
Java
4
star
76

CSharpTestProject

C#
4
star
77

HashRoute.scala

Scala
4
star
78

Tensor.scala

A totally functional DSL for general purpose GPU programming
4
star
79

akka-http-rpc

Turn akka-http to a RPC server
Scala
4
star
80

monadic-deep-learning

TeX
4
star
81

wxapp-workshop

3
star
82

sde

Scala
3
star
83

clojuregoat

A goat, in Clojure
Clojure
3
star
84

sonic

React UI Components
JavaScript
3
star
85

go-maven-poller

Go plugin that polls Maven (Nexus) repositories
3
star
86

mooncake

A D-CENT project: Secure notifications combined with w3 activity streams
Clojure
3
star
87

Binding.scala-activator-template

Scala
3
star
88

akka-http-webjars

Serve static assets from WebJars
Scala
3
star
89

scala-project-template

3
star
90

offnet

The Unified Neural Network
Jupyter Notebook
3
star
91

aem-training-2016

Code repo for AEM training in August 2016.
Java
3
star
92

hackerbrasileiro

Java
3
star
93

helsinki

[Main repo found at https://github.com/d-cent/decisionsproto] Spike for indexing data from the Open Ahjo API in elasticsearch
Python
3
star
94

FallbackLookupStrategy.java

Java
2
star
95

cep-conference

Core Engineering Practices "Conference" exercise
Java
2
star
96

Cifar10.scala

Scala
2
star
97

Kaleidoscopez

JavaScript
2
star
98

infra-code-devopslabs01

TechRadar Academy - Cloud Workshop - IaC
Python
2
star
99

android-test-project

Kotlin
2
star
100

FunctionalPattern

Scala
2
star