• Stars
    star
    8,004
  • Rank 4,366 (Top 0.09 %)
  • Language
    JavaScript
  • License
    MIT License
  • Created almost 14 years ago
  • Updated over 3 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

[UNMAINTAINED] Simple feed-forward neural network in JavaScript

This project has reached the end of its development as a simple neural network library. Feel free to browse the code, but please use other JavaScript neural network libraries in development like brain.js and convnetjs.

brain

brain is a JavaScript neural network library. Here's an example of using it to approximate the XOR function:

var net = new brain.NeuralNetwork();

net.train([{input: [0, 0], output: [0]},
           {input: [0, 1], output: [1]},
           {input: [1, 0], output: [1]},
           {input: [1, 1], output: [0]}]);

var output = net.run([1, 0]);  // [0.987]

There's no reason to use a neural network to figure out XOR however (-: so here's a more involved, realistic example: Demo: training a neural network to recognize color contrast

Using in node

If you have node you can install with npm:

npm install brain

Using in the browser

Download the latest brain.js. Training is computationally expensive, so you should try to train the network offline (or on a Worker) and use the toFunction() or toJSON() options to plug the pre-trained network in to your website.

Training

Use train() to train the network with an array of training data. The network has to be trained with all the data in bulk in one call to train(). The more training patterns, the longer it will probably take to train, but the better the network will be at classifiying new patterns.

Data format

Each training pattern should have an input and an output, both of which can be either an array of numbers from 0 to 1 or a hash of numbers from 0 to 1. For the color constrast demo it looks something like this:

var net = new brain.NeuralNetwork();

net.train([{input: { r: 0.03, g: 0.7, b: 0.5 }, output: { black: 1 }},
           {input: { r: 0.16, g: 0.09, b: 0.2 }, output: { white: 1 }},
           {input: { r: 0.5, g: 0.5, b: 1.0 }, output: { white: 1 }}]);

var output = net.run({ r: 1, g: 0.4, b: 0 });  // { white: 0.99, black: 0.002 }

Options

train() takes a hash of options as its second argument:

net.train(data, {
  errorThresh: 0.005,  // error threshold to reach
  iterations: 20000,   // maximum training iterations
  log: true,           // console.log() progress periodically
  logPeriod: 10,       // number of iterations between logging
  learningRate: 0.3    // learning rate
})

The network will train until the training error has gone below the threshold (default 0.005) or the max number of iterations (default 20000) has been reached, whichever comes first.

By default training won't let you know how its doing until the end, but set log to true to get periodic updates on the current training error of the network. The training error should decrease every time. The updates will be printed to console. If you set log to a function, this function will be called with the updates instead of printing to the console.

The learning rate is a parameter that influences how quickly the network trains. It's a number from 0 to 1. If the learning rate is close to 0 it will take longer to train. If the learning rate is closer to 1 it will train faster but it's in danger of training to a local minimum and performing badly on new data. The default learning rate is 0.3.

Output

The output of train() is a hash of information about how the training went:

{
  error: 0.0039139985510105032,  // training error
  iterations: 406                // training iterations
}

Failing

If the network failed to train, the error will be above the error threshold. This could happen because the training data is too noisy (most likely), the network doesn't have enough hidden layers or nodes to handle the complexity of the data, or it hasn't trained for enough iterations.

If the training error is still something huge like 0.4 after 20000 iterations, it's a good sign that the network can't make sense of the data you're giving it.

JSON

Serialize or load in the state of a trained network with JSON:

var json = net.toJSON();

net.fromJSON(json);

You can also get a custom standalone function from a trained network that acts just like run():

var run = net.toFunction();

var output = run({ r: 1, g: 0.4, b: 0 });

console.log(run.toString()); // copy and paste! no need to import brain.js

Options

NeuralNetwork() takes a hash of options:

var net = new brain.NeuralNetwork({
  hiddenLayers: [4],
  learningRate: 0.6 // global learning rate, useful when training using streams
});

hiddenLayers

Specify the number of hidden layers in the network and the size of each layer. For example, if you want two hidden layers - the first with 3 nodes and the second with 4 nodes, you'd give:

hiddenLayers: [3, 4]

By default brain uses one hidden layer with size proportionate to the size of the input array.

Streams

The network now has a WriteStream. You can train the network by using pipe() to send the training data to the network.

Example

Refer to stream-example.js for an example on how to train the network with a stream.

Initialization

To train the network using a stream you must first create the stream by calling net.createTrainStream() which takes the following options:

  • floodCallback() - the callback function to re-populate the stream. This gets called on every training iteration.
  • doneTrainingCallback(info) - the callback function to execute when the network is done training. The info param will contain a hash of information about how the training went:
{
  error: 0.0039139985510105032,  // training error
  iterations: 406                // training iterations
}

Transform

Use a Transform to coerce the data into the correct format. You might also use a Transform stream to normalize your data on the fly.

More Repositories

1

kittydar

Face detection for cats in JavaScript - demo for TXJS 2012 talk
JavaScript
1,416
star
2

replace

[UNMAINTAINED] Command line search and replace utility
JavaScript
749
star
3

classifier

Bayesian classifier with Redis backend
JavaScript
626
star
4

clustering

[UNMAINTAINED] K-means and hierarchical clustering
JavaScript
483
star
5

nomnom

[UNMAINTAINED] Option parser for node with generated usage and commands
JavaScript
471
star
6

glossary

[UNMAINTAINED] Extract terms and keywords from a piece of text
JavaScript
167
star
7

hog-descriptor

[UNMAINTAINED] Histogram of Oriented Gradients (HOG) descriptor extractor
JavaScript
167
star
8

fxconsole

[UNMAINTAINED] Remote JavaScript console for Firefox
JavaScript
128
star
9

firefox-client

[UNMAINTAINED] Node.js remote debugging client for Firefox
JavaScript
101
star
10

js-select

[UNMAINTAINED] Traverse and modify objects using JSONSelect selectors
JavaScript
56
star
11

costco

UI for bulk editing CouchDB docs
JavaScript
55
star
12

rainbow

Color tools for Firefox
JavaScript
39
star
13

txjs-slides

dzslides deck for my txjs talk
CSS
14
star
14

mac-sounds

Play default OS X sounds from node
JavaScript
10
star
15

bzhome

Bugzilla dashboard
JavaScript
8
star
16

passion-project-slides

Slides for my Passion Projects talk on Machine Learning and JS
JavaScript
7
star
17

detect-indent

CSS
6
star
18

tcm

Mozilla's new testcase manager
JavaScript
5
star
19

searchbugs

Search for Bugzilla bugs by component
JavaScript
5
star
20

celestial-snips-app

Snips voice assistant that answers celestial questions
Python
3
star
21

mini-devtools

mini in-content devtools
JavaScript
3
star
22

test-pages

Testcase websites for devtools
JavaScript
3
star
23

artfulimage

The Artful Image fine printing
3
star
24

devtools-guide

3
star
25

showsearch

Jetpack that shows search terms in Firefox's awesomebar
JavaScript
3
star
26

firefontfamily

A Firebug extension that highlights the rendered font-family
JavaScript
2
star
27

contextfont

Firefox addon to find fonts on websites
JavaScript
2
star
28

bztweaks

mirror of Bugzilla Tweaks Firefox addon (https://bitbucket.org/ehsan/bugzilla-tweaks)
JavaScript
2
star
29

fluent-talk

JavaScript
1
star
30

bugidhelper

Bugzilla bug id linkifier and tooltipifier extension
JavaScript
1
star
31

brooklynjs-slides

Write your own in-content devtools with web APIs
CSS
1
star
32

harth

harth's no.de code
1
star
33

test-snips-calc

Test Snips Calc Action
Python
1
star
34

wwcode

A small website for Women Who Code SF
JavaScript
1
star
35

celestial-jupyter

Jupyter Notebook
1
star
36

test.js

Whatever's in my ~/test.js
JavaScript
1
star