• Stars
    star
    577
  • Rank 77,363 (Top 2 %)
  • Language
    JavaScript
  • License
    Apache License 2.0
  • Created over 8 years ago
  • Updated over 7 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Networked Process Manager to execute functions in a Computer Grid
Build Status

GridControl

GridControl provisions and links multiple servers together to form a Grid.

Files are synchronized, Opinionated Pub/Sub system is implemented, Servers get linked together.

You develop, you play, in a scalable way. The more Servers you add to the Grid, the more calculation power you get.

Welcome to the Grid

5 minutes to get started. By the authors of PM2.

Behind the scenes: GridControl is a network layer built on top of PM2 allowing file synchronization, inter-process communication via an opionated PUB/SUB system and a wide-range system discovery

Features

  • 0 conf Auto discovery system via multiple DNS
  • 0 conf P2P application source sharing
  • Ecosystem Grid management toolbox (CLI, provisioning, Logs, Monitoring)
  • Secure Diffie Hellman key exchange and password authentication
  • Decentralized Each Node can trigger actions executed by another Nodes
  • Fast Grid interconnected via TCP sockets
  • Fast Services are started once, then stay alive waiting for inputs. This saves non-negligible startup time
  • Polyglot Services can be written in any language
  • Compatible with Amazon Lambda, Google Cloud Functions
  • Rock Solid PM2 behind the scene for process management and cluster capabilities
  • And a lot more like Buffering, Retry on Failure...

Creating a Grid

Install your Swiss Army Knife to manage a Grid:

$ npm install grid-cli -g

Now the bin grid is available via the CLI.

Commands

1/ Generate a new Gridfile in the current path that contains grid name, grid password, host and SSH keys:

$ grid new

The Gridfile will look like this:

grid_name     = 'grid-name'
grid_password = 'xxxx'

servers = [
  'user@ip1',
  'user@ip2',
  'user@ip3'
]

ssh_key = '''
1024_PRIVATE_SSH_KEY
'''

ssh_public_key = '''
PUBLIC_SSH_KEY
'''

Change each attribute to the desired value. Note that an SSH client should be running on the defaut port (22) on each remote machine

2/ Provision every host listed in the Gridfile:

$ grid provision

This will copy the SSH pub key and install NVM, Node.js, PM2 and Gridcontrol This installation does not need ROOT access rights at any time

3/ Grid management

$ grid dash

Grid dashboard

Commands to manage your grid:

# List all nodes linked to the grid
$ grid ls

# Display Dashboard
$ grid dash

# Execute a command on each server
$ grid multissh <bash_command>

# Restart/Recover the current Grid
$ grid restart

# Upgrade Gridcontrol to latest version
$ grid upgrade

# Display realtime logs of all tasks
$ grid logs

# Monitor the whole Grid with Keymetrics
$ grid monitor <secret> <public>
$ grid unmonitor

# Interactively SSH into desired machine
$ grid ssh

Interact with the Grid

Now let's play with the Grid. You can generate a sample project by typing:

$ grid sample [project-name]
$ cd [project-name]
$ npm install

Now you'll have a project that looks like this:

.
├── index.js
├── package.json
└── tasks
    └── get-ip.js

Let's look at the content of tasks/get-ip.js, this is a task that will be propagated in the grid:

var request = require('request');

module.exports = function(context, cb) {
  request('https://api.ipify.org/?format=json', function (error, response, body) {
    if (error)
      return cb(error);

    if (!error && response.statusCode == 200) {
      return cb(null, body);
    }
  });
};

To call this function, look at how it's done in index.js:

var grid = require('grid-api').init({
  instances   : 1,
  env         : {
    NODE_ENV  : 'development'
  }
});

function triggerTask() {
  // 'get-ip' is the filename
  grid.exec('get-ip', function(err, data, server) {
    if (err) {
      console.log(err);
      return false;
    }
	  console.log('Got response from server pub_ip=(%s) priv_ip=(%s):',
                server.public_ip,
                server.private_ip);
    console.log(data);
  });
}

grid.on('ready', function() {
  console.log('Gridcontrol Ready');
  setInterval(triggerTask, 1000);
});

Start the main application:

$ node index.js

At the beginning, only the local gridcontrol will respond. Once the other peers are synchronized they will also process the queries:

Got response from server pub_ip=(88.123.12.21) priv_ip=(10.2.2.1):
$HTML
Got response from server pub_ip=(88.123.12.22) priv_ip=(10.2.2.8):
$HTML
Got response from server pub_ip=(88.125.120.20) priv_ip=(10.2.2.10):
$HTML
Got response from server pub_ip=(88.123.23.42) priv_ip=(10.2.2.12):
$HTML

Distributed processing, on-premise!

Contributing

If you find any issues please open an issue on Github

For Contributing please refer to docs/CONTRIBUTING.md

License

Apache V2 (see LICENSE.txt)

More Repositories

1

pm2

Node.js Production Process Manager with a built-in Load Balancer.
JavaScript
41,442
star
2

angular-bridge

Link models easily via a REST interface between Mongoose/Node-Express/Angular.js
JavaScript
208
star
3

pm2-deploy

Deploy part of PM2
JavaScript
178
star
4

node-logman

Log client side/server side messages and monitor them with a realtime interface
CSS
113
star
5

pm2-interface

Deprecated
JavaScript
90
star
6

multissh

SSH into multiple machine to execute a command in parallel, with a convenient terminal UX
JavaScript
51
star
7

Skytoop

A Nice Desktop in Cloud with a filemanager and a lot of features ! Fork it ! Web2Py
JavaScript
30
star
8

cryptoengine

Monitor and verify transactions from a local or remote bitcoin deamon.
JavaScript
20
star
9

pm2-axon

JavaScript
19
star
10

pm2-axon-rpc

Modified version of visionmedia/axon-rpc
JavaScript
15
star
11

express-repl

Interact with express internal data (routes, settings...) with an auto reconnect remote repl
JavaScript
11
star
12

yeoman-jade-angularjs

Yeoman template with nice jade integration (Haml like)
JavaScript
10
star
13

scaleway-commander

Manage a fleet of scaleway servers
JavaScript
10
star
14

WebGL-solar-system

WebGL - Three.js v53 - Realistic solar system in WebGL
JavaScript
9
star
15

spiderlink

Simple, yet optimized application intercommunication within the same machine. Based on WS.
JavaScript
8
star
16

pm2-openstack

Nginx load balancing and hot edit + load monitoring and create VM triggers
7
star
17

gc-refresh-mode

Google Chrome & Emacs & File Save = Automatic Refresh. This plugin permits you to refresh AUTOMATICALLY google chrome when saving a file ! - Fell free to fork it and enhance it. It worth (I do not have enough time to maitain it) -
Python
6
star
18

pm2-dev

Development dashboard for pm2
JavaScript
6
star
19

ProjectManager

J2EE - Spring (MVC, Secutiry, Autowiring), Hibernate, PostgreSQL, ExtJS. Project manager
JavaScript
5
star
20

Web2Py-application-template

Application template for Web2Py including JS/CSS minification, logger...
JavaScript
5
star
21

Speak.js

Make your software speak, naturally
JavaScript
4
star
22

afflux-server.js

Standalone log server, that routes and send message logs to different db/bucket/table/realtime interface
JavaScript
4
star
23

InterBenchmark

Benchmarks script : Node.Js vs Ruby Rails vs Django vs WebPy vs J2EE-Spring vs Express vs Php - Nginx FastCgi Passenger Tomcat
Shell
3
star
24

coinvest

coinvest
JavaScript
3
star
25

youtube-to-mp3-UI

Your own youtubetomp3
JavaScript
3
star
26

micro-dot-files

dot files for low resources systems (.bashrc/.emacs.el/nodejs)
Shell
3
star
27

Empathy

Baby steps with Raspberry Pi & Node.js
JavaScript
3
star
28

Css-Sprite-AutoGen

Transform multiple images in one and gives css position - Avoid multiple request for each images - Web2Py application
JavaScript
2
star
29

test-repo

JavaScript
2
star
30

easy-voice

JavaScript
2
star
31

lensesly

lensesly
Ruby
2
star
32

companystats

JavaScript
2
star
33

GeonameLocal

GeonameLocal
Shell
2
star
34

Eschool

LMS
JavaScript
2
star
35

afflux-logger.js

Logger client for afflux server - namespaced and custom logger for logging specific events
JavaScript
2
star
36

high-summary

Retrieve title, image and summary of a web page
JavaScript
2
star
37

synapsis

JavaScript
2
star
38

Speak

Speak
JavaScript
2
star
39

node-v8tools

V8 profiler bindings
C++
2
star
40

Sputnik

Library to control Drone over 4G (Hybrid System) / Drone as API / Expose the world as an API
Python
2
star
41

Gsplitter

Gsplitter extension allows you to have a big preview of Google search results !
JavaScript
2
star
42

SoundcloudCommentsHider

Chrome extension - Automatically hide comments on Soundcloud.
JavaScript
1
star
43

nurse-api

Interface for seccomp-nurse
JavaScript
1
star
44

auto-ap

Shell
1
star
45

DeportedChat

JavaScript
1
star
46

.i3

C
1
star
47

storydata-htp

JavaScript
1
star
48

pm2-hub-rd

R&D on pub sub system
JavaScript
1
star
49

rails_snippets

Some rails snippets
1
star
50

Unification

Ruby
1
star
51

Quizzy

Quizzy
JavaScript
1
star
52

rails_template

rails new my_new_app -m rails_template/generate_template.rb
Ruby
1
star
53

carcass-auth-app

JavaScript
1
star
54

carcass-memoray

In memory storage with array.js
JavaScript
1
star
55

TAH-project

JavaScript
1
star
56

Monetify

Monetify
Ruby
1
star
57

mavlinkjs

JavaScript
1
star
58

release-test-check

JavaScript
1
star
59

Arduino_LoRa_SX12XX_Demo_Sensor-heltec-ab01

C
1
star
60

thinblog

Thin Wordpress Theme for Wordpress - Example : http://blog.hemca.fr
PHP
1
star
61

reach

1
star
62

jquery.heat-progressbar

Jquery plugin for drawing easily a progressbar with heat color
JavaScript
1
star
63

clem-remote

JavaScript
1
star
64

afflux-listener.js

JavaScript
1
star
65

Gamification

Ruby
1
star
66

OpenStreetMap

OpenStreetMap
JavaScript
1
star
67

AMNBS-boilerplate

Boilerplate/Structure App - Mongoose + Nodejs + Express + AngularJS + Boostrap + Compass
JavaScript
1
star
68

gc-refresh-mode-

Google Chrome & Emacs & File Save = Automatic Refresh. This plugin permits you to refresh AUTOMATICALLY google chrome when saving a file !
1
star
69

MDNotes

Live markdown editor with multiple accounts
JavaScript
1
star
70

youtube-download

JavaScript
1
star
71

afflux-express.js

Middlewares for Express to log access and errors to afflux-server.js
1
star