azat-co/node-patterns azat-co/node-patterns

  • Stars
    star
    158
  • Rank 235,760 (Top 5 %)
  • Language
    JavaScript
  • Created over 8 years ago
  • Updated over 7 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
azat-co/node-patterns
github

azat-co

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Node Patterns: From Callbacks to Observer

footer: © NodeProgram.com, 2016 slidenumbers: true

Node Patterns

From Callbacks to Observer

The one true #JavaScript exception handler pattern

inline

The one true Node exception handler pattern

process.on('uncaughtException',
  e => require('opn')(`http://stackoverflow.com/search?q=[node.js ]+${e.message}`)
)

Why Turn off Your IM and Care?

  1. You want to write and organize code better
  2. You want to become a go-to Node person in your team
  3. You want to understand Node things a bit deeper

Slides 📄

https://github.com/azat-co/node-patterns

git clone https://github.com/azat-co/node-patterns

About Presenter

Azat Mardan

inline

Twitter: @azat_co Email: [email protected] Blog: webapplog.com

About Presenter

  • Work: Technology Fellow at Capital One
  • Experience: FDIC, NIH, DocuSign, HackReactor and Storify
  • Books: Practical Node.js, Pro Express.js, Express.js API and 8 others
  • Teach: NodeProgram.com

inline

Node Basics

  • JavaScript, but not "==="
  • Asynchronous + Event Driven
  • Non-Blocking I/O

JavaScript? 😒

  • Async code is hard
  • Code complexity grows exponentially
  • Good code organization is important

^So JavaScript can be tough

Problem

How to schedule something in the future?

Callbacks All the Way!

Functions are First-Class Citizens

var t = function(){...}
setTimeout(t, 1000)

t is a callback

Callback Convention

var fs = require('fs')
var callback = function(error, data){...}
fs.readFile('data.csv', 'utf-8', callback)

Conventions

  • error 1st argument, null if everything is okay
  • data is the second argument
  • callback is the last argument

Note

Naming doesn't matter but order matters.

Node.js won't enforce the arguments.

Convention is not a guarantee. It's just a style. — Read documentation or source code.

Problem

How to ensure the right sequence? Control flow 😕

Example

HTTP requests to:

  1. Get an auth token
  2. Fetch data
  3. PUT an update

They must be executed in a certain order.

... // callback is defined, callOne, callTwo, and callThree are defined
callOne({...}, function(error, data1) {
    if (error) return callback(error, null)
    // work to parse data1 to get auth token
    // fetch the data from the API
    callTwo(data1, function(error, data2) {
        if (error) return callback(error, null)
        // data2 is the response, transform it and make PUT call
        callThree(data2, function(error, data3) {
            //
            if (error) return callback(error, null)
            // parse the response
            callback(null, data3)
        })
    })
})

Welcome to callback hell

fs.readdir(source, function (err, files) {
  if (err) {
    console.log('Error finding files: ' + err)
  } else {
    files.forEach(function (filename, fileIndex) {
      console.log(filename)
      gm(source + filename).size(function (err, values) {
        if (err) {
          console.log('Error identifying file size: ' + err)
        } else {
          console.log(filename + ' : ' + values)
          aspect = (values.width / values.height)
          widths.forEach(function (width, widthIndex) {
            height = Math.round(width / aspect)
            console.log('resizing ' + filename + 'to ' + height + 'x' + height)
            this.resize(width, height).write(dest + 'w' + width + '_' + filename, function(err) {
              if (err) console.log('Error writing file: ' + err)
            })
          }.bind(this))
        }
      })
    })
  }
}

Callback Hell

  • Hard to read
  • Hard to modify/maintain/enhance
  • Easy for devs to make bugs
  • Closing parens - 👿

callbackhell.com

Solutions

  • Abstract into named functions (hoisted or variables)
  • Use obververs
  • Use advanced libraries and techniques

Named Functions

callOne({...}, processResponse1)

function processResponse1(error, data1) {
  callTwo(data1, processResponse2)
}

function processResponse2(error, data2) {
  callThere(data2, processResponse3)
}

function processResponse3(error, data1) {
  ...
}

Modular Functions

var processResponse1 = require('./response1.js')
callOne({...}, processResponse1)
// response1.js
var processResponse2 = require('./response2.js')
module.exports = function processResponse1(error, data1) {
  callTwo(data1, processResponse2)
}
// response2.js
var processResponse3 = require('./response3.js')
module.exports = function processResponse2(error, data2) {
  callThere(data2, processResponse3)
}
// response3.js
module.exports = function processResponse3(error, data3) {
  ...
}

Problem

How to modularize code properly?

  • module.exports = {...}
  • module.exports.obj = {...}
  • exports.obj = {...}

Note: exports = {...} is anti-pattern.

Problem

How to modularize dynamic code or where to initialize?

Solution

  • module.exports = function(options) {...}
  • module.exports.func = function(options) {...}
  • exports.func = function(options) {...}

Import

// code A
module.exports = function(options){
  // code B
}

When you require, code A is run and code B is not. Code A is run only once, no matter how many times you require. You need to invoke the object to run code B.

Demo

node import-main

Importing Folders / Plugin Pattern

// main.js
var routes = require('./routes')
// routes/index.js
module.exports = {
  users: require('./users.js'),
  accounts: require('./accounts.js')
  ...
}

Singletons

  • require: modules are cached
// module.js
var a = 1 // Private
module.exports = {
  b: 2 // Public
}
// program.js
var m = require('./module')
console.log(m.a) // undefined
console.log(m.b) // 2
m.b ++
require('./main')
// main.js
var m = require('./module')
console.log(m.b) // 3

Demo

node main.js
node program.js

Problem

Modules are cached on based on their resolved filename.

Filename will break the caching

var m = require('./MODULE')
var m = require('./module')

Or different paths

Solution

global

global.name

or

GLOBAL.name

_log = global.console.log
global.console.log = function(){
  var args = arguments
  args[0] = '\033[31m' +args[0] + '\x1b[0m'
  return _log.apply(null, args)
}
require('./logs.js')

Color Logs

global.error = global.console.error = msg =>
  console.log( '\x1b[31m\x1b[1mError:\x1b[22m \x1b[93m' + msg + '\x1b[0m' )
global.info = global.console.info = msg =>
  console.log( '\x1b[31m\x1b[36mInfo:\x1b[22m \x1b[93m\x1b[0m' + msg )
global.log = console.log

200% inline

global is powerful... anti-pattern

similar window.jQuery = jQuery

use it sparringly

^with a lot of power comes a lot of responsibility

Problem: How to organize your modular code into classes?

  • ES5 Classes are too complex (new, prototype, this)
  • ES6 Classes don't allow define property and other issues

inline

Sidenote: Prototypes

Objects inherit from other objects

Functions are objects too.

Solution

Function factory for objects

module.exports = function(options) {
  // initialize
  return {
    getUsers: function() {...},
    findUserById: function(){...},
    limit: options.limit || 10,
    // ...
  }
}

Solution 2

require('util').inherits(child, parent)

Problem

Enhance functionality "on the fly"

Decorator: Enhances an object

let userModel = function(options = {}) {
 return {
   getUsers: function() {},
   findUserById: function() {},
   limit: options.limit || 10
 }
}
let user = userModel()
console.log(user.limit)
let adminModel = (userModel) => {
  userModel.limit += 20
  userModel.removeUser = () => {}
  userModel.addUser = () => {}
  return userModel
}
console.log(adminModel(user).limit)

// 10 30

Problem

How to enhance classes defined wit prototypal inheritance?

Prototype Decorator: Enhance global object

Object.prototype.toPrettyJSON = function() {
  console.log(this)
  return JSON.stringify(this, null, 2)
}
let obj = new Object({a: 1})
console.log(obj.toPrettyJSON())

Should: https://github.com/shouldjs/should.js

Problem

Non-blocking I/O can be blocked 😢

Have you ever seen this code?

setTimeout(function timeout() {
  console.log('Hello Node')
}, 0)
// setimmediate.js
setImmediate(function A() {
  setImmediate(function B() {
    console.log('Step 1')
  })
  setImmediate(function C() {
    console.log('Step 2')
    setImmediate(function F() { console.log('Step 3') })
    setImmediate(function G() { console.log('Step 4') })
  })
})
console.log('Step 0')
setTimeout(function timeout() {
  console.log('Timeout!')
}, 0)
console.log('Step 0.5')

// Step 0, Step 0.5, Timeout!, Step 1, Step 2, Step 3, Step 4

^setTimeout is on the next iteration of the event loop ^setImmediate is also, after I/O and before timers (official docs). setImmediate allows you to distribute computation over many turns of the event loop while ensuring that I/O doesn't get starved ^setTimeout is slower

process.nextTick(function A() {
  process.nextTick(function B() {
    console.log('Step 1')
  })
  process.nextTick(function C() {
    console.log('Step 2')
    process.nextTick(function F() { console.log('Step 3') })
    process.nextTick(function G() { console.log('Step 4') })
  })
})
console.log('Step 0')
setTimeout(function timeout() {
  console.log('Timeout!')
}, 0)
console.log('Step 0.5')

// Step 0, Step 0.5, Step 1, Step 2, Step 3, Step 4, Timeout!

^nextTick happens before I/O callbacks. So in a case where you're trying to break up a long running, CPU-bound job using recursion, you would now want to use setImmediate rather than process.nextTick to queue the next iteration as otherwise any I/O event callbacks wouldn't get the chance to run between iterations.

inline

Problem

Insure continuity

Node.js Middleware Pattern

Middleware pattern is a series of processing units connected together, where the output of one unit is the input for the next one. In Node.js, this often means a series of functions in the form:

function(args, next) {
  // ... Run some code
  next(output) // Error or real output
}

Express Example

Request is coming from a client and response is sent back to the client.

request->middleware1->middleware2->...middlewareN->route->response

Express.js Middleware

app.use(function(request, response, next) {
  // ...
  next()
}, function(request, response, next) {
  next()
}, function(request, response, next) {
  next()
})

Problems

  • Callbacks are still hard to manage even in modules!
  • Callbacks fire just once
  • Callbacks fire only at the end
  • No way to remove a callback or add a new one "on the fly"

No it's not promises. 😉

^Promises fire just once just as callbacks

Example

  1. Module Job is performing a task.
  2. In the main file, we import Job.

How do we specify a callback (some future logic) on the Job's task completion?

Maybe:

var job = require('./job.js')(callback)

What about multiple callbacks?

Not very scalable 😢

Solution

Observer pattern with event emitters!

// module.js
var util = require('util')
var Job = function Job() {
  // ...
  this.process = function() {
    // ...
    this.emit('done', { completedOn: new Date() })
  }
}

util.inherits(Job, require('events').EventEmitter)
module.exports = Job

^ module

// main.js
var Job = require('./module.js')
var job = new Job()

job.on('done', function(details){
  console.log('Job was completed at', details.completedOn)
  job.removeAllListeners()
})

job.process()

^main

emitter.listeners(eventName)
emitter.on(eventName, listener)
emitter.once(eventName, listener)
emitter.removeListener(eventName, listener)

More Async

  • async and neo-async
  • co and bluebird
  • Promises - not really helping much
  • Generators - promising
  • Async await - nice wrapper for promises

Let's leave managing async for another talk.

Dependency Injection

Express Middleware

var express = require('express')  
var app = express()  
var session = require('express-session')

app.use(session({  
  store: require('connect-session-knex')()
}))

Hapi

server.views({  
  engines: {
    html: require('handlebars')
  },
  relativeTo: __dirname,
  path: 'templates'
})

Express Routes

// server.js
var app = express()
//...
app.use(logger('dev'))
//...
app = require('./routes')(app)
app.listen(3000)
// routes/index.js
module.exports = function(app) {
  app.get('/users', require('./users.js').getUsers)
  app.post('/order', require('./orders.js').addOrder)
  //...
  return app
}

Problem

let f = ?
let a = f('Portland')
let b = f('Oakland')
console.log(a('Hi')) // Hi Portland
console.log(b('Hey')) // Hey Oakland

Function which returns a function (monad?)

// routes/index.js
module.exports = function(app){
  return function(options, callback) {
    app.listen(app.get('port'), options, callback)
  }
}

Useful modules

^http://thenodeway.io and https://darrenderidder.github.io/talks/ModulePatterns/#/14

There are more patterns!

Node Patterns by Mario Casciaro

inline

http://amzn.to/21hXxTy

30-second Summary

  1. Callbacks
  2. Observer
  3. Singleton
  4. Plugins
  5. Middleware
  6. Bunch of other stuff 💥

THE END

  1. Start with what you need
  2. Try and see what works for you (func vs. proto)
  3. Don't over engineers following the latest fad — stick to fundamentals!

Don't fight with JavaScript/Node, use it!

Rate This Talk 👍

Scale 1-10 (10 is highest)

Anyone below 8?

This is your chance ask a question to make it 10! (If you don't ✋, then you rate 8+, right?)

Code and Slides

https://github.com/azat-co/node-patterns/issues

Twitter: @azat_co Email: [email protected]

Also, checkout NodeProgram.com

More Repositories

1

practicalnode

Practical Node.js, 1st and 2nd Editions [Apress] 📓
JavaScript
3,802
star
2

you-dont-know-node

You Don't Know Node.js
Python
1,504
star
3

cheatsheets

JavaScript and Node.js cheatsheets
HTML
1,341
star
4

expressworks

Learn Express.js from the author of one of the best books on Express.js—Pro Express.js— with this workshop that will teach you basics of Express.js.
JavaScript
684
star
5

mongoui

MongoDB admin UI server written in Node.js 🎮
JavaScript
586
star
6

react-quickly

Source code for React Quickly [Manning, 2017]: Painless Web Apps with React, JSX, Redux, and GraphQL 📕
JavaScript
540
star
7

fullstack-javascript

Source code for the Fullstack JavaScript book
JavaScript
513
star
8

react

Examples for the React Quickly book.
JavaScript
248
star
9

rest-api-express

REST API built with Express.js, Mongoskin and Mocha
JavaScript
198
star
10

rpjs

Rapid Prototyping with JS examples. Rapid Prototyping with JS is a JavaScript and Node.js book that will teach you how to build mobile and web apps fast.
JavaScript
187
star
11

nodeframework

Hand-picked registry of Node.js frameworks.
HTML
161
star
12

hackhall

Invite-only community and social collaboration platform for programmers, startupers, JavaScript professionals and pirates.
JavaScript
152
star
13

blog-express

Express.js Blog App
JavaScript
133
star
14

expressjsguide

Express.js Guide code examples
JavaScript
110
star
15

todo-express

Express.js Todo App
CSS
108
star
16

proexpressjs

Examples for the Practical Node.js book [Apress, 2014]
JavaScript
106
star
17

react-native-quickly

JavaScript
100
star
18

node-advanced

Node Advanced Courseware
JavaScript
82
star
19

super-simple-backbone-starter-kit

Super Simple Backbone Starter Kit and Boilerplate
CSS
74
star
20

http2-express

JavaScript
61
star
21

proto-buffer-api

Google Protocol Buffers Node.js/Express.js API Example
JavaScript
58
star
22

meanworks

MEANWorks: MongoDB/Mongoose, Express, Angular and Node Workshop with Angular Fullstack Generator
JavaScript
41
star
23

openlist

Open source PHP web application for ad classified websites
PHP
37
star
24

http2-node-server-push

JavaScript
23
star
25

node-in-production

JavaScript
21
star
26

editor

Collaborative Online Real-Time Code Editor
JavaScript
20
star
27

universal-web

JavaScript
14
star
28

node-react

JavaScript
13
star
29

aws-intermediate

Shell
12
star
30

codedeploy-codepipeline-node

Shell
11
star
31

es7-es8

ES7/ES2016 and ES8/ES2017 Features
JavaScript
9
star
32

react-foundation

JavaScript
9
star
33

relay-modern

Relay Modern: Declarative Data-Driven React Apps Development with GraphQL
JavaScript
8
star
34

ga-backbone

Introduction to Backbone.js
JavaScript
7
star
35

simple-message-board

Simple Message Board with Parse.com REST API and jQuery
JavaScript
7
star
36

expressapiref

Express.js Deep API Reference
JavaScript
7
star
37

state-js

State of Javascript: Overview of recent ECMAScript standard and libraries (end of 2017, early 2018)
6
star
38

node-toolchain

https://node.university/p/node-toolchain
JavaScript
6
star
39

h2-node

6
star
40

mern

JavaScript
5
star
41

node-testing

Node Testing Courseware
JavaScript
5
star
42

copy-paste

Atom package for pasting code with delay
CoffeeScript
5
star
43

graphql-apollo

CSS
5
star
44

5-reasons

JavaScript
4
star
45

weather-app

JavaScript
3
star
46

aws-intro

Build Solid Foundation of Main AWS Concepts and Services Start Learning
Shell
3
star
47

nodejs-hello-world

nodejs-hello-world
JavaScript
3
star
48

ato-2018-writting

3
star
49

azat-co

Source code for http://azat.co personal website
HTML
3
star
50

es6

3
star
51

nodejs-message-board

nodejs-message-board
JavaScript
2
star
52

vote-app

JavaScript
2
star
53

swe-coaching

Software Engineering Mastermind and Coaching
2
star
54

mongo-message-board

mongo-message-board
JavaScript
2
star
55

rpjs-co

Rapid Prototyping with JS website (<http://rpjs.co>).
HTML
2
star
56

mongoose

JavaScript
2
star
57

react-native-timer

JavaScript
2
star
58

crud-rest-api

JavaScript
2
star
59

serverless

1
star
60

globalog

Colored logs without having to type "console".
JavaScript
1
star
61

Tasks

Various Tasks
Ruby
1
star
62

twitter-stream

JavaScript
1
star
63

practicalnodebook

Website for the Practical Node.js [Apress, 2014]
HTML
1
star
64

message-board

Message Board: Backbone + Parse.com
JavaScript
1
star
65

react-start

JavaScript
1
star
66

rest-api-koa-mongo

JavaScript
1
star
67

SquareQ

Square Coding Question
1
star
68

graphql-relay

JavaScript
1
star
69

ic

Image Conversion with ImageMagick
JavaScript
1
star
70

react-16

Demo project of all things new in React 16
JavaScript
1
star
71

150x

150x.co
JavaScript
1
star
72

azatblog

CSS
1
star
73

mobile-dapp-wrapper

Like Metamask but for mobile
JavaScript
1
star
74

rn-workshop

1
star