• Stars
    star
    490
  • Rank 89,811 (Top 2 %)
  • Language
    JavaScript
  • Created about 14 years ago
  • Updated over 9 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

lazy lists for node.js

Lazy lists for node

Table of contents:

Introduction

Documentation

# Introduction Lazy comes really handy when you need to treat a stream of events like a list. The best use case currently is returning a lazy list from an asynchronous function, and having data pumped into it via events. In asynchronous programming you can't just return a regular list because you don't yet have data for it. The usual solution so far has been to provide a callback that gets called when the data is available. But doing it this way you lose the power of chaining functions and creating pipes, which leads to not that nice interfaces. (See the 2nd example below to see how it improved the interface in one of my modules.)

Check out this toy example, first you create a Lazy object:

    var Lazy = require('lazy');

    var lazy = new Lazy;
    lazy
      .filter(function (item) {
        return item % 2 == 0
      })
      .take(5)
      .map(function (item) {
        return item*2;
      })
      .join(function (xs) {
        console.log(xs);
      });

This code says that 'lazy' is going to be a lazy list that filters even numbers, takes first five of them, then multiplies all of them by 2, and then calls the join function (think of join as in threads) on the final list.

And now you can emit 'data' events with data in them at some point later,

    [0,1,2,3,4,5,6,7,8,9,10].forEach(function (x) {
      lazy.emit('data', x);
    });

The output will be produced by the 'join' function, which will output the expected [0, 4, 8, 12, 16].

And here is a real-world example. Some time ago I wrote a hash database for node.js called node-supermarket (think of key-value store except greater). Now it had a similar interface as a list, you could .forEach on the stored elements, .filter them, etc. But being asynchronous in nature it lead to the following code, littered with callbacks and temporary lists:

    var Store = require('supermarket');

    var db = new Store({ filename : 'users.db', json : true });

    var users_over_20 = [];
    db.filter(
      function (user, meta) {
        // predicate function
        return meta.age > 20;
      },
      function (err, user, meta) {
        // function that gets executed when predicate is true
        if (users_over_20.length < 5)
          users_over_20.push(meta);
      },
      function () {
        // done function, called when all records have been filtered

        // now do something with users_over_20
      }
    )

This code selects first five users who are over 20 years old and stores them in users_over_20.

But now we changed the node-supermarket interface to return lazy lists, and the code became:

    var Store = require('supermarket');

    var db = new Store({ filename : 'users.db', json : true });

    db.filter(function (user, meta) {
        return meta.age > 20;
      })
      .take(5)
      .join(function (xs) {
        // xs contains the first 5 users who are over 20!
      });

This is so much nicer!

Here is the latest feature: .lines. Given a stream of data that has \n's in it, .lines converts that into a list of lines.

Here is an example from node-iptables that I wrote the other week,

    var Lazy = require('lazy');
    var spawn = require('child_process').spawn;
    var iptables = spawn('iptables', ['-L', '-n', '-v']);

    Lazy(iptables.stdout)
        .lines
        .map(String)
        .skip(2) // skips the two lines that are iptables header
        .map(function (line) {
            // packets, bytes, target, pro, opt, in, out, src, dst, opts
            var fields = line.trim().split(/\s+/, 9);
            return {
                parsed : {
                    packets : fields[0],
                    bytes : fields[1],
                    target : fields[2],
                    protocol : fields[3],
                    opt : fields[4],
                    in : fields[5],
                    out : fields[6],
                    src : fields[7],
                    dst : fields[8]
                },
                raw : line.trim()
            };
        });

This example takes the iptables -L -n -v command and uses .lines on its output. Then it .skip's two lines from input and maps a function on all other lines that creates a data structure from the output.

# Documentation

Supports the following operations:

  • lazy.filter(f)
  • lazy.forEach(f)
  • lazy.map(f)
  • lazy.take(n)
  • lazy.takeWhile(f)
  • lazy.bucket(init, f)
  • lazy.lines
  • lazy.sum(f)
  • lazy.product(f)
  • lazy.foldr(op, i, f)
  • lazy.skip(n)
  • lazy.head(f)
  • lazy.tail(f)
  • lazy.join(f)

The Lazy object itself has a .range property for generating all the possible ranges.

Here are several examples:

  • Lazy.range('10..') - infinite range starting from 10
  • Lazy.range('(10..') - infinite range starting from 11
  • Lazy.range(10) - range from 0 to 9
  • Lazy.range(-10, 10) - range from -10 to 9 (-10, -9, ... 0, 1, ... 9)
  • Lazy.range(-10, 10, 2) - range from -10 to 8, skipping every 2nd element (-10, -8, ... 0, 2, 4, 6, 8)
  • Lazy.range(10, 0, 2) - reverse range from 10 to 1, skipping every 2nd element (10, 8, 6, 4, 2)
  • Lazy.range(10, 0) - reverse range from 10 to 1
  • Lazy.range('5..50') - range from 5 to 49
  • Lazy.range('50..44') - range from 50 to 45
  • Lazy.range('1,1.1..4') - range from 1 to 4 with increment of 0.1 (1, 1.1, 1.2, ... 3.9)
  • Lazy.range('4,3.9..1') - reverse range from 4 to 1 with decerement of 0.1
  • Lazy.range('[1..10]') - range from 1 to 10 (all inclusive)
  • Lazy.range('[10..1]') - range from 10 to 1 (all inclusive)
  • Lazy.range('[1..10)') - range grom 1 to 9
  • Lazy.range('[10..1)') - range from 10 to 2
  • Lazy.range('(1..10]') - range from 2 to 10
  • Lazy.range('(10..1]') - range from 9 to 1
  • Lazy.range('(1..10)') - range from 2 to 9
  • Lazy.range('[5,10..50]') - range from 5 to 50 with a step of 5 (all inclusive)

Then you can use other lazy functions on these ranges.

More Repositories

1

the-little-schemer

All the Scheme code examples from the book "The Little Schemer"
Scheme
662
star
2

nodejs-proxy

A HTTP proxy server written in node.js
JavaScript
416
star
3

node-tree-kill

kill trees of processes
JavaScript
330
star
4

bash-redirections-cheat-sheet

Bash redirections cheat sheet
309
star
5

perl1line.txt

collection of handy perl one-liner scripts
241
star
6

xgoogle

Python library to Google services (google search, google sets, google translate, sponsored links)
Python
217
star
7

node-gif

A node.js C++ module for creating GIF images and animated GIFs from RGB or RGBA buffers.
C++
212
star
8

node-video

A node.js module for streaming and recording HTML5 Theora videos
C++
202
star
9

node-png

A nodejs C++ module that given a buffer with RGB or RGBA values creates a PNG image (in memory).
C
193
star
10

stackvm

Configure, network, and interact with virtual machines entirely over the web
JavaScript
141
star
11

hacker-top

A top-like program for monitoring hacker news from the console
Python
133
star
12

node-iptables

basic iptables control via nodejs
JavaScript
112
star
13

node-image

Unifies node-png, node-jpeg and node-gif (for great good)
JavaScript
99
star
14

the-seasoned-schemer

All the Scheme code examples from the book "The Seasoned Schemer"
Scheme
97
star
15

bash-vi-editing-mode-cheat-sheet

Bash has two input modes - emacs and vi. This is vi input/editing mode keyboard shortcut cheat sheet.
91
star
16

the-little-mler

All the ML code examples from the book "The Little MLer"
OCaml
84
star
17

awk-cheat-sheet

This is AWK programming language cheat sheet.
76
star
18

bithacks.h

bithacks.h is a C header file containing useful bit manipulation macros
C
73
star
19

node-jpeg

A nodejs C++ module that given a buffer with RGB or RGBA values creates a JPEG image in memory.
C++
65
star
20

node-base64

A base64 encoding and decoding C++ module for node.js that actually works! (node now has it's own base64 encoding, see docs!)
C
64
star
21

perl-tcp-proxy

A simple TCP proxy written in Perl. Uses IO::Socket::INET and IO::Select for multiplexing.
Perl
60
star
22

node-jsmin

javascript minimizer for node.js
JavaScript
52
star
23

reddit-top

A top-like program for monitoring reddit from the console
Python
52
star
24

bash-history-cheat-sheet

This is the bash history cheat sheet. It summarizes everything there is to know about working efficiently with command line history in bash.
52
star
25

the-reasoned-schemer

All the logic programming code examples from the book "The Reasoned Schemer"
Scheme
49
star
26

catonmat.net

The new catonmat.net website.
Python
46
star
27

node-browser

Provides a Browser for easy web browsing from node.js
JavaScript
44
star
28

sed-cheat-sheet

This is sed (unix stream editor) cheat sheet.
44
star
29

node-supermarket

A key/value store based on sqlite for node.js that actually works.
JavaScript
43
star
30

screen-cheat-sheet

This is the screen terminal emulator cheat sheet. It lists the default keyboard shortcuts for working with screen.
42
star
31

bash-emacs-editing-mode-cheat-sheet

Bash has two input modes - emacs and vi. This is emacs input/editing mode keyboard shortcut cheat sheet.
33
star
32

set-operations-in-unix-shell

This is an implementation of 14 set operations by using only Unix utilities such as sort, uniq, diff, comm, cat, head, tail, awk, and others.
32
star
33

social-scraper

Social scraper is a Perl program that scrapes reddit, digg, stumbleupon, delicious, furl, flickr, simpy, boingboing, wired for content that matches the given patterns.
Perl
29
star
34

node-async

An example async library for node.js
C++
27
star
35

bash-one-liners

Bash one-liners
26
star
36

perl-tcp-proxy2

Program for my "A TCP Proxy in Perl" article
Perl
24
star
37

the-little-prover

All code examples from "The Little Prover" book
Scheme
22
star
38

adns

Asynchronous DNS resultion in Python by using adns C library.
21
star
39

busy-beaver

Implementation of a Turing Machine that runs the Busy Beaver programs.
C++
19
star
40

social-submitter

Submit your stories to Reddit, Hacker News, Twitter, Plurk, Identi.ca, Facebook at once!
JavaScript
19
star
41

youtube-uploader

A Perl program that uploads videos to YouTube without any APIs.
Perl
18
star
42

invoice

generate pdf invoices from latex via pdflatex
JavaScript
17
star
43

node-passwd

Node.js module to manage /etc/passwd
JavaScript
17
star
44

gnu-awk-youtube-downloader

A program written in GNU Awk that downloads YouTube Videos. Proof of concept that AWK can do binary IO and networking effectively.
17
star
45

ed-cheat-sheet

This is ed (the unix text editor) cheat sheet. It lists all the commands and how to do line addressing.
16
star
46

gnu-coreutils-cheat-sheet

Gnu Coreutils Cheat Sheet
15
star
47

load-status-server

Windows load status server that returns cpu load, memory usage and disk usage through JSON
C++
15
star
48

util-linux-cheat-sheet

Util-Linux Cheat Sheet
15
star
49

node-chess

Node chess - Node.js knockout competition
JavaScript
12
star
50

ssh-key-manager

manage ssh keys on the server side (can be used with ssh-key-widget)
JavaScript
12
star
51

node-des

A C++ node.js module that does DES encryption and actually works (node's crypto module didn't work.)
C++
12
star
52

node-rfb

implements the client-side of the rfb protocol that vnc uses
JavaScript
12
star
53

perl-youtube-downloader-one-liner

This is a Perl one-liner that downloads YouTube videos.
12
star
54

node-bufferdiff

A C++ module for node-js to test if two buffers are equal, fast (could add diff later).
C++
12
star
55

vbscript-youtube-downloader

A program written in VBScript that downloads YouTube videos.
Visual Basic
12
star
56

supermarket-cart

Connect session store using supermarket
JavaScript
11
star
57

keyboard

provides english keyboard (used as a widget for browserling)
JavaScript
11
star
58

speak-text-files-to-wav

Speaks text files to wav using Microsoft Speech API
C++
10
star
59

perl-predefined-variable-cheat-sheet

This is Perl special variable (predefined variable) cheat sheet. It lists all the Perl variables.
10
star
60

dnode

Simple asynchronous remote method invocation for node.js
JavaScript
10
star
61

node-time

This module provides some time functions for node.js (i forgot about Date() so this module is useless)
C++
8
star
62

reddit-comment-finder

A program that finds all the comments a given reddit user has made.
Python
8
star
63

dotfiles

Vim Script
8
star
64

bitly

shorten urls with bitly without api
Perl
7
star
65

node-quine

A node.js module that exports a function that prints itself
7
star
66

firefox-update-dialog-killer

a simple win32 program that detects and closes the firefox update dialog
C++
6
star
67

node-number-range

number ranges
JavaScript
6
star
68

node-png-sync

sync part of node-png that works on windows with node-gyp on node 0.6
C
6
star
69

youtube-video-downloader-in-perl

Wrote this real quick as I needed to get some vids
Perl
6
star
70

chrome-dialog-killer

Kills the nasty "your profile can not be used because it is from a newer version of Google Chrome" dialog by clicking OK button
C++
5
star
71

hwnd-finder

find hwnds easily
C++
5
star
72

rfb-protocols

A node.js module for various RFB encoding (RRE, Hextile, Tight, etc.) conversion to RGB buffers.
C++
5
star
73

TextToWav

Converts text files to wav using Loquendo text-to-speech sdk
C
5
star
74

perl-pack-unpack-printf-sprintf-cheat-sheet

This is Perl pack/unpack template parameter and printf/sprintf format specifier cheat sheet
4
star
75

quick-cd

Quick cd is a utility for bash that keeps track of your most often used directories and allows you to cd to them with as few keystrokes as possible.
4
star
76

feedburner-graph-generator

Current feedburner graphs suck. I wrote this Perl program to generate the nice graphs they used to have in 2008.
4
star
77

install-computer

Shell
3
star
78

plurk-command-line-plurker

A Perl program for plurking from command line
Perl
3
star
79

node-bufferlist

Create linked lists of Buffer objects
JavaScript
3
star
80

webdev-template

Web development template with reset.css
3
star
81

reddit-river

The old reddit river website for mobile devices
3
star
82

http-async-retry

HTTP::Async with retry
Perl
2
star
83

quick-history

Better interface to bash's (and other shell) CTRL+R for reverse-search-history
2
star
84

dockerfiles

Dockerfile
2
star
85

node-parse-users-exe-output

parses output from users.exe on windows
JavaScript
2
star
86

rackspace-tools

some tools to manage rackspace cloud servers
JavaScript
2
star
87

codinghorror-keyword-analyzer

A Perl program that parses public statcounter data for codinghorror.com blog and stores the search keywords in an SQLite database.
2
star
88

node-rdesktop

Client side of the RDP protocol that Windows uses for remote deskop
JavaScript
2
star
89

node-crashing-async-buf

for ry - a crashing Buffer::New example in eio_custom.
C++
2
star
90

stacked-linux

A small linux distribution for use on routers
2
star
91

utf8-length

return the number of bytes in a unicode string
JavaScript
2
star
92

lulzbot

An IRC bot for node.js
JavaScript
2
star
93

latex2html

convert latex to html (works for me)
Perl
2
star
94

utf8-bytes

return an array of bytes from a unicode string
JavaScript
2
star
95

lwp-protocol-http-socketeer

http protocol implementation for LWP that uses a proxy
Perl
1
star
96

startupsupper.github.com

Recipes for Bootstrappers & Hungry Hackers
JavaScript
1
star
97

testling-ci-badge-checker

This perl program checks github repos for testling-ci badges
Perl
1
star
98

testling-ci-test-example

testling-ci-test-example
JavaScript
1
star
99

picurls.com

This is repository of picurls.com website. picurls: picture buzz! buzziest pictures on the net!
1
star
100

digpicz

The old digpicz.com website
1
star