• Stars
    star
    3,815
  • Rank 10,970 (Top 0.3 %)
  • Language
    JavaScript
  • License
    MIT License
  • Created about 11 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A JavaScript implementation of Git.

JS-Git

Gitter

This project is a collection of modules that helps in implementing git powered applications in JavaScript. The original purpose for this is to enable better developer tools for authoring code in restricted environments like ChromeBooks and tablets. It also enables using git as a database to replace SQL and no-SQL data stores in many applications.

This project was initially funded by two crowd-sourced fundraisers. See details in BACKERS.md and BACKERS-2.md. Thanks to all of you who made this possible!

Usage

Detailed API docs are contained in the doc subfolder of this repository.

In general the way you use js-git is you create a JS object and then mixin the functionality you need. Here is an example of creating an in-memory database, creating some objects, and then walking that tree using the high-level walker APIs.

Creating a repo object.

// This provides symbolic names for the octal modes used by git trees.
var modes = require('js-git/lib/modes');

// Create a repo by creating a plain object.
var repo = {};

// This provides an in-memory storage backend that provides the following APIs:
// - saveAs(type, value) => hash
// - loadAs(type, hash) => hash
// - saveRaw(hash, binary) =>
// - loadRaw(hash) => binary
require('js-git/mixins/mem-db')(repo);

// This adds a high-level API for creating multiple git objects by path.
// - createTree(entries) => hash
require('js-git/mixins/create-tree')(repo);

// This provides extra methods for dealing with packfile streams.
// It depends on
// - unpack(packStream, opts) => hashes
// - pack(hashes, opts) => packStream
require('js-git/mixins/pack-ops')(repo);

// This adds in walker algorithms for quickly walking history or a tree.
// - logWalk(ref|hash) => stream<commit>
// - treeWalk(hash) => stream<object>
require('js-git/mixins/walkers')(repo);

// This combines parallel requests for the same resource for efficiency under load.
require('js-git/mixins/read-combiner')(repo);

// This makes the object interface less strict.  See its docs for details
require('js-git/mixins/formats')(repo);

Generators vs Callbacks

There are two control-flow styles that you can use to consume js-git APIs. All the examples here use yield style and assume the code is contained within a generator function that's yielding to a tool like gen-run.

This style requires ES6 generators. This feature is currently in stable Firefox, in stable Chrome behind a user-configurable flag, in node.js 0.11.x or greater with a command-line flag.

Also you can use generators on any ES5 platform if you use a source transform like Facebook's regenerator tool.

You read more about how generators work at Generators vs Fibers.

var run = require('gen-run');

run(function*() {
 // Blocking logic goes here.  You can use yield
 var result = yield someAction(withArgs);
 // The generator pauses at yield and resumes when the data is available.
 // The rest of your process is not blocked, just this generator body.
 // If there was an error, it will throw into this generator.
});

If you can't use this new feature or just plain prefer node-style callbacks, all js-git APIs also support that. The way this works is actually quite simple. If you don't pass in the callback, the function will return a partially applied version of your call expecting just the callback.

someAction(withArgs, function (err, value) {
  if (err) return handleMyError(err);
  // do something with value
});

// The function would be implemented to support both style like this.
function someAction(arg, callback) {
  if (!callback) return someAction.bind(this, arg);
  // We now have callback and arg
}

Basic Object Creation

Now we have an in-memory git repo useful for testing the network operations or just getting to know the available APIs.

In this example, we'll create a blob, create a tree containing that blob, create a commit containing that tree. This shows how to create git objects manually.

  // First we create a blob from a string.  The `formats` mixin allows us to
  // use a string directly instead of having to pass in a binary buffer.
  var blobHash = yield repo.saveAs("blob", "Hello World\n");

  // Now we create a tree that is a folder containing the blob as `greeting.txt`
  var treeHash = yield repo.saveAs("tree", {
    "greeting.txt": { mode: modes.file, hash: blobHash }
  });

  // With that tree, we can create a commit.
  // Again the `formats` mixin allows us to omit details like committer, date,
  // and parents.  It assumes sane defaults for these.
  var commitHash = yield repo.saveAs("commit", {
    author: {
      name: "Tim Caswell",
      email: "[email protected]"
    },
    tree: treeHash,
    message: "Test commit\n"
  });

Basic Object Loading

We can read objects back one at a time using loadAs.

// Reading the file "greeting.txt" from a commit.

// We first read the commit.
var commit = yield repo.loadAs("commit", commitHash);
// We then read the tree using `commit.tree`.
var tree = yield repo.loadAs("tree", commit.tree);
// We then read the file using the entry hash in the tree.
var file = yield repo.loadAs("blob", tree["greeting.txt"].hash);
// file is now a binary buffer.

When using the formats mixin there are two new types for loadAs, they are "text" and "array".

// When you're sure the file contains unicode text, you can load it as text directly.
var fileAsText = yield repo.loadAs("text", blobHash);

// Also if you prefer array format, you can load a directory as an array.
var entries = yield repo.loadAs("array", treeHash);
entries.forEach(function (entry) {
  // entry contains {name, mode, hash}
});

Using Walkers

Now that we have a repo with some minimal data in it, we can query it. Since we included the walkers mixin, we can walk the history as a linear stream or walk the file tree as a depth-first linear stream.

// Create a log stream starting at the commit we just made.
// You could also use symbolic refs like `refs/heads/master` for repos that
// support them.
var logStream = yield repo.logWalk(commitHash);

// Looping through the stream is easy by repeatedly calling waiting on `read`.
var commit, object;
while (commit = yield logStream.read(), commit !== undefined) {

  console.log(commit);

  // We can also loop through all the files of each commit version.
  var treeStream = yield repo.treeWalk(commit.tree);
  while (object = yield treeStream.read(), object !== undefined) {
    console.log(object);
  }

}

Filesystem Style Interface

If you feel that creating a blob, then creating a tree, then creating the parent tree, etc is a lot of work to save just one file, I agree. While writing the tedit app, I discovered a nice high-level abstraction that you can mixin to make this much easier. This is the create-tree mixin referenced in the above config.

// We wish to create a tree that contains `www/index.html` and `README.me` files.
// This will create these two blobs, create a tree for `www` and then create a
// tree for the root containing `README.md` and the newly created `www` tree.
var treeHash = yield repo.createTree({
  "www/index.html": {
    mode: modes.file,
    content: "<h1>Hello</h1>\n<p>This is an HTML page?</p>\n"
  },
  "README.md": {
    mode: modes.file,
    content: "# Sample repo\n\nThis is a sample\n"
  }
});

This is great for creating several files at once, but it can also be used to edit existing trees by adding new files, changing existing files, or deleting existing entries.

var changes = [
  {
    path: "www/index.html" // Leaving out mode means to delete the entry.
  },
  {
    path: "www/app.js", // Create a new file in the existing directory.
    mode: modes.file,
    content: "// this is a js file\n"
  }
];

// We need to use array form and specify the base tree hash as `base`.
changes.base = treeHash;

treeHash = yield repo.createTree(changes);

Creating Composite Filesystems

The real fun begins when you create composite filesystems using git submodules.

The code that handles this is not packaged as a repo mixin since it spans several independent repos. Instead look to the git-tree repo for the code. It's interface is still slightly unstable and undocumented but is used in production by tedit and my node hosting service that complements tedit.

Basically this module allows you to perform high-level filesystem style commands on a virtual filesystem that consists of many js-git repos. Until there are proper docs, you can see how tedit uses it at https://github.com/creationix/tedit-app/blob/master/src/data/fs.js#L11-L21.

Mounting Github Repos

I've been asking Github to enable CORS headers to their HTTPS git servers, but they've refused to do it. This means that a browser can never clone from github because the browser will disallow XHR requests to the domain.

They do, however, offer a REST interface to the raw git data.

Using this I wrote a mixin for js-git that uses github as the backend store.

Code at https://github.com/creationix/js-github. Usage in tedit can be seen at https://github.com/creationix/tedit-app/blob/master/src/data/fs.js#L31.

More Repositories

1

step

An async control-flow library that makes stepping through logic easy.
JavaScript
2,214
star
2

wheat

Wheat is a blog engine for coders written in node.JS
JavaScript
1,364
star
3

haml-js

Haml ported to server-side Javascript. This is a traditional server-side templating language. Tested with node-js
JavaScript
901
star
4

howtonode.org

This repo contains the actual articles for HowToNode.org
JavaScript
886
star
5

topcube

Webkit Bindings for node
JavaScript
404
star
6

nstore

nStore is a simple, in-process key/value database for node.js
JavaScript
393
star
7

jsonparse

A streaming JSON parser written in pure JavaScript for node.js
JavaScript
349
star
8

do

Do is a simple library for managing async actions in node.JS.
JavaScript
294
star
9

dukluv

LibUV bindings for duktape JS engine
C
265
star
10

msgpack-js

The msgpack protocol implemented in pure javascript.
JavaScript
252
star
11

http-parser-js

A pure JS HTTP parser for node.
JavaScript
237
star
12

node-gir

Node bindings to libgirepository
C++
229
star
13

node-git

Node.JS library to read git repositories.
JavaScript
205
star
14

node-persistence

A high level persistance/database system for node.js
JavaScript
193
star
15

tedit

Chrome app for tedit
JavaScript
188
star
16

node-sdl

Minimal sdl bindings for maximum fun
C++
180
star
17

brozula

VM that runs in the browser and interprets luajit bytecode.
JavaScript
168
star
18

node-router

A simple http server for node.js that has sinatra like qualities. Ideal for generating web services via node.
JavaScript
166
star
19

js-github

An implementation of the js-git interface that mounts a live github repo.
JavaScript
162
star
20

node-webgl

OpenGLES 2.0 bindings for nodeJS with a webGL API
JavaScript
161
star
21

postgres-js

The postgres backend protocol implemented in pure JS for use with node.js
JavaScript
121
star
22

jsgit

A command-line git client powered by js-git and node.js
JavaScript
114
star
23

node-leveldb

NodeJS bindings to levelDB - a fast and lightweight key/value database library
C++
112
star
24

jquery-haml

jQuery-haml is a haml like language written in JSON. This allows for easy dom building so that web apps can do more work independent of the server for a better user experience.
JavaScript
111
star
25

jack

Another new language. The name will probably change.
JavaScript
109
star
26

gen-run

Generator Async Runner. Makes it possible to yield and wait for callbacks and continuables.
JavaScript
105
star
27

stack

Stack is a minimal http module system for node.js
JavaScript
103
star
28

couch-client

A minimal CouchDB client that easy and powerful
JavaScript
101
star
29

weblit

A web framework for luvit 2.0 and lit
Lua
101
star
30

node-gamepad

node.js bindings for Alex Diener's cross-platform gamepad code
C
95
star
31

msgpack-js-browser

A browser port of msgpack-js using DataView and ArrayBuffer.
JavaScript
92
star
32

git-browser

Browse Git Repos offline.
JavaScript
88
star
33

js-git-app

A js-git packaged app for chrome and chromebooks.
JavaScript
77
star
34

node-blog

This repo is the code behind the blog HowToNode.org
JavaScript
76
star
35

domchanger

dombuilder that applies diffs only to the real dom
JavaScript
70
star
36

nuv

N-API bindings to libuv with async/await friendly JS wrapping on top.
JavaScript
66
star
37

ivy

Ivy is Tim Caswell's personal node.js distribution, fork and customize!
Shell
59
star
38

conductor

Conducts async functions into orchestrated masterpieces
JavaScript
52
star
39

lua-git

Git implementation in pure lua for luvit.
Lua
51
star
40

nstore-session

This is a connect session store using nStore
JavaScript
51
star
41

libco

Mirror of http://byuu.org/library/libco/
C
51
star
42

rack-php

A php 5.3 framework with built in rack support. You can serve the php site using thin, mongrel, webrick, passenger, mod_php, or mod_fcgid.
JavaScript
51
star
43

jack-old

A new programming language that sits on top of JavaScript in order to iron out the rough spots and please the programmer
JavaScript
47
star
44

wheaty

JS-Git based application hosting platform
JavaScript
45
star
45

sousaball

social networking game project using html5 canvas for the front end and node.js/V8 for the backend.
JavaScript
45
star
46

adventure

Adventure is the start of a real-time RPG world using node.js and Websockets.
JavaScript
41
star
47

movies

The server I use to stream my kids dvd backups over http using node.js from a mac mini running linux.
JavaScript
40
star
48

luvmonkey

Spidermonkey shell with libuv bindings built-in.
C
39
star
49

safereturn

Functional helpers for callback based code.
JavaScript
38
star
50

moonslice-luv

A port of moonslice running on top of luv and lhttp_parser
Lua
38
star
51

seaduk

An implementation of nucleus-js using duktape and libuv implemented in C
C
35
star
52

dombuilder

An easy dombuilder using json-ml style syntax
JavaScript
35
star
53

min-stream

A meta-package for min-stream helper modules.
JavaScript
33
star
54

node-web

A small and fast web/http library for nodejs. (replaces the built-in http module)
JavaScript
33
star
55

refresh

Refreshing REST/Webdav for node
JavaScript
33
star
56

proto

Adds nice utilities to Object.prototype for JavaScript using Object.defineProperty
JavaScript
32
star
57

creationix

My personal sandbox for growing packages.
JavaScript
31
star
58

pattern

An experiment with my new prototypal style programming
JavaScript
30
star
59

rec

A tool for recording CLI programs and posting their output.
JavaScript
29
star
60

uscript

μScript is a simple programming language for μControllers co-designed by a 9-year-old.
C
29
star
61

simple-mime

A small, simple mime database for node.js
JavaScript
29
star
62

grain

Grain is an async framework for node-js templating languages.
JavaScript
28
star
63

fastqueue

A fast push/shift sync queue
JavaScript
28
star
64

sftp-server

sftp-server implemented in pure JS using node
JavaScript
28
star
65

websocket

A simple drop-in replacement for node's TCP server that supports the WebSocket protocol.
JavaScript
26
star
66

nodemcu-webide

A websocket based IDE for nodemcu devices.
JavaScript
25
star
67

LESS.tmbundle

a textmate bundle for less css using the node version of the library at http://github.com/cloudhead/less.js
25
star
68

boxcode

A tiling code editor
JavaScript
23
star
69

firmata-chrome

Firmata driver for chrome.serial
JavaScript
23
star
70

nibs

Lua
21
star
71

topcloud

High level application framework for Javascript (built on top of jquery-haml and modeled somewhat after SproutCore)
JavaScript
21
star
72

basic-games

An archive of the best games I wrote for Q-Basic over a 10 year span.
C++
21
star
73

experiments

Random collection of my experiments
JavaScript
20
star
74

drop-dat

A quick file sharing mechanism using dat libraries.
JavaScript
20
star
75

uvrun

Tiny node module to expose uv_run and uv_run_once to JavaScript
C++
19
star
76

candor.io

candor.io is a server framework that uses the candor language combined with libuv to make fast scripted network servers.
C++
19
star
77

chrome-app-module-loader

A module loader that lets you load npm modules from a chrome app.
JavaScript
19
star
78

nodeconf2012

My slides and content from nodeconf 2012
Lua
18
star
79

world-db

A specialized database for massive tilemaps
JavaScript
18
star
80

git-node-fs

A node adapter for the fs-db mixin for js-git.
JavaScript
18
star
81

lshell

A remote shell using luvit, websockets, and term.js
JavaScript
17
star
82

mine

Miner digs into a javascript file looking for require calls. Used to statically extract common js dependencies.
JavaScript
17
star
83

coffeepot

A compiler for coffee-script written in coffee-script.
JavaScript
17
star
84

git-node

A convenience wrapper for js-git preconfigured for node.js
JavaScript
17
star
85

moonslice

A fast web framework for luvit
JavaScript
16
star
86

git-sha1

A pure js SHA1 implementation created for js-git.
JavaScript
16
star
87

bench-suite

A benchmark skeleton for something better than static hello world http servers.
C
15
star
88

candor.js

An alternate vm for candor that runs in your browser!
JavaScript
14
star
89

ujkl

Micro Jack Lisp - For ESP8266 and Raspberry PI
C
14
star
90

staff

A port of the excellent Backbone.JS with minor API style changes and no dependencies. ES5 support only.
JavaScript
14
star
91

revision

TypeScript
14
star
92

git-repo

A local git repository using any pluggable backend.
JavaScript
14
star
93

msgpack-es

Msgpack implemented in pure ecmascript.
JavaScript
14
star
94

dukcurl

libcurl bindings for duktape
JavaScript
13
star
95

redis-luvit

A redis client for luvit
Lua
12
star
96

jack-lang

C VM for Jack Language
C
12
star
97

wscat

A netcat like client for websockets
Lua
12
star
98

ts-git

TypeScript re-implementation of js-git (aka js-git 2.0)
TypeScript
12
star
99

git-tree

Create a composite tree of js-git repos and work via paths.
JavaScript
11
star
100

luvit-quest

An interactive workshop that teaches network programming with luvit.
JavaScript
11
star