• This repository has been archived on 18/Feb/2021
  • Stars
    star
    775
  • Rank 58,632 (Top 2 %)
  • Language
    JavaScript
  • License
    MIT License
  • Created over 10 years ago
  • Updated over 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A consistent shrinkwrap tool

npm-shrinkwrap

(This project is deprecated and not maintained.)

A consistent shrinkwrap tool Note: npm >= 3 is currently not supported.

Usage

$ npm-shrinkwrap

This runs shrinkwrap, which verifies your package.json & node_modules tree are in sync. If they are it runs shrinkwrap then fixes the resolved fields and trims from fields

When you run npm-shrinkwrap it will either:

  • fail because your package.json & node_modules disagree, i.e. you installed something without --save or hand edited your package.json
  • succeed, and add all top level dependencies to your npm-shrinkwrap.json file and then runs npm-shrinkwrap sync which writes the npm-shrinkwrap.json back into node_modules

Motivation

Verify local correctness

We need to verify that package.json, npm-shrinkwrap.json and node_modules all have the same content.

Currently npm verifies most things but doesn't verify git completely.

The edge case npm doesn't handle is if you change the tag in your package.json. npm happily says that the dependency in your node_modules tree is valid regardless of what tag it is.

Consistently set a resolved field.

NPM shrinkwrap serializes your node_modules folder. Depending on whether you installed a module from cache or not it will either have or not have a resolved field.

npm-shrinkwrap will put a resolved field in for everything in your shrinkwrap.

Reduce diff churn

There are a few tricks to ensuring there is no unneeded churn in the output of npm shrinkwrap.

This first is to ensure you install with npm cache clean so that an npm ls output is going to consistently give you the resolved and from fields.

The second is to just delete all from fields from the generated shrinkwrap file since they change a lot but are never used. However you can only delete some from fields, not all.

Human readable diff

When you run shrinkwrap and check it into git you have an unreadable git diff.

npm-shrinkwrap comes with an npm-shrinkwrap diff command.

npm-shrinkwrap diff master HEAD
npm-shrinkwrap diff HEAD npm-shrinkwrap.json --short

You can use this command to print out a readable context specific diff of your shrinkwrap changes.

Custom shrinkwrap validators

npm-shrinkwrap can be programmatically configured with an array of validators.

These validators run over every node in the shrinkwrap file and can do assertions.

Useful assertions are things like assertion all dependencies point at your private registry instead of the public one.

Example

var npmShrinkwrap = require("npm-shrinkwrap");

npmShrinkwrap({
    dirname: process.cwd()
}, function (err, optionalWarnings) {
    if (err) {
        throw err;
    }

    optionalWarnings.forEach(function (err) {
        console.warn(err.message)
    })

    console.log("wrote npm-shrinkwrap.json")
})

Algorithm

npm-shrinkwrap algorithm

  • run npm ls to verify that node_modules & package.json agree.

  • run verifyGit() which has a similar algorithm to npm ls and will verify that node_modules & package.json agree for all git links.

  • read the old npm-shrinkwrap.json into memory

  • run npm shrinkwrap

  • copy over excess non-standard keys from old shrinkwrap into new shrinkwrap and write new shrinkwrap with extra keys to disk.

  • run setResolved() which will ensure that the new npm-shrinkwrap.json has a "resolved" field for every package and writes it to disk.

  • run trimFrom() which normalizes or removes the "from" field from the new npm-shrinkwrap.json. It also sorts the new npm-shrinkwrap.json deterministically then writes that to disk

  • run trimNested() which will trim any changes in the npm-shrinkwrap.json to dependencies at depth >=1. i.e. any changes to nested dependencies without changes to the direct parent dependency just get deleted

  • run sync() to the new npm-shrinkwrap.json back into the node_modules folder

npm-shrinkwrap NOTES:

  • verifyGit() only has a depth of 0, where as npm ls has depth infinity.

  • verifyGit() is only sound for git tags. This means that for non git tags it gives warnings / errors instead.

  • trimFrom() also sorts and rewrites the package.json for consistency

  • By default, the npm-shrinkwrap algorithm does not dedupe nested dependencies. This means that the shrinkwrap is closer to the installed dependencies by default. If this is not desired --keepNested=false can be passed to the shrinkwrap cli

Cli Documentation

npm-shrinkwrap [options]

Verifies your package.json and node_modules are in sync. Then runs npm shrinkwrap and cleans up the npm-shrinkwrap.json file to be consistent.

Basically like npm shrinkwrap but better

Options:
  --dirname           sets the directory location of the package.json
                      defaults to `process.cwd()`.
  --keep-nested       If set, will not remove nested changes.
  --warnOnNotSemver   If set, will downgrade invalid semver errors
                      to warnings
  --dev               If set, will shrinkwrap dev dependencies
  --silent            If set, will be silent.

npm-shrinkwrap --help

Prints this message

npm-shrinkwrap sync

Syncs your npm-shrinkwrap.json file into the node_modules directory.

This will ensure that your local node_modules matches the npm-shrinkwrap.json file verbatim. Any excess modules in your node_modules folder will be removed if they are not in the npm-shrinkwrap.json file.

Options: --dirname sets the directory of the npm-shrinkwrap.json

  • --dirname defaults to process.cwd()

npm-shrinkwrap install

Will write a shrinkwrap script to your package.json file.

{
    "scripts": {
        "shrinkwrap": "npm-shrinkwrap"
    }
}

Options: --dirname sets the directory location of the package.json

npm-shrinkwrap diff [OldShaOrFile] [NewShaOrfile]

This will show a human readable for the shrinkwrap file.

You can pass it either a path to a file or a git shaism.

Example:

npm-shrinkwrap diff HEAD npm-shrinkwrap.json
npm-shrinkwrap diff origin/master HEAD
Options:
    --depth     configure the depth at which it prints
    --short     when set it will print add/remove tersely
    --dirname   configure which folder to run within
  • --depth defaults to 0
  • --short defaults to false
  • --dirname defaults to process.cwd()

Installation

For usage with npm@2

npm install npm-shrinkwrap

For usage with npm@1

npm install [email protected]

Note: npm >= 3 is not supported.

Tests

npm test

Contributors

  • Raynos

More Repositories

1

go-torch

Stochastic flame graph profiler for Go programs
Go
3,958
star
2

pyflame

🔥 Pyflame: A Ptracing Profiler For Python. This project is deprecated and not maintained.
C++
2,974
star
3

image-diff

Create image differential between two images
JavaScript
2,453
star
4

makisu

Fast and flexible Docker image building tool, works in unprivileged containerized environments like Mesos and Kubernetes.
Go
2,409
star
5

cpustat

high frequency performance measurements for Linux. This project is deprecated and not maintained.
Go
1,659
star
6

cherami-server

Distributed, scalable, durable, and highly available message queue system. This project is deprecated and not maintained.
Go
1,416
star
7

AthenaX

SQL-based streaming analytics platform at scale
Java
1,224
star
8

plato-research-dialogue-system

This is the Plato Research Dialogue System, a flexible platform for developing conversational AI agents.
Python
977
star
9

chaperone

A Kafka audit system
Java
640
star
10

coding-challenge-tools

Uber's tools team coding challenge
562
star
11

hyperbahn

Service discovery and routing for large scale microservice operations
JavaScript
394
star
12

sql-differential-privacy

Dataflow analysis & differential privacy for SQL queries. This project is deprecated and not maintained.
Scala
391
star
13

phabricator-jenkins-plugin

Jenkins plugin to integrate with Phabricator, Harbormaster, and Uberalls
Java
367
star
14

ohana-ios

Contacts simplified. This project is deprecated and not maintained.
Objective-C
362
star
15

rave

A data model validation framework that uses java annotation processing.
Java
355
star
16

jetstream-ios

An elegant model framework written in Swift
Swift
333
star
17

node-stap

Tools for analyzing Node.js programs with SystemTap. This project is deprecated and not maintained.
JavaScript
291
star
18

r-dom

React DOM wrapper
JavaScript
263
star
19

focuson

A tool to surface security issues in python code
Python
226
star
20

cherami-client-go

Go Client Implementation of Cherami - A distributed, scalable, durable, and highly available message queue system. This project is deprecated and not maintained.
Go
207
star
21

viewport-mercator-project

NOTE: The viewport-mercator-project repo is archived and code has moved to
JavaScript
137
star
22

infer-plugin

Gradle plugin that allows easy integration with the infer static analyzer.
Groovy
126
star
23

express-statsd

Statsd route monitoring middleware for connect/express
JavaScript
126
star
24

android-build-environment

Docker repository for android build environment
122
star
25

in-n-out

A library to perform point-in-geofence searches.
JavaScript
106
star
26

buck-http-cache

An Implementation of Buck's HTTP Cache API as a distributed cache service. This project is deprecated and not maintained.
Shell
101
star
27

statsrelay

A consistent-hashing relay for statsd and carbon metrics
C
101
star
28

hacheck

HAproxy healthcheck proxying service
Python
86
star
29

potter

a CLI to create node.js services
JavaScript
83
star
30

opentracing-go

A general-purpose instrumentation API for distributed tracing systems
Go
82
star
31

idl

A CLI for managing Thrift IDL files
JavaScript
78
star
32

jetstream

Jetstream Sync server framework
JavaScript
73
star
33

canduit

Node.js Phabricator Conduit API client. This project is deprecated and not maintained.
JavaScript
65
star
34

kafka-spraynozzle

A nozzle to spray a kafka topic at an HTTP endpoint. This project is deprecated and not maintained.
Java
49
star
35

usb2fac

Enabling 2fac confirmation for newly connected USB devices
Python
44
star
36

nanny

Cluster management for Node processes
JavaScript
40
star
37

auto-value-bundle

Extends Autovalue to extract data from a bundle into a value object.
Java
36
star
38

node-flame

Tools for analyzing Node.js programs with ptrace. This project is deprecated and not maintained.
JavaScript
29
star
39

Bug-Bounty-Page

A repo to make our changes more transparent to bug bounty researchers in our program (so they can see commits, etc).
29
star
40

paranoid-request

An SSRF-preventing wrapper around Node's request module
JavaScript
26
star
41

lint-trap

JavaScript linter module for Uber projects
JavaScript
26
star
42

thriftify

JavaScript implementation of Thrift encoding and decoding
JavaScript
25
star
43

HackerOneAlchemy

A tool to generate statistics and help manage bug bounty reports in HackerOne.
Python
23
star
44

express-translate

Add simple translation support to Express
JavaScript
21
star
45

cherami-thrift

Thrift APIs for Cherami - A distributed, scalable, durable, and highly available message queue system. This project is deprecated and not maintained.
Go
20
star
46

h1-python

A HackerOne API client for Python
Python
19
star
47

cidrtrie

Trie implementation of a CIDR lookup table
Python
19
star
48

ios-template

This template provides a starting point for open source iOS projects at Uber.
Ruby
18
star
49

tcheck

TChannel health check utility
Go
17
star
50

job_progress

Store the progress of a job
Python
16
star
51

java-code-styles

IntelliJ IDEA code style settings for Uber's Java and Android projects.
15
star
52

fixed-server

Server for HTTP fixtures
JavaScript
14
star
53

vis-academy

A set of tutorials on how our frameworks make effective data visualization applications.
JavaScript
13
star
54

shared-docs

Shared Markdown Documents from Uber Engineering
12
star
55

typed-request-stack

Middleware stack runner for typed HTTP requests
JavaScript
11
star
56

cherami-client-python

Python Client for Cherami - A distributed, scalable, durable, and highly available message queue system. This project is deprecated and not maintained.
Python
11
star
57

failpointsjs

JavaScript
10
star
58

instafork

JavaScript
8
star
59

py-find-unicode

Find incorrect unicode() invocations
Python
8
star
60

shallow-settings

Shallow inheritance-based settings for your application
JavaScript
7
star
61

clusto-query

Silly CLI for querying clusto more quickly
Python
7
star
62

gg

Go dependency debugger
Go
7
star
63

connect-csrf-lite

CSRF validation middleware for Connect/Express
JavaScript
7
star
64

javax-extras

(DEPRECATED) Extra utilities for javax
Java
6
star
65

fixtures-fs

Create a temporary fs with JSON fixtures
JavaScript
6
star
66

redis-delete-pattern

Delete a set of keys from a pattern in Redis
6
star
67

opentracing-python

NOTE: This repository has been retired. The latest OpenTracing APIs can be found in the official repository.
Python
5
star
68

tchannel-gen

Scaffolding for new TChannel w/ Hyperbahn applications
JavaScript
5
star
69

node-dot-arcanist

Uber's .arcanist folder as an npm module
PHP
5
star
70

cherami-client-java

Java Client for Cherami. This project is deprecated and not maintained.
Java
5
star
71

pyrehol

Python wrapper for Firehol
Python
4
star
72

dubstep

This repo is DEPRECATED. See https://github.com/dubstepjs/core
JavaScript
4
star
73

ottr

Easy, robust end-to-end UI tests for web apps
JavaScript
3
star
74

clouseau

A Node.js performance profiler by Uber
JavaScript
3
star
75

vertica-aesgcm-udx

C++
2
star
76

stacked

Go
2
star
77

request-redis-cache

Make requests and cache them in Redis
JavaScript
2
star
78

nodesol-write

Kafka producer.
JavaScript
2
star
79

request-mocha

Request utilities for Mocha
JavaScript
2
star
80

UberBuilder

Make building flexible, immutable objects a simple task
Objective-C
2
star
81

uLeak

DEPRECATED: This is continued in https://github.com/behroozkhorashadi/uLeak
Java
2
star
82

fusion-orchestrate

Tools and scripts for working across multiple fusion repos at once
JavaScript
2
star
83

deck.gl-data-osm

OSM data for the data visualization library deck.gl examples (https://uber.github.io/deck.gl/#/)
1
star
84

uberclass-clouseau

A subclass of uberclass that adds profiling support
JavaScript
1
star
85

backbone-api-client

Backbone mixin built for interacting with API clients
JavaScript
1
star
86

fusion-release

Releases and verifies FusionJS packages
JavaScript
1
star
87

cache-redis

An ES6 Map-like cache with redis backing
JavaScript
1
star
88

redis-broadcast

Write redis commands to a set of redises efficiently
JavaScript
1
star