• Stars
    star
    330
  • Rank 127,657 (Top 3 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created over 10 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

🌀 JS standard CRC-32 and CRC32C implementation

crc32

Standard CRC-32 algorithm implementation in JS (for the browser and nodejs). Emphasis on correctness, performance, and IE6+ support.

Installation

With a node package manager like npm:

$ npm i --save https://cdn.sheetjs.com/crc-32-latest/crc-32-latest.tgz

When installed globally, npm installs a script crc32 that computes the checksum for a specified file or standard input.

Hosted versions are available at https://cdn.sheetjs.com/:

Integration

Using NodeJS or a bundler with require:

var CRC32 = require("crc-32");

Using NodeJS or a bundler with import:

import { bstr, buf, str } from "crc-32";

In the browser, the crc32.js script can be loaded directly:

<script src="crc32.js"></script>

The browser script exposes a variable CRC32.

The script will manipulate module.exports if available . This is not always desirable. To prevent the behavior, define DO_NOT_EXPORT_CRC.

CRC32C (Castagnoli)

The module and CDNs also include a parallel script for CRC32C calculations.

Using NodeJS or a bundler:

var CRC32C = require("crc-32/crc32c");

Using NodeJS or a bundler with import:

import { bstr, buf, str } from "crc-32/crc32c";

In the browser, the crc32c.js script can be loaded directly:

<script src="crc32c.js"></script>

The browser exposes a variable CRC32C.

The script will manipulate module.exports if available . This is not always desirable. To prevent the behavior, define DO_NOT_EXPORT_CRC.

Usage

In all cases, the relevant function takes an argument representing data and an optional second argument representing the starting "seed" (for rolling CRC).

The return value is a signed 32-bit integer!

  • CRC32.buf(byte array or buffer[, seed]) assumes the argument is a sequence of 8-bit unsigned integers (nodejs Buffer, Uint8Array or array of bytes).

  • CRC32.bstr(binary string[, seed]) assumes the argument is a binary string where byte i is the low byte of the UCS-2 char: str.charCodeAt(i) & 0xFF

  • CRC32.str(string[, seed]) assumes the argument is a standard JS string and calculates the hash of the UTF-8 encoding.

For example:

// var CRC32 = require('crc-32');               // uncomment this line if in node
CRC32.str("SheetJS")                            // -1647298270
CRC32.bstr("SheetJS")                           // -1647298270
CRC32.buf([ 83, 104, 101, 101, 116, 74, 83 ])   // -1647298270

crc32 = CRC32.buf([83, 104])                    // -1826163454  "Sh"
crc32 = CRC32.str("eet", crc32)                 //  1191034598  "Sheet"
CRC32.bstr("JS", crc32)                         // -1647298270  "SheetJS"

[CRC32.str("\u2603"),  CRC32.str("\u0003")]     // [ -1743909036,  1259060791 ]
[CRC32.bstr("\u2603"), CRC32.bstr("\u0003")]    // [  1259060791,  1259060791 ]
[CRC32.buf([0x2603]),  CRC32.buf([0x0003])]     // [  1259060791,  1259060791 ]

// var CRC32C = require('crc-32/crc32c');       // uncomment this line if in node
CRC32C.str("SheetJS")                           // -284764294
CRC32C.bstr("SheetJS")                          // -284764294
CRC32C.buf([ 83, 104, 101, 101, 116, 74, 83 ])  // -284764294

crc32c = CRC32C.buf([83, 104])                  // -297065629   "Sh"
crc32c = CRC32C.str("eet", crc32c)              //  1241364256  "Sheet"
CRC32C.bstr("JS", crc32c)                       // -284764294   "SheetJS"

[CRC32C.str("\u2603"),  CRC32C.str("\u0003")]   // [  1253703093,  1093509285 ]
[CRC32C.bstr("\u2603"), CRC32C.bstr("\u0003")]  // [  1093509285,  1093509285 ]
[CRC32C.buf([0x2603]),  CRC32C.buf([0x0003])]   // [  1093509285,  1093509285 ]

Best Practices

Even though the initial seed is optional, for performance reasons it is highly recommended to explicitly pass the default seed 0.

In NodeJS with the native Buffer implementation, it is oftentimes faster to convert binary strings with Buffer.from(bstr, "binary") first:

/* Frequently slower in NodeJS */
crc32 = CRC32.bstr(bstr, 0);
/* Frequently faster in NodeJS */
crc32 = CRC32.buf(Buffer.from(bstr, "binary"), 0);

This does not apply to browser Buffer shims, and thus is not implemented in the library directly.

Signed Integers

Unconventional for a CRC32 checksum, this library uses signed 32-bit integers. This is for performance reasons. Standard JS operators can convert between signed and unsigned 32-bit integers:

CRC32.str("SheetJS")                            // -1647298270 (signed)
CRC32.str("SheetJS") >>> 0                      //  2647669026 (unsigned)
(CRC32.str("SheetJS")>>>0).toString(16)         //  "9dd03922" (hex)

(2647669026 | 0)                                // -1647298270
  • x >>> 0 converts a number value to unsigned 32-bit integer.

  • x | 0 converts a number value to signed 32-bit integer.

Testing

make test will run the nodejs-based test.

To run the in-browser tests, run a local server and go to the ctest directory. make ctestserv will start a python SimpleHTTPServer server on port 8000.

To update the browser artifacts, run make ctest.

To generate the bits file, use the crc32 function from python zlib:

>>> from zlib import crc32
>>> x="foo bar baz٪☃🍣"
>>> crc32(x)
1531648243
>>> crc32(x+x)
-218791105
>>> crc32(x+x+x)
1834240887

The included crc32.njs script can process files or standard input:

$ echo "this is a test" > t.txt
$ bin/crc32.njs t.txt
1912935186

For comparison, the included crc32.py script uses python zlib:

$ bin/crc32.py t.txt
1912935186

On OSX the command cksum generates unsigned CRC-32 with Algorithm 3:

$ cksum -o 3 < IE8.Win7.For.Windows.VMware.zip
1891069052 4161613172
$ crc32 --unsigned ~/Downloads/IE8.Win7.For.Windows.VMware.zip
1891069052

Performance

make perf will run algorithmic performance tests (which should justify certain decisions in the code).

The adler-32 project has more performance notes

License

Please consult the attached LICENSE file for details. All rights not explicitly granted by the Apache 2.0 license are reserved by the Original Author.

Badges

Sauce Test Status

Build Status Coverage Status Dependencies Status NPM Downloads ghit.me Analytics

More Repositories

1

sheetjs

📗 SheetJS Spreadsheet Data Toolkit -- New home https://git.sheetjs.com/SheetJS/sheetjs
JavaScript
33,614
star
2

js-word

✒️ Word Processing Document Library
Rich Text Format
1,302
star
3

j

❌ Multi-format spreadsheet CLI (now merged in http://github.com/sheetjs/js-xlsx )
JavaScript
345
star
4

SheetJS.github.io

:goberserk: SheetJS Spreadsheet Parser/Writer tests and demos
HTML
262
star
5

printj

📜 sprintf for JS
JavaScript
196
star
6

test_files

📚 SheetJS Test Files (XLS/XLSX/XLSB and other spreadsheet formats)
HTML
162
star
7

ssf

📝 Spreadsheet Number Formatter
JavaScript
157
star
8

js-codepage

💱 Codepages for JS
JavaScript
147
star
9

js-adler32

☑️ ADLER-32 checksum
Python
137
star
10

js-ppt

Pure JS PowerPoint 97-2003 (PPT) Parser
JavaScript
107
star
11

k

❌ Spreadsheet Differ
JavaScript
85
star
12

frac

➗ rational approximation with bounded denominator
JavaScript
72
star
13

sgds

Simple REST Server that emulates Google Docs interface using your Excel files (currently read-only)
JavaScript
71
star
14

js-cfb

💾 OLE File Container Format
JavaScript
59
star
15

js-harb

❌ Host of Archaic Representations of Books (now merged in http://github.com/sheetjs/js-xlsx )
JavaScript
55
star
16

jxls

Snapshot for test files. https://github.com/jxlsteam/jxls is the current repo for the project
Java
51
star
17

sheetaki

🔣 Spreadsheet CSV conversion microservice
HTML
50
star
18

pb

📋 Access HTML and other pasteboards from JS and command line
JavaScript
35
star
19

bessel

Bessel Functions in JS
JavaScript
31
star
20

sheets

generate pretty ascii tables from XLS/XLSX/XLSB/XLSM/XML workbooks
JavaScript
23
star
21

wk

🔍 Preview spreadsheets in your terminal!
TypeScript
20
star
22

enron_xls

Spreadsheets from the Enron Corpus
JavaScript
20
star
23

voc

👷 A Literate Programming Framework for JS and compile-to-JS languages.
JavaScript
20
star
24

maths

Collection of Math Functions for NodeJS
JavaScript
18
star
25

py-xls

PyPI xls module
Python
16
star
26

sheet.js.org

sheet.js.org
15
star
27

js-wmf

Windows MetaFile (wmf) processor
TypeScript
14
star
28

bz2

bzip2 for JavaScript
JavaScript
13
star
29

js-vdc

🎧 van der Corput low-discrepancy sequences
HTML
13
star
30

cfb-editor

💼 ZIP/CFB/MIME Archive Editor
JavaScript
9
star
31

node-exit-on-epipe

💥 Cleanly exit on pipe errors
JavaScript
9
star
32

xlsx-nw-demo

node-webkit XLSX demo
JavaScript
7
star
33

rooster

🐓 File filter for version control systems.
Go
7
star
34

docs.sheetjs.com

SheetJS Community Edition Docs repo
HTML
6
star
35

notes

Various file format notes
TypeScript
5
star
36

js-funzip

`funzip` for nodejs
TypeScript
4
star
37

libreoffice_test-files

Mirror of LO Test Files (see https://bugs.freedesktop.org/show_bug.cgi?id=85756)
Python
3
star
38

flat-sheet

demo for https://docs.sheetjs.com/docs/demos/hosting/github
TypeScript
2
star
39

sheetjs-npm-placeholder

Placeholder for the `sheetjs` package on npm
1
star
40

test_files_pres

Presentation Test Files
1
star