• Stars
    star
    194
  • Rank 200,219 (Top 4 %)
  • Language
    JavaScript
  • License
    MIT License
  • Created over 11 years ago
  • Updated over 6 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A http file downloader made in nodejs

mt-downloader

Join the chat at https://gitter.im/tusharmath/Multi-threaded-downloader Build Status npm Commitizen friendly semantic-release js-standard-style Coverage Status

This is a nodejs based utility library that helps in performing resumable, multi-threaded downloads over Http. The module is highly inspired by Speedbit's — Download Accelerator Plus.

Features

  1. Multi connection downloads: In a conventional download we don't completely utilize our bandwidth. With multiple connections there is always one thread which is getting data thus minimizing the wait period between data packets. This doesn't mean that we will be able to download faster than what our ISP allows.

  2. Stop and start from the last downloaded byte: You don't have to worry about internet getting disconnected or your computer shutting down while downloading. You can quite easily start from the last byte that was downloaded.

  3. Console application: If installed globally, mtd command would be available.

Installation

The conventional npm installation process needs to be followed.

npm install mt-downloader --save

CLI Installation

$ npm install -g mt-downloader
$ mtd --help

Objects

Functions

FILE : object

Provides wrappers over the async utils inside the fs module. The wrappers take in an input stream of arguments and returns the result of function call as another stream.

Kind: global namespace

FILE.open(params$) ⇒ Observable

Kind: static method of FILE

Param Type
params$ Observable

FILE.fstat(params$) ⇒ Observable

https://nodejs.org/api/fs.html#fs_fs_open_path_flags_mode_callback

Kind: static method of FILE

Param Type
params$ Observable

FILE.read(params$) ⇒ Observable

https://nodejs.org/api/fs.html#fs_fs_read_fd_buffer_offset_length_position_callback

Kind: static method of FILE

Param Type
params$ Observable

FILE.write(params$) ⇒ Observable

https://nodejs.org/api/fs.html#fs_fs_write_fd_buffer_offset_length_position_callback

Kind: static method of FILE

Param Type
params$ Observable

FILE.close(params$) ⇒ Observable

https://nodejs.org/api/fs.html#fs_fs_close_fd_callback

Kind: static method of FILE

Param Type
params$ Observable

FILE.truncate(params$) ⇒ Observable

https://nodejs.org/api/fs.html#fs_fs_truncate_path_len_callback

Kind: static method of FILE

Param Type
params$ Observable

FILE.rename(params$) ⇒ Observable

https://nodejs.org/api/fs.html#fs_fs_rename_oldpath_newpath_callback

Kind: static method of FILE

Param Type
params$ Observable

HTTP : object

Kind: global namespace

HTTP.request(params) ⇒ Observable

Stream based wrapper over npm/request

Kind: static method of HTTP
Returns: Observable - multiplex stream

Param Type Description
params object request module params.

CreateMTDFile(options) ⇒ Observable

Creates a new .mtd file that is a little larger in size than the original file. The file is initially empty and has all the relevant meta information regarding the download appended to the end.

Kind: global function
Returns: Observable - A multiplexed stream containing ~

  • written$ - Bytes being saved on disk.
  • meta$ - Meta information about the download.
  • remoteFileSize$ - Size of the content that is to be downloaded.
  • fdW$ - File descriptor in w mode.
Param Type Default Description
options object The options must have mtdPath and url.
options.url string Download url.
options.path string Relative path where the file needs to be saved.
[options.range] number 3 Number of concurrent downloads.
[options.metaWrite] number 300 Throttles the write frequency of meta data.

DownloadFromMTDFile(mtdPath, [meta]) ⇒ Observable

Reads a .mtd file and resumes the download from the last successfully saved byte.

Kind: global function
Returns: Observable - A multiplexed stream containing ~

  • metaWritten$ - Meta data buffer stream.
  • response$ - HTTP response object.
  • responses$ - List of all the HTTP response objects.
  • localFileSize$ - Size of the .mtd file on disk.
  • fdR$ - File Descriptor in r+ mode.
  • meta$ - Download meta information.
Param Type Description
mtdPath String Relative path to the .mtd file.
[meta] Object Optional meta data to override the one that's being loaded from the .mtd file.

FinalizeDownload(params) ⇒ Observable

Removes the meta information and the .mtd extension from the file once the download is successfully completed.

Kind: global function
Returns: Observable - A multiplexed stream containing ~

  • truncated$ - Fired when the meta data is removed.
  • renamed$ - Fired when the .mtd extension is removed.
Param Type Description
params object {fd$, meta$}
params.fd$ Observable File descriptor Observable
params.meta$ Observable Download meta information

Completion(meta$) ⇒ Observable

Util method that calculates the total completion percentage (between 0-100).

Kind: global function
Returns: Observable - Value between 0-100

Param Type Description
meta$ Observable Meta data stream ie. exposed by DownloadFromMTDFile

.mtd file

Once the download starts the library will create a file with a .mtd extension. This file contains some meta information related to the download and is a little bigger (around 4kb) than the original file size. The .mtd file can be used later to restart downloads from where the last byte that was downloaded. After the download is completed the downloader will truncate the .mtd file to remove that meta information.

Releases

There are two release channels viz. — latest and next. The next is unstable and must be used with caution.

Latest

npm i mt-downloader

Next

npm i mt-downloader@next

More Repositories

1

qio

TypeScript
72
star
2

ts-codemod

Typescript based codemods
TypeScript
66
star
3

node-config-ts

A simple node configuration manager
TypeScript
57
star
4

rwc

Reactive Web Components (RWC)
TypeScript
50
star
5

observable-air

Observable for the calorie conscious
TypeScript
34
star
6

Super-Cache

Enables caching of static content on client side irrespective of the implemented server side caching behavior
JavaScript
27
star
7

ioc

Dependency injection with support for inheritence
JavaScript
11
star
8

tts-schema

Reify your types in Typescript
TypeScript
10
star
9

mtd-console

A console application for mt-downloader
JavaScript
10
star
10

reactive-dom

A reactive ui library
TypeScript
9
star
11

funjector

Dependency injection for functional paradigm
JavaScript
7
star
12

scala-operator-names

List of all the popular Scala operators, their names.
6
star
13

hoe

Module to emit higher order events (HOE)
TypeScript
5
star
14

muxer

stream multiplexer for es6 observables
JavaScript
5
star
15

funag

Testing the limits of a progressive web application
JavaScript
5
star
16

scalafix-alphabetize

Alphabetically sorts fields and methods.
Scala
5
star
17

ts-curry

Proper type safe way of currying functions in typescript
TypeScript
4
star
18

html-scraper

Scrape data using method chaining
CoffeeScript
4
star
19

react-announce-fetch

a react-announce declarative to create REST based data stores
JavaScript
3
star
20

react-announce

a declarative approach to creating react components
JavaScript
3
star
21

react-render-if

declarative way to conditionally render a react component
JavaScript
3
star
22

histable

A no external dependency module that maintains the history of state changes.
JavaScript
3
star
23

react-announce-connect

Applies a stream to a component's state.
JavaScript
3
star
24

reactive-storage

a simple state based store that is exposed as a stream
JavaScript
3
star
25

ts-scheduler

A job scheduler for node and browser environment
TypeScript
2
star
26

super-gmail

Make gmail like superhuman
TypeScript
2
star
27

react-announce-size

a react-announce declarative that exposes component size as a stream
JavaScript
2
star
28

flex-jss

flexbox fun with jss
JavaScript
2
star
29

logistic-progressor

A sigmoid function based progress bar
JavaScript
2
star
30

rx-raf-throttle

requestAnimationFrame based stream throttler
JavaScript
2
star
31

standard-data-structures

https://tusharmath.com/standard-data-structures
TypeScript
1
star
32

side-nav-vanilla

side nav web component
TypeScript
1
star
33

action-type

Type class for actions
TypeScript
1
star
34

adb-fastboot-install

Automatically exported from code.google.com/p/adb-fastboot-install
1
star
35

movable-tabs

webcomponent for material design like tabs
JavaScript
1
star
36

vagrant-node

A set of simple vagrant provisioning tools for node.js
Shell
1
star
37

zio-zeromq

A ZIO-based interface to ZeroMQ.
Scala
1
star
38

kips

Yeoman Generator for Typescript Based Projects
HTML
1
star
39

mean-ads

CoffeeScript
1
star
40

react-announce-draggable

expose component drag and drop events as a stream
JavaScript
1
star
41

react-announce-collapse

a reusable decorator to create collapsable components
JavaScript
1
star
42

dbl-linked-list-ds

A typescript based doubly linked list data structure
TypeScript
1
star
43

node-vs-vertx

Scala
1
star
44

update-function-types

Provides a specification and basic type classes for update function.
TypeScript
1
star
45

interview-scheduler

An interview scheduler
TypeScript
1
star
46

checked-exceptions

A utility library to create and manage checked exceptions in typescript
TypeScript
1
star
47

ComponentNext

POC Repo for ComponentNext | ActionLand
TypeScript
1
star
48

chaining-tatum

A module to enable method chaining of async tasks
CoffeeScript
1
star
49

action-component

Action based components
TypeScript
1
star