mt-downloader
This is a nodejs based utility library that helps in performing resumable, multi-threaded downloads over Http. The module is highly inspired by Speedbit's β Download Accelerator Plus.
Features
-
Multi connection downloads: In a conventional download we don't completely utilize our bandwidth. With multiple connections there is always one thread which is getting data thus minimizing the wait period between data packets. This doesn't mean that we will be able to download faster than what our ISP allows.
-
Stop and start from the last downloaded byte: You don't have to worry about internet getting disconnected or your computer shutting down while downloading. You can quite easily start from the last byte that was downloaded.
-
Console application: If installed globally,
mtdcommand would be available.
Installation
The conventional npm installation process needs to be followed.
npm install mt-downloader --saveCLI Installation
$ npm install -g mt-downloader
$ mtd --helpObjects
Functions
- CreateMTDFile(options) β
Observable - DownloadFromMTDFile(mtdPath, [meta]) β
Observable - FinalizeDownload(params) β
Observable - Completion(meta$) β
Observable
FILE : object
Provides wrappers over the async utils inside the fs module. The wrappers take in an input stream of arguments and returns the result of function call as another stream.
Kind: global namespace
- FILE :
object
FILE.open(params$) β Observable
Kind: static method of FILE
| Param | Type |
|---|---|
| params$ | Observable |
FILE.fstat(params$) β Observable
https://nodejs.org/api/fs.html#fs_fs_open_path_flags_mode_callback
Kind: static method of FILE
| Param | Type |
|---|---|
| params$ | Observable |
FILE.read(params$) β Observable
https://nodejs.org/api/fs.html#fs_fs_read_fd_buffer_offset_length_position_callback
Kind: static method of FILE
| Param | Type |
|---|---|
| params$ | Observable |
FILE.write(params$) β Observable
https://nodejs.org/api/fs.html#fs_fs_write_fd_buffer_offset_length_position_callback
Kind: static method of FILE
| Param | Type |
|---|---|
| params$ | Observable |
FILE.close(params$) β Observable
https://nodejs.org/api/fs.html#fs_fs_close_fd_callback
Kind: static method of FILE
| Param | Type |
|---|---|
| params$ | Observable |
FILE.truncate(params$) β Observable
https://nodejs.org/api/fs.html#fs_fs_truncate_path_len_callback
Kind: static method of FILE
| Param | Type |
|---|---|
| params$ | Observable |
FILE.rename(params$) β Observable
https://nodejs.org/api/fs.html#fs_fs_rename_oldpath_newpath_callback
Kind: static method of FILE
| Param | Type |
|---|---|
| params$ | Observable |
HTTP : object
HTTP.request(params) β Observable
Stream based wrapper over npm/request
Kind: static method of HTTP
Returns: Observable - multiplex stream
| Param | Type | Description |
|---|---|---|
| params | object |
request module params. |
CreateMTDFile(options) β Observable
Creates a new .mtd file that is a little larger in size than the original file. The file is initially empty and has all the relevant meta information regarding the download appended to the end.
Kind: global function
Returns: Observable - A multiplexed stream containing ~
written$- Bytes being saved on disk.meta$- Meta information about the download.remoteFileSize$- Size of the content that is to be downloaded.fdW$- File descriptor inwmode.
| Param | Type | Default | Description |
|---|---|---|---|
| options | object |
The options must have mtdPath and url. |
|
| options.url | string |
Download url. | |
| options.path | string |
Relative path where the file needs to be saved. | |
| [options.range] | number |
3 |
Number of concurrent downloads. |
| [options.metaWrite] | number |
300 |
Throttles the write frequency of meta data. |
DownloadFromMTDFile(mtdPath, [meta]) β Observable
Reads a .mtd file and resumes the download from the last successfully saved
byte.
Kind: global function
Returns: Observable - A multiplexed stream containing ~
metaWritten$- Meta data buffer stream.response$- HTTP response object.responses$- List of all the HTTP response objects.localFileSize$- Size of the.mtdfile on disk.fdR$- File Descriptor inr+mode.meta$- Download meta information.
| Param | Type | Description |
|---|---|---|
| mtdPath | String |
Relative path to the .mtd file. |
| [meta] | Object |
Optional meta data to override the one that's being loaded from the .mtd file. |
FinalizeDownload(params) β Observable
Removes the meta information and the .mtd extension from the file once the
download is successfully completed.
Kind: global function
Returns: Observable - A multiplexed stream containing ~
truncated$- Fired when the meta data is removed.renamed$- Fired when the.mtdextension is removed.
| Param | Type | Description |
|---|---|---|
| params | object |
{fd$, meta$} |
| params.fd$ | Observable |
File descriptor Observable |
| params.meta$ | Observable |
Download meta information |
Completion(meta$) β Observable
Util method that calculates the total completion percentage (between 0-100).
Kind: global function
Returns: Observable - Value between 0-100
| Param | Type | Description |
|---|---|---|
| meta$ | Observable |
Meta data stream ie. exposed by DownloadFromMTDFile |
.mtd file
Once the download starts the library will create a file with a .mtd extension. This file contains some meta information related to the download and is a little bigger (around 4kb) than the original file size. The .mtd file can be used later to restart downloads from where the last byte that was downloaded. After the download is completed the downloader will truncate the .mtd file to remove that meta information.
Releases
There are two release channels viz. β latest and next. The next is unstable and must be used with caution.
Latest
npm i mt-downloaderNext
npm i mt-downloader@next