live-plugin-manager
live-plugin-manager
is a Node.js module that allows you to
install, uninstall and load any node package at runtime from npm registry.
My main goal is to allow any application to be easily extensible by installing and running any node package at runtime, without deployments or server customization. You can basically allow your users to decide what plugin/extension to install.
Main features are:
- Install and run Node packages at runtime (no deployment)
- Install from npm registry (private or public)
- Install from filesystem
- Install from github (branch, commit or tag)
- Update/uninstall packages
- Any Node.js packages can be installed
- No special configuration is required
- Installed packages can have dependencies
- dependencies are automatically installed
- when updating a dependencies all dependents are reloaded
- Support for concurrency operation on filesystem (cloud/web farm scenario where file system is shared)
- A filesystem lock is used to prevent multiple instances to work on the same filesystem in the same moment
- Each package run in an semi isolated environment (VM sandbox)
- Set different environment variables for each package
- Implemented in TypeScript
- Fully tested (mocha tests)
There are some known limitations, see section at the end.
Installation
npm install live-plugin-manager
Usage
import {PluginManager} from "live-plugin-manager";
const manager = new PluginManager();
async function run() {
await manager.install("moment");
const moment = manager.require("moment");
console.log(moment().format());
await manager.uninstall("moment");
}
run();
In the above code I install moment
package at runtime, load and execute it.
Plugins are installed inside the directory specified in the PluginManager
constructor or in the plugin_packages
directory if not specified.
Each time your application start you should reinstall any packages that you need; already downloaded packages are not automatically installed, but installation is faster because no new file is downloaded. Typically I suggest to put the list of the installed packages in a database or any other central repository.
Here another more complex scenario where I install express
with all it's dependencies, just to demonstrate how many possibilities you can have:
import {PluginManager} from "live-plugin-manager";
const manager = new PluginManager();
async function run() {
await manager.install("express");
const express = manager.require("express");
const app = express();
app.get("/", function(req: any, res: any) {
res.send("Hello World!");
});
const server = app.listen(3000, function() {
});
}
run();
Load plugins
live-plugin-manager
doesn't have any special code or convention to load plugins.
When you require a plugin it just load the main file (taken from package.json
) and execute it, exactly like standard node.js require
.
Often when working with plugins you need some extension point or convention to actually integrate your plugins inside your host application. Here are some possible solutions:
Another solution is to load your plugins inside a Dependency Injection container.
I'm working also on shelf-dependency, a simple dependency injection/inversion of control container that can be used to load plugins.
Samples
- See
./samples
and./test
directories - Sample of an extensible web application: lpm-admin
Reference
PluginManager.constructor(options?: Partial<PluginManagerOptions>)
Create a new instance of PluginManager
. Takes an optional options
parameter with the following properties:
cwd
: current working directory (default toprocess.cwd()
)pluginsPath
: plugins installation directory (default to.\plugin_packages
, seecwd
). Directory is created if not existsnpmRegistryUrl
: npm registry to use (default to https://registry.npmjs.org)npmRegistryConfig
: npm registry configuration see npm-registry-client configignoredDependencies
: array of string or RegExp with the list of dependencies to ignore, default to@types/*
staticDependencies
: object with an optional list of static dependencies that can be used to force a dependencies to be ignored (not installed when a plugin depend on it) and loaded from this listgithubAuthentication
: Github authentication configuration (optional). SeeinstallFromGithub
or Github api authentication for more info.lockWait
: A number of milliseconds to wait for locks when installing modules to expire before giving up. (default 2 min)lockStale
: A number of milliseconds before installations locks are considered to have expired. (default 3 min)
pluginManager.install(name: string, version?: string): Promise<IPluginInfo>
Install the specified package from npm registry or directly from github repository.
version
parameter can be a version like 1.0.3
or a github repository in the format owner/repository_name#ref
(like expressjs/express#351396f
).
If a version is specified package is downloaded from NPM.
Dependencies are automatically installed (devDependencies are ignored).
pluginManager.installFromNpm(name: string, version = "latest"): Promise<IPluginInfo>
Install the specified package from npm registry. Dependencies are automatically installed (devDependencies are ignored).
By default is the package is already available in the download folder then and version is compatible then it is not requested again from npm. Change this behavior by setting npmInstallMode: "noCache"
options.
To setup authentication for private npm registry use:
const manager = new PluginManager({
npmRegistryUrl: "http://your-private-registry",
npmRegistryConfig: {
auth: {
token: "your-token"
}
}
});
pluginManager.installFromGithub(repository: string): Promise<IPluginInfo>
Install the specified package from github. repository
is specified in the format owner/repository_name#ref
(like expressjs/express#351396f
).
Dependencies are automatically installed (devDependencies are ignored).
Note: Github has an API rate limit of 60 calls if not authenticated. To authenticate calls just set the githubAuthentication
property in the options:
manager = new PluginManager({
githubAuthentication: {
"type": "basic",
"username": "YOUR_USER",
"password": "YOUR_PERSONAL_TOKEN"
}
});
See Github api authentication for more info.
installFromPath(location: string, options: {force: boolean} = {}): Promise<IPluginInfo>
Install the specified package from a filesystem location. Dependencies are automatically installed from npm. node_modules
folder is excluded from source location.
installFromCode(name: string, code: string, version?: string): Promise<IPluginInfo>
Install a package by specifying code directly. If no version is specified it will be always reinstalled.
uninstall(name: string): Promise<void>
Uninstall the package. Dependencies are not uninstalled automatically.
uninstallAll(): Promise<void>
Uninstall all installed packages.
list(): Promise<IPluginInfo[]>
Get the list of installed packages.
require(name: string): any
Load and get the instance of the plugin. Node.js require
rules are used to load modules.
Calling require multiple times gives always the same instance until plugins changes.
getInfo(name: string): IPluginInfo | undefined
Get information about an installed package.
runScript(code: string): any
Run the specified Node.js javascript code with the same context of plugins. Script are executed using vm
as with each plugin.
async queryPackage(name: string, version?: string): Promise
Get package/module info from npm registry or github depending of the version format.
async queryPackageFromNpm(name: string, version = "latest"): Promise
Get package/module info from npm registry.
async queryPackageFromGithub(repository: string): Promise
Get package/module info from github registry.
Security
Often is a bad idea for security to allow installation and execution of any node.js package inside your application. When installing a package it's code is executed with the same permissions of your host application and can potentially damage your entire server. I suggest usually to allow to install only a limited sets of plugins or only allow trusted administrator to install plugins.
Sandbox
For advanced scenarios you can customize the sandbox where each plugin run. Sandbox defined NodeJS global
object.
A typical scenario is to customize the environment variables global.process.env
. To customize the sandbox you can set a single
sandbox for all plugins using options.sandbox
in the PluginManager
constructor or set a different sandbox for each plugin, see
PluginManager.setSandboxTemplate
and PluginManager.getSandboxTemplate
functions.
Under to hood
This project use the following dependencies to do it's job:
- vm: compiling and running plugin code within V8 Virtual Machine contexts
- lockfile: file system locking to prevent concurrent operations (see below)
- tar.gz: extract package file
- fs-extra: file system operations
- debug: debug information
- (removed for now because I have problems with getArchiveLink api, 302 ...) github
- (removed for now because for some dependencies issues) npm-registry-client: npm registry handling
While I have tried to mimic the standard Node.js module and package architecture there are some differences.
First of all is the fact that plugins by definition are installed at runtime in contrast with a standard Node.js application where modules are installed before executing the node.js process.
Modules can be loaded one or more times, instead in standard Node.js they are loaded only the first time that you require
it.
Only one version of a plugin can be installed, also for dependencies, while in Node.js multiple version can be installed (each module can have it's own node_modules
).
Locking
A file system locking is implemented using lockfile library to allow multiple app instances to share the file system. This is common in some cloud scenario (for example Azure) where file system is shared between instances.
Locking is used to ensure that only one instance is installing/uninstalling plugins in a given moment.
You can configure lockWait
inside constructor to configure lock timeout, and lockStale
to consider a file lock no more valid after that period.
Git integration
Remember to git ignore the directory where packages are installed, see pluginsPath
.
Usually you should add this to your .gitignore
file:
plugin_packages
Known limitations
There are some known limitations when installing a package:
- Different plugins cannot depend on different/incompatible modules. If plugin A require module x at version 1 and plugin B require the same module X at version 2 then plugin A and plugin B cannot be installed simultaneously. Version 2 of module X will be used and this can cause problems on your code.
process.on
has been stubbed, so plugins that use these events may not work as expected.- No
pre/post-install
scripts are executed (some packages use this scripts to build assets or for platform specific installation, so for this reason some packages are not supported) - C/C++ packages (
.node
) are not supported - Plugin dependencies can be specified only as NPM dependencies (version number) or github dependencies (owner/repo), url or other kind of dependencies are not supported
If you find other problems please open an issue.
Development
Compile typescript using:
npm run src-build
Run tests using:
npm run test
Due to some github rate limits, github related tests can fail if you don't specify a github token. To specify it use:
github_auth_username=YOURUSER github_auth_token=YOUR_TOKEN npm test
The token must only have public access permission.
License (MIT)
MIT License
Copyright (c) 2021 Davide Icardi
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.