๐ Vanilla Clipper
Vanilla Clipper is a Node.js library to completely save a webpage to local with Puppeteer. You can save all the contents in the page such as images, videos, CSS, web fonts, iframes, and Shadow DOMs with one command.
Dependencies
- Node.js (>= 8.10)
- Chrome or Chromium (Latest version)
Installation
yarn global add vanilla-clipper
# or
npm i -g vanilla-clipperUsage
CLI
Note: If it fails to launch, try adding --no-sandbox (-n) option.
-
Save https://example.com:
vanilla-clipper https://example.com
-
Save
.timelineelement in https://example.com totechdirectory (Set browser language to Japanese):vanilla-clipper -d tech -s .timeline -l ja-JP https://example.com
-
Login with
subaccount in the config file:vanilla-clipper -a sub https://example.com
See here for details of the options.
๐ Directory structure in ~/.vanilla-clipper
๐ .vanilla-clipper
๐ pages
๐ main
๐ 20190213-page1.html
๏ธ
๐ {SOME_FOLDER}
๐ 20190213-page2.html
๐ 20190214-page3.html
๏ธ
๐ resources
๐ 20190213
๐ {ulid}.jpg
๐ {ulid}.svg
๏ธ
๐ 20190214
๐ {ulid}.woff2
๏ธ
๐ resources.json
๐ config.json
โ๏ธ Config file example
{YOUR_HOME_DIRECTORY}/.vanilla-clipper/config.js
module.exports = {
resource: { maxSize: 50 * 1024 * 1024 },
sites: [
{
url: 'example.com', // site URL
accounts: {
default: {
// โ account label
username: 'main', // or () => 'main'
password: 'password1',
},
sub: {
// โ account label
username: 'sub_account',
password: 'password2',
},
},
login: [
// [action, arg1, arg2, ...]
[
'goto',
'https://example.com/login', // URL
],
[
'input',
'input[name="session[username_or_email]"]', // selector
'$username', // -> accounts.{ACCOUNT_LABEL}.username
],
[
'input',
'input[name="session[password]"]', // selector
'$password', // -> accounts.{ACCOUNT_LABEL}.password
],
[
'submit',
'[role=button]', // selector
],
],
},
],
}