scheduleR

scheduleR is a framework that can be used to deploy R tasks, reports and apps.

• Tasks are 'regular' R scripts that you want to schedule to be executed on a regular basis (often ETL related scripts).
• Reports are Rmarkdown (.Rmd) reports that can be converted to a PDF or HTML. See rmarkdown for more info.
• Apps are Shiny apps, support for these in scheduleR is experimental.

An easy web interface for scheduling is provided for adding tasks, maintenance and viewing logs. scheduleR provides extensive logging support and error/success notifications.

scheduleR is built to be used on a server. It can be used locally but that mean that you have to keep a mongodb server and scheduleR running at all times.

Requirements

Minimal dependencies:

• Node.js (with npm)
• R
• mongodb

Optional dependencies (necessary for generating Rmarkdown reports):

• Pandoc (> v1.13.0)
• rmarkdown (most recent version installed using devtools)
• knitr

scheduleR's web interface is built using Node.js and tested under GNU/Linux and Windows. Feedback on Mac compatibility is appreciated.

Installation

Make sure you have access to a running MongoDB server. You can set one up locally or use a service such as mongolab for testing.

First download the repository using the latest tarball / zip file and extract it or simply clone the repository. Cloning the repository is advised as this makes updating scheduleR a piece of cake.

git clone https://github.com/Bart6114/scheduleR.git

Next, run the following command to install dependencies (might require admin rights).

npm install

Known issues

If when using npm install you get an error like Error: ENOENT ... try updating npm to the latest version: npm update -g npm (might require admin rights).

Configuration

To configure scheduleR go to the directory containing the installation and edit the user.config.json file to your likings:

• uploadDir path to upload your scripts to (is essential!)
• RScriptExecutable how to call the Rscript executable (best to simply put Rscript on the path)
• RstandardArguments standard arguments to add to Rscript (should be fine as is)
• errorNotificationMailAddresses (array of) email addresses that will be added to the recipients of error notification mails
• mailer.from the from address to use in notification/report mails
• mailer.options settings for nodemailer to user
• db.url the address of the mongodb server
• db.suffix the name used to create a db on the mongodb server
• port the port to serve scheduleR on

Running scheduleR

You can start scheduleR by simply running in the directory where it was downloaded.

npm start

You can then point your browser at the configured port & server address (default is http://localhost:3000).

User management

At the first start of scheduleR an initial signup option is available. After the initial signup new users can be invited (an account will be created for them). An invite email will be send to them with their username and initial password. If something goes wrong in sending this email (e.g. when you don't have access to a mail server) the new user's username and initial password are shown in the server-side console.

Tasks

The list tasks view gives an overview of all the scheduled tasks. One can choose a specific task to get a detailed overview on run times and logs. The edit mode is also available via the detailed view.

Using new task one can schedule a new script:

• name the name of the task
• description a description of the task
• script file upload the .R file
• extra arguments potential extra arguments to call the script with
• enabled should the task be enabled after saving
• ignore lock if disabled, tasks are locked when executing, meaning that no other instance can be started when previous instance hasn't finished
• mail report to addresses to the resulting report to (as attachment) - make sure to press enter after each address
• schedule the desired schedule (see the cron manual)
• on success addresses to send successful execution notification to - make sure to press enter after each address
• on error addresses to send unsuccesful execution notification to - make sure to press enter after each address

Reports

The list reports view gives an overview of all the scheduled reports. One can choose a specific report to get a detailed overview on run times and logs. The edit mode is also available via the detailed view.

Markdown reports (pdf / html) can be copied to a specified path (either on the server on one that is available on the network) or be sent by mail to a number of recipients.

Using new report one can schedule a new report:

• name the name of the report
• description a description of the script
• script file upload the .Rmd file
• extra arguments potential extra arguments to call the script with
• enabled should the task be enabled after saving
• Rmd output path where to copy the resulting report (optional)
• Rmd mail message a message for the body of the report mail (optional)
• mail report to addresses to the resulting report to (as attachment) - make sure to press enter after each address
• schedule the desired schedule (see the cron manual)
• on success addresses to send successful execution notification to - make sure to press enter after each address
• on error addresses to send unsuccesful execution notification to - make sure to press enter after each address

Shiny app

Beware, shiny app support is very experimental.

When adding new app, the following input fields are available:

• name the name of the app
• description a description of the app
• script file the script file should be a zipped (.zip) file with at is root the required file to call shiny::runApp() on (e.g. server.r and ui.r).
• URL suffix the URL to access the app by, if the suffix is for example test-app you can access the app by using http://address-to-scheduleR/app/test-app.

See an example below:

Test scripts

In the examples directory you can find a few example scripts to test scheduleR's functionality with.

Browser compatibility

Up to now only tested on a recent Chrome browser. Testing on different browsers is appreciated.

Updating

When a new version is available it will be shown in the footer. Updating is easiest when you've done an install using git clone, in this case you can simply do a pull:

git pull

It is a good idea to always run the following after an update:

npm install
npm prune

Issues, questions, feedback

For support and bug reports create a new issue at scheduleR's GitHub site.

Bug fixes are very welcome.

Changes

v0.0.18

• switched scheduling engine from node-cron to schedule-node (cron string specification now more in line with crontab)
• some extra refactoring of the scheduling code

v.0.0.17

• removed 'detached' setting when spawning processes from node (caused some problems on Windows machines)

v0.0.16

• fixed bug where some files didn't get properly renamed before copying
• fixed up some jslint warnings

v0.0.15

• report generation bugfix: files could only be copied if email addresses were present

v.0.0.14

• tasks are locked when executing, meaning that no other instance of the task can be started if the previous instance hasn't finished (behaviour can be overruled in the task's settings, see docs)

v.0.0.13

• fixed bug that didn't correctly show report status in list view
• correct link to reports from 'enabled reports' button
• added list of mail-to addresses to detailed report view

v.0.0.12

• Added experimental support for Shiny apps
• Separated R tasks from R markdown reports
• A lot of refactoring

v.0.0.11

• Added (limited) pagination to logs in detailed task view in order to improve performance
• Improved file renaming method
• Functionality to add timestamps to the filename of generated Rmd reports

v0.0.10

• Small optimizations to retrieving big log lists

v0.0.9

• Inviting users depended on the ability to send emails (hence have access to a mail server). Quick fix: if there is an error in sending the account details to the user the username and initial password will be printed on the server-side console.

v0.0.8

• The filenames of Rmarkdown reports are renamed back to the original filename (instead of the randomly generated upload filename)

v0.0.7

• Added a "notificationMailAddresses" variable to user.config.json, email addresses specified here will be added to the recipients of all notification mails

v0.0.6

• Added a "run once" button to test scheduled tasks
• Added status (success / error) to notification mails