Online course starter: R

This is a starter repo based on the course framework I developed for my spaCy course. The front-end is powered by Gatsby and Reveal.js and the back-end code execution uses Binder 💖

This repo could use some better code examples. Also, if you have experience with R, feel free to suggest improvements to the test logic and template. It all works as expected, but there might be ways to make it more elegant.

✅ Quickstart

1. Import this repo, install it and make sure the app is running locally.
2. Customize the meta.json and binder/install.R.
3. Build a Binder from the binder branch of this repo.
4. Add content (chapters, exercises and slides) and optionally add separate content license.
5. Customize the UI theme in theme.sass and update images in static as needed.
6. Deploy the app, e.g. to Netlify.

Running the app

To start the local development server, install Gatsby and then all other dependencies. This should serve up the app on localhost:8000.

npm install -g gatsby-cli  # Install Gatsby globally
npm install                # Install dependencies
npm run dev                # Run the development server

💡Introduction

This section was contributed by @laderast. Thanks! ✨

How does this repo work?

The course repository works with two components: Gatsby (front-end), and Binder (back-end). We'll go over both of these to understand how it works as a whole.

What is Gatsby?

Gatsby is a JavaScript/react.js based web page building framework, like Hugo, or Jekyll. The nice thing about it being JavaScript is that JavaScript widgets that you build are tightly integrated.

You can think of Gatsby as being the client side of the lesson framework. All code, solutions, tests, and chapter.md files are handled by Gatsby.

What is Binder?

Binder is a way of building Docker containers from repositories that can be launched on a remote server/cluster (such as mybinder.org). This Docker container can be based on a Dockerfile, or R image. The thing about Binder is that the containers are ephemeral - if they're not used, they're deleted off the Binder servers. The main applications of Binder are:

1. Reproducible Research (shareable notebooks) and
2. Education (shareable notebooks)

You can think of Binder as being the server side of the lesson framework. It needs the instructions on how to build the docker container (which is in the binder/ folder), and the data you want to use for the lessons (in the data/ folder).

The only thing you need to get started with R and Binder is a repo that has a runtime.txt file, or a Dockerfile. The rest, such as datafiles, are optional, but are usually contained in a Binder repository.

How does Gatsby work with Binder?

Ines was super clever and designed a JavaScript plugin for Gatsby called Juniper to handle communication to and from the Binder container using Jupyter kernels. That's how code gets executed on the Binder container, and how code output (such as terminal messages, images, etc) are received from the Binder container.

Branches of this repo

There are two branches of this repo, which are used for different tasks:

• master - this is what the course is served out of via netlify: http://r-bootcamp.netlify.com - any changes to exercises in this branch will show up on the netlify page. The netlify page uses a JavaScript framework called Gatsby to build the pages. Gatsby submits code to binder and receives the output. It also handles the code checking. The parts of the repo that are handled by Gatsby include:

1. exercises/solutions/tests
2. chapter.md files
3. slides (using reveal.js)

• binder - this is what the Binder image is built from. The reason they're different is that binder forces a Docker container rebuild when a branch is updated. So, if we served our container out of master, it would rebuild everytime we modified a chapter.md or an exercise. If you need to add packages, you will add them to the binder/install.R for this branch, and if you need to add datasets, you can add them to the data/ folder. The parts of the repo handled by Binder include:

1. datasets in data/ folder (the container needs access to these to load data from submitted code)
2. installation instructions in the binder/ folder for installing dependencies

I would say that the easiest thing to do is to occasionally merge master into binder when you need to update the data:

Note that rebuilding the binder container can take a little bit of time (usually on the order of 5 or 10 minutes or so), since it is installing/compiling tidyverse for the container. You can always check the build status of the container by clicking the badge below and looking at the log.

You can view the binder container here: or at: https://mybinder.org/v2/gh/ines/course-starter-r/binder

• you can launch an Rstudio instance to test the container by using the "new" tab in the top right corner, and selecting 'Rstudio'. This is super helpful if you want to test if code will work in the binder container.

Adding Packages

If you need to add packages, add the appropriate install.packages() statement into binder/install.R. When you do, check that the container was built properly by clicking the binder link above.

Currently, tidyverse is installed in the binder container.

data/ folder

If you want to access datasets in the data folder, you can always refer to this folder as data/. For example, to use data/pets.csv:

pets <- read.csv("data/pets.csv")

Remember, if you need to add a dataset to the repo, it needs to be done in the binder branch into the data/ folder.

Using decampr to transfer your DataCamp repository

If you would like to transfer your courses from DataCamp, there is a package made for that: decampr. It will scan your repository and attempt to extract exercise instructions, quizzes, exercise code, and solutions and write them to the appropriate directory for your project. For more info, please check out the decampr repo: http://github.com/laderast/decampr

🎨 Customization

The app separates its source and content – so you usually shouldn't have to dig into the JavaScript source to change things. The following points of customization are available:

meta.json

The following meta settings are available. Note that you have to re-start Gatsby to see the changes if you're editing it while the server is running.

Static assets

All files added to /static will become available at the root of the deployed site. So /static/image.jpg can be referenced in your course as /image.jpg. The following assets need to be available and can be customized:

✏️ Content

File formats

Chapters

Chapters are placed in /chapters and are Markdown files consisting of <exercise> components. They'll be turned into pages, e.g. /chapter1. In their frontmatter block at the top of the file, they need to specify type: chapter, as well as the following meta:

---
title: The chapter title
description: The chapter description
prev: /chapter1 # exact path to previous chapter or null to not show a link
next: /chapter3 # exact path to next chapter or null to not show a link
id: 2 # unique identifier for chapter
type: chapter # important: this creates a standalone page from the chapter
---

Slides

Slides are placed in /slides and are markdown files consisting of slide content, separated by ---. They need to specify the following frontmatter block at the top of the file:

---
type: slides
---

The first and last slide use a special layout and will display the headline in the center of the slide. Speaker notes (in this case, the script) can be added at the end of a slide, prefixed by Notes:. They'll then be shown on the right next to the slides. Here's an example slides file:

---
type: slides
---

# Processing pipelines

Notes: This is a slide deck about processing pipelines.

---

# Next slide

- Some bullet points here
- And another bullet point

<img src="/image.jpg" alt="An image located in /static" />

Custom Elements

When using custom elements, make sure to place a newline between the opening/closing tags and the children. Otherwise, Markdown content may not render correctly.

<exercise>

Container of a single exercise.

<exercise id="1" title="Introduction to spaCy">

Content goes here...

</exercise>

<codeblock>

<codeblock id="02_03">

This is a hint!

</codeblock>

<slides>

Container to display slides interactively using Reveal.js and a Markdown file.

<slides source="chapter1_01_introduction-to-spacy">
</slides>

<choice>

Container for multiple-choice question.

<choice>

<opt text="Option one">You have selected option one! This is not good.</opt>
<opt text="Option two" correct="true">Yay! </opt>

</choice>

<opt>

A multiple-choice option.

Setting up Binder

The install.R in the repository defines the packages that are installed when building it with Binder. You can specify the binder settings like repo, branch and kernel type in the "juniper" section of the meta.json. I'd recommend running the very first build via the interface on the Binder website, as this gives you a detailed build log and feedback on whether everything worked as expected. Enter your repository URL, click "launch" and wait for it to install the dependencies and build the image.

Adding tests

To validate the code when the user hits "Submit", we're currently using a slightly hacky trick. Since the R code is sent back to the kernel as a string, we can manipulate it and add tests – for example, exercise exc_01_02_01.R will be validated using test_01_02_01.R (if available). The user code and test are combined using a string template. At the moment, the testTemplate in the meta.json looks like this:

success <- function(text) {
    cat(paste("\033[32m", text, "\033[0m", sep = ""))
}

.solution <- "${solutionEscaped}"

${solution}

${test}
tryCatch({
    test()
}, error = function(e) {
    cat(paste("\033[31m", e[1], "\033[0m", sep = ""))
})

If present, ${solution} will be replaced with the string value of the submitted user code, and ${solutionEscaped} with the code but with all " replaced by \", so we can assign it to a variable as a string and check whether the submission includes something. We also insert the regular solution, so we can actually run it and check the objects it creates. ${test} is replaced by the contents of the test file. The template also defines a success function, which prints a formatted green message and can be used in the tests. Finally, the tryCatch expression checks if the test function raises a stop and if so, it outputs the formatted error message. This also hides the full error traceback (which can easily leak the correct answers).

A test file could then look like this:

test <- function() {
    if (some_var != length(mtcars)) {
        stop("Are you getting the correct length?")
    }
    if (!grepl("print(mtcars$gear)", .solution, fixed = TRUE)) {
        stop("Are you printing the correct variable?")
    }
    success("Well done!")
}

The string answer is available as .solution, and the test also has access to the solution code.

For more details on how it all works behind the scenes, see the original course repo.