• Stars
    star
    111
  • Rank 307,628 (Top 7 %)
  • Language
    Shell
  • License
    MIT License
  • Created over 5 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

System requirements for R packages

System Requirements for R Packages

CI Status

R packages can depend on one another, but they can also depend on software external to the R ecosystem. On Ubuntu 18.04, for example, in order to install the curl R package, you must have previously run apt-get install libcurl. R packages often note these dependencies inside their DESCRIPTION files, but this information is free-form text that varies by package.

This repository contains a catalog of "rules" that can be used to systematically identify these dependencies and generate commands to install them.

You may be expecting to see a list like:

Package SystemRequirements Field Dependency
rgdal "for building from source: GDAL >= ..." libgdal-dev

Storing this information as a table in this format is not efficient. Many R packages do not have any system dependencies, so the table would be very sparse. Moreover, R packages are added at an exponential rate, so maintaining this data would be nearly impossible.

Instead, this repository contains a set of rules that map a SystemRequirements field, e.g. rgdal's "for building from source: GDAL >= 1.11.4 and <= 2.5.0, library from ..." to a platform specific install command: apt-get install libgdal-dev gdal-bin libproj-dev.

Usage

The primary purpose of this catalog is to support RStudio Package Manager which knows how to translate these rules into install steps for specific packages or repositiories. However, the community is free to use and contribute to these rules subject to the MIT license.

RStudio Package Manager is professionally supported, but RStudio does not offer support for these rules. Please file questions in RStudio Community or open an issue in this repository.

A similar project is maintained by R-Hub. The two catalogs have different data formats, test coverage, and target different operating systems.

Rule Coverage

The rules presented in this repository are extensively tested with the following process:

  1. A Docker container is started with a minimal base R image.
  2. A target R package is identified. The catalog of rules is applied to install any known requirements for the package into the Docker container.
  3. The package is installed.

If the package install is successful, there is a high chance the existing rules are sufficient. If the install fails, there is an indication that a rule is missing. This process is repeated for all CRAN packages across 6 Linux distributions: Ubuntu 16/18, CentOS 7/8, openSUSE 42/15.

The results are summarized below:

Percentage of CRAN Packages that Install Successfully

Ubuntu 16 Ubuntu 18 CentOS 7 CentOS 8 openSUSE 42.3 openSUSE 15.0
No Rules 78% 78.1% 77.8% 77.7% 78.2%
With Rules 93.5% 95.8% 93.7% 88.5% 89.7%

Percentage Weighted by Downloads

This table contains similar results as the table above, but adjusted by download. This metric indicates how good the rules are for the majority of packages R users are likely to install, discounting the long tail of packages that have system requirements but are not frequently used.

Ubuntu 16 Ubuntu 18 CentOS 7 CentOS 8 openSUSE 42.3 openSUSE 15.0
No Rules 90.1% 90.1% 90.1% 90% 90.2%
With Rules 98.5% 99.2% 98.6% 96.1% 96.3%

Both tests run with R 3.5.3 for all CRAN packages as of April 4, 2019.

Operating Systems

The rules in this catalog support the following operating systems:

  • Ubuntu 20.04, 22.04
  • CentOS 7
  • Rocky Linux 8*, 9
  • Red Hat Enterprise Linux 7, 8, 9
  • openSUSE 15.4, 15.5
  • SUSE Linux Enterprise 15 SP4, 15 SP5
  • Debian 10, 11, 12, unstable
  • Fedora 36, 37, 38
  • Windows (for R 4.0+ only)

* Rocky Linux 8 is specified as centos8 for backward compatibility. CentOS 8 reached end of support on December 31, 2021.


For Developers

We welcome contributions to this catalog! To report a bug or request a rule, please open an issue in this repository. To add or update a rule, fork this repository and submit a pull request.

Overview

Each system requirement rule is described by a JSON file in the rules directory. The file is named rule-name.json, where rule-name is typically the name of the system dependency.

For example, here's an excerpt from a rule for the Protocol Buffers (protobuf) library at rules/libprotobuf.json.

{
  "patterns": ["\\blibprotobuf\\b"],  // regex which matches "libprotobuf" or "LIBPROTOBUF; libxml2"
  "dependencies": [
    {
      "packages": ["protobuf-devel"],  // to install the package: "yum install protobuf-devel"
      "pre_install": [
        {
          "command": "yum install -y epel-release"  // add the EPEL repository before installing
        }
      ],
      "constraints": [
        {
          "os": "linux",
          "distribution": "centos",  // make these instructions specific to CentOS 7
          "versions": ["7"]
        }
      ]
    }
  ]
}

Other examples:

  • Simple rule: git.json
  • OS version constraints (package names vary by OS version): libmysqlclient.json
  • Pre-install steps (adding the EPEL repo on CentOS/RHEL): gdal.json
  • Post-install steps (reconfiguring R for Java): java.json

JSON Fields

{
  "patterns": [...],
  "dependencies": [
    {
      "packages": [...],
      "constraints": [
        {
          "os": ...,
          "distribution": ...,
          "versions": [...]
        }
      ],
      "pre_install": [
        {
          "command": ...,
          "script": ...
        }
      ],
      "post_install": [
        {
          "command": ...,
          "script": ...
        }
      ]
    }
  ]
}

Top-level fields

Field Type Description
patterns Array Regular expressions to match SystemRequirements fields. Case-insensitive. Note that the escape character must be escaped itself (\\. to match a dot). Use word boundaries (\\b) for more accurate matches.
Example: ["\\bgnu make\\b", "\\bgmake\\b"] to match GNU Make or gmake; OpenSSL
dependencies Array Rules for installing the dependency on one or more operating systems. See dependencies.

Dependencies

Field Type Description
packages Array Packages installed through the default system package manager (e.g. apt, yum, zypper). Examples: ["libxml2-dev"], ["tcl", "tk"]
constraints Array One or more operating system constraints. See constraints.
pre_install Array Optional commands or scripts to run before installing packages (e.g. adding a third-party repository). See pre/post-install actions.
post_install Array Optional commands or scripts to run after installing packages (e.g. cleaning up). See pre/post-install actions.

Constraints

Field Type Description
os String Operating system. Only "linux" is supported for now.
distribution String Linux distribution. One of "ubuntu", "debian", "centos", "redhat", "opensuse", "sle", "fedora"
versions Array Optional set of OS versions. If unspecified, the rule applies to all supported versions. See systems.json for supported values by OS. Example: ["18.04"] for Ubuntu.

Pre/post-install actions

Pre-install and post-install actions can be specified as either a command or script. Commands are preferred unless there's complicated logic involved.

Field Type Description
command String A shell command. Example: "yum install -y epel-release"
script String A shell script found in the scripts directory. Example: "centos_epel.sh"

Adding a rule

A typical workflow for adding a new rule:

  1. Come up with regular expressions to match all R packages with the system dependency. See sysreqs.json for a sample list of CRAN packages and their SystemRequirements fields. Note that the applicable R packages don't have to be on CRAN; they can be on GitHub or other repositories, such as Bioconductor and rOpenSci.

  2. Determine the system packages and any pre/post-install steps if needed. The more operating systems covered, the better, but it's fine if only some operating systems are covered.

    Useful resources for finding packages across different OSs:

    Or to search for packages on each OS:

    # Ubuntu/Debian
    apt-cache search <package-name>
    
    # CentOS/RHEL/Fedora
    yum search <package-name>
    
    # openSUSE/SLE
    zypper search <package-name>
  3. Add the new rule as a rule-name.json file in the rules directory.

  4. Run the schema tests and (optionally) the system package tests locally.

  5. Submit a pull request.

Testing

Schema tests

To lint and validate rules against the schema, you'll need Node.js.

# Install dependencies
npm install

# Run the tests
npm test

To list R packages and system requirements matched by a rule:

# List matching system requirements for a rule
npm run test-patterns -- rules/libcurl.json --verbose

# List matching system requirements for all rules
npm run test-patterns-all -- --verbose

# Fail if a rule doesn't match any system requirements
npm run test-patterns-all -- --strict

To update the list of R packages and system requirements used for testing, run:

make update-sysreqs

System package tests

Docker images are provided to help validate system packages on supported OSs.

Available tags:

  • focal (Ubuntu 20.04)
  • jammy (Ubuntu 22.04)
  • buster (Debian 10)
  • bullseye (Debian 11)
  • bookworm (Debian 12)
  • sid (Debian unstable)
  • centos7 (CentOS 7)
  • centos8 (Rocky Linux 8)
  • rockylinux9 (Rocky Linux 9)
  • opensuse154 (openSUSE 15.4)
  • opensuse155 (openSUSE 15.5)
  • fedora36 (Fedora 36)
  • fedora37 (Fedora 37)
  • fedora38 (Fedora 38)

To build the images:

# Build a specific image (e.g. focal)
make build-focal

# Build all images
make build-all

To test the rules:

# Test a specific rule on an OS (e.g. focal)
make test-focal RULES=rules/libcurl.json

# Test a specific rule on all OSs
make test-all RULES=rules/libcurl.json

# Test all rules on all OSs
make test-all

Schema

The JSON schema is defined in the file schema.json. Do not modify this file directly, since it is automatically generated. Instead, modify schema.template.json and then run npm run generate-schema. The generate-schema target is automatically run when running npm test.

If you need to modify the distros and/or versions supported in the schema definitions, modify systems.json.

More Repositories

1

cheatsheets

Posit Cheat Sheets - Can also be found at https://posit.co/resources/cheatsheets/.
TeX
5,673
star
2

shiny

Easy interactive web applications with R
R
5,281
star
3

rstudio

RStudio is an integrated development environment (IDE) for R
Java
4,432
star
4

bookdown

Authoring Books and Technical Documents with R Markdown
JavaScript
3,690
star
5

rmarkdown

Dynamic Documents for R
R
2,737
star
6

shiny-examples

JavaScript
1,945
star
7

gt

Easily generate information-rich, publication-quality tables from R
R
1,934
star
8

blogdown

Create Blogs and Websites with R Markdown
R
1,710
star
9

reticulate

R Interface to Python
R
1,634
star
10

webinars

Code and slides for RStudio webinars
HTML
1,510
star
11

rticles

LaTeX Journal Article Templates for R Markdown
TeX
1,402
star
12

plumber

Turn your R code into a web API.
R
1,374
star
13

tensorflow

TensorFlow for R
R
1,322
star
14

renv

renv: Project environments for R.
R
975
star
15

pagedown

Paginate the HTML Output of R Markdown with CSS for Print
R
873
star
16

shinydashboard

Shiny Dashboarding framework
CSS
852
star
17

pointblank

Data quality assessment and metadata reporting for data frames and database tables
R
826
star
18

keras

R Interface to Keras
R
823
star
19

flexdashboard

Easy interactive dashboards for R
JavaScript
804
star
20

leaflet

R Interface to Leaflet Maps
JavaScript
793
star
21

rmarkdown-book

R Markdown: The Definitive Guide (published by Chapman & Hall/CRC in July 2018)
RMarkdown
738
star
22

ggvis

Interactive grammar of graphics for R
R
709
star
23

shiny-server

Host Shiny applications over the web.
JavaScript
707
star
24

learnr

Interactive Tutorials with R Markdown
R
704
star
25

rstudio-conf

Materials for rstudio::conf
HTML
699
star
26

RStartHere

A guide to some of the most useful R Packages that we know about
R
661
star
27

py-shiny

Shiny for Python
Python
627
star
28

DT

R Interface to the jQuery Plug-in DataTables
JavaScript
581
star
29

rmarkdown-cookbook

R Markdown Cookbook. A range of tips and tricks to make better use of R Markdown.
RMarkdown
568
star
30

blastula

Easily send great-looking HTML email messages from R
R
530
star
31

r2d3

R Interface to D3 Visualizations
R
514
star
32

bookdown-demo

A minimal book example using bookdown
CSS
476
star
33

bslib

Tools for theming Shiny and R Markdown via Bootstrap 3, 4, or 5.
SCSS
441
star
34

hex-stickers

RStudio hex stickers
R
434
star
35

distill

Distill for R Markdown
HTML
419
star
36

packrat

Packrat is a dependency management system for R
R
394
star
37

tufte

Tufte Styles for R Markdown Documents
R
385
star
38

dygraphs

R interface to dygraphs
JavaScript
361
star
39

revealjs

R Markdown Format for reveal.js Presentations
JavaScript
316
star
40

pins-r

Pin, Discover and Share Resources
R
301
star
41

fontawesome

Easily insert FontAwesome icons into R Markdown docs and Shiny apps
R
290
star
42

profvis

Visualize R profiling data
JavaScript
285
star
43

crosstalk

Inter-htmlwidget communication for R (with and without Shiny)
JavaScript
284
star
44

config

config package for R
R
247
star
45

tinytex-releases

Windows/macOS/Linux binaries and installation methods of TinyTeX
PowerShell
245
star
46

pool

Object Pooling in R
R
244
star
47

thematic

Theme ggplot2, lattice, and base graphics based on a few simple settings.
R
242
star
48

Intro

Course materials for "Introduction to Data Science with R", a video course by RStudio and O'Reilly Media
R
234
star
49

shinytest

Automated testing for shiny apps
JavaScript
224
star
50

shinymeta

Record and expose Shiny app logic using metaprogramming
R
220
star
51

nomnoml

Sassy 'UML' Diagrams for R
JavaScript
218
star
52

httpuv

HTTP and WebSocket server package for R
C
217
star
53

shinyuieditor

A GUI for laying out a Shiny application that generates clean and human-readable UI code
JavaScript
208
star
54

htmltools

Tools for HTML generation and output
R
201
star
55

promises

A promise library for R
R
195
star
56

vetiver-r

Version, share, deploy, and monitor models
R
176
star
57

rstudioapi

Safely access RStudio's API (when available)
R
161
star
58

gradethis

Tools for teachers to use with learnr
R
160
star
59

concept-maps

Concept maps for all things data science
HTML
158
star
60

master-the-tidyverse

Course contents for Master the Tidyverse
155
star
61

shinythemes

Themes for Shiny
R
152
star
62

ShinyDeveloperConference

Materials collected from the First Shiny Developer Conference Palo Alto, CA January 30-31 2016
HTML
152
star
63

chromote

Chrome Remote Interface for R
R
151
star
64

shiny-gallery

Code and other documentation for apps in the Shiny Gallery ✨
HTML
145
star
65

sortable

R htmlwidget for Sortable.js
R
124
star
66

reactlog

Shiny Reactivity Visualizer
JavaScript
121
star
67

rsconnect

Publish Shiny Applications, RMarkdown Documents, Jupyter Notebooks, Plumber APIs, and more
R
120
star
68

redx

dynamic nginx configuration
Lua
118
star
69

r-docker

Docker images for R
Dockerfile
117
star
70

bigdataclass

Two-day workshop that covers how to use R to interact databases and Spark
R
114
star
71

shinyapps

Deploy Shiny applications to ShinyApps
110
star
72

webshot2

Take screenshots of web pages from R
R
109
star
73

shinyloadtest

Tools for load testing Shiny applications
HTML
108
star
74

shinyvalidate

Input validation package for the Shiny web framework
JavaScript
108
star
75

miniUI

R
102
star
76

sass

Sass compiler package for R
C++
102
star
77

shinytest2

R
100
star
78

keras-customer-churn

Customer Churn Shiny Application
R
99
star
79

r-builds

an opinionated environment for compiling R
Shell
92
star
80

r-manuals

A re-styled version of the R manuals
R
85
star
81

addinexamples

An R package showcasing how RStudio addins can be registered and used.
R
85
star
82

shinyapps-package-dependencies

Collection of bash scripts that install R package system dependencies
R
74
star
83

markdown

The first generation of Markdown rendering for R (born in 2012). Originally based on the C library sundown. Now based on commonmark. Note that this package is markdown, not *rmarkdown*.
R
72
star
84

R-Websockets

HTML 5 Websockets implementation for R
R
68
star
85

webdriver

WebDriver client in R
R
68
star
86

beyond-dashboard-fatigue

Materials for the RStudio webinar 'Beyond Dashboard Fatigue'
R
66
star
87

cloudml

R interface to Google Cloud Machine Learning Engine
R
65
star
88

rstudio-docker-products

Docker images for RStudio Professional Products
Shell
62
star
89

shinylive

Run Shiny on Python (compiled to wasm) in the browser
TypeScript
61
star
90

rstudio-conf-2022-program

rstudio::conf(2022, "program")
R
60
star
91

bookdown.org

Source documents to generate the bookdown.org website
R
59
star
92

vetiver-python

Version, share, deploy, and monitor models.
Python
58
star
93

education.rstudio.com

CSS
57
star
94

tfestimators

R interface to TensorFlow Estimators
R
57
star
95

connections

https://rstudio.github.io/connections/
R
54
star
96

tfprobability

R interface to TensorFlow Probability
R
54
star
97

sparkDemos

HTML
53
star
98

shiny-incubator

Examples and ideas that don't belong in the core Shiny package and aren't officially supported.
JavaScript
53
star
99

swagger

Swagger is a collection of HTML, Javascript, and CSS assets that dynamically generate beautiful documentation from a Swagger-compliant API.
HTML
51
star
100

leaflet.mapboxgl

Extends the R Leaflet package with a Mapbox GL JS plugin to allow easy drawing of vector tile layers.
R
49
star