• Stars
    star
    183
  • Rank 202,306 (Top 5 %)
  • Language
    R
  • Created about 2 years ago
  • Updated about 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

CRAN as Ubuntu Binaries

r2u: CRAN as Ubuntu Binaries

Key features

  • Full integration with apt as every binary resolves all its dependencies: No more installations (of pre-built archives) only to discover that a shared library is missing. No more surprises.

  • Full integration with apt so that an update of a system library cannot break an R package: if a (shared) library is used by a CRAN, the package manager knows, and will not remove it. No more (R package) breakage from (system) library updates.

  • Simpler and lighter than some alternatives as only run-time library packages are installed as dependencies (instead of generally heavier development packages).

  • Installations are fast, automated and reversible thanks to the package management layer.

  • Fast and well-connected mirror at r2u.stat.illinois.edu on the Internet2

  • Complete coverage with (currently) ~ 21,700 CRAN packages (and 260+ from BioConductor) using current versions: We use R 4.3.1, and BioConductor 3.17.

  • Complete support for both Ubuntu 20.04 ("focal") and Ubuntu 22.04 ("jammy").

  • Optional (but recommended) bspm use automagically connects R functions like install.packages() to apt for access to binaries and dependencies.

  • Docker containers rocker/r2u from the Rocker Project for both 'focal' and 'jammy'.

Brief Demo

The gif below shows how one install.packages("tidyverse") command on an Ubuntu 20.04 system installs all packages and dependencies as binaries in 18 seconds (by passing the R package installation to apt using bspm).

This uses the Docker container referenced below, which has been set up with the five easy setup steps detailed here.

What is Covered ?

We generally support amd64 (i.e. standard 64-bit Intel/AMD cpus, sometimes also called x86_64) for the current Ubuntu LTS release and its predecessor release. We use 'r-release' just like CRAN. So currently the 'focal' 20.04 LTS and 'jammy' 22.04 LTS releases are fully supported.

Support for other cpu architectures is certainly possible but somewhat unlikely due to a lack of (additional hardware) resources and time. Support for other distributions is possible but unlikely right now (due to a lack of resources and time). RSPM/PPM now appears to also support Debian which could be added at some later point.

Current versions are based on R 4.3.1, and BioConductor release 3.17 packages are provided when required by CRAN packages. Binaries are generally R 4.3.* based. Some older packages released when we used R 4.2.* may have been built with R 4.2., they will still work the same with R 4.3. as R is generally forward-compatible.

What is Selected ?

Everything :)

Initially, we started from cran-logs by picking the N most-downloaded packages, along with their dependencies from BioConductor. (It should be noted that for example the first 100 packages already account for approximately half the total downloads: it is a very skewed distribution.) We iterated, and fairly soon arrived of full coverage of CRAN.

So we now cover

  • all CRAN packages (modulo at best handful of blacklisted ones) including all packages needing compilation
  • all BioConductor package that are implied by these (and build for us).

This currently resuls in 21833 and 21737 binary packages from CRAN in "focal" and "jammy", respectively, and 267 and 262 BioConductor packages, respectively, from the 3.17 release.

The sole exception are two packages we cannot build (as we do not have the required commercial software it accessess) plus less than a handful of 'odd builds' that fail and are skipped.

What is it Based On?

For the CRAN binaries we either repackage RSPM/PPM builds (where available) or build natively. All selected BioConductor 3.17 packages are built natively. For all of these, full dependency resolution and integration with the system is a key feature.

Everything is provided as .deb binary files with proper dependency resolution by using a proper apt repo which also has a signed Release file.

Usage and Setup

(Note that you could use one of the two scripts add_cranapt_jammy.sh (for Ubuntu 22.04), or add_cranapt_focal.sh (for the older Ubuntu 20.04) to facilitate the setup. They are tested on 'empty' Ubuntu containers of the corresponding release. However, you may prefer to execute the steps outlined here by hand.)

We show the setup step by step for 'jammy' aka Ubuntu 22.04. You should run all these commands as root to carefully review each one.

Step 1: Update apt, install tools, fetch key

First add the repository key so that apt knows it (this is optional but recommended)

apt update -qq && apt install --yes --no-install-recommends wget \
    ca-certificates gnupg
wget -q -O- https://eddelbuettel.github.io/r2u/assets/dirk_eddelbuettel_key.asc \
    | tee -a /etc/apt/trusted.gpg.d/cranapt_key.asc

Step 2: Add the apt repo

Second, add the repository to the apt registry. We recommend the well-connected main mirror provide at University of Illinois:

echo "deb [arch=amd64] https://r2u.stat.illinois.edu/ubuntu jammy main" \
     > /etc/apt/sources.list.d/cranapt.list
apt update -qq

Step 3: Ensure you have current R binaries (optional)

Third, and optionally, if you do not yet have the current R version, run these two lines (or use the standard CRAN repo setup)

wget -q -O- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc \
    | tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc
echo "deb [arch=amd64] https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/" \
    > /etc/apt/sources.list.d/cran_r.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys \
    67C2D66C4B1D4339 51716619E084DAB9
apt update -qq
DEBIAN_FRONTEND=noninteractive apt install --yes --no-install-recommends \
    r-base-core

Step 4: Use pinning for the r2u repo (optional)

Fourth, add repository 'pinning' as apt might get confused by some older packages (in the Ubuntu distro) which accidentally appear with a higher version number. See the next section for a short discussion how it ensures 'CRANapt' sorts highest.

echo "Package: *" > /etc/apt/preferences.d/99cranapt
echo "Pin: release o=CRAN-Apt Project" >> /etc/apt/preferences.d/99cranapt
echo "Pin: release l=CRAN-Apt Packages" >> /etc/apt/preferences.d/99cranapt
echo "Pin-Priority: 700"  >> /etc/apt/preferences.d/99cranapt

After that the package are known (under their r-cran-* and r-bioc-* names). You can install them on the command-line using apt and apt-get, via aptitude as well as other front-ends.

Step 5: Use bspm (optional)

Fifth, and also optional, install and enable the bspm package so that the r2u (or CRANapt) as well as other R packages (available as r-*.deb binaries) become available via install.packages() and update.packages(). Note that you may need to install it directly from source via sudo Rscript -e 'install.packages("bspm")' to ensure it integrates correctly with the packaging system. You should also install Python components used internally by bspm via the sudo apt-get install python3-{dbus,gi,apt} command.

apt install --yes --no-install-recommends python3-{dbus,gi,apt}
## Then install bspm (as root) and enable it, and enable a speed optimization
Rscript -e 'install.packages("bspm")'
RHOME=$(R RHOME)
echo "suppressMessages(bspm::enable())" >> ${RHOME}/etc/Rprofile.site
echo "options(bspm.version.check=FALSE)" >> ${RHOME}/etc/Rprofile.site

That's it! Now try it out!

About Pinning

Packages can be found in different repositories, and generally the highest available version is the one we what---and apt picks it for us. Now, because we let apt (and related tools) pick the packages based on versions, we may want to ensure that the CRANapt repo sorts higher than the default repo as (older) package builds in the distribution itself may appear (to apt) to be newer via a quirk in the sorting algorithm. A case in point was package gtable whose version in Ubuntu was 0.3.0+dfsg-1 which accidentally sorts higher than the rebuild we made under a newer and more consistent version number 0.3.0-1.ca2004.1.

For this issue, one possible and popular fix is to use 'apt pinning'. It can give 'higher weight' to packages from a particular repositor or tag. In the suggested example above, we give the r2u / cranapt repo a weight of 700 which is higher than the package default value of 500.

Docker

There are also two Docker containers for Ubuntu 20.04 'focal' and 22.04 'jammy', respectively. Initially published as eddelbuettel/r2u, these are now also available also as rocker/r2u. They all have the features detailed above, including pinning and bspm support, already set up.

Each of the Ubuntu LTS flavors, i.e., 'focal' and 'jammy' is also available as an identical image using the release version, i.e., '20.04' and '22.04', respectively.

Note that with some builds of Docker (and possibly related to Ubuntu hosts) you may have to add the --security-opt seccomp=unconfined option to your Docker invocation to take advantage of bspm and the full system integration inside the container. This is also documented in the FAQ.

Try It

Via codespaces

See the vignette Codespaces about how to launch a 'Codespace' directly in your browser, launched from the gitrepo within minutes.

This also works from your [vscode][https://code.visualstudio.com/] installation as a remote codespace.

The vignette has more details.

Via gitpod.io

Use this link below (after possibly signing up for gitpod.io first)

Open in Gitpod

and run one of the three example scripts, or just start R in the terminal window.

The gif below display running one such example to install brms from binaries in a few seconds. Using this requires only (free) GitHub and GitPod accounts.

Support

Please file issues at the GitHub issues for r2u.

Known Issues

  • The littler package reflects build-time configuration, the RSPM/PPM binary is then expecting a different R location so it needs a binary rebuild. Added a 'force' flag, may need a list similar to the blacklist to always compiled.

  • A small number of packages do not build for lack required components; examples are ROracle and Rcplex. They, and their reverse dependencies, are blacklisted and not built.

  • r2u is an apt repo, which via bspm becomes used "automagically" via standard R calls of install.packages() and alike. That last part is important: package installations that do not use install.packages() (such as renv, rig, ...) do not benefit from install.packages() calling apt for you, and cannot take advantage of r2u via bspm.

  • bspm traces calls to install.packages() and maps them system-wide installation via apt. By choice, it does not map the remove.packages() for package removal, see this issue for more discussion. Packages can be uninstalled via the system package manager using, respectively, apt, dpkg or one of graphical frontends as well as via the R function bspm::remove_sys().

Author

Dirk Eddelbuettel

License

The repository-building code in this package is released under the GPL (>= 2).

All CRAN and BioConductor packages are released under their respective licenses.

Acknowledgment

This was made possible by the generous support of endless coffee thanks to my GitHub Sponsors.

More Repositories

1

littler

A scripting and command-line front-end for GNU R
R
293
star
2

tint

Tint is not Tufte
R
259
star
3

gsir-te

Getting Started in R -- Tinyverse Edition
R
226
star
4

mkl4deb

Adding the Intel MKL to a Debian / Ubuntu system via one simple script
Shell
204
star
5

binb

Binb is not Beamer
TeX
188
star
6

anytime

Anything to POSIXct or Date Converter
R
153
star
7

drat

Drat R Archive Template
HTML
148
star
8

pinp

Pinp Is Not PNAS -- Two-Column PDF Template
TeX
146
star
9

rinside

Seamless embedding of R in C++ programs
C++
130
star
10

rcppsimdjson

Rcpp Bindings for the 'simdjson' Header Library
C++
113
star
11

rquantlib

R interface to the QuantLib library
C++
111
star
12

linl

Linl Is Not Letter -- Markdown-based LaTeX Letter Template
TeX
110
star
13

digest

R package to create compact hash digests of R objects
C
108
star
14

rpushbullet

R interface to the awesome Pushbullet service
R
94
star
15

bh

R package providing Boost Header files
C++
80
star
16

crp

Archived copies of the CRAN Repo Policy
HTML
72
star
17

rcppannoy

Rcpp bindings for Annoy
C++
69
star
18

rprotobuf

R Interface to Protocol Buffers
C++
68
star
19

ctv-finance

CRAN Task View: Empirical Finance
R
55
star
20

samples-rmarkdown-metropolis

RMarkdown with Metropolis/Mtheme for Beamer
Makefile
52
star
21

nanotime

Nanosecond Resolution Time Functionality for R
R
52
star
22

rcppredis

R interface to Redis using the hiredis library
C
49
star
23

rcppexamples

Examples of using Rcpp to interface R and C++
C++
45
star
24

inline

Inline C, C++ or Fortran functions in R
R
40
star
25

tidycpp

Tidy C++ wrapping of the C API of R
C++
38
star
26

ctv-hpc

CRAN Task View: High-Performance Computing with R
R
38
star
27

rcppkalman

Kalman filtering via RcppArmadillo
MATLAB
35
star
28

pkgkitten

Create simple packages which pass R CMD check
R
33
star
29

rcpptoml

Rcpp Bindings to C++ parser for TOML files
C++
31
star
30

rcppgsl

Rcpp integration for GNU GSL vectors and matrices
TeX
30
star
31

rfoaas

R interface to FOAAS service
R
28
star
32

rcppcnpy

Rcpp bindings for NumPy files
C++
26
star
33

rvowpalwabbit

R interface to the Vowpal Wabbit
C++
24
star
34

rcppcctz

Rcpp interface to CCTZ library
C++
22
star
35

r-ci

CI for R at Travis, GitHub Actions, Azure Pipelines, ...
22
star
36

ttdo

Extend tinytest with diffobj
R
21
star
37

tinythemes

Lightweight Repackaging of 'Themes' for 'ggplot2'
R
20
star
38

rcppmsgpack

MsgPack Headers for R / msgpack.org[R]
C++
19
star
39

rcppfastfloat

Rcpp Bindings for the 'fastfloat' Header-Only Library
C++
19
star
40

rcppbdt

Rcpp bindings for Boost Date_Time
C++
17
star
41

docker-ubuntu-r

Docker images for R on Ubuntu
Shell
17
star
42

rcppstreams

A C++ DSEL for real-time event stream processing
C++
15
star
43

t4

Support repo for T^4 Video Lightning Talks
Shell
15
star
44

rticles-gallery

Preview of Templates in the rticles Package
R
14
star
45

dtts

Time-series functionality based on nanotime and data.table
R
14
star
46

rcppspdlog

Bundling of spdlog for use from R and Rcpp
C++
14
star
47

filter-journal-spam

spamassassin blocked-list to avoid traffic from predatory publishers
Makefile
13
star
48

tiledb-user2021

Repository for useR! 2021 TileDB Tutorial Helper Package
R
13
star
49

rcppde

Rcpp port of Differential Evolution
C++
13
star
50

pgapack

A general-purpose, data-structure-neutral, and parallel genetic algorithm library
C
12
star
51

rcppnloptexample

Rcpp Example for accessing NLopt
C++
12
star
52

ppa-rstudio

apt install rstudio rstudio-server quarto
R
12
star
53

prrd

Parallel Running of Reverse Depends
R
12
star
54

rcppdate

R package providing date C++ library header files
C++
11
star
55

lwplot

(Experimental but working) LightWeight Plot / Leland Wilkinson Plot -- a ggplot2 2.1.0 fork aiming for lighter weight
R
11
star
56

rcppziggurat

Rcpp bindings for different Ziggurat RNG implementations
C++
11
star
57

pkg-fonts-fira

(Unofficial) Debian packaging for Mozilla Fira fonts
Makefile
11
star
58

asioheaders

R package providing Asio C++ library header files
C++
11
star
59

td

R interface to 'twelvedata' API
R
11
star
60

rapiserialize

Serialization from the C API for R
C++
10
star
61

minm

Minm Is Not Meta: One way to get several RMarkdown-using packages
10
star
62

rapidatetime

Datetime functionality from the C API for R
C
10
star
63

samples-uzuerich-2017-06

Some Examples for Rcpp Workshop
9
star
64

rcppxts

Rcpp interface to xts objects
C++
9
star
65

rcppapt

Rcpp Interface to the APT Package Manager
C++
9
star
66

dang

A collection of utility functions for R
R
8
star
67

samplecode

Example code for talks or workshops
HTML
8
star
68

earthmovdist

Earth Mover's Distance for R via the Emd-L1 library
C++
8
star
69

rcppfastad

Rcpp Bindings to FastAD Automatic Differentiation
C++
8
star
70

rcpputs

Rcpp bindings for algorithms for unevenly spaced time series
C++
8
star
71

random

R package for true random numbers from random.org
R
8
star
72

dieharder-rgb

Random number generator tester
C
7
star
73

beancounter

Stock Portfolio Performance Tool
Perl
7
star
74

gcbd

R package for GPU/CPU benchmarking on Debian-based systems
R
7
star
75

rdieharder

R interface to the Dieharder RNG test suite
C
7
star
76

ldlasb

Lies, Damned Lies, and Selective Benchmarks
C++
6
star
77

user2022-r-core-panel

useR! 2022 R Core Keynote and Panel
6
star
78

r-travis

Tools for using R with Travis, GitHub Actions and other CI backends
Shell
6
star
79

docker-debian-r

Docker image for R on Debian
Shell
6
star
80

dieharder

C
6
star
81

spotifytop50us

Visualize Most Popular Songs from the Top50
R
6
star
82

rf2

Iteration Two of RinFinance.com
5
star
83

rcpp_comparison_convolution

Rcpp versus C on the Standard Convolution Example
R
5
star
84

rbenchmark

Benchmarking routine for R
R
5
star
85

safy

Show the Global Environment Some Appreciation
R
5
star
86

snap-r-base

Snapcraft for R
R
5
star
87

pkg-latex-metropolis

[deprecated as Metropolis is now in TeXLive and hence Debian] (Unofficial) Debian packaging for Metropolis theme for Beamer
TeX
5
star
88

sanitizers

Sample R package with C/C++ code to trigger Address and Undefined Behaviour Sanitizers
C++
5
star
89

stackoverflow

Grabbag repo collecting some answers I posted. If only I had started this repo a decade ago...
HTML
5
star
90

data-examples

Unpacked data sets with Gapminder, Titanic and more
R
4
star
91

rocker-tiledb

Unofficial Development Docker files for R and TileDB
Makefile
4
star
92

rmsfact

Amazing Random Facts About the World's Greatest Hacker
R
4
star
93

arch

Arrow R and C Helpers
C
4
star
94

gaussfacts

The Greatest Mathematician since Antiquity
R
4
star
95

pkg-fonts-jetbrains-mono

(Unofficial) Debian packaging for Jetbrains Mono fonts
Makefile
4
star
96

pkg-fonts-plex

(Unofficial) Debian packaging for IBM Plex fonts
Makefile
4
star
97

drr35

Temporary repository for Debian packages built with R 3.5.0
3
star
98

docker-swc

Docker image for Software Carpentry
3
star
99

crc32c

R Package for crc32c with hardware-acceleration and software fallback
C++
3
star
100

cook-county-tax-model

Modeling Cook County Property Taxes
R
3
star