• Stars
    star
    207
  • Rank 189,164 (Top 4 %)
  • Language
    C
  • License
    Other
  • Created almost 10 years ago
  • Updated 10 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A Modern and Flexible Web Client for R

curl

A Modern and Flexible Web Client for R

CRAN_Status_Badge CRAN RStudio mirror downloads

The curl() and curl_download() functions provide highly configurable drop-in replacements for base url() and download.file() with better performance, support for encryption (https, ftps), gzip compression, authentication, and other libcurl goodies. The core of the package implements a framework for performing fully customized requests where data can be processed either in memory, on disk, or streaming via the callback or connection interfaces. Some knowledge of libcurl is recommended; for a more-user-friendly web client see the 'httr' package which builds on this package with http specific tools and logic.

Documentation

About the R package:

Other resources:

Hello World

There are three download interfaces (memory, disk and streaming). Always start by setting up a request handle:

library(curl)
h <- new_handle(copypostfields = "moo=moomooo")
handle_setheaders(h,
  "Content-Type" = "text/moo",
  "Cache-Control" = "no-cache",
  "User-Agent" = "A cow"
)

Perform request and download response in memory:

# Perform the request
req <- curl_fetch_memory("http://httpbin.org/post", handle = h)

# Show some outputs
parse_headers(req$headers)
cat(rawToChar(req$content))
str(req)

Or alternatively, write response to disk:

tmp <- tempfile()
curl_download("https://httpbin.org/post", tmp, handle = h)
readLines(tmp)

Or stream response via Connection interface:

con <- curl("https://httpbin.org/post", handle = h)
open(con)

# Get 3 lines
readLines(con, n = 3)

# Get remaining lines and close connection
readLines(con)
close(con)

Installation

Binary packages for OS-X or Windows can be installed directly from CRAN:

install.packages("curl")

Installation from source on Linux requires libcurl. On Debian or Ubuntu use libcurl4-openssl-dev:

sudo apt-get install -y libcurl-dev

On Fedora, CentOS or RHEL use libcurl-devel:

sudo yum install libcurl-devel

MacOS using curl from homebrew

On MacOS libcurl is included with the system, so usually nothing extra is needed. However if you want to build against the very most recent version of libcurl, which also has many extra features enabled (sftp, http2), install curl from homebrew and then recompile the R package.

brew install curl pkg-config

You need to set the PKG_CONFIG_PATH environment variable to help R find the non-default curl, when building from source. Run this in a clean R session which does not have the curl package loaded already:

Sys.setenv(PKG_CONFIG_PATH="/usr/local/opt/curl/lib/pkgconfig")
install.packages("curl", type = "source")

Afterwards confirm the version using curl::curl_version().

Development version

Because devtools and httr depend on curl, installing with install_github does not work well. The easiest way to install the development version of curl is a clean R session:

install.packages("https://github.com/jeroen/curl/archive/master.tar.gz", repos = NULL)

Of course windows users need Rtools to compile from source.

More Repositories

1

jsonlite

A Robust, High Performance JSON Parser and Generator for R
C
361
star
2

mongolite

Fast and Simple MongoDB Client for R
C
281
star
3

resttesttest

A native in-browser tool for testing REST/CORS services. Builds on jQuery and Bootstrap.
HTML
195
star
4

V8

Embedded JavaScript Engine for R
C++
188
star
5

sys

Powerful replacements for base::system2
C
102
star
6

js

Tools for Working with JavaScript in R
R
67
star
7

openssl

OpenSSL bindings for R
R
63
star
8

RAppArmor

R interfaces to Linux and AppArmor security methods
TeX
52
star
9

protolite

Fast and Simple Object Serialization to Protocol Buffers
C++
47
star
10

opencpu-legacy

This version is no longer maintained. Switch to "opencpu" repository.
R
41
star
11

solarisvm

VMware image with R on Solaris
32
star
12

webutils

Utility functions for web applications
R
23
star
13

shinymagick

Simple Demo of using Magick with Shiny
R
22
star
14

unix

R bindings to system utilities found in most Unix systems
R
21
star
15

agent

Store sensitive data such as API tokens
R
19
star
16

gpg

Bindings to gpgme for R
C
19
star
17

bcrypt

Wrapper for bcrypt in R
C
17
star
18

webp

R bindings for libwebp
R
14
star
19

tidyhtml

R bindings to html-tidy C interface
C
14
star
20

maketools

Exploring and Testing your Toolchain Configuration and System Packages
R
14
star
21

ipfs

R bindings to IPFS
R
10
star
22

popplertest

Example code to test string encoding in poppler-cpp
C++
10
star
23

build-v8-static

Building static v8-monolith library for various distros
Dockerfile
10
star
24

imwidgets

HTML widgets for viewing images
JavaScript
9
star
25

user2021

Slides from UseR-2021 keynote presentation about R-universe
HTML
7
star
26

snapshot

Create snapshots of R package repositories
R
6
star
27

rjade

R Bindings to the Jade Templating Engine
R
6
star
28

yeroon

yeroon.net webapps
JavaScript
5
star
29

badgen

R
5
star
30

betty

Some experimental rOpenSci infrastructure plumbing
R
5
star
31

Ohmage

Ohmage R package
R
5
star
32

ndjson-benchmark

Simple benchmark to compare jsonlite, ndjson, corpus
4
star
33

yeroon-setup

JavaScript
4
star
34

r-dependency-versioning

CSS
4
star
35

opencpu.tools

some tools for the opencpu project
R
4
star
36

backup

4
star
37

markdownit

Bindings to markdown-it JavaScript library
R
3
star
38

minimist

R binding to minimist JavaScript library
R
3
star
39

slides

Slides for the defense presentation
CSS
2
star
40

Mobilize

Mobilize R package
R
2
star
41

lmergm

lmergm R package
R
2
star
42

ohmage-ubuntu

ohmage-ubuntu
Shell
2
star
43

base64

R
2
star
44

debugcurl

How to debug curl (because I keep forgetting)
2
star
45

pacmantools

Tools for pacman repos
R
1
star
46

maintest

Test with main repository
HTML
1
star
47

JJcorr

Git repo for the JJcorr package
R
1
star
48

interactivity

cran-interactivity
C
1
star
49

r-security

R
1
star
50

r-universe

1
star
51

autobrew-old

Scripts for brewing static libs on CRAN MacOS builder
Roff
1
star
52

brotli

C
1
star
53

testinterrupt

Package + readme to demonstrate the user interruption problem in R
C
1
star
54

grenoble2015

Automatically exported from code.google.com/p/io-2012-slides
CSS
1
star
55

uptest

C++
1
star