• Stars
    star
    985
  • Rank 44,896 (Top 1.0 %)
  • Language
    R
  • License
    Other
  • Created almost 11 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Read flat files (csv, tsv, fwf) into R

readr

CRAN status R-CMD-check Codecov test coverage

Overview

The goal of readr is to provide a fast and friendly way to read rectangular data from delimited files, such as comma-separated values (CSV) and tab-separated values (TSV). It is designed to parse many types of data found in the wild, while providing an informative problem report when parsing leads to unexpected results. If you are new to readr, the best place to start is the data import chapter in R for Data Science.

Installation

# The easiest way to get readr is to install the whole tidyverse:
install.packages("tidyverse")

# Alternatively, install just readr:
install.packages("readr")
# Or you can install the development version from GitHub:
# install.packages("pak")
pak::pak("tidyverse/readr")

Cheatsheet

<img src="https://github.com/rstudio/cheatsheets/raw/main/pngs/thumbnails/data-import-cheatsheet-thumbs.png" height="252" alt="thumbnail of tidyverse data import cheatsheet"//>

Usage

readr is part of the core tidyverse, so you can load it with:

library(tidyverse)
#> โ”€โ”€ Attaching core tidyverse packages โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ tidyverse 2.0.0 โ”€โ”€
#> โœ” dplyr     1.1.4          โœ” readr     2.1.4.9000
#> โœ” forcats   1.0.0          โœ” stringr   1.5.1     
#> โœ” ggplot2   3.4.3          โœ” tibble    3.2.1     
#> โœ” lubridate 1.9.3          โœ” tidyr     1.3.0     
#> โœ” purrr     1.0.2          
#> โ”€โ”€ Conflicts โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ tidyverse_conflicts() โ”€โ”€
#> โœ– dplyr::filter() masks stats::filter()
#> โœ– dplyr::lag()    masks stats::lag()
#> โ„น Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

Of course, you can also load readr as an individual package:

library(readr)

To read a rectangular dataset with readr, you combine two pieces: a function that parses the lines of the file into individual fields and a column specification.

readr supports the following file formats with these read_*() functions:

  • read_csv(): comma-separated values (CSV)
  • read_tsv(): tab-separated values (TSV)
  • read_csv2(): semicolon-separated values with , as the decimal mark
  • read_delim(): delimited files (CSV and TSV are important special cases)
  • read_fwf(): fixed-width files
  • read_table(): whitespace-separated files
  • read_log(): web log files

A column specification describes how each column should be converted from a character vector to a specific data type (e.g.ย character, numeric, datetime, etc.). In the absence of a column specification, readr will guess column types from the data. vignette("column-types") gives more detail on how readr guesses the column types. Column type guessing is very handy, especially during data exploration, but itโ€™s important to remember these are just guesses. As any data analysis project matures past the exploratory phase, the best strategy is to provide explicit column types.

The following example loads a sample file bundled with readr and guesses the column types:

(chickens <- read_csv(readr_example("chickens.csv")))
#> Rows: 5 Columns: 4
#> โ”€โ”€ Column specification โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
#> Delimiter: ","
#> chr (3): chicken, sex, motto
#> dbl (1): eggs_laid
#> 
#> โ„น Use `spec()` to retrieve the full column specification for this data.
#> โ„น Specify the column types or set `show_col_types = FALSE` to quiet this message.
#> # A tibble: 5 ร— 4
#>   chicken                 sex     eggs_laid motto                               
#>   <chr>                   <chr>       <dbl> <chr>                               
#> 1 Foghorn Leghorn         rooster         0 That's a joke, ah say, that's a jokโ€ฆ
#> 2 Chicken Little          hen             3 The sky is falling!                 
#> 3 Ginger                  hen            12 Listen. We'll either die free chickโ€ฆ
#> 4 Camilla the Chicken     hen             7 Bawk, buck, ba-gawk.                
#> 5 Ernie The Giant Chicken rooster         0 Put Captain Solo in the cargo hold.

Note that readr prints the column types โ€“ the guessed column types, in this case. This is useful because it allows you to check that the columns have been read in as you expect. If they havenโ€™t, that means you need to provide the column specification. This sounds like a lot of trouble, but luckily readr affords a nice workflow for this. Use spec() to retrieve the (guessed) column specification from your initial effort.

spec(chickens)
#> cols(
#>   chicken = col_character(),
#>   sex = col_character(),
#>   eggs_laid = col_double(),
#>   motto = col_character()
#> )

Now you can copy, paste, and tweak this, to create a more explicit readr call that expresses the desired column types. Here we express that sex should be a factor with levels rooster and hen, in that order, and that eggs_laid should be integer.

chickens <- read_csv(
  readr_example("chickens.csv"),
  col_types = cols(
    chicken   = col_character(),
    sex       = col_factor(levels = c("rooster", "hen")),
    eggs_laid = col_integer(),
    motto     = col_character()
  )
)
chickens
#> # A tibble: 5 ร— 4
#>   chicken                 sex     eggs_laid motto                               
#>   <chr>                   <fct>       <int> <chr>                               
#> 1 Foghorn Leghorn         rooster         0 That's a joke, ah say, that's a jokโ€ฆ
#> 2 Chicken Little          hen             3 The sky is falling!                 
#> 3 Ginger                  hen            12 Listen. We'll either die free chickโ€ฆ
#> 4 Camilla the Chicken     hen             7 Bawk, buck, ba-gawk.                
#> 5 Ernie The Giant Chicken rooster         0 Put Captain Solo in the cargo hold.

vignette("readr") gives an expanded introduction to readr.

Editions

readr got a new parsing engine in version 2.0.0 (released July 2021). In this so-called second edition, readr calls vroom::vroom(), by default.

The parsing engine in readr versions prior to 2.0.0 is now called the first edition. If youโ€™re using readr >= 2.0.0, you can still access first edition parsing via the functions with_edition(1, ...) and local_edition(1). And, obviously, if youโ€™re using readr < 2.0.0, you will get first edition parsing, by definition, because thatโ€™s all there is.

We will continue to support the first edition for a number of releases, but the overall goal is to make the second edition uniformly better than the first. Therefore the plan is to eventually deprecate and then remove the first edition code. New code and actively-maintained code should use the second edition. The workarounds with_edition(1, ...) and local_edition(1) are offered as a pragmatic way to patch up legacy code or as a temporary solution for infelicities identified as the second edition matures.

Alternatives

There are two main alternatives to readr: base R and data.tableโ€™s fread(). The most important differences are discussed below.

Base R

Compared to the corresponding base functions, readr functions:

  • Use a consistent naming scheme for the parameters (e.g.ย col_names and col_types not header and colClasses).

  • Are generally much faster (up to 10x-100x) depending on the dataset.

  • Leave strings as is by default, and automatically parse common date/time formats.

  • Have a helpful progress bar if loading is going to take a while.

  • All functions work exactly the same way regardless of the current locale. To override the US-centric defaults, use locale().

data.table and fread()

data.table has a function similar to read_csv() called fread(). Compared to fread(), readr functions:

  • Are sometimes slower, particularly on numeric heavy data.

  • Can automatically guess some parameters, but basically encourage explicit specification of, e.g., the delimiter, skipped rows, and the header row.

  • Follow tidyverse-wide conventions, such as returning a tibble, a standard approach for column name repair, and a common mini-language for column selection.

Acknowledgements

Thanks to:

  • Joe Cheng for showing me the beauty of deterministic finite automata for parsing, and for teaching me why I should write a tokenizer.

  • JJ Allaire for helping me come up with a design that makes very few copies, and is easy to extend.

  • Dirk Eddelbuettel for coming up with the name!

More Repositories

1

ggplot2

An implementation of the Grammar of Graphics in R
R
6,324
star
2

dplyr

dplyr: A grammar of data manipulation
R
4,627
star
3

tidyverse

Easily install and load packages from the tidyverse
R
1,575
star
4

rvest

Simple web scraping for R
R
1,455
star
5

tidyr

Tidy Messy Data
R
1,323
star
6

purrr

A functional programming toolkit for R
R
1,211
star
7

magrittr

Improve the readability of R code with the pipe
R
952
star
8

datascience-box

Data Science Course in a Box
JavaScript
895
star
9

reprex

Render bits of R code for sharing, e.g., on GitHub or StackOverflow.
R
726
star
10

readxl

Read excel files (.xls and .xlsx) into R ๐Ÿ–‡
C++
713
star
11

lubridate

Make working with dates in R just that little bit easier
R
712
star
12

glue

Glue strings to data in R. Small, fast, dependency free interpreted string literals.
R
685
star
13

dtplyr

Data table backend for dplyr
R
656
star
14

tibble

A modern re-imagining of the data frame
R
641
star
15

multidplyr

A dplyr backend that partitions a data frame over multiple processes
R
636
star
16

vroom

Fast reading of delimited files
C++
604
star
17

stringr

A fresh approach to string manipulation in R
R
565
star
18

forcats

๐Ÿˆ๐Ÿˆ๐Ÿˆ๐Ÿˆ: tools for working with categorical variables (factors)
R
533
star
19

dbplyr

Database (DBI) backend for dplyr
R
455
star
20

haven

Read SPSS, Stata and SAS files from R
C
421
star
21

modelr

Helper functions for modelling
R
398
star
22

googlesheets4

Google Spreadsheets R API (reboot of the googlesheets package)
R
347
star
23

googledrive

Google Drive R API
R
312
star
24

style

The tidyverse style guide for R code
HTML
285
star
25

design

Tidyverse design principles
R
208
star
26

tidyverse.org

Source of tidyverse.org
HTML
189
star
27

hms

A simple class for storing time-of-day values
R
136
star
28

nycflights13

An R data package containing all out-bound flights from NYC in 2013 + useful metdata
R
121
star
29

tidyversedashboard

Tidyverse activity dashboard
R
71
star
30

tidy-dev-day

Tidyverse developer day
59
star
31

tidyeval

A guide to tidy evaluation
CSS
54
star
32

dsbox

Companion R package to Data Science Course in a Box
R
47
star
33

tidytemplate

A pkgdown template for core tidyverse packages
SCSS
45
star
34

blob

A simple S3 class for representing BLOBs
R
44
star
35

code-review

32
star
36

funs

Collection of low-level functions for working with vctrs
R
31
star
37

website-analytics

Web analytics for tidyverse + r-lib sites
R
28
star
38

tidyups

20
star
39

ggplot2-docs

ggplot2 documentation. Auto-generated from ggplot2 sources by pkgdown
HTML
10
star