The tidyverse is an opinionated collection of R packages designed for data science. All packages share an underlying design philosophy, grammar, and data structures (tidyverse.org).
Sepal.Length Sepal.Width petal_length Petal.Width Species
1 5.1 3.5 1.4 0.2 setosa
2 4.9 3.0 1.4 0.2 setosa
3 4.7 3.2 1.3 0.2 setosa
4 4.6 3.1 1.5 0.2 setosa
5 5.0 3.6 1.4 0.2 setosa
6 5.4 3.9 1.7 0.4 setosa
7 4.6 3.4 1.4 0.3 setosa
8 5.0 3.4 1.5 0.2 setosa
9 4.4 2.9 1.4 0.2 setosa
10 4.9 3.1 1.5 0.1 setosa
11 5.4 3.7 1.5 0.2 setosa
12 4.8 3.4 1.6 0.2 setosa
13 4.8 3.0 1.4 0.1 setosa
14 4.3 3.0 1.1 0.1 setosa
15 5.8 4.0 1.2 0.2 setosa
16 5.7 4.4 1.5 0.4 setosa
17 5.4 3.9 1.3 0.4 setosa
18 5.1 3.5 1.4 0.3 setosa
19 5.7 3.8 1.7 0.3 setosa
20 5.1 3.8 1.5 0.3 setosa
21 5.4 3.4 1.7 0.2 setosa
22 5.1 3.7 1.5 0.4 setosa
23 4.6 3.6 1.0 0.2 setosa
24 5.1 3.3 1.7 0.5 setosa
25 4.8 3.4 1.9 0.2 setosa
26 5.0 3.0 1.6 0.2 setosa
27 5.0 3.4 1.6 0.4 setosa
28 5.2 3.5 1.5 0.2 setosa
29 5.2 3.4 1.4 0.2 setosa
30 4.7 3.2 1.6 0.2 setosa
31 4.8 3.1 1.6 0.2 setosa
32 5.4 3.4 1.5 0.4 setosa
33 5.2 4.1 1.5 0.1 setosa
34 5.5 4.2 1.4 0.2 setosa
35 4.9 3.1 1.5 0.2 setosa
36 5.0 3.2 1.2 0.2 setosa
37 5.5 3.5 1.3 0.2 setosa
38 4.9 3.6 1.4 0.1 setosa
39 4.4 3.0 1.3 0.2 setosa
40 5.1 3.4 1.5 0.2 setosa
41 5.0 3.5 1.3 0.3 setosa
42 4.5 2.3 1.3 0.3 setosa
43 4.4 3.2 1.3 0.2 setosa
44 5.0 3.5 1.6 0.6 setosa
45 5.1 3.8 1.9 0.4 setosa
46 4.8 3.0 1.4 0.3 setosa
47 5.1 3.8 1.6 0.2 setosa
48 4.6 3.2 1.4 0.2 setosa
49 5.3 3.7 1.5 0.2 setosa
50 5.0 3.3 1.4 0.2 setosa
51 7.0 3.2 4.7 1.4 versicolor
52 6.4 3.2 4.5 1.5 versicolor
53 6.9 3.1 4.9 1.5 versicolor
54 5.5 2.3 4.0 1.3 versicolor
55 6.5 2.8 4.6 1.5 versicolor
56 5.7 2.8 4.5 1.3 versicolor
57 6.3 3.3 4.7 1.6 versicolor
58 4.9 2.4 3.3 1.0 versicolor
59 6.6 2.9 4.6 1.3 versicolor
60 5.2 2.7 3.9 1.4 versicolor
61 5.0 2.0 3.5 1.0 versicolor
62 5.9 3.0 4.2 1.5 versicolor
63 6.0 2.2 4.0 1.0 versicolor
64 6.1 2.9 4.7 1.4 versicolor
65 5.6 2.9 3.6 1.3 versicolor
66 6.7 3.1 4.4 1.4 versicolor
67 5.6 3.0 4.5 1.5 versicolor
68 5.8 2.7 4.1 1.0 versicolor
69 6.2 2.2 4.5 1.5 versicolor
70 5.6 2.5 3.9 1.1 versicolor
71 5.9 3.2 4.8 1.8 versicolor
72 6.1 2.8 4.0 1.3 versicolor
73 6.3 2.5 4.9 1.5 versicolor
74 6.1 2.8 4.7 1.2 versicolor
75 6.4 2.9 4.3 1.3 versicolor
76 6.6 3.0 4.4 1.4 versicolor
77 6.8 2.8 4.8 1.4 versicolor
78 6.7 3.0 5.0 1.7 versicolor
79 6.0 2.9 4.5 1.5 versicolor
80 5.7 2.6 3.5 1.0 versicolor
81 5.5 2.4 3.8 1.1 versicolor
82 5.5 2.4 3.7 1.0 versicolor
83 5.8 2.7 3.9 1.2 versicolor
84 6.0 2.7 5.1 1.6 versicolor
85 5.4 3.0 4.5 1.5 versicolor
86 6.0 3.4 4.5 1.6 versicolor
87 6.7 3.1 4.7 1.5 versicolor
88 6.3 2.3 4.4 1.3 versicolor
89 5.6 3.0 4.1 1.3 versicolor
90 5.5 2.5 4.0 1.3 versicolor
91 5.5 2.6 4.4 1.2 versicolor
92 6.1 3.0 4.6 1.4 versicolor
93 5.8 2.6 4.0 1.2 versicolor
94 5.0 2.3 3.3 1.0 versicolor
95 5.6 2.7 4.2 1.3 versicolor
96 5.7 3.0 4.2 1.2 versicolor
97 5.7 2.9 4.2 1.3 versicolor
98 6.2 2.9 4.3 1.3 versicolor
99 5.1 2.5 3.0 1.1 versicolor
100 5.7 2.8 4.1 1.3 versicolor
101 6.3 3.3 6.0 2.5 virginica
102 5.8 2.7 5.1 1.9 virginica
103 7.1 3.0 5.9 2.1 virginica
104 6.3 2.9 5.6 1.8 virginica
105 6.5 3.0 5.8 2.2 virginica
106 7.6 3.0 6.6 2.1 virginica
107 4.9 2.5 4.5 1.7 virginica
108 7.3 2.9 6.3 1.8 virginica
109 6.7 2.5 5.8 1.8 virginica
110 7.2 3.6 6.1 2.5 virginica
111 6.5 3.2 5.1 2.0 virginica
112 6.4 2.7 5.3 1.9 virginica
113 6.8 3.0 5.5 2.1 virginica
114 5.7 2.5 5.0 2.0 virginica
115 5.8 2.8 5.1 2.4 virginica
116 6.4 3.2 5.3 2.3 virginica
117 6.5 3.0 5.5 1.8 virginica
118 7.7 3.8 6.7 2.2 virginica
119 7.7 2.6 6.9 2.3 virginica
120 6.0 2.2 5.0 1.5 virginica
121 6.9 3.2 5.7 2.3 virginica
122 5.6 2.8 4.9 2.0 virginica
123 7.7 2.8 6.7 2.0 virginica
124 6.3 2.7 4.9 1.8 virginica
125 6.7 3.3 5.7 2.1 virginica
126 7.2 3.2 6.0 1.8 virginica
127 6.2 2.8 4.8 1.8 virginica
128 6.1 3.0 4.9 1.8 virginica
129 6.4 2.8 5.6 2.1 virginica
130 7.2 3.0 5.8 1.6 virginica
131 7.4 2.8 6.1 1.9 virginica
132 7.9 3.8 6.4 2.0 virginica
133 6.4 2.8 5.6 2.2 virginica
134 6.3 2.8 5.1 1.5 virginica
135 6.1 2.6 5.6 1.4 virginica
136 7.7 3.0 6.1 2.3 virginica
137 6.3 3.4 5.6 2.4 virginica
138 6.4 3.1 5.5 1.8 virginica
139 6.0 3.0 4.8 1.8 virginica
140 6.9 3.1 5.4 2.1 virginica
141 6.7 3.1 5.6 2.4 virginica
142 6.9 3.1 5.1 2.3 virginica
143 5.8 2.7 5.1 1.9 virginica
144 6.8 3.2 5.9 2.3 virginica
145 6.7 3.3 5.7 2.5 virginica
146 6.7 3.0 5.2 2.3 virginica
147 6.3 2.5 5.0 1.9 virginica
148 6.5 3.0 5.2 2.0 virginica
149 6.2 3.4 5.4 2.3 virginica
150 5.9 3.0 5.1 1.8 virginica
datascience_exercises.Rmd
When you start programming for yourself:
.RProj
file will be createdTo read text data, you need:
,
, ;
or \t
Read from file | Write to file | Value separator |
---|---|---|
read_delim() |
write_delim() |
catch-all: user needs to specify |
read_csv() |
write_csv() |
comma separated |
read_csv2() |
write_csv2() |
semicolon separated |
read_tsv() |
write_tsv() |
tab separated |
For non-text data files, there are other R packages:
readxl
: Excel fileshaven
: SPSS & STATA filesgooglesheets4
: Google Sheetsrvest
: HTML filesRead a file located in the same folder as your script:
Or from the web:
dplyr
contains functions for many types of data manipulation, such as:
filter()
: select rows that meet one or several logical criteriaselect()
: select (or drop) columnsrename()
: change column namemutate()
: transform column values or create new columngroup_by()
: group data on one or more columnssummarize()
: reduces a group of data into a single rowSelects rows in your dataframe.
Use:
From the morning session: “From your dataframe df
, return complete rows for everyone living in a country of your choice.”
Select or drop columns in your dataframe.
Basic use:
Use special selecting functions
From the morning session: “Return the columns name
and age
together.”
penguins
to leave out the NAs.Or
Transform column values, or create new column.
Basic use:
Renaming columns.
Basic use:
mutate()
to create a new column culmen_ratio
.A key tidyverse component that chains all data science steps together:
%>%
1
Why?
<-
Note:
df
is only mentioned once at the beginning%>%
Make a workflow that starts with the data penguins
and subsequently applies your filter
, select
, mutate
and rename
operations.
penguins_subset_5 <- penguins %>%
# Filter out NAs
filter(!is.na(Sex)) %>%
# Select only relevant columns
select(Individual_ID, Species, Sex, Island, starts_with("Culmen")) %>%
# Add a new columns culmen_ratio
mutate(culmen_ratio = Culmen_Length_mm / Culmen_Depth_mm) %>%
# Rename Culmen measurement columns
rename(length = Culmen_Length_mm,
depth = Culmen_Depth_mm)
Tidy data sets are all alike; but every messy data set is messy in its own way (Wickham/Grolemund, 2017)
Tidy Data Principles: principles for structuring tabular data sets:
name age country mood_wk1 mood_wk2
1 Ann 35 UK 4 2
2 Bob 22 USA 3 3
3 Chloe 50 USA 4 5
4 Dan 51 UK 2 4
Wide or long?
Wide! Why is this not tidy?
Wide
name age country mood_wk1 mood_wk2
1 Ann 35 UK 4 2
2 Bob 22 USA 3 3
3 Chloe 50 USA 4 5
4 Dan 51 UK 2 4
Long
# A tibble: 4 × 5
name age country week mood
<chr> <dbl> <chr> <dbl> <dbl>
1 Ann 35 UK 1 4
2 Ann 35 UK 2 2
3 Bob 22 USA 1 3
4 Bob 22 USA 2 3
Tidy data is a consistent way of storing data + most R functions work with vectors of values (columns). Tidyverse packages are designed to work with tidy data (dplyr
, ggplot2
, etc.)
Do It Yourself:
pivot_longer()
: lengthen data: more rows, fewer columns (long format, tidy)pivot_wider()
: widen data: fewer rows, more columns (wide format)Basic use:
Check ?pivot_longer()
and Google! for examples and other function arguments.
Source: Allison Hill
Transform the dataframe from wide to long format using the function pivot_longer()
.
Group by one or more columns and perform some summarizing operation.
Basic use:
Read more about joins in R for Data Science.
group_by()
and summarize()
to calculate the mean and standard deviation of all measurements, grouped by species and the type of measurement.ggplot2
is Hadley Wickham’s reimplementation of The Grammar of Graphics (Leland Wilkinson, 2005).Source: towardsdatascience
Remember df_ext
?
name age country mood_wk1 mood_wk2
1 Ann 35 UK 4 2
2 Bob 22 USA 3 3
3 Chloe 50 USA 4 5
4 Dan 51 UK 2 4
First layer: the data
Second layer: aesthetics
Third layer: geom
Another layer: labels
Turn week into character (to prevent ugly plotting) + plot data.
Add aesthetics.
Add a geom.
Create a separate plot per country.
Use a nicer theme.
Larger text
Tip: Choose a visualization -> Get example code: https://www.data-to-viz.com/
A scatterplot of Culmen_Length_mm against Flipper_Length_mm per Island.
What have we learned this afternoon?
read_csv()
, read_tsv()
, read_excel()
write_csv
filter()
, select()
, mutate()
, rename()
%>%
ggplot2
to make some basic plotsSee also the What’s next page.