ChartifyR https://noahweidig.github.io/chartifyr/ quarto-1.6.42 Sat, 28 Mar 2026 00:00:00 GMT Multi-Panel Plots in R Noah Weidig https://noahweidig.github.io/chartifyr/posts/multiplot/ One plot is rarely enough. Whether you are comparing species across islands, aligning a scatter plot next to a box plot, or building a manuscript figure with labeled panels, multi-panel layouts are a daily tool in data visualization. R gives you two broad strategies: faceting (splitting one plot by a variable) and plot composition (gluing separate plots together). This post covers both, using the Palmer Penguins dataset throughout.

Setup

Code 
# Uncomment to install any missing packages
# install.packages(c("ggplot2", "dplyr", "palmerpenguins", "patchwork", "cowplot"))

library(ggplot2)
library(dplyr)
library(palmerpenguins)
library(patchwork)
library(cowplot)

penguins <- palmerpenguins::penguins

Faceting: Splitting One Plot by a Variable

Faceting is built into ggplot2. You write a single ggplot() call, then add facet_wrap() or facet_grid() to automatically replicate the plot for each level of a grouping variable. All panels share the same axes by default, making comparisons easy.

facet_wrap

facet_wrap() lays panels out in a ribbon that wraps to a new row once it hits a set number of columns. It is the go-to choice when you have one grouping variable and want ggplot2 to handle the layout for you.

Code 
ggplot(penguins, aes(x = bill_length_mm, y = bill_depth_mm, color = species)) +
  geom_point(alpha = 0.7, na.rm = TRUE) +
  facet_wrap(~species) +
  scale_color_brewer(palette = "Set2") +
  labs(
    title = "Bill dimensions by species",
    x = "Bill length (mm)",
    y = "Bill depth (mm)",
    color = "Species"
  ) +
  theme_bw() +
  theme(legend.position = "none")

The ~species formula tells ggplot2 which variable to facet on. By default it picks the number of columns automatically, but you can override that with ncol or nrow.

Code 
ggplot(penguins, aes(x = body_mass_g, fill = sex)) +
  geom_histogram(bins = 20, na.rm = TRUE, position = "identity", alpha = 0.7) +
  facet_wrap(~species, ncol = 1) +
  scale_fill_brewer(palette = "Pastel1", na.value = "grey70") +
  labs(
    title = "Body mass distribution by species",
    x = "Body mass (g)",
    y = "Count",
    fill = "Sex"
  ) +
  theme_bw()

Free scales

By default all panels share the same axis limits (scales = "fixed"). Set scales = "free", "free_x", or "free_y" when the ranges differ dramatically between groups and you care more about within-panel pattern than cross-panel comparison.

Code 
ggplot(penguins, aes(x = flipper_length_mm, y = body_mass_g)) +
  geom_point(aes(color = island), alpha = 0.7, na.rm = TRUE) +
  geom_smooth(method = "lm", se = FALSE, na.rm = TRUE, color = "grey30") +
  facet_wrap(~species, scales = "free") +
  scale_color_brewer(palette = "Dark2") +
  labs(
    title = "Flipper length vs body mass (free scales)",
    x = "Flipper length (mm)",
    y = "Body mass (g)",
    color = "Island"
  ) +
  theme_bw()

facet_grid

facet_grid() creates a strict rows × columns grid defined by two variables. Use it when you want to cross-tabulate two grouping variables and keep the layout perfectly aligned.

Code 
ggplot(
  penguins |> filter(!is.na(sex)),
  aes(x = bill_length_mm, y = bill_depth_mm)
) +
  geom_point(aes(color = species), alpha = 0.7, na.rm = TRUE) +
  facet_grid(sex ~ species) +
  scale_color_brewer(palette = "Set2") +
  labs(
    title = "Bill dimensions by species and sex",
    x = "Bill length (mm)",
    y = "Bill depth (mm)",
    color = "Species"
  ) +
  theme_bw() +
  theme(legend.position = "bottom")

The formula sex ~ species puts sex levels along rows and species levels across columns. Use . to suppress one dimension: ~ species gives one row of columns, sex ~ . gives one column of rows.

Code 
# One row of columns (equivalent to facet_wrap with fixed layout)
ggplot(penguins, aes(x = body_mass_g, y = flipper_length_mm)) +
  geom_point(aes(color = sex), alpha = 0.7, na.rm = TRUE) +
  facet_grid(. ~ island) +
  scale_color_brewer(palette = "Set1", na.value = "grey70") +
  labs(
    title = "Body mass vs flipper length by island",
    x = "Body mass (g)",
    y = "Flipper length (mm)",
    color = "Sex"
  ) +
  theme_bw()

facet_wrap vs facet_grid at a glance

Feature facet_wrap facet_grid
Variables One Two (rows × columns)
Layout Wraps automatically Strict grid
Best for One grouping variable Crossing two variables
Free scales Yes (scales =) Yes (scales =)
Space control nrow / ncol Fixed by variable levels

Plot Composition: Combining Separate Plots

Sometimes you need panels that show different variables, geoms, or even different datasets side by side. That is where patchwork and cowplot come in — they let you take any ggplot objects and arrange them into a single figure.

Let us build a few reusable plots to combine throughout these examples.

Code 
p_scatter <- ggplot(penguins, aes(x = bill_length_mm, y = bill_depth_mm,
                                   color = species)) +
  geom_point(alpha = 0.7, na.rm = TRUE) +
  scale_color_brewer(palette = "Set2") +
  labs(x = "Bill length (mm)", y = "Bill depth (mm)", color = "Species") +
  theme_bw()

p_box <- ggplot(penguins, aes(x = species, y = body_mass_g, fill = species)) +
  geom_boxplot(na.rm = TRUE, show.legend = FALSE) +
  scale_fill_brewer(palette = "Set2") +
  labs(x = "Species", y = "Body mass (g)") +
  theme_bw()

p_hist <- ggplot(penguins, aes(x = flipper_length_mm, fill = species)) +
  geom_histogram(bins = 25, alpha = 0.7, na.rm = TRUE, position = "identity") +
  scale_fill_brewer(palette = "Set2") +
  labs(x = "Flipper length (mm)", y = "Count", fill = "Species") +
  theme_bw()

p_bar <- ggplot(penguins, aes(x = island, fill = species)) +
  geom_bar(position = "dodge") +
  scale_fill_brewer(palette = "Set2") +
  labs(x = "Island", y = "Count", fill = "Species") +
  theme_bw()

patchwork

patchwork extends ggplot2 with a simple operator-based syntax. After loading the package, you can combine plots with +, /, and | exactly like arithmetic.

Code 
# Side by side
p_scatter + p_box

Code 
# Stacked
p_scatter / p_box

Layouts with plot_layout()

plot_layout() gives you fine-grained control over the grid, including relative widths, heights, and shared guides.

Code 
(p_scatter | p_box | p_hist) +
  plot_layout(widths = c(2, 1, 1))

Code 
# A 2×2 grid with shared legend collected at the bottom
(p_scatter + p_box) / (p_hist + p_bar) +
  plot_layout(guides = "collect") &
  theme(legend.position = "bottom")

Panel labels with plot_annotation()

plot_annotation() adds a title and subtitle to the whole figure, plus automatic A, B, C… labels on each panel — essential for publications.

Code 
(p_scatter + p_box) / (p_hist + p_bar) +
  plot_annotation(
    title = "Palmer Penguins: a four-panel summary",
    subtitle = "Bill dimensions, body mass, flipper distribution, and island counts",
    tag_levels = "A"
  ) +
  plot_layout(guides = "collect") &
  theme(legend.position = "bottom")

Nesting layouts

You can nest patchwork assemblies inside each other using wrap_plots() or by grouping with parentheses.

Code 
right_column <- p_box / p_bar

p_scatter | right_column

cowplot

cowplot (by Claus Wilke) takes a slightly different approach. Its main function is plot_grid(), which accepts a list of plots and arranges them with precise alignment control. It is especially useful when panels have different axis structures and you need exact pixel-level alignment.

Code 
plot_grid(p_scatter, p_box, ncol = 2)

Code 
# Add panel labels
plot_grid(p_scatter, p_box, p_hist, p_bar,
          labels = c("A", "B", "C", "D"),
          label_size = 12,
          ncol = 2)

Controlling relative sizes

Use rel_widths and rel_heights to weight panels.

Code 
plot_grid(p_scatter, p_box,
          ncol = 2,
          rel_widths = c(2, 1),
          labels = "AUTO")

Shared legends with get_legend()

A common cowplot pattern is to pull the legend out of one plot and place it as its own element using get_legend().

Code 
# Build plots without their own legends
p_scatter_noleg <- p_scatter + theme(legend.position = "none")
p_hist_noleg    <- p_hist    + theme(legend.position = "none")

# Extract legend from the original scatter
legend <- get_legend(p_scatter)

# Arrange plots + legend
top_row <- plot_grid(p_scatter_noleg, p_hist_noleg,
                     ncol = 2, labels = c("A", "B"))

plot_grid(top_row, legend,
          ncol = 1,
          rel_heights = c(1, 0.15))

Aligning axes

align = "hv" forces horizontal and vertical axis alignment across panels — critical when mixing plots with different title heights or legend sizes.

Code 
plot_grid(p_scatter, p_box, p_hist,
          align  = "hv",
          ncol   = 3,
          labels = "AUTO")

patchwork vs cowplot at a glance

Feature patchwork cowplot
Syntax Operators (+, /, |) plot_grid() function
Learning curve Very low Low
Shared legend plot_layout(guides = "collect") get_legend() + plot_grid()
Axis alignment Automatic align = "hv"
Panel labels plot_annotation(tag_levels =) labels = argument
Nesting Parentheses / wrap_plots() Nested plot_grid() calls
Best for Quick layouts, publications Precise alignment, mixed axes

Putting It All Together

You can even combine faceting and composition. Here we facet a scatter plot by island, then place it alongside a summary box plot.

Code 
p_faceted <- ggplot(penguins, aes(x = bill_length_mm, y = bill_depth_mm,
                                   color = species)) +
  geom_point(alpha = 0.7, na.rm = TRUE) +
  facet_wrap(~island) +
  scale_color_brewer(palette = "Set2") +
  labs(
    title = "Bill dimensions by island",
    x = "Bill length (mm)",
    y = "Bill depth (mm)",
    color = "Species"
  ) +
  theme_bw() +
  theme(legend.position = "none")

p_summary <- ggplot(penguins, aes(x = species, y = bill_length_mm,
                                   fill = species)) +
  geom_violin(na.rm = TRUE, alpha = 0.7, show.legend = FALSE) +
  geom_boxplot(width = 0.15, na.rm = TRUE, show.legend = FALSE) +
  scale_fill_brewer(palette = "Set2") +
  labs(
    title = "Bill length by species",
    x = "Species",
    y = "Bill length (mm)"
  ) +
  theme_bw()

p_faceted + p_summary +
  plot_layout(widths = c(2, 1)) +
  plot_annotation(
    title   = "Penguin bill dimensions: island facets + species summary",
    tag_levels = "A"
  )

Quick Reference

Goal Tool Key argument
Split one plot by one variable facet_wrap(~var) ncol, nrow, scales
Cross two grouping variables facet_grid(r ~ c) scales, space
Place plots side by side p1 \| p2 (patchwork) —
Stack plots vertically p1 / p2 (patchwork) —
Control widths/heights plot_layout() widths, heights
Add figure title + labels plot_annotation() title, tag_levels
Arrange with precise alignment plot_grid() (cowplot) align, rel_widths
Shared legend (cowplot) get_legend() —

Multi-panel layouts take a little practice, but once you have the patterns down they become second nature. Start with facet_wrap whenever a single variable is doing the grouping, reach for facet_grid when you are crossing two factors, and pull in patchwork or cowplot whenever you need to combine fundamentally different plots into one cohesive figure.

]]> code ggplot2 dataviz https://noahweidig.github.io/chartifyr/posts/multiplot/ Sat, 28 Mar 2026 00:00:00 GMT Everything Palettes in R: Viridis, Brewer, and Beyond Noah Weidig https://noahweidig.github.io/chartifyr/posts/palettes/ Color can make or break a figure. The right palette draws attention to the story in your data; the wrong one buries it under visual noise. In this post we will walk through the major palette ecosystems available in R, learn when to reach for each one, and pick up practical rules that keep figures readable, accessible, and honest.

Setup

We will use several packages throughout this tutorial. Install any you are missing, then load them all at once.

Code 
# Uncomment to install any missing packages
# install.packages(c(
#   "ggplot2", "dplyr", "scales", "viridis",
#   "RColorBrewer", "colorspace", "paletteer",
#   "scico", "MetBrewer", "wesanderson", "ggsci",
#   "rcartocolor", "palmerpenguins", "khroma"
# ))

library(ggplot2)
library(dplyr)
library(scales)
library(viridis)
library(RColorBrewer)
library(colorspace)
library(paletteer)
library(scico)
library(MetBrewer)
library(wesanderson)
library(ggsci)
library(rcartocolor)
library(palmerpenguins)
library(khroma)

We will rely on the palmerpenguins dataset for most examples. It gives us both categorical variables (species, island) and continuous ones (bill_length_mm, body_mass_g).

Code 
penguins <- palmerpenguins::penguins
glimpse(penguins)
Rows: 344
Columns: 8
$ species           <fct> Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adel…
$ island            <fct> Torgersen, Torgersen, Torgersen, Torgersen, Torgerse…
$ bill_length_mm    <dbl> 39.1, 39.5, 40.3, NA, 36.7, 39.3, 38.9, 39.2, 34.1, …
$ bill_depth_mm     <dbl> 18.7, 17.4, 18.0, NA, 19.3, 20.6, 17.8, 19.6, 18.1, …
$ flipper_length_mm <int> 181, 186, 195, NA, 193, 190, 181, 195, 193, 190, 186…
$ body_mass_g       <int> 3750, 3800, 3250, NA, 3450, 3650, 3625, 4675, 3475, …
$ sex               <fct> male, female, female, NA, female, male, female, male…
$ year              <int> 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007…

Continuous vs Discrete Color Scales

Before choosing a palette, decide whether the variable you are mapping to color is continuous (numeric) or discrete (categorical). In ggplot2:

  • Continuous variables use scale_color_*() / scale_fill_*() functions that interpolate between colors.
  • Discrete variables use functions that assign one distinct color per level.

Many palette packages offer both variants. Here is a quick comparison.

Code 
# Continuous: bill length mapped to a gradient
p_cont <- ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g,
                                color = bill_length_mm)) +
  geom_point(size = 2, na.rm = TRUE) +
  scale_color_viridis_c() +
  labs(title = "Continuous scale", color = "Bill length (mm)") +
  theme_classic()

# Discrete: species mapped to distinct colors
p_disc <- ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g,
                                color = species)) +
  geom_point(size = 2, na.rm = TRUE) +
  scale_color_viridis_d() +
  labs(title = "Discrete scale", color = "Species") +
  theme_classic()

p_cont

Code 
p_disc

Rule of thumb: continuous scales for numbers, discrete scales for categories.

Viridis (viridis / viridisLite)

The viridis family is the gold standard for perceptually uniform, colorblind-safe palettes. The colors change smoothly in lightness and chroma, so a plot printed in grayscale still makes sense.

Sub-palettes: viridis, magma, inferno, plasma, cividis, rocket, mako, turbo.

When to use: any time you need a safe default for continuous data, heatmaps, or ordered categories.

Code 
ggplot(faithfuld, aes(waiting, eruptions, fill = density)) +
  geom_tile() +
  scale_fill_viridis_c(option = "magma") +
  labs(title = "Old Faithful eruptions — viridis magma") +
  theme_classic()

Code 
ggplot(penguins, aes(x = species, y = flipper_length_mm,
                      fill = species)) +
  geom_boxplot(na.rm = TRUE, show.legend = FALSE) +
  scale_fill_viridis_d(option = "cividis") +
  labs(title = "Flipper length by species — viridis cividis",
       x = "Species", y = "Flipper length (mm)") +
  theme_classic()

RColorBrewer

RColorBrewer provides three palette families designed by cartographer Cynthia Brewer:

Type Purpose Example palettes
Sequential Low-to-high values Blues, YlOrRd, Greens
Diverging Deviation from a midpoint RdBu, BrBG, PiYG
Qualitative Unordered categories Set2, Paired, Dark2

When to use: you need a well-tested, publication-ready palette and want to choose the family explicitly.

Code 
# Display available palettes
display.brewer.all(n = 8)

Code 
ggplot(penguins, aes(x = bill_length_mm, y = bill_depth_mm,
                      color = body_mass_g)) +
  geom_point(size = 2, na.rm = TRUE) +
  scale_color_distiller(palette = "YlOrRd", direction = 1) +
  labs(title = "Body mass — Brewer YlOrRd (sequential)",
       x = "Bill length (mm)", y = "Bill depth (mm)",
       color = "Mass (g)") +
  theme_classic()

Code 
ggplot(penguins, aes(x = bill_length_mm, y = bill_depth_mm,
                      color = species)) +
  geom_point(size = 2, na.rm = TRUE) +
  scale_color_brewer(palette = "Set2") +
  labs(title = "Species — Brewer Set2 (qualitative)",
       x = "Bill length (mm)", y = "Bill depth (mm)") +
  theme_classic()

colorspace

The colorspace package lets you build palettes from scratch using perceptual color spaces (HCL). It ships ready-made palettes and diagnostic tools.

When to use: you want fine control over hue, chroma, and luminance or need to evaluate existing palettes.

Code 
ggplot(penguins, aes(x = species, y = body_mass_g, fill = species)) +
  geom_violin(na.rm = TRUE, show.legend = FALSE) +
  scale_fill_discrete_qualitative(palette = "Dark 3") +
  labs(title = "Body mass — colorspace Dark 3",
       x = "Species", y = "Body mass (g)") +
  theme_classic()

Code 
ggplot(faithfuld, aes(waiting, eruptions, fill = density)) +
  geom_tile() +
  scale_fill_continuous_sequential(palette = "ag_Sunset") +
  labs(title = "Old Faithful — colorspace ag_Sunset") +
  theme_classic()

A powerful diagnostic: colorspace::specplot() shows how lightness, chroma, and hue vary across a palette.

Code 
colorspace::specplot(colorspace::sequential_hcl(256, palette = "ag_Sunset"))

paletteer

paletteer is a meta-package that gives you a unified interface to hundreds of palettes from dozens of packages.

When to use: you want to browse or switch between palettes without loading each individual package.

Code 
ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g,
                      color = species)) +
  geom_point(size = 2, na.rm = TRUE) +
  paletteer::scale_color_paletteer_d("ggsci::default_jco") +
  labs(title = "Species — paletteer (ggsci JCO)",
       x = "Bill length (mm)", y = "Body mass (g)") +
  theme_classic()

Code 
ggplot(penguins, aes(x = bill_length_mm, y = bill_depth_mm,
                      color = body_mass_g)) +
  geom_point(size = 2, na.rm = TRUE) +
  paletteer::scale_color_paletteer_c("scico::batlow", direction = -1) +
  labs(title = "Body mass — paletteer (scico batlow)",
       x = "Bill length (mm)", y = "Bill depth (mm)",
       color = "Mass (g)") +
  theme_classic()

scico

scico provides perceptually uniform, colorblind-safe scientific colour maps developed by Fabio Crameri.

When to use: earth science, oceanography, or any continuous data where perceptual uniformity is critical.

Code 
ggplot(faithfuld, aes(waiting, eruptions, fill = density)) +
  geom_tile() +
  scico::scale_fill_scico(palette = "roma") +
  labs(title = "Old Faithful — scico roma") +
  theme_classic()

Code 
ggplot(penguins, aes(x = species, y = flipper_length_mm,
                      fill = species)) +
  geom_boxplot(na.rm = TRUE, show.legend = FALSE) +
  scico::scale_fill_scico_d(palette = "batlow") +
  labs(title = "Flipper length — scico batlow (discrete)",
       x = "Species", y = "Flipper length (mm)") +
  theme_classic()

MetBrewer

MetBrewer offers palettes inspired by works at the Metropolitan Museum of Art. They are visually striking and still colorblind-friendly.

When to use: presentations, posters, or publications where you want an artistic aesthetic without sacrificing readability.

Code 
ggplot(penguins, aes(x = bill_length_mm, y = bill_depth_mm,
                      color = species)) +
  geom_point(size = 2, na.rm = TRUE) +
  MetBrewer::scale_color_met_d("Lakota") +
  labs(title = "Species — MetBrewer Lakota",
       x = "Bill length (mm)", y = "Bill depth (mm)") +
  theme_classic()

Code 
ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g,
                      color = bill_depth_mm)) +
  geom_point(size = 2, na.rm = TRUE) +
  MetBrewer::scale_color_met_c("Hiroshige") +
  labs(title = "Bill depth — MetBrewer Hiroshige",
       x = "Bill length (mm)", y = "Body mass (g)",
       color = "Bill depth (mm)") +
  theme_classic()

wesanderson

wesanderson provides palettes inspired by the films of Wes Anderson — charming pastels and earthy tones.

When to use: informal presentations, blog graphics, or any context where personality is a plus. These are primarily discrete palettes.

Code 
ggplot(penguins, aes(x = species, y = body_mass_g, fill = species)) +
  geom_boxplot(na.rm = TRUE, show.legend = FALSE) +
  scale_fill_manual(values = wes_palette("Darjeeling1", 3)) +
  labs(title = "Body mass — wesanderson Darjeeling1",
       x = "Species", y = "Body mass (g)") +
  theme_classic()

Code 
ggplot(penguins, aes(x = bill_length_mm, y = flipper_length_mm,
                      color = island)) +
  geom_point(size = 2, na.rm = TRUE) +
  scale_color_manual(values = wes_palette("GrandBudapest1", 3)) +
  labs(title = "Islands — wesanderson GrandBudapest1",
       x = "Bill length (mm)", y = "Flipper length (mm)") +
  theme_classic()

ggsci

ggsci provides color palettes from scientific journals and sci-fi franchises (JAMA, Lancet, NEJM, Nature, Star Trek, and more).

When to use: journal submissions where you want colors that match the publication’s own figures, or when you need a conservative, professional palette.

Code 
ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g,
                      color = species)) +
  geom_point(size = 2, na.rm = TRUE) +
  ggsci::scale_color_jco() +
  labs(title = "Species — ggsci JCO",
       x = "Bill length (mm)", y = "Body mass (g)") +
  theme_classic()

Code 
ggplot(penguins, aes(x = species, fill = island)) +
  geom_bar(position = "dodge") +
  ggsci::scale_fill_nejm() +
  labs(title = "Island counts by species — ggsci NEJM",
       x = "Species", y = "Count") +
  theme_classic()

rcartocolor

rcartocolor provides the CARTOColors palettes, designed for cartography. They include sequential, diverging, and qualitative schemes that work well in maps and standard plots alike.

When to use: maps, geographic data, or when you want palettes designed for spatial visualization.

Code 
ggplot(penguins, aes(x = bill_length_mm, y = bill_depth_mm,
                      color = species)) +
  geom_point(size = 2, na.rm = TRUE) +
  rcartocolor::scale_color_carto_d(palette = "Vivid") +
  labs(title = "Species — rcartocolor Vivid",
       x = "Bill length (mm)", y = "Bill depth (mm)") +
  theme_classic()

Code 
ggplot(faithfuld, aes(waiting, eruptions, fill = density)) +
  geom_tile() +
  rcartocolor::scale_fill_carto_c(palette = "SunsetDark") +
  labs(title = "Old Faithful — rcartocolor SunsetDark") +
  theme_classic()

khroma

khroma provides colour schemes for scientific data visualization, with a strong focus on accessibility. All palettes are designed to be distinct for people with colour vision deficiencies.

When to use: scientific figures where accessibility is non-negotiable and you want assurance the palette has been tested for CVD safety.

Code 
ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g,
                      color = species)) +
  geom_point(size = 2, na.rm = TRUE) +
  khroma::scale_color_bright() +
  labs(title = "Species — khroma bright",
       x = "Bill length (mm)", y = "Body mass (g)") +
  theme_classic()

Code 
ggplot(penguins, aes(x = bill_length_mm, y = bill_depth_mm,
                      color = body_mass_g)) +
  geom_point(size = 2, na.rm = TRUE) +
  khroma::scale_color_batlow() +
  labs(title = "Body mass — khroma batlow",
       x = "Bill length (mm)", y = "Bill depth (mm)",
       color = "Mass (g)") +
  theme_classic()

Handling NA Colors

Missing values appear in almost every real dataset. By default ggplot2 maps NA to grey, but you can control this.

Code 
# Create some NAs for demonstration
penguins_na <- penguins |>
  mutate(species_na = if_else(row_number() %% 10 == 0, NA_character_, as.character(species)))

ggplot(penguins_na, aes(x = bill_length_mm, y = body_mass_g,
                         color = species_na)) +
  geom_point(size = 2, na.rm = TRUE) +
  scale_color_viridis_d(na.value = "red") +
  labs(title = "NA values shown in red",
       x = "Bill length (mm)", y = "Body mass (g)",
       color = "Species") +
  theme_classic()

Use na.value = inside any scale_color_*() or scale_fill_*() call to set the NA color. Common choices:

  • "grey50" — subtle, does not draw attention
  • "red" or "black" — highlights missing data for QC
  • NA — removes NA points from the legend entirely

Practical Rules for Number of Colors

The 6–8 Rule

Humans can reliably distinguish roughly 6 to 8 hues at a glance. Beyond that, colors start to blur together, especially in scatterplots where points overlap. Here is what happens when you push it.

Code 
# mtcars has many cylinder + gear combos — let's create a busy variable
mtcars_busy <- mtcars |>
  mutate(car_name = rownames(mtcars))

ggplot(mtcars_busy, aes(x = wt, y = mpg, color = car_name)) +
  geom_point(size = 3) +
  labs(title = "Too many categories — the legend is unreadable",
       x = "Weight (1000 lbs)", y = "Miles per gallon") +
  theme_classic() +
  theme(legend.text = element_text(size = 5))

That is a mess. Here are your escape hatches:

Fix 1: Lump rare categories

Code 
penguins_islands <- penguins |>
  count(island, species) |>
  mutate(label = paste(island, species, sep = " — "))

# Works fine with a manageable number of groups
ggplot(penguins_islands, aes(x = reorder(label, n), y = n, fill = island)) +
  geom_col(show.legend = TRUE) +
  coord_flip() +
  scale_fill_brewer(palette = "Set2") +
  labs(title = "Lumped into a few groups",
       x = NULL, y = "Count", fill = "Island") +
  theme_classic()

Fix 2: Facet instead of coloring

Code 
ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g)) +
  geom_point(size = 1.5, na.rm = TRUE, color = "steelblue") +
  facet_wrap(~ species) +
  labs(title = "Faceting removes the need for color",
       x = "Bill length (mm)", y = "Body mass (g)") +
  theme_classic()

Fix 3: Use shape or linetype in addition to color

Code 
ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g,
                      color = species, shape = species)) +
  geom_point(size = 2, na.rm = TRUE) +
  scale_color_brewer(palette = "Dark2") +
  labs(title = "Shape + color = double encoding",
       x = "Bill length (mm)", y = "Body mass (g)") +
  theme_classic()

Fix 4: Highlight one group, grey the rest

Code 
penguins_hl <- penguins |>
  mutate(highlight = if_else(species == "Chinstrap", "Chinstrap", "Other"))

ggplot(penguins_hl, aes(x = bill_length_mm, y = body_mass_g,
                         color = highlight)) +
  geom_point(size = 2, na.rm = TRUE) +
  scale_color_manual(values = c("Chinstrap" = "#E63946", "Other" = "grey70")) +
  labs(title = "Highlighting one group against the rest",
       x = "Bill length (mm)", y = "Body mass (g)",
       color = NULL) +
  theme_classic()

Accessibility: Contrast & Colorblind-Friendly Design

Approximately 8% of men and 0.5% of women have some form of color vision deficiency (CVD). Designing for accessibility is not optional — it is good science.

Key Principles

  1. Never rely on color alone. Pair color with shape, size, or labels.
  2. Use perceptually uniform palettes. Viridis, scico, and khroma are safe bets.
  3. Avoid red-green contrasts. The most common CVD type (deuteranopia) confuses red and green.
  4. Test your palette. The colorspace package can simulate what your plot looks like under different CVD types.

Simulating Color Vision Deficiency

Code 
# Create a base plot
p_base <- ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g,
                                color = species)) +
  geom_point(size = 2, na.rm = TRUE) +
  scale_color_manual(values = c("Adelie" = "#E63946",
                                 "Chinstrap" = "#2A9D8F",
                                 "Gentoo" = "#264653")) +
  labs(x = "Bill length (mm)", y = "Body mass (g)") +
  theme_classic()

# Original
p_base + labs(title = "Original")

Code 
# Simulate deuteranopia (red-green deficiency)
# cvd_emulator() expects a file path, so save the plot to a temp file first
tmp_plot <- tempfile(fileext = ".png")
ggsave(tmp_plot, p_base + labs(title = "Simulated deuteranopia"),
       width = 7, height = 5, dpi = 150)
colorspace::cvd_emulator(tmp_plot)
Convert deutan          -> /var/folders/03/d3dmjds91_vckyhkgdt677300000gn/T//RtmpSVvF9F/deutan_file7e027e5356fa.png
Convert protan          -> /var/folders/03/d3dmjds91_vckyhkgdt677300000gn/T//RtmpSVvF9F/protan_file7e027e5356fa.png
Convert tritan          -> /var/folders/03/d3dmjds91_vckyhkgdt677300000gn/T//RtmpSVvF9F/tritan_file7e027e5356fa.png
Convert desaturate      -> /var/folders/03/d3dmjds91_vckyhkgdt677300000gn/T//RtmpSVvF9F/desaturate_file7e027e5356fa.png

If the simulated plot makes it hard to distinguish groups, switch to a CVD-safe palette such as viridis, "Okabe-Ito" (khroma), or Brewer’s qualitative palettes.

A Safer Version

Code 
ggplot(penguins, aes(x = bill_length_mm, y = body_mass_g,
                      color = species, shape = species)) +
  geom_point(size = 2, na.rm = TRUE) +
  scale_color_viridis_d(option = "cividis") +
  labs(title = "CVD-safe: viridis cividis + shape encoding",
       x = "Bill length (mm)", y = "Body mass (g)") +
  theme_classic()

Common Pitfalls

Pitfall 1: The Rainbow Palette

Rainbow (jet) palettes are not perceptually uniform — yellow appears brighter than blue, creating false visual emphasis. They also fail under grayscale printing and for colorblind viewers.

Code 
# Don't do this
ggplot(faithfuld, aes(waiting, eruptions, fill = density)) +
  geom_tile() +
  scale_fill_gradientn(colours = rainbow(256)) +
  labs(title = "Rainbow palette — avoid this") +
  theme_classic()

Code 
# Do this instead
ggplot(faithfuld, aes(waiting, eruptions, fill = density)) +
  geom_tile() +
  scale_fill_viridis_c() +
  labs(title = "Viridis — perceptually uniform") +
  theme_classic()

Pitfall 2: Low-Contrast Palettes

Pastel-on-white palettes look washed out on screens and vanish when printed.

Code 
# Low contrast
ggplot(penguins, aes(x = species, y = body_mass_g, fill = species)) +
  geom_boxplot(na.rm = TRUE, show.legend = FALSE) +
  scale_fill_manual(values = c("#FFF5F5", "#F5FFF5", "#F5F5FF")) +
  labs(title = "Low contrast — hard to read") +
  theme_classic()

Code 
# Better contrast
ggplot(penguins, aes(x = species, y = body_mass_g, fill = species)) +
  geom_boxplot(na.rm = TRUE, show.legend = FALSE) +
  scale_fill_brewer(palette = "Set2") +
  labs(title = "Brewer Set2 — clear contrast") +
  theme_classic()

Pitfall 3: Diverging Scales on Non-Diverging Data

Diverging palettes imply a meaningful midpoint (zero, average, threshold). Using them for data without a natural center misleads the reader.

Code 
# Misleading: diverging scale on body mass (no natural midpoint)
ggplot(penguins, aes(x = bill_length_mm, y = bill_depth_mm,
                      color = body_mass_g)) +
  geom_point(size = 2, na.rm = TRUE) +
  scale_color_distiller(palette = "RdBu") +
  labs(title = "Diverging scale on non-diverging data — misleading",
       x = "Bill length (mm)", y = "Bill depth (mm)",
       color = "Mass (g)") +
  theme_classic()

Code 
# Correct: sequential scale
ggplot(penguins, aes(x = bill_length_mm, y = bill_depth_mm,
                      color = body_mass_g)) +
  geom_point(size = 2, na.rm = TRUE) +
  scale_color_viridis_c() +
  labs(title = "Sequential scale — appropriate for body mass",
       x = "Bill length (mm)", y = "Bill depth (mm)",
       color = "Mass (g)") +
  theme_classic()

Pitfall 4: Too Many Categories

We covered this above, but it bears repeating: if your legend has more than 8 entries, rethink your encoding strategy. Lump, facet, highlight, or use a different visual channel.

Palette Checklist

Before you finalize a figure, run through this checklist:

# Question Action if “No”
1 Is the variable continuous or discrete? Switch scale type
2 Does the palette type match the data? (sequential / diverging / qualitative) Change palette family
3 Are there 8 or fewer color categories? Lump, facet, or highlight
4 Is the palette colorblind-safe? Use viridis, scico, or khroma
5 Does the plot read in grayscale? Use a perceptually uniform palette
6 Is there enough contrast against the background? Darken colors or switch palette
7 Are NAs handled intentionally? Set na.value = explicitly
8 Is color the only encoding? Add shape, size, or labels as backup

Quick Reference: Which Package When?

Package Best for Continuous Discrete
viridis Safe default, heatmaps Yes Yes
RColorBrewer Publication-ready, classic Yes Yes
colorspace Custom HCL palettes, diagnostics Yes Yes
paletteer Browsing many packages at once Yes Yes
scico Scientific maps, earth science Yes Yes
MetBrewer Artistic, striking figures Yes Yes
wesanderson Fun, informal graphics No Yes
ggsci Journal-style figures No Yes
rcartocolor Cartography, spatial data Yes Yes
khroma Accessibility-first science Yes Yes

Wrapping Up

There is no single “best” palette — but there are wrong ones. Start with viridis as a safe default, reach for RColorBrewer when you need classic publication palettes, and explore MetBrewer, scico, or khroma when you want something that is both beautiful and accessible. Always test for colorblind safety, keep your categories under 8, and never rely on color alone.

Happy plotting!

]]> code ggplot2 dataviz color https://noahweidig.github.io/chartifyr/posts/palettes/ Tue, 24 Feb 2026 00:00:00 GMT Mastering Joins in dplyr Noah Weidig https://noahweidig.github.io/chartifyr/posts/dplyr-joins/ Combining two datasets sounds simple — until you get back twice as many rows as you expected, or half your data vanishes. Joins are one of the most useful operations in data analysis, but they’re also one of the easiest places to introduce silent bugs. This tutorial will walk you through the main dplyr join functions, show you exactly what each one does, and teach you how to spot problems before they ruin your analysis.

Introduction

A join merges two tables together based on one or more shared columns (called keys). If you’ve ever used VLOOKUP in Excel or merge() in base R, you’ve already done something like a join.

Why do joins trip people up?

  • You assume every key in one table has a match in the other — often it doesn’t.
  • You don’t realize a key appears more than once, causing row multiplication.
  • Missing matches silently introduce NA values that propagate through your analysis.

Joins are everywhere in real work. You might link survey responses to participant demographics, attach weather data to field observations, or connect sales records to product details. Getting them right is essential.

Setup

Code 
# Uncomment and run once if needed
# install.packages("tidyverse")

library(tidyverse)

Example Data

We’ll use two small tibbles throughout this tutorial. Think of them as a field study: one table records which bird species were observed at each site, and the other records habitat information for each site.

Code 
# Bird observations at field sites
observations <- tibble(
  site_id = c("A", "B", "C", "D"),
  species = c("Warbler", "Sparrow", "Hawk", "Finch"),
  count   = c(12, 7, 3, 15)
)

observations
# A tibble: 4 x 3
  site_id species count
  <chr>   <chr>   <dbl>
1 A       Warbler    12
2 B       Sparrow     7
3 C       Hawk        3
4 D       Finch      15
Code 
# Habitat data for field sites
habitat <- tibble(
  site_id     = c("A", "B", "C", "E"),
  habitat_type = c("Forest", "Grassland", "Wetland", "Desert"),
  elevation_m  = c(450, 120, 30, 900)
)

habitat
# A tibble: 4 x 3
  site_id habitat_type elevation_m
  <chr>   <chr>              <dbl>
1 A       Forest               450
2 B       Grassland            120
3 C       Wetland               30
4 E       Desert               900

Notice the mismatch: observations has site D but not E, while habitat has site E but not D. This is realistic — data sources rarely line up perfectly.

left_join()

left_join() keeps every row from the left table and attaches matching columns from the right table. If there’s no match, the right-side columns are filled with NA.

Think of it this way: you start with your primary dataset and add information from a second source.

Code 
# Keep all observations, attach habitat info where available
observations |>
  left_join(habitat, by = "site_id")
# A tibble: 4 x 5
  site_id species count habitat_type elevation_m
  <chr>   <chr>   <dbl> <chr>              <dbl>
1 A       Warbler    12 Forest               450
2 B       Sparrow     7 Grassland            120
3 C       Hawk        3 Wetland               30
4 D       Finch      15 <NA>                  NA

What happened:

  • Sites A, B, and C matched, so their habitat data was attached.
  • Site D had no match in habitat, so habitat_type and elevation_m are NA.
  • Site E (only in habitat) does not appear — left_join() only preserves left-table rows.

This is the most common join in day-to-day analysis. Use it when your left table is the “main” dataset and you want to enrich it with extra columns.

inner_join()

inner_join() keeps only rows that have a match in both tables. No match, no row.

Code 
# Keep only sites present in both tables
observations |>
  inner_join(habitat, by = "site_id")
# A tibble: 3 x 5
  site_id species count habitat_type elevation_m
  <chr>   <chr>   <dbl> <chr>              <dbl>
1 A       Warbler    12 Forest               450
2 B       Sparrow     7 Grassland            120
3 C       Hawk        3 Wetland               30

What happened:

  • Only sites A, B, and C appear — they exist in both tables.
  • Site D (only in observations) and site E (only in habitat) are both dropped.
  • No NA values are introduced from the join itself.

Use inner_join() when you need complete data from both sources and are okay discarding non-matching rows. Be aware that you might silently lose data if your keys don’t overlap as much as you expect.

full_join()

full_join() keeps every row from both tables. Where there’s no match, missing values are filled with NA.

Code 
# Keep everything from both tables
observations |>
  full_join(habitat, by = "site_id")
# A tibble: 5 x 5
  site_id species count habitat_type elevation_m
  <chr>   <chr>   <dbl> <chr>              <dbl>
1 A       Warbler    12 Forest               450
2 B       Sparrow     7 Grassland            120
3 C       Hawk        3 Wetland               30
4 D       Finch      15 <NA>                  NA
5 E       <NA>       NA Desert               900

What happened:

  • Sites A, B, and C matched fully.
  • Site D appears with NA for habitat columns.
  • Site E appears with NA for observation columns.
  • Nothing is lost — every row from both tables is preserved.

Use full_join() when you can’t afford to drop any data and want to see the complete picture of both datasets, gaps included.

Duplicate Row Explosions

This is the join pitfall that catches most people off guard. When a key appears multiple times in one or both tables, the join produces every combination of matching rows. This is called a many-to-many join, and it can silently multiply your row count.

Code 
# Multiple observations per site
obs_multi <- tibble(
  site_id = c("A", "A", "B", "B", "B"),
  species = c("Warbler", "Robin", "Sparrow", "Jay", "Wren"),
  count   = c(12, 5, 7, 3, 9)
)

obs_multi
# A tibble: 5 x 3
  site_id species count
  <chr>   <chr>   <dbl>
1 A       Warbler    12
2 A       Robin       5
3 B       Sparrow     7
4 B       Jay         3
5 B       Wren        9
Code 
# Multiple habitat records per site (e.g., sub-habitats)
hab_multi <- tibble(
  site_id     = c("A", "A", "B"),
  habitat_type = c("Forest-canopy", "Forest-understory", "Grassland"),
  elevation_m  = c(450, 445, 120)
)

hab_multi
# A tibble: 3 x 3
  site_id habitat_type      elevation_m
  <chr>   <chr>                   <dbl>
1 A       Forest-canopy             450
2 A       Forest-understory         445
3 B       Grassland                 120
Code 
# Many-to-many join: row explosion
obs_multi |>
  left_join(hab_multi, by = "site_id", relationship = "many-to-many")
# A tibble: 7 x 5
  site_id species count habitat_type      elevation_m
  <chr>   <chr>   <dbl> <chr>                   <dbl>
1 A       Warbler    12 Forest-canopy             450
2 A       Warbler    12 Forest-understory         445
3 A       Robin       5 Forest-canopy             450
4 A       Robin       5 Forest-understory         445
5 B       Sparrow     7 Grassland                 120
6 B       Jay         3 Grassland                 120
7 B       Wren        9 Grassland                 120

Site A had 2 observation rows and 2 habitat rows, producing 2 x 2 = 4 rows. Site B had 3 observation rows and 1 habitat row, producing 3 x 1 = 3 rows. Our 5-row table became 7 rows.

In a real dataset with thousands of rows, this can balloon your data without any warning if you aren’t checking row counts.

How to detect duplicates before joining:

Code 
# Count how many times each key appears
obs_multi |>
  count(site_id) |>
  filter(n > 1)
# A tibble: 2 x 2
  site_id     n
  <chr>   <int>
1 A           2
2 B           3
Code 
hab_multi |>
  count(site_id) |>
  filter(n > 1)
# A tibble: 1 x 2
  site_id     n
  <chr>   <int>
1 A           2

If both tables show duplicates for the same key, you have a many-to-many situation. Decide whether to aggregate first (e.g., summarize to one row per key) or whether the duplication is intentional.

Diagnosing Join Problems

anti_join()

anti_join() returns rows from the left table that have no match in the right table. It’s the best way to find what gets lost in a join.

Code 
# Which observations have no habitat data?
observations |>
  anti_join(habitat, by = "site_id")
# A tibble: 1 x 3
  site_id species count
  <chr>   <chr>   <dbl>
1 D       Finch      15
Code 
# Which habitats have no observations?
habitat |>
  anti_join(observations, by = "site_id")
# A tibble: 1 x 3
  site_id habitat_type elevation_m
  <chr>   <chr>              <dbl>
1 E       Desert               900

Run anti_join() in both directions before doing your actual join. This tells you exactly what data is missing.

Counting keys

Always check whether your join key is unique before joining. If it isn’t, you may get unexpected row multiplication.

Code 
# Check if site_id is unique in each table
observations |>
  count(site_id) |>
  filter(n > 1)
# A tibble: 0 x 2
# i 2 variables: site_id <chr>, n <int>

An empty result means every key is unique — safe for a one-to-one or one-to-many join.

Checking row counts before and after

A simple but effective diagnostic: compare nrow() before and after your join.

Code 
# Before
nrow(observations)
[1] 4
Code 
# After left_join
result <- observations |>
  left_join(habitat, by = "site_id")

nrow(result)
[1] 4

For a left_join() with unique keys on the right, the row count should stay the same. If it goes up, you likely have duplicate keys in the right table. If it goes down (shouldn’t happen with left_join()), something unusual is going on.

Handling Missing Values

Joins frequently introduce NA values — any time a key doesn’t match, the unmatched columns get NA. It’s important to handle them explicitly rather than letting them propagate through calculations.

Code 
# left_join introduces NAs for site D
joined <- observations |>
  left_join(habitat, by = "site_id")

joined
# A tibble: 4 x 5
  site_id species count habitat_type elevation_m
  <chr>   <chr>   <dbl> <chr>              <dbl>
1 A       Warbler    12 Forest               450
2 B       Sparrow     7 Grassland            120
3 C       Hawk        3 Wetland               30
4 D       Finch      15 <NA>                  NA
Code 
# Replace NA habitat_type with "Unknown"
joined |>
  mutate(habitat_type = replace_na(habitat_type, "Unknown"))
# A tibble: 4 x 5
  site_id species count habitat_type elevation_m
  <chr>   <chr>   <dbl> <chr>              <dbl>
1 A       Warbler    12 Forest               450
2 B       Sparrow     7 Grassland            120
3 C       Hawk        3 Wetland               30
4 D       Finch      15 Unknown               NA
Code 
# Replace multiple columns at once using across()
joined |>
  mutate(
    habitat_type = replace_na(habitat_type, "Unknown"),
    elevation_m  = replace_na(elevation_m, 0)
  )
# A tibble: 4 x 5
  site_id species count habitat_type elevation_m
  <chr>   <chr>   <dbl> <chr>              <dbl>
1 A       Warbler    12 Forest               450
2 B       Sparrow     7 Grassland            120
3 C       Hawk        3 Wetland               30
4 D       Finch      15 Unknown                0

Choosing what to replace NA with depends on your analysis. Sometimes "Unknown" makes sense; sometimes 0 is appropriate; sometimes you should filter out the NA rows entirely. The key is making a deliberate choice rather than ignoring the NA values.

Best Practices Checklist

Before and after every join, run through this list:

  • Inspect your keys. Use count() to check for duplicates in your join columns. Unique keys prevent row explosions.
  • Run anti_join() both ways. Know what doesn’t match before you join so you aren’t surprised by missing data.
  • Compare row counts. Check nrow() before and after. If the count changes unexpectedly, investigate.
  • Use explicit by arguments. Always specify by = "key_column" instead of relying on dplyr to guess. This makes your code readable and avoids surprises when column names change.
  • Handle NA values deliberately. After a left_join() or full_join(), decide what to do with introduced NA values — replace, filter, or flag them.
  • Choose the right join type. Use left_join() to enrich a primary dataset, inner_join() when you need complete cases from both sides, and full_join() when you can’t afford to lose any rows.
  • Aggregate before joining when possible. If you only need one summary row per key, use summarize() before the join to avoid many-to-many blowups.
  • Document your joins. A short comment explaining why you chose a particular join type helps future you (and your collaborators) understand the logic.

Conclusion

Joins are a fundamental skill for working with real data, and dplyr makes them straightforward — once you know the rules. Here’s what to remember:

  • left_join() keeps all left-side rows and fills gaps with NA.
  • inner_join() keeps only rows that match in both tables.
  • full_join() keeps everything from both sides.
  • Duplicate keys cause row multiplication — always check with count().
  • anti_join() is your best diagnostic tool for finding mismatches.
  • Handle NA values explicitly after every join.

The difference between a reliable analysis and a broken one often comes down to whether you checked your join. Build the habit of inspecting keys, comparing row counts, and running anti_join() — it will save you from subtle, hard-to-find bugs.

]]> code dplyr tidyverse https://noahweidig.github.io/chartifyr/posts/dplyr-joins/ Tue, 25 Feb 2025 00:00:00 GMT Data Cleaning in the Real World Noah Weidig https://noahweidig.github.io/chartifyr/posts/data-cleaning-tidyr-stringr/

Artwork by @allison_horst

Artwork by @allison_horst

You know the drill. Someone hands you a spreadsheet with columns like cases_2019, cases_2020, cases_2021, values crammed into single cells, and mysterious NAs scattered everywhere. Before you can analyze anything, you need to clean it. This is where tidyr and stringr come in — and where most of your time as a data analyst actually goes.

This tutorial builds directly on Basics of dplyr. If you’re comfortable with filter(), mutate(), and the pipe, you’re ready.

Setup

Code 
library(tidyverse)

We’ll use two built-in datasets throughout this post:

  • who2 — World Health Organization tuberculosis data (comes with tidyr). Wide, messy, and realistic.
  • starwars — Character data from the Star Wars films (comes with dplyr). Has strings to clean and list-columns to untangle.
Code 
who2
# A tibble: 7,240 × 58
   country      year sp_m_014 sp_m_1524 sp_m_2534 sp_m_3544 sp_m_4554 sp_m_5564
   <chr>       <dbl>    <dbl>     <dbl>     <dbl>     <dbl>     <dbl>     <dbl>
 1 Afghanistan  1980       NA        NA        NA        NA        NA        NA
 2 Afghanistan  1981       NA        NA        NA        NA        NA        NA
 3 Afghanistan  1982       NA        NA        NA        NA        NA        NA
 4 Afghanistan  1983       NA        NA        NA        NA        NA        NA
 5 Afghanistan  1984       NA        NA        NA        NA        NA        NA
 6 Afghanistan  1985       NA        NA        NA        NA        NA        NA
 7 Afghanistan  1986       NA        NA        NA        NA        NA        NA
 8 Afghanistan  1987       NA        NA        NA        NA        NA        NA
 9 Afghanistan  1988       NA        NA        NA        NA        NA        NA
10 Afghanistan  1989       NA        NA        NA        NA        NA        NA
# ℹ 7,230 more rows
# ℹ 50 more variables: sp_m_65 <dbl>, sp_f_014 <dbl>, sp_f_1524 <dbl>,
#   sp_f_2534 <dbl>, sp_f_3544 <dbl>, sp_f_4554 <dbl>, sp_f_5564 <dbl>,
#   sp_f_65 <dbl>, sn_m_014 <dbl>, sn_m_1524 <dbl>, sn_m_2534 <dbl>,
#   sn_m_3544 <dbl>, sn_m_4554 <dbl>, sn_m_5564 <dbl>, sn_m_65 <dbl>,
#   sn_f_014 <dbl>, sn_f_1524 <dbl>, sn_f_2534 <dbl>, sn_f_3544 <dbl>,
#   sn_f_4554 <dbl>, sn_f_5564 <dbl>, sn_f_65 <dbl>, ep_m_014 <dbl>, …
Code 
starwars |> select(name, height, mass, skin_color, homeworld)
# A tibble: 87 × 5
   name               height  mass skin_color  homeworld
   <chr>               <int> <dbl> <chr>       <chr>    
 1 Luke Skywalker        172    77 fair        Tatooine 
 2 C-3PO                 167    75 gold        Tatooine 
 3 R2-D2                  96    32 white, blue Naboo    
 4 Darth Vader           202   136 white       Tatooine 
 5 Leia Organa           150    49 light       Alderaan 
 6 Owen Lars             178   120 light       Tatooine 
 7 Beru Whitesun Lars    165    75 light       Tatooine 
 8 R5-D4                  97    32 white, red  Tatooine 
 9 Biggs Darklighter     183    84 light       Tatooine 
10 Obi-Wan Kenobi        182    77 fair        Stewjon  
# ℹ 77 more rows

Reshaping with pivot_longer()

The most common mess you’ll run into is data that’s wide when it should be long. The who2 dataset is a textbook example — it has separate columns for every combination of diagnosis method, sex, and age group.

Code 
who2 |> colnames() |> head(20)
 [1] "country"   "year"      "sp_m_014"  "sp_m_1524" "sp_m_2534" "sp_m_3544"
 [7] "sp_m_4554" "sp_m_5564" "sp_m_65"   "sp_f_014"  "sp_f_1524" "sp_f_2534"
[13] "sp_f_3544" "sp_f_4554" "sp_f_5564" "sp_f_65"   "sn_m_014"  "sn_m_1524"
[19] "sn_m_2534" "sn_m_3544"

Each of those sp_m_014, sp_f_1524 columns encodes three variables in a single column name. That’s not tidy. Let’s pivot.

Code 
who_long <- who2 |>
  pivot_longer(
    cols = !c(country, year),
    names_to = "category",
    values_to = "count",
    values_drop_na = TRUE
  )

who_long
# A tibble: 76,046 × 4
   country      year category  count
   <chr>       <dbl> <chr>     <dbl>
 1 Afghanistan  1997 sp_m_014      0
 2 Afghanistan  1997 sp_m_1524    10
 3 Afghanistan  1997 sp_m_2534     6
 4 Afghanistan  1997 sp_m_3544     3
 5 Afghanistan  1997 sp_m_4554     5
 6 Afghanistan  1997 sp_m_5564     2
 7 Afghanistan  1997 sp_m_65       0
 8 Afghanistan  1997 sp_f_014      5
 9 Afghanistan  1997 sp_f_1524    38
10 Afghanistan  1997 sp_f_2534    36
# ℹ 76,036 more rows

The values_drop_na = TRUE argument quietly drops all the rows where count is NA. Without it, you’d end up with thousands of empty rows — a common gotcha that balloons your data for no reason.

Pivoting with names_sep

Those category values still pack three pieces of information into one string. We can split them during the pivot itself.

Code 
who_tidy <- who2 |>
  pivot_longer(
    cols = !c(country, year),
    names_to = c("diagnosis", "sex", "age_group"),
    names_sep = "_",
    values_to = "count",
    values_drop_na = TRUE
  )

who_tidy
# A tibble: 76,046 × 6
   country      year diagnosis sex   age_group count
   <chr>       <dbl> <chr>     <chr> <chr>     <dbl>
 1 Afghanistan  1997 sp        m     014           0
 2 Afghanistan  1997 sp        m     1524         10
 3 Afghanistan  1997 sp        m     2534          6
 4 Afghanistan  1997 sp        m     3544          3
 5 Afghanistan  1997 sp        m     4554          5
 6 Afghanistan  1997 sp        m     5564          2
 7 Afghanistan  1997 sp        m     65            0
 8 Afghanistan  1997 sp        f     014           5
 9 Afghanistan  1997 sp        f     1524         38
10 Afghanistan  1997 sp        f     2534         36
# ℹ 76,036 more rows

One call and we went from a 56-column mess to a clean, long-format table with clearly named variables. That’s the power of pivot_longer().

Going wide with pivot_wider()

Sometimes you need the opposite — spreading long data into a wider format for summary tables or specific analyses. Let’s say we want a quick comparison table of total TB cases by diagnosis method and sex.

Code 
who_tidy |>
  group_by(diagnosis, sex) |>
  summarize(total = sum(count), .groups = "drop") |>
  pivot_wider(
    names_from = sex,
    values_from = total
  )
# A tibble: 4 × 3
  diagnosis        f        m
  <chr>        <dbl>    <dbl>
1 ep          941880  1044299
2 rel        1201596  2018976
3 sn         2439139  3840388
4 sp        11324409 20586831

pivot_wider() is the inverse of pivot_longer(). You’ll reach for it less often, but it’s essential for creating cross-tabulations and reporting tables.

Splitting and combining columns

separate_wider_delim()

Sometimes a single column contains multiple values separated by a delimiter. Let’s manufacture a quick example to see separate_wider_delim() in action.

Code 
messy_locations <- tibble(
  id = 1:4,
  location = c("USA-New York", "CAN-Toronto", "GBR-London", "AUS-Sydney")
)

messy_locations |>
  separate_wider_delim(
    location,
    delim = "-",
    names = c("country_code", "city")
  )
# A tibble: 4 × 3
     id country_code city    
  <int> <chr>        <chr>   
1     1 USA          New York
2     2 CAN          Toronto 
3     3 GBR          London  
4     4 AUS          Sydney

This cleanly splits one column into two. If the number of pieces isn’t consistent across rows, use too_few and too_many to control the behavior:

Code 
tricky <- tibble(
  id = 1:3,
  value = c("A-B-C", "D-E", "F-G-H-I")
)

tricky |>
  separate_wider_delim(
    value,
    delim = "-",
    names = c("first", "second", "third"),
    too_few = "align_start",
    too_many = "merge"
  )
# A tibble: 3 × 4
     id first second third
  <int> <chr> <chr>  <chr>
1     1 A     B      C    
2     2 D     E      <NA> 
3     3 F     G      H-I

The too_few = "align_start" fills missing pieces with NA from the right. The too_many = "merge" lumps extra pieces into the last column. This keeps your pipeline from crashing on messy, inconsistent data.

unite()

unite() is the reverse — gluing columns together.

Code 
who_tidy |>
  unite("demographic", sex, age_group, sep = "_") |>
  select(country, year, diagnosis, demographic, count) |>
  head()
# A tibble: 6 × 5
  country      year diagnosis demographic count
  <chr>       <dbl> <chr>     <chr>       <dbl>
1 Afghanistan  1997 sp        m_014           0
2 Afghanistan  1997 sp        m_1524         10
3 Afghanistan  1997 sp        m_2534          6
4 Afghanistan  1997 sp        m_3544          3
5 Afghanistan  1997 sp        m_4554          5
6 Afghanistan  1997 sp        m_5564          2

This is handy when you need to create an interaction label for plotting or joining.

String manipulation with stringr

Real-world data is full of inconsistent text. The stringr package gives you a consistent set of functions (all starting with str_) for detecting, extracting, and replacing patterns.

str_detect() — finding patterns

str_detect() returns TRUE or FALSE, making it perfect inside filter().

Code 
# Which Star Wars characters have "Skywalker" in their name?
starwars |>
  filter(str_detect(name, "Skywalker"))
# A tibble: 3 × 14
  name      height  mass hair_color skin_color eye_color birth_year sex   gender
  <chr>      <int> <dbl> <chr>      <chr>      <chr>          <dbl> <chr> <chr> 
1 Luke Sky…    172    77 blond      fair       blue            19   male  mascu…
2 Anakin S…    188    84 blond      fair       blue            41.9 male  mascu…
3 Shmi Sky…    163    NA black      fair       brown           72   fema… femin…
# ℹ 5 more variables: homeworld <chr>, species <chr>, films <list>,
#   vehicles <list>, starships <list>
Code 
# Find characters with multiple skin colors (contain a comma)
starwars |>
  filter(str_detect(skin_color, ",")) |>
  select(name, skin_color)
# A tibble: 14 × 2
   name                  skin_color         
   <chr>                 <chr>              
 1 R2-D2                 white, blue        
 2 R5-D4                 white, red         
 3 Jabba Desilijic Tiure green-tan, brown   
 4 Watto                 blue, grey         
 5 Sebulba               grey, red          
 6 Ratts Tyerel          grey, blue         
 7 Dud Bolt              blue, grey         
 8 Gasgano               white, blue        
 9 Ben Quadinaros        grey, green, yellow
10 Zam Wesell            fair, green, yellow
11 R4-P17                silver, red        
12 Wat Tambor            green, grey        
13 Shaak Ti              red, blue, white   
14 Grievous              brown, white

str_replace() and str_replace_all()

str_replace() swaps the first match; str_replace_all() swaps every match. This is essential for cleaning up inconsistent labels.

Code 
starwars |>
  mutate(skin_color = str_replace_all(skin_color, ", ", "/")) |>
  filter(str_detect(skin_color, "/")) |>
  select(name, skin_color)
# A tibble: 14 × 2
   name                  skin_color       
   <chr>                 <chr>            
 1 R2-D2                 white/blue       
 2 R5-D4                 white/red        
 3 Jabba Desilijic Tiure green-tan/brown  
 4 Watto                 blue/grey        
 5 Sebulba               grey/red         
 6 Ratts Tyerel          grey/blue        
 7 Dud Bolt              blue/grey        
 8 Gasgano               white/blue       
 9 Ben Quadinaros        grey/green/yellow
10 Zam Wesell            fair/green/yellow
11 R4-P17                silver/red       
12 Wat Tambor            green/grey       
13 Shaak Ti              red/blue/white   
14 Grievous              brown/white

A common use case: standardizing messy category labels.

Code 
raw_labels <- c("United States", "united states", "US", "U.S.", "usa")

raw_labels |>
  str_to_lower() |>
  str_replace_all(c(
    "^us$"      = "united states",
    "^u\\.s\\.$" = "united states",
    "^usa$"      = "united states"
  ))
[1] "united states" "united states" "united states" "united states"
[5] "united states"

str_extract() — pulling out pieces

str_extract() grabs the first matching portion of a string. Let’s pull the numeric age boundaries from our cleaned WHO data.

Code 
who_tidy |>
  mutate(
    age_start = str_extract(age_group, "^\\d+") |> as.integer()
  ) |>
  distinct(age_group, age_start) |>
  arrange(age_start)
# A tibble: 7 × 2
  age_group age_start
  <chr>         <int>
1 014              14
2 65               65
3 1524           1524
4 2534           2534
5 3544           3544
6 4554           4554
7 5564           5564

str_trim() and str_squish()

Whitespace is the silent killer of joins and group-bys. Two rows that look identical can fail to match because one has a trailing space.

Code 
messy_names <- c("  Alice ", "Bob", " Charlie  ", "  Alice")

# str_trim removes leading/trailing whitespace
str_trim(messy_names)
[1] "Alice"   "Bob"     "Charlie" "Alice"
Code 
# str_squish also collapses internal whitespace
str_squish("  too   many   spaces  ")
[1] "too many spaces"

Always trim your strings before joining or grouping. This one habit will save you hours of debugging.

Common gotchas

Factor explosions

When you read a CSV, character columns sometimes get read as factors. This means levels() bakes in the exact set of unique values. If you then filter() down to a subset, the unused levels stick around as ghosts — inflating your legend in plots, adding empty groups in summaries, and generally causing confusion.

Code 
# Simulate the problem
species_factor <- factor(c("cat", "dog", "bird", "cat", "dog"))
filtered <- species_factor[species_factor != "bird"]

# "bird" is gone from the data but still in the levels
levels(filtered)
[1] "bird" "cat"  "dog"
Code 
# Fix: drop unused levels
levels(droplevels(filtered))
[1] "cat" "dog"

The lesson: use droplevels() after filtering factor data, or better yet, keep text as character columns with stringsAsFactors = FALSE (the default since R 4.0) and convert to factors only when you need explicit ordering.

NA handling

Missing values propagate silently. Any arithmetic with NA returns NA. Any comparison with NA returns NA. This means filter(x == NA) never returns rows — use is.na() instead.

Code 
# This returns nothing — NA == NA is NA, not TRUE
starwars |>
  filter(mass == NA) |>
  nrow()
[1] 0
Code 
# This is what you actually want
starwars |>
  filter(is.na(mass)) |>
  select(name, mass) |>
  head()
# A tibble: 6 × 2
  name            mass
  <chr>          <dbl>
1 Wilhuff Tarkin    NA
2 Mon Mothma        NA
3 Arvel Crynyd      NA
4 Finis Valorum     NA
5 Rugor Nass        NA
6 Ric Olié          NA

For summaries, always pass na.rm = TRUE.

Code 
# Without na.rm — returns NA
starwars |>
  summarize(avg_height = mean(height))
# A tibble: 1 × 1
  avg_height
       <dbl>
1         NA
Code 
# With na.rm — returns the actual mean
starwars |>
  summarize(avg_height = mean(height, na.rm = TRUE))
# A tibble: 1 × 1
  avg_height
       <dbl>
1       175.

If you want to replace NAs with a default value, use replace_na() from tidyr.

Code 
starwars |>
  mutate(hair_color = replace_na(hair_color, "unknown")) |>
  count(hair_color, sort = TRUE) |>
  head()
# A tibble: 6 × 2
  hair_color     n
  <chr>      <int>
1 none          38
2 brown         18
3 black         13
4 unknown        5
5 white          4
6 blond          3

Duplicate rows

Before any analysis, always check for duplicates. distinct() keeps unique rows, and get_dupes() from the janitor package can identify which rows are repeated. Here’s the tidyverse approach:

Code 
who_tidy |>
  group_by(country, year, diagnosis, sex, age_group) |>
  filter(n() > 1) |>
  nrow()
[1] 0

Zero duplicates — good. When you do find them, decide whether to keep the first, last, or aggregate.

Putting it all together

Let’s chain everything into a real pipeline. Starting from the raw who2 data, we’ll clean, reshape, and summarize in one shot.

Code 
who2 |>
  # Reshape: wide to long, splitting column names
  pivot_longer(
    cols = !c(country, year),
    names_to = c("diagnosis", "sex", "age_group"),
    names_sep = "_",
    values_to = "count",
    values_drop_na = TRUE
  ) |>
  # Clean: standardize sex labels
  mutate(
    sex = str_replace_all(sex, c("m" = "male", "f" = "female")),
    age_start = str_extract(age_group, "^\\d+") |> as.integer()
  ) |>
  # Filter: focus on recent data
  filter(year >= 2010) |>
  # Summarize: total cases by country and sex
  group_by(country, sex) |>
  summarize(total_cases = sum(count), .groups = "drop") |>
  # Reshape: make a comparison table
  pivot_wider(names_from = sex, values_from = total_cases) |>
  # Sort: highest total burden first
  mutate(total = male + female) |>
  arrange(desc(total)) |>
  head(10)
# A tibble: 10 × 4
   country                                female    male   total
   <chr>                                   <dbl>   <dbl>   <dbl>
 1 China                                 1059090 2381098 3440188
 2 India                                  585702 1316362 1902064
 3 South Africa                           593847  669632 1263479
 4 Indonesia                              530997  727340 1258337
 5 Bangladesh                             243335  399146  642481
 6 Pakistan                               206403  206493  412896
 7 Russian Federation                     121867  273245  395112
 8 Philippines                            111062  262920  373982
 9 Democratic People's Republic of Korea  130943  211636  342579
10 Kenya                                  137336  188480  325816

That’s eight operations piped together, and every step reads like a sentence. This kind of pipeline is what your daily R work will actually look like.

Quick reference

Function Package What it does
pivot_longer() tidyr Reshape wide data to long format
pivot_wider() tidyr Reshape long data to wide format
separate_wider_delim() tidyr Split one column into many by delimiter
unite() tidyr Combine multiple columns into one
replace_na() tidyr Replace NA with a specified value
str_detect() stringr Test if a pattern exists in a string
str_replace() stringr Replace first match of a pattern
str_replace_all() stringr Replace all matches of a pattern
str_extract() stringr Pull out the first match of a pattern
str_trim() stringr Remove leading/trailing whitespace
str_squish() stringr Trim + collapse internal whitespace

This is the unglamorous core of data analysis — the cleaning that happens before any chart or model. Master these tools and you’ll spend less time fighting your data and more time learning from it.

]]> code tidyverse https://noahweidig.github.io/chartifyr/posts/data-cleaning-tidyr-stringr/ Sat, 22 Feb 2025 00:00:00 GMT Basics of dplyr Noah Weidig https://noahweidig.github.io/chartifyr/posts/dplyr-basics/

Artwork by @allison_horst

Artwork by @allison_horst

If you’ve ever worked with messy data in R, you know how painful base R subsetting can be. The dplyr package makes data wrangling intuitive and readable. In this tutorial, we’ll walk through the six core dplyr verbs using the Palmer Penguins dataset.

Setup

Let’s load the packages we need.

Code 
# Uncomment and run these once
#install.packages("tidyverse")
#install.packages("palmerpenguins")

library(tidyverse)
library(palmerpenguins)

Let’s take a look at the data.

Code 
penguins
# A tibble: 344 × 8
   species island    bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
   <fct>   <fct>              <dbl>         <dbl>             <int>       <int>
 1 Adelie  Torgersen           39.1          18.7               181        3750
 2 Adelie  Torgersen           39.5          17.4               186        3800
 3 Adelie  Torgersen           40.3          18                 195        3250
 4 Adelie  Torgersen           NA            NA                  NA          NA
 5 Adelie  Torgersen           36.7          19.3               193        3450
 6 Adelie  Torgersen           39.3          20.6               190        3650
 7 Adelie  Torgersen           38.9          17.8               181        3625
 8 Adelie  Torgersen           39.2          19.6               195        4675
 9 Adelie  Torgersen           34.1          18.1               193        3475
10 Adelie  Torgersen           42            20.2               190        4250
# ℹ 334 more rows
# ℹ 2 more variables: sex <fct>, year <int>

We have 344 penguins with 8 variables: species, island, bill measurements, flipper length, body mass, sex, and year.

The Pipe Operator

Before we dive into the verbs, let’s talk about the pipe: |> (or %>%). The pipe takes the output of one function and passes it as the first argument to the next. This lets us chain operations together in a readable way.

Code 
# Without the pipe
nrow(penguins)
[1] 344
Code 
# With the pipe — same result
penguins |> nrow()
[1] 344

You’ll see the pipe used throughout this tutorial. Think of it as saying “and then…”

filter()

filter() keeps rows that match a condition. Let’s grab only the Adelie penguins.

Code 
penguins |>
  filter(species == "Adelie")
# A tibble: 152 × 8
   species island    bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
   <fct>   <fct>              <dbl>         <dbl>             <int>       <int>
 1 Adelie  Torgersen           39.1          18.7               181        3750
 2 Adelie  Torgersen           39.5          17.4               186        3800
 3 Adelie  Torgersen           40.3          18                 195        3250
 4 Adelie  Torgersen           NA            NA                  NA          NA
 5 Adelie  Torgersen           36.7          19.3               193        3450
 6 Adelie  Torgersen           39.3          20.6               190        3650
 7 Adelie  Torgersen           38.9          17.8               181        3625
 8 Adelie  Torgersen           39.2          19.6               195        4675
 9 Adelie  Torgersen           34.1          18.1               193        3475
10 Adelie  Torgersen           42            20.2               190        4250
# ℹ 142 more rows
# ℹ 2 more variables: sex <fct>, year <int>

You can combine multiple conditions. Let’s find Adelie penguins that weigh more than 4000 grams.

Code 
penguins |>
  filter(species == "Adelie", body_mass_g > 4000)
# A tibble: 35 × 8
   species island    bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
   <fct>   <fct>              <dbl>         <dbl>             <int>       <int>
 1 Adelie  Torgersen           39.2          19.6               195        4675
 2 Adelie  Torgersen           42            20.2               190        4250
 3 Adelie  Torgersen           34.6          21.1               198        4400
 4 Adelie  Torgersen           42.5          20.7               197        4500
 5 Adelie  Torgersen           46            21.5               194        4200
 6 Adelie  Dream               39.2          21.1               196        4150
 7 Adelie  Dream               39.8          19.1               184        4650
 8 Adelie  Dream               44.1          19.7               196        4400
 9 Adelie  Dream               39.6          18.8               190        4600
10 Adelie  Dream               42.3          21.2               191        4150
# ℹ 25 more rows
# ℹ 2 more variables: sex <fct>, year <int>

Use | (or) for either/or conditions.

Code 
penguins |>
  filter(island == "Biscoe" | island == "Dream")
# A tibble: 292 × 8
   species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
   <fct>   <fct>           <dbl>         <dbl>             <int>       <int>
 1 Adelie  Biscoe           37.8          18.3               174        3400
 2 Adelie  Biscoe           37.7          18.7               180        3600
 3 Adelie  Biscoe           35.9          19.2               189        3800
 4 Adelie  Biscoe           38.2          18.1               185        3950
 5 Adelie  Biscoe           38.8          17.2               180        3800
 6 Adelie  Biscoe           35.3          18.9               187        3800
 7 Adelie  Biscoe           40.6          18.6               183        3550
 8 Adelie  Biscoe           40.5          17.9               187        3200
 9 Adelie  Biscoe           37.9          18.6               172        3150
10 Adelie  Biscoe           40.5          18.9               180        3950
# ℹ 282 more rows
# ℹ 2 more variables: sex <fct>, year <int>

A cleaner way to filter for multiple values is %in%.

Code 
penguins |>
  filter(island %in% c("Biscoe", "Dream"))
# A tibble: 292 × 8
   species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
   <fct>   <fct>           <dbl>         <dbl>             <int>       <int>
 1 Adelie  Biscoe           37.8          18.3               174        3400
 2 Adelie  Biscoe           37.7          18.7               180        3600
 3 Adelie  Biscoe           35.9          19.2               189        3800
 4 Adelie  Biscoe           38.2          18.1               185        3950
 5 Adelie  Biscoe           38.8          17.2               180        3800
 6 Adelie  Biscoe           35.3          18.9               187        3800
 7 Adelie  Biscoe           40.6          18.6               183        3550
 8 Adelie  Biscoe           40.5          17.9               187        3200
 9 Adelie  Biscoe           37.9          18.6               172        3150
10 Adelie  Biscoe           40.5          18.9               180        3950
# ℹ 282 more rows
# ℹ 2 more variables: sex <fct>, year <int>

select()

select() picks specific columns. This is useful when you only need a few variables from a wide dataset.

Code 
penguins |>
  select(species, island, body_mass_g)
# A tibble: 344 × 3
   species island    body_mass_g
   <fct>   <fct>           <int>
 1 Adelie  Torgersen        3750
 2 Adelie  Torgersen        3800
 3 Adelie  Torgersen        3250
 4 Adelie  Torgersen          NA
 5 Adelie  Torgersen        3450
 6 Adelie  Torgersen        3650
 7 Adelie  Torgersen        3625
 8 Adelie  Torgersen        4675
 9 Adelie  Torgersen        3475
10 Adelie  Torgersen        4250
# ℹ 334 more rows

You can also remove columns with a minus sign.

Code 
penguins |>
  select(-year)
# A tibble: 344 × 7
   species island    bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
   <fct>   <fct>              <dbl>         <dbl>             <int>       <int>
 1 Adelie  Torgersen           39.1          18.7               181        3750
 2 Adelie  Torgersen           39.5          17.4               186        3800
 3 Adelie  Torgersen           40.3          18                 195        3250
 4 Adelie  Torgersen           NA            NA                  NA          NA
 5 Adelie  Torgersen           36.7          19.3               193        3450
 6 Adelie  Torgersen           39.3          20.6               190        3650
 7 Adelie  Torgersen           38.9          17.8               181        3625
 8 Adelie  Torgersen           39.2          19.6               195        4675
 9 Adelie  Torgersen           34.1          18.1               193        3475
10 Adelie  Torgersen           42            20.2               190        4250
# ℹ 334 more rows
# ℹ 1 more variable: sex <fct>

There are handy helper functions too. starts_with() grabs columns that start with a string.

Code 
penguins |>
  select(species, starts_with("bill"))
# A tibble: 344 × 3
   species bill_length_mm bill_depth_mm
   <fct>            <dbl>         <dbl>
 1 Adelie            39.1          18.7
 2 Adelie            39.5          17.4
 3 Adelie            40.3          18  
 4 Adelie            NA            NA  
 5 Adelie            36.7          19.3
 6 Adelie            39.3          20.6
 7 Adelie            38.9          17.8
 8 Adelie            39.2          19.6
 9 Adelie            34.1          18.1
10 Adelie            42            20.2
# ℹ 334 more rows

mutate()

mutate() creates new columns or modifies existing ones. Let’s convert body mass from grams to kilograms.

Code 
penguins |>
  mutate(body_mass_kg = body_mass_g / 1000) |>
  select(species, body_mass_g, body_mass_kg)
# A tibble: 344 × 3
   species body_mass_g body_mass_kg
   <fct>         <int>        <dbl>
 1 Adelie         3750         3.75
 2 Adelie         3800         3.8 
 3 Adelie         3250         3.25
 4 Adelie           NA        NA   
 5 Adelie         3450         3.45
 6 Adelie         3650         3.65
 7 Adelie         3625         3.62
 8 Adelie         4675         4.68
 9 Adelie         3475         3.48
10 Adelie         4250         4.25
# ℹ 334 more rows

You can create multiple columns at once.

Code 
penguins |>
  mutate(
    body_mass_kg = body_mass_g / 1000,
    bill_ratio = bill_length_mm / bill_depth_mm
  ) |>
  select(species, body_mass_kg, bill_ratio)
# A tibble: 344 × 3
   species body_mass_kg bill_ratio
   <fct>          <dbl>      <dbl>
 1 Adelie          3.75       2.09
 2 Adelie          3.8        2.27
 3 Adelie          3.25       2.24
 4 Adelie         NA         NA   
 5 Adelie          3.45       1.90
 6 Adelie          3.65       1.91
 7 Adelie          3.62       2.19
 8 Adelie          4.68       2   
 9 Adelie          3.48       1.88
10 Adelie          4.25       2.08
# ℹ 334 more rows

arrange()

arrange() sorts rows. By default, it sorts in ascending order.

Code 
penguins |>
  arrange(body_mass_g)
# A tibble: 344 × 8
   species   island   bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
   <fct>     <fct>             <dbl>         <dbl>             <int>       <int>
 1 Chinstrap Dream              46.9          16.6               192        2700
 2 Adelie    Biscoe             36.5          16.6               181        2850
 3 Adelie    Biscoe             36.4          17.1               184        2850
 4 Adelie    Biscoe             34.5          18.1               187        2900
 5 Adelie    Dream              33.1          16.1               178        2900
 6 Adelie    Torgers…           38.6          17                 188        2900
 7 Chinstrap Dream              43.2          16.6               187        2900
 8 Adelie    Biscoe             37.9          18.6               193        2925
 9 Adelie    Dream              37.5          18.9               179        2975
10 Adelie    Dream              37            16.9               185        3000
# ℹ 334 more rows
# ℹ 2 more variables: sex <fct>, year <int>

Use desc() for descending order.

Code 
penguins |>
  arrange(desc(body_mass_g))
# A tibble: 344 × 8
   species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
   <fct>   <fct>           <dbl>         <dbl>             <int>       <int>
 1 Gentoo  Biscoe           49.2          15.2               221        6300
 2 Gentoo  Biscoe           59.6          17                 230        6050
 3 Gentoo  Biscoe           51.1          16.3               220        6000
 4 Gentoo  Biscoe           48.8          16.2               222        6000
 5 Gentoo  Biscoe           45.2          16.4               223        5950
 6 Gentoo  Biscoe           49.8          15.9               229        5950
 7 Gentoo  Biscoe           48.4          14.6               213        5850
 8 Gentoo  Biscoe           49.3          15.7               217        5850
 9 Gentoo  Biscoe           55.1          16                 230        5850
10 Gentoo  Biscoe           49.5          16.2               229        5800
# ℹ 334 more rows
# ℹ 2 more variables: sex <fct>, year <int>

You can sort by multiple columns. This sorts by species first, then by body mass within each species.

Code 
penguins |>
  arrange(species, desc(body_mass_g))
# A tibble: 344 × 8
   species island    bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
   <fct>   <fct>              <dbl>         <dbl>             <int>       <int>
 1 Adelie  Biscoe              43.2          19                 197        4775
 2 Adelie  Biscoe              41            20                 203        4725
 3 Adelie  Torgersen           42.9          17.6               196        4700
 4 Adelie  Torgersen           39.2          19.6               195        4675
 5 Adelie  Dream               39.8          19.1               184        4650
 6 Adelie  Dream               39.6          18.8               190        4600
 7 Adelie  Biscoe              45.6          20.3               191        4600
 8 Adelie  Torgersen           42.5          20.7               197        4500
 9 Adelie  Dream               37.5          18.5               199        4475
10 Adelie  Torgersen           41.8          19.4               198        4450
# ℹ 334 more rows
# ℹ 2 more variables: sex <fct>, year <int>

summarize()

summarize() (or summarise()) collapses the data into a summary. Let’s find the average body mass.

Code 
penguins |>
  summarize(mean_mass = mean(body_mass_g, na.rm = TRUE))
# A tibble: 1 × 1
  mean_mass
      <dbl>
1     4202.

The na.rm = TRUE argument tells R to ignore missing values. You can compute multiple summaries at once.

Code 
penguins |>
  summarize(
    mean_mass = mean(body_mass_g, na.rm = TRUE),
    sd_mass = sd(body_mass_g, na.rm = TRUE),
    n = n()
  )
# A tibble: 1 × 3
  mean_mass sd_mass     n
      <dbl>   <dbl> <int>
1     4202.    802.   344

group_by()

group_by() is where dplyr really shines. It splits the data into groups so that subsequent operations are performed per group. Pair it with summarize() for powerful grouped summaries.

Code 
penguins |>
  group_by(species) |>
  summarize(
    mean_mass = mean(body_mass_g, na.rm = TRUE),
    sd_mass = sd(body_mass_g, na.rm = TRUE),
    n = n()
  )
# A tibble: 3 × 4
  species   mean_mass sd_mass     n
  <fct>         <dbl>   <dbl> <int>
1 Adelie        3701.    459.   152
2 Chinstrap     3733.    384.    68
3 Gentoo        5076.    504.   124

You can group by multiple variables.

Code 
penguins |>
  group_by(species, island) |>
  summarize(
    mean_mass = mean(body_mass_g, na.rm = TRUE),
    n = n(),
    .groups = "drop"
  )
# A tibble: 5 × 4
  species   island    mean_mass     n
  <fct>     <fct>         <dbl> <int>
1 Adelie    Biscoe        3710.    44
2 Adelie    Dream         3688.    56
3 Adelie    Torgersen     3706.    52
4 Chinstrap Dream         3733.    68
5 Gentoo    Biscoe        5076.   124

The .groups = "drop" argument ungroups the data after summarizing, which is good practice.

Putting It All Together

The real power of dplyr is chaining verbs together. Let’s find the average flipper length for each species, but only for female penguins weighing over 3500 grams, sorted from longest to shortest.

Code 
penguins |>
  filter(sex == "female", body_mass_g > 3500) |>
  group_by(species) |>
  summarize(
    mean_flipper = mean(flipper_length_mm, na.rm = TRUE),
    n = n()
  ) |>
  arrange(desc(mean_flipper))
# A tibble: 3 × 3
  species   mean_flipper     n
  <fct>            <dbl> <int>
1 Gentoo            213.    58
2 Chinstrap         192.    19
3 Adelie            190.    22

Each step is readable on its own, and the pipe makes the full pipeline easy to follow.

Quick Reference

Verb What it does
filter() Keep rows that match a condition
select() Pick or remove columns
mutate() Create or modify columns
arrange() Sort rows
summarize() Collapse data into summaries
group_by() Group data for per-group operations

And that’s it! These six verbs will cover the vast majority of your data wrangling needs. Thanks for reading — keep an eye out for more R tutorials!

]]> code dplyr tidyverse https://noahweidig.github.io/chartifyr/posts/dplyr-basics/ Sat, 15 Feb 2025 00:00:00 GMT Mapping with ggplot2 and sf Noah Weidig https://noahweidig.github.io/chartifyr/posts/maps-ggplot-sf/

Artwork by @allison_horst

Artwork by @allison_horst

Have you ever made a map in ArcGIS, only to later realize that you made a mistake with your data and will have to remake the entire thing? That’s where the sf package in R comes in!

Let’s load the sf and tidyverse packages.

Code 
# Uncomment and run this only once
#install.packages("sf")
library(sf)
library(tidyverse)

Now, we’ll need to load our data. We’ll use the North Carolina counties data built into the sf package.

Code 
nc <- st_read(system.file("shape/nc.shp", package = "sf"))
Reading layer `nc' from data source 
  `/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/sf/shape/nc.shp' 
  using driver `ESRI Shapefile'
Simple feature collection with 100 features and 14 fields
Geometry type: MULTIPOLYGON
Dimension:     XY
Bounding box:  xmin: -84.32385 ymin: 33.88199 xmax: -75.45698 ymax: 36.58965
Geodetic CRS:  NAD27

Every sf object has a “geometry” column, which stores the coordinates of the point, line, or polygon. Now, let’s plot it. The ggplot2 package has a geom_sf option for simple features.

Code 
ggplot() + geom_sf(data = nc)

Code 
# if that does not work, you may have to specify the geometry in the aesthetic
ggplot() + geom_sf(data = nc, aes(geometry = geometry))

Not bad. I prefer to not see the latitude/longitude along the axes, so I frequently use theme_void().

Code 
ggplot() + geom_sf(data = nc) + theme_void()

Great! This dataset has several attributes related to births and infant deaths. We’ll use the SID79, which is the number of sudden infant deaths in the year 1979 for each county.

Code 
ggplot() + geom_sf(data = nc, aes(fill = SID79)) + theme_void()

The ggplot default colors are not my favorite. I prefer using brewer colors. The direction argument reverses the direction of color darkening so that darker indicates a higher value.

Code 
ggplot() + geom_sf(data = nc, aes(fill = SID79)) + theme_void() +
  scale_fill_distiller(palette = "YlOrBr", direction = 1)

We can see quite clearly that a few counties in central North Carolina had a large number of sudden infant deaths in 1979. To clean up this map, let’s add a scale bar and north arrow (using the ggspatial package), and refine the labels.

Code 
library(ggspatial)

ggplot() +
  geom_sf(data = nc, aes(fill = SID79)) +
  theme_void() +
  scale_fill_distiller(palette = "YlOrBr", direction = 1) +
  annotation_scale(location = "bl", line_width = 1) +
  annotation_north_arrow(style = north_arrow_fancy_orienteering(),
                         location = "tl", 
                         height = unit(0.8, "cm"),
                         width = unit(0.8, "cm")) +
  labs(fill = "SIDS 1979")

Much better! The scale bar and north arrow size can be adjusted as needed. This mostly depends on how large you save the figure. These ggplot objects can be saved the same way as a regular plot.

Code 
ggsave("nc.jpg", width = 5, height = 3, dpi = 300)

And you’re done! You’ve created your first official map in R!

Thanks for joining me. Keep an eye out for more tutorials in R!

]]> code ggplot2 dataviz sf https://noahweidig.github.io/chartifyr/posts/maps-ggplot-sf/ Fri, 14 Feb 2025 00:00:00 GMT Basics of ggplot2 Noah Weidig https://noahweidig.github.io/chartifyr/posts/ggplot-basics/

Artwork by @allison_horst

Artwork by @allison_horst

Nothing is worse than reading a scientific publication only to encounter muddled, overly busy figures. Clear storytelling is a cornerstone of scientific writing, and the saying “a picture is worth a thousand words” has never been more relevant. Let’s dive into creating clean, effective figures in R!

Import Data

First, we need to make sure that tidyverse is installed. ggplot is a package within tidyverse, which is one of the most useful tools in R for data wrangling and visualization.

Code 
# Uncomment and run this line once
#install.packages("tidyverse")

# load the package
library(tidyverse)

Next, lets read in our data. We’ll be working with the palmer penguins dataset, which is free and publicly available.

Code 
# Uncomment and run this line once
#install.packages("palmerpenguins")
library(palmerpenguins)

data <- penguins

First, let’s look at the data in a tabular form.

Code 
print(data)
# A tibble: 344 × 8
   species island    bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
   <fct>   <fct>              <dbl>         <dbl>             <int>       <int>
 1 Adelie  Torgersen           39.1          18.7               181        3750
 2 Adelie  Torgersen           39.5          17.4               186        3800
 3 Adelie  Torgersen           40.3          18                 195        3250
 4 Adelie  Torgersen           NA            NA                  NA          NA
 5 Adelie  Torgersen           36.7          19.3               193        3450
 6 Adelie  Torgersen           39.3          20.6               190        3650
 7 Adelie  Torgersen           38.9          17.8               181        3625
 8 Adelie  Torgersen           39.2          19.6               195        4675
 9 Adelie  Torgersen           34.1          18.1               193        3475
10 Adelie  Torgersen           42            20.2               190        4250
# ℹ 334 more rows
# ℹ 2 more variables: sex <fct>, year <int>

So there are eight columns and 344 rows. Let’s say we are interested in penguin weight. We’ll need to look at the “body_mass_g” variable.

Basic Syntax

Let’s create a basic histogram to look at the distribution of this continuous variable.

To start, you call ggplot(). The first argument is our data.

I could also code it with data = data, although the first argument with ggplot is always the “data =” argument.

Next we have to specify the aesthetics. Since a histogram only has an x argument (our continuous variable: body_mass_g, in our case), that’s all we need.

Code 
ggplot(data = data, aes(x = body_mass_g))

Code 
# or
ggplot(data, aes(body_mass_g))

However, this is just a blank graph. We need to specify which type of graph. In this example, we will create a histogram. Each additional “geom” is added by a “+” sign. I generally like to start a new line after each “+”.

Code 
ggplot(data, aes(x = body_mass_g)) +
  geom_histogram()

Not bad! Now let’s customize it. I personally do not like the default ggplot look, so let’s try some built in themes.

Code 
ggplot(data, aes(x = body_mass_g)) +
  geom_histogram() +
  theme_bw()

Code 
ggplot(data, aes(x = body_mass_g)) +
  geom_histogram() +
  theme_minimal()

Code 
ggplot(data, aes(x = body_mass_g)) +
  geom_histogram() +
  theme_classic()

Code 
ggplot(data, aes(x = body_mass_g)) +
  geom_histogram() +
  theme_linedraw()

I prefer theme_bw() or theme_minimal(). Let’s stick with those going forward.

Get Colorful

Maybe you want to change the fill color of the histogram bars. Colors in R are always specified within quotes.

Code 
ggplot(data, aes(x = body_mass_g)) +
  geom_histogram(fill = "red") +
  theme_minimal()

Maybe you want you want each species to have a different color. We’ll need to specify which variable is our fill within the aes().

Code 
ggplot(data, aes(x = body_mass_g, fill = species)) +
  geom_histogram() +
  theme_minimal()

Great! But these are stacked bars. We want them to overlap.

Code 
ggplot(data, aes(x = body_mass_g, fill = species)) +
  geom_histogram(position = "identity") +
  theme_minimal()

But we can’t quite see some of the bars that overlap. Let’s change the transparency (alpha).

Code 
ggplot(data, aes(x = body_mass_g, fill = species)) +
  geom_histogram(position = "identity", alpha = 0.7) +
  theme_minimal()

Much better!

Labeling Your Graph

The x and y-axes aren’t very pretty right now. Let’s change that. We’ll use labs().

Code 
ggplot(data, aes(x = body_mass_g, fill = species)) +
  geom_histogram(position = "identity",
                              alpha = 0.7) +
  theme_minimal() +
  labs(x = "Body Mass (g)",
          y = "Frequency",
          fill = "Species")

Other geoms

Since this is a basic introduction to ggplot, we will not go into all the other geoms. However, there are several other types of graphs you can make:

  • geom_bar: bar graphs

  • geom_point: scatterplots

  • geom_line: linegraphs

  • geom_text: adding text to your figure

  • And so many more!

Exporting Figures

Now let’s save our figure. Most journals require between 300-600 dots per inch (dpi).

Code 
ggsave("penguins_histogram.jpg", width = 5, height = 3, dpi = 300)

And that’s it! Thanks for reading! Look out for more R tutorials.

]]> code ggplot2 dataviz https://noahweidig.github.io/chartifyr/posts/ggplot-basics/ Thu, 13 Feb 2025 00:00:00 GMT Welcome To ChartifyR Noah Weidig https://noahweidig.github.io/chartifyr/posts/welcome/ I’m excited to share my knowledge of data science and visualization using R, specifically the tidyverse suite of packages, as well as other languages and tools.

Artwork by @allison_horst

Artwork by @allison_horst
]]> news https://noahweidig.github.io/chartifyr/posts/welcome/ Mon, 10 Feb 2025 00:00:00 GMT