Read data from URL in R

Problem

How can we read data from an URL in R? We have a URL pointing to a text file, and we'd like to read it directly in R without previously downloading it to our computer.

http://www-bcf.usc.edu/~gareth/ISL/Advertising.csv

Solution

We just need to pass the URL string inside the appropriate function. Apart from read.csv from the base package, I include a couple of examples from two popular packages: data.table y readr.

• Base

ad <- read.csv("http://www-bcf.usc.edu/~gareth/ISL/Advertising.csv")
head(ad)

  X    TV radio newspaper sales
1 1 230.1  37.8      69.2  22.1
2 2  44.5  39.3      45.1  10.4
3 3  17.2  45.9      69.3   9.3
4 4 151.5  41.3      58.5  18.5
5 5 180.8  10.8      58.4  12.9
6 6   8.7  48.9      75.0   7.2

• data.table

library(data.table)
ad <- fread("http://www-bcf.usc.edu/~gareth/ISL/Advertising.csv")
head(ad)

V1    TV radio newspaper sales
1:  1 230.1  37.8      69.2  22.1
2:  2  44.5  39.3      45.1  10.4
3:  3  17.2  45.9      69.3   9.3
4:  4 151.5  41.3      58.5  18.5
5:  5 180.8  10.8      58.4  12.9
6:  6   8.7  48.9      75.0   7.2

• readr

library(readr)
ad <- read_csv("http://www-bcf.usc.edu/~gareth/ISL/Advertising.csv")
head(ad)

# A tibble: 6 x 5
     X1    TV radio newspaper sales
          
1     1 230.1  37.8      69.2  22.1
2     2  44.5  39.3      45.1  10.4
3     3  17.2  45.9      69.3   9.3
4     4 151.5  41.3      58.5  18.5
5     5 180.8  10.8      58.4  12.9
6     6   8.7  48.9      75.0   7.2

Related posts

• Spanish version

References

• Importar datos en R: ficheros de texto
• stackoverflow

Drop unused levels from a factor in R

Problem

If we filter a data frame containing a factor and then perform any operation, such as creating a contingency table, R will still show the unused levels. Subsetting does not in general drop unused levels.

df <- data.frame(name = c("a", "a", "a", "b", "b", "c", "c", "c", "c"), x = 1:9)
library(dplyr)
aa <-  df %>%
  group_by(name) %>%
  filter(n() < 4) %>% 
  droplevels()
table(aa$name)

In our example, the level c is still included in the results. We'd like to remove it and display only the used levels a and b.

# Resultado
a b c 
3 2 0
# Resultado deseado
a b 
3 2

Solution

There are two alternatives, the function droplevels or factor.

table(droplevels(aa$name))
table(factor(aa$name))

If we are using dplyr and the pipe operator:

aa <-  df %>%
  group_by(name) %>%
  filter(n() < 4) %>% 
  droplevels()
table(aa$name)

# Better still
df %>%
  group_by(name) %>%
  filter(n() < 4) %>% 
  droplevels() %>% 
  {table(.$name)}

Related posts

Spanish version

References

• stackoverflow
• droplevels
• factor

Plot median and quartiles in ggplot2 using geom_pointrange

Problem

We would like to plot the interquartile range (IQR) and the median in ggplot2.

Solution

In this case we use the function geom_pointrange. We need to specify the interval (interquartile range) with the arguments fun.ymin and fun.ymax, and fun.y to plot the median.

library(ggplot2)
ggplot(data = diamonds) +
  geom_pointrange(mapping = aes(x = cut, y = depth),
                  stat = "summary",
                  fun.ymin = function(z) {quantile(z,0.25)},
                  fun.ymax = function(z) {quantile(z,0.75)},
                  fun.y = median)

Results

Related posts

Spanish version

References

• Vertical intervals: lines, crossbars & errorbars
• stat_summary
• stackoverflow

Quantile-quantile (Q-Q) plots with ggplot2

Problem

We'd like to create quantile-quantile (Q-Q) plots using ggplot2.

Solution

• Option 1: one column.
We create two examples using one column of a data frame. In the first example, we previously transform the vector rivers into a data frame. In the second, we use the column Volume from the data frame trees.

library(tidyverse)
ggplot(data.frame(rivers), aes(sample = rivers)) + stat_qq() + stat_qq_line()
ggplot(trees, aes(sample = Volume)) + stat_qq() + stat_qq_line()



• Option 2: multiple columns.
We compare the distribution of Miles/(US) gallon by number of cylinders (cyl).

ggplot(mtcars, aes(sample = mpg, colour = factor(cyl))) +
  stat_qq() +
  stat_qq_line()


• Option 3: multiple panels.

In this first example no transformation is needed because a column of the data frame contains the groups..

ggplot(mtcars, aes(sample = mpg)) +
  facet_wrap( ~ factor(cyl)) +
  stat_qq() +
  stat_qq_line()


In the second example, we convert the data frame from wide format to long format for faceting into multiple panels with facet_wrap. Instead of 31 observations for 3 variables, we will have one column condition containing 93 observaciones for the 3 variables.

gather(trees, condition, measurement, Girth:Volume, factor_key = TRUE) %>%
  ggplot(aes(sample = measurement)) +
  facet_wrap( ~ condition, scales = "free") +
  stat_qq() +
  stat_qq_line()


In the third example, we convert from wide to long, facet by chemical composition, and use a colour for each kiln.

library(HSAUR2)
gather(pottery, condition, measurement, Al2O3:BaO, factor_key = TRUE) %>%
  ggplot(aes(sample = measurement, colour = kiln)) +
  facet_wrap(~ condition, scales = "free") +
  stat_qq() +
  stat_qq_line()

References

• A quantile-quantile plot
• stackoverflow
• Spanish version

Gráficos Q-Q con ggplot2

Problema

En entradas anteriores explicamos como crear un gráfico Q-Q con el paquete stats preinstalado por defecto. En esa ocasión mostraremos cómo crear estos gráficos con ggplot2.

Solución

• Opción 1: única columna.
Creamos dos ejemplos de una sola columna de un data frame. En el primero el vector rivers que transformamos previamente en un data frame, y en el segundo la columna Volume del data frame trees.

library(tidyverse)
ggplot(data.frame(rivers), aes(sample = rivers)) + stat_qq() + stat_qq_line()
ggplot(trees, aes(sample = Volume)) + stat_qq() + stat_qq_line()



• Opción 2: múltiples columnas.
Comparamos las distribuciones de Miles/(US) gallon por número de cilindros (cyl).

ggplot(mtcars, aes(sample = mpg, colour = factor(cyl))) +
  stat_qq() +
  stat_qq_line()


• Opción 3: múltiples paneles.

En este primer ejemplo no es necesario realizar ninguna transformación porque el data frame tiene una columna con los grupos.

ggplot(mtcars, aes(sample = mpg)) +
  facet_wrap( ~ factor(cyl)) +
  stat_qq() +
  stat_qq_line()


En este segundo ejemplo transformamos el data frame de formato ancho a largo para poder crear los paneles con facet_wrap. En lugar de tener 31 observaciones para 3 variables, tendremos una columna condition con las 3 variables y 93 observaciones.

gather(trees, condition, measurement, Girth:Volume, factor_key = TRUE) %>%
  ggplot(aes(sample = measurement)) +
  facet_wrap( ~ condition, scales = "free") +
  stat_qq() +
  stat_qq_line()


En este tercer ejemplo empleamos múltiples colores en función del kiln en cada panel por sustancia química encontrada.

library(HSAUR2)
gather(pottery, condition, measurement, Al2O3:BaO, factor_key = TRUE) %>%
  ggplot(aes(sample = measurement, colour = kiln)) +
  facet_wrap(~ condition, scales = "free") +
  stat_qq() +
  stat_qq_line()

Entradas relacionadas

• Construir un gráfico Q-Q desde cero en R
• Gráfico Q-Q en R

Referencias

• A quantile-quantile plot
• stackoverflow
• Versión en inglés

Calcular y representar la duración del día con R

Problema

Queremos calcular y representar la duración del día con R en función de unas coordinadas geográficas.

Solución

1. Calculamos la salida y puesta de sol

Primero calculamos la salida y la puesta de sol con la función getSunlightTimes del paquete suncalc. Indicamos el intervalo deseado, las coordinadas (latitud y longitud), y el huso horario (tz, time zone) correspondiente.

library(suncalc) 
library(tidyverse)
library(scales)
df <-
  getSunlightTimes(
    date = seq.Date(as.Date("2017-12-01"), as.Date("2018-12-31"), by = 1),
    keep = c("sunrise", "sunriseEnd", "sunset", "sunsetStart"),
    lat = 39.8628,
    lon = 4.0273,
    tz = "CET"
  )

2. Gráfico de la salida y puesta de sol

Necesitamos manipular los datos originales para calcular la diferencia entre el inicio del día, y la salida y puesta de sol. Después representamos las dos nuevas variables usando geom_ribbon de ggplot2. Luego personalizamos los ejes y el título.

# Amanecer/ocaso
df %>%
  mutate(
    date = as.POSIXct(date) - 12 * 60 * 60 ,
    sunrise = sunrise - date,
    sunset =  sunset - date,
  ) %>%
  ggplot() +
  geom_ribbon(aes(x = date, ymin = sunrise, ymax = sunset),
              fill = "#FDE725FF",
              alpha = .8) + # "#ffeda0"
  scale_x_datetime(
    breaks = seq(as.POSIXct(min(df$date)), as.POSIXct(max(df$date)), "month"),
    expand = c(0, 0),
    labels = date_format("%b %y"),
    minor_breaks = NULL
  ) +
  scale_y_continuous(
    limits = c(0, 24),
    breaks = seq(0, 24, 2),
    expand = c(0, 0),
    minor_breaks = NULL
  ) +
  labs(
    x = "Date",
    y = "Hours",
    title = sprintf(
      "Sunrise and Sunset for %s\n%s ",
      "Toledo (Spain)",
      paste0(as.Date(range(df$date)), sep = " ", collapse = "to ")
    )
  ) +
  theme(
    panel.background = element_rect(fill = "#180F3EFF"),
    panel.grid = element_line(colour = "grey", linetype = "dashed")
  )

3. Duración del día

Muy similar al gráfico anterior. Ahora solamente necesitamos calcular la duración del día day_length y representar los resultados con geom_area and geom_line.

df %>%
  mutate(
    date = as.POSIXct(date),
    day_length = as.numeric(sunset - sunrise)
  ) %>%
  ggplot(aes(x = date, y = day_length)) +
  geom_area(fill = "#FDE725FF", alpha = .4) +
  geom_line(color = "#525252") +
  scale_x_datetime(
    expand = c(0, 0),
    labels = date_format("%b '%y"),
    breaks =  seq(as.POSIXct(min(df$date)), as.POSIXct(max(df$date)), "month"),
    minor_breaks = NULL
  ) +
  scale_y_continuous(
    limits = c(0, 24),
    breaks = seq(0, 24, 2),
    expand = c(0, 0),
    minor_breaks = NULL
  ) +
  labs(x = "Date", y = "Hours", title = "Toledo (Spain) - Daytime duration") +
  theme_bw()

Entradas relacionadas

• Versión en inglés: Calculate and plot sunrise and sunset times with R
• English posts

Referencias

• suncalc
• ggplot2 formats