Nube de datos: summary

Mostrando entradas con la etiqueta summary. Mostrar todas las entradas

2020-03-05

Descriptive statistics by group in R

Title

Problem

We'd like to report descriptive statistics in R by a grouping variable and subsetting the output statistics.

Solution

We will use the data frame iris, columns Sepal.Length and Sepal.Width and grouping by Species. In our example, we want to return the mean, the standard deviation, the skewness and kurtosis.

Subset of descriptive statistics by group

library(psych)
# Variables by index
d <- describeBy(iris[1:2], group = iris$Species)
# Two options to subset the statistics:
lapply(d, "[", , c(3, 4, 11, 12))
lapply(d, subset, , c(3, 4, 11, 12))

# Variables by name
i <- match(c("Sepal.Length", "Petal.Length"), names(iris))
d <- describeBy(iris[i], group = iris$Species)
lapply(d, subset, , c("mean", "sd", "skew", "kurtosis"))

$setosa
             mean   sd skew kurtosis
Sepal.Length 5.01 0.35 0.11    -0.45
Sepal.Width  3.43 0.38 0.04     0.60

$versicolor
             mean   sd  skew kurtosis
Sepal.Length 5.94 0.52  0.10    -0.69
Sepal.Width  2.77 0.31 -0.34    -0.55

$virginica
             mean   sd skew kurtosis
Sepal.Length 6.59 0.64 0.11    -0.20
Sepal.Width  2.97 0.32 0.34     0.38

Subset of descriptive statistics without grouping

# Seleccionamos las columnas deseadas de la tabla
d <- describe(iris[1:2])
# Subsetting output statistics
d[, c(3, 4, 11, 12)]

             mean   sd skew kurtosis
Sepal.Length 5.84 0.83 0.31    -0.61
Sepal.Width  3.06 0.44 0.31     0.14

References

Descriptive statistics in R

2020-03-04

Descriptive statistics in R

Title

Problem

We'd like to compute descriptive statistics in R.

Solution

The summary function returns a set of summary statistics for the input (a vector, data frame or model).

# For a variable
summary(iris$Sepal.Length)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  4.300   5.100   5.800   5.843   6.400   7.900

# For a data frame
summary(iris)

  Sepal.Length    Sepal.Width     Petal.Length    Petal.Width          Species  
 Min.   :4.300   Min.   :2.000   Min.   :1.000   Min.   :0.100   setosa    :50  
 1st Qu.:5.100   1st Qu.:2.800   1st Qu.:1.600   1st Qu.:0.300   versicolor:50  
 Median :5.800   Median :3.000   Median :4.350   Median :1.300   virginica :50  
 Mean   :5.843   Mean   :3.057   Mean   :3.758   Mean   :1.199                  
 3rd Qu.:6.400   3rd Qu.:3.300   3rd Qu.:5.100   3rd Qu.:1.800                  
 Max.   :7.900   Max.   :4.400   Max.   :6.900   Max.   :2.500

The fivenum function returns Tukey's five number summary (minimum, lower-hinge, median, upper-hinge, maximum) for the input data.

# For a variable
fivenum(iris$Sepal.Width)

[1] 2.0 2.8 3.0 3.3 4.4

The boxplot.stats function returns the statistics necessary for producing box plots.

boxplot.stats(iris$Sepal.Width)

$stats
[1] 2.2 2.8 3.0 3.3 4.0

$n
[1] 150

$conf
[1] 2.935497 3.064503

$out
[1] 4.4 4.1 4.2 2.0

To return a specific statistic we type boxplot.stats(iris$Sepal.Width) followed by:

$stats - vector with Tukey's five number summary.
$n - the number of non-NA observations.
$conf - the lower and upper extremes of the ‘notch’.
$out- outliers.

The psych package

For a data frame

library(psych)
describe(iris)

             vars   n mean   sd median trimmed  mad
Sepal.Length    1 150 5.84 0.83   5.80    5.81 1.04
Sepal.Width     2 150 3.06 0.44   3.00    3.04 0.44
Petal.Length    3 150 3.76 1.77   4.35    3.76 1.85
Petal.Width     4 150 1.20 0.76   1.30    1.18 1.04
Species*        5 150  NaN   NA     NA     NaN   NA
             min  max range  skew kurtosis   se
Sepal.Length 4.3  7.9   3.6  0.31    -0.61 0.07
Sepal.Width  2.0  4.4   2.4  0.31     0.14 0.04
Petal.Length 1.0  6.9   5.9 -0.27    -1.42 0.14
Petal.Width  0.1  2.5   2.4 -0.10    -1.36 0.06
Species*     Inf -Inf  -Inf    NA       NA   NA

Statistics by group

describeBy(iris, group = iris$Species)

group: setosa
             vars  n mean   sd median trimmed  mad
Sepal.Length    1 50 5.01 0.35    5.0    5.00 0.30
Sepal.Width     2 50 3.43 0.38    3.4    3.42 0.37
Petal.Length    3 50 1.46 0.17    1.5    1.46 0.15
Petal.Width     4 50 0.25 0.11    0.2    0.24 0.00
Species*        5 50  NaN   NA     NA     NaN   NA
             min  max range skew kurtosis   se
Sepal.Length 4.3  5.8   1.5 0.11    -0.45 0.05
Sepal.Width  2.3  4.4   2.1 0.04     0.60 0.05
Petal.Length 1.0  1.9   0.9 0.10     0.65 0.02
Petal.Width  0.1  0.6   0.5 1.18     1.26 0.01
Species*     Inf -Inf  -Inf   NA       NA   NA
--------------------------------------- 
group: versicolor
             vars  n mean   sd median trimmed  mad
Sepal.Length    1 50 5.94 0.52   5.90    5.94 0.52
Sepal.Width     2 50 2.77 0.31   2.80    2.78 0.30
Petal.Length    3 50 4.26 0.47   4.35    4.29 0.52
Petal.Width     4 50 1.33 0.20   1.30    1.32 0.22
Species*        5 50  NaN   NA     NA     NaN   NA
             min  max range  skew kurtosis   se
Sepal.Length 4.9  7.0   2.1  0.10    -0.69 0.07
Sepal.Width  2.0  3.4   1.4 -0.34    -0.55 0.04
Petal.Length 3.0  5.1   2.1 -0.57    -0.19 0.07
Petal.Width  1.0  1.8   0.8 -0.03    -0.59 0.03
Species*     Inf -Inf  -Inf    NA       NA   NA
--------------------------------------- 
group: virginica
             vars  n mean   sd median trimmed  mad
Sepal.Length    1 50 6.59 0.64   6.50    6.57 0.59
Sepal.Width     2 50 2.97 0.32   3.00    2.96 0.30
Petal.Length    3 50 5.55 0.55   5.55    5.51 0.67
Petal.Width     4 50 2.03 0.27   2.00    2.03 0.30
Species*        5 50  NaN   NA     NA     NaN   NA
             min  max range  skew kurtosis   se
Sepal.Length 4.9  7.9   3.0  0.11    -0.20 0.09
Sepal.Width  2.2  3.8   1.6  0.34     0.38 0.05
Petal.Length 4.5  6.9   2.4  0.52    -0.37 0.08
Petal.Width  1.4  2.5   1.1 -0.12    -0.75 0.04
Species*     Inf -Inf  -Inf    NA       NA   NA

2019-12-28

How to convert a continuous variable to discrete in R?

Problem

We want to convert continuous variable to discrete in R:

'Create a new qualitative variable, called Elite, by binning the Top10perc variable. We are going to divide universities into two groups based on whether or not the proportion of students coming from the top 10% of their high school classes exceeds 50%'.

library(ISLR)
library(tidyverse)
glimpse(College)

Observations: 777
Variables: 18
$ Private      Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Ye...
$ Apps         1660, 2186, 1428, 417, 193, 587, 353, 1899, 1038, 582, 17...
$ Accept       1232, 1924, 1097, 349, 146, 479, 340, 1720, 839, 498, 142...
$ Enroll       721, 512, 336, 137, 55, 158, 103, 489, 227, 172, 472, 484...
$ Top10perc    23, 16, 22, 60, 16, 38, 17, 37, 30, 21, 37, 44, 38, 44, 2...
$ Top25perc    52, 29, 50, 89, 44, 62, 45, 68, 63, 44, 75, 77, 64, 73, 4...
$ F.Undergrad  2885, 2683, 1036, 510, 249, 678, 416, 1594, 973, 799, 183...
$ P.Undergrad  537, 1227, 99, 63, 869, 41, 230, 32, 306, 78, 110, 44, 63...
$ Outstate     7440, 12280, 11250, 12960, 7560, 13500, 13290, 13868, 155...
$ Room.Board   3300, 6450, 3750, 5450, 4120, 3335, 5720, 4826, 4400, 338...
$ Books        450, 750, 400, 450, 800, 500, 500, 450, 300, 660, 500, 40...
$ Personal     2200, 1500, 1165, 875, 1500, 675, 1500, 850, 500, 1800, 6...
$ PhD          70, 29, 53, 92, 76, 67, 90, 89, 79, 40, 82, 73, 60, 79, 3...
$ Terminal     78, 30, 66, 97, 72, 73, 93, 100, 84, 41, 88, 91, 84, 87, ...
$ S.F.Ratio    18.1, 12.2, 12.9, 7.7, 11.9, 9.4, 11.5, 13.7, 11.3, 11.5,...
$ perc.alumni  12, 16, 30, 37, 2, 11, 26, 37, 23, 15, 31, 41, 21, 32, 26...
$ Expend       7041, 10527, 8735, 19016, 10922, 9727, 8861, 11487, 11644...
$ Grad.Rate    60, 56, 54, 59, 15, 55, 63, 73, 80, 52, 73, 76, 74, 68, 5...

Solution

Option 1: form ISLR's book.

Elite = rep("No", nrow(College))
Elite[College$Top10perc > 50] = "Yes"
Elite <- as.factor(Elite)
college <- data.frame(College,  Elite)
summary(college[, c("Top10perc", "Elite")])

  Top10perc     Elite    
 Min.   : 1.00   No :699  
 1st Qu.:15.00   Yes: 78  
 Median :23.00            
 Mean   :27.56            
 3rd Qu.:35.00            
 Max.   :96.00

Option 2: ifelse from base package and dplyr

# base 
College$Elite <- factor(ifelse(College$Top10perc > 50, "Yes", "No"))
# dplyr
library(dplyr)
College <-
  college %>%
  mutate(Elite = factor(ifelse(College$Top10perc > 50, "Yes", "No")))

Option 3: creating a logical vector.

There are multiple options. I show two examples.

college$Elite <- transform(College, Elite = Top10perc > 50)
College$Elite <- College$Top10perc > 50

References

An Introduction to Statistical Learning - Page 54.

From 'An Introduction to Statistical Learning' (ISLR), page 54.

2018-12-08

Discretización de variables en R

Problema

Deseamos discretizar una variable, es decir, convertir una variable continua en discreta. Utilizamos el conjunto de datos College del paquete ISLR. Crearemos una nueva variable cualitativa llamada Elite, discretizando la variable Top10perc. Vamos a dividir las universidades en dos grupos basados en si la proporción de nuevos estudiantes provienen de entre el 10% de los mejores alumnos de sus institutos excede o no el 50%.

library(ISLR)
library(tidyverse)
glimpse(College)

Observations: 777
Variables: 18
$ Private      Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Yes, Ye...
$ Apps         1660, 2186, 1428, 417, 193, 587, 353, 1899, 1038, 582, 17...
$ Accept       1232, 1924, 1097, 349, 146, 479, 340, 1720, 839, 498, 142...
$ Enroll       721, 512, 336, 137, 55, 158, 103, 489, 227, 172, 472, 484...
$ Top10perc    23, 16, 22, 60, 16, 38, 17, 37, 30, 21, 37, 44, 38, 44, 2...
$ Top25perc    52, 29, 50, 89, 44, 62, 45, 68, 63, 44, 75, 77, 64, 73, 4...
$ F.Undergrad  2885, 2683, 1036, 510, 249, 678, 416, 1594, 973, 799, 183...
$ P.Undergrad  537, 1227, 99, 63, 869, 41, 230, 32, 306, 78, 110, 44, 63...
$ Outstate     7440, 12280, 11250, 12960, 7560, 13500, 13290, 13868, 155...
$ Room.Board   3300, 6450, 3750, 5450, 4120, 3335, 5720, 4826, 4400, 338...
$ Books        450, 750, 400, 450, 800, 500, 500, 450, 300, 660, 500, 40...
$ Personal     2200, 1500, 1165, 875, 1500, 675, 1500, 850, 500, 1800, 6...
$ PhD          70, 29, 53, 92, 76, 67, 90, 89, 79, 40, 82, 73, 60, 79, 3...
$ Terminal     78, 30, 66, 97, 72, 73, 93, 100, 84, 41, 88, 91, 84, 87, ...
$ S.F.Ratio    18.1, 12.2, 12.9, 7.7, 11.9, 9.4, 11.5, 13.7, 11.3, 11.5,...
$ perc.alumni  12, 16, 30, 37, 2, 11, 26, 37, 23, 15, 31, 41, 21, 32, 26...
$ Expend       7041, 10527, 8735, 19016, 10922, 9727, 8861, 11487, 11644...
$ Grad.Rate    60, 56, 54, 59, 15, 55, 63, 73, 80, 52, 73, 76, 74, 68, 5...

Solución

Opción 1:Propuesta en el libro ISLR.

Elite = rep("No", nrow(College))
Elite[College$Top10perc > 50] = "Yes"
Elite <- as.factor(Elite)
college <- data.frame(College,  Elite)
summary(college[, c("Top10perc", "Elite")])

  Top10perc     Elite    
 Min.   : 1.00   No :699  
 1st Qu.:15.00   Yes: 78  
 Median :23.00            
 Mean   :27.56            
 3rd Qu.:35.00            
 Max.   :96.00

Opción 2: ifelse con paquete base y dplyr

# base 
College$Elite <- factor(ifelse(College$Top10perc > 50, "Yes", "No"))
# dplyr
library(dplyr)
College <-
  college %>%
  mutate(Elite = factor(ifelse(College$Top10perc > 50, "Yes", "No")))

Opción 3: vector lógico.

Hay múltiples opciones. Presento dos ejemplos.

college$Elite <- transform(College, Elite = Top10perc > 50)
College$Elite <- College$Top10perc > 50

Entradas relacionadas

2015-08-13

Analizar subgrupos de un data frame con la función aggregate en R

Title

Problema

Deseamos calcular para varios grupos de un data frame diferentes indicadores. En nuestro ejemplo usaremos los datos de TootGrowth.

[,1] len numeric Tooth length
[,2] supp factor Supplement type (VC or OJ).
[,3] dose numeric Dose in milligrams/day

Calcularemos la media y la mediana de la longitud de los dientes (len) para cada uno de los suplementos (supp) de vitaminas (VC o OJ).

Solución

Empleamos la función aggregate.

aggregate(len ~ supp, data = ToothGrowth, 
          FUN = function(x) c(media =mean(x), mediana = median(x)))

Empleamos la estructura de fórmula y ~ x, donde y es la variable numérica de la que queremos el resultado y x es la variable por la que agruparemos.

Resultado

  supp len.media len.mediana
1   OJ  20.66333    22.70000
2   VC  16.96333    16.50000

Más niveles de agrupación

Si deseamos añadir el nivel de agrupación por dosis (dose).

aggregate(len ~ supp + dose, data = ToothGrowth, 
          FUN = function(x) c(media =mean(x), mediana = median(x)))

 supp dose len.media len.mediana
1   OJ  0.5     13.23       12.25
2   VC  0.5      7.98        7.15
3   OJ  1.0     22.70       23.45
4   VC  1.0     16.77       16.50
5   OJ  2.0     26.06       25.95
6   VC  2.0     26.14       25.95

Más variables numéricas

Utilizamos ahora el conjunto de datos iris.

aggregate(cbind(Sepal.Length, Sepal.Width) ~ Species, data = iris, 
          FUN = function(x) c(media =mean(x), mediana = median(x)))

     Species Sepal.Length.media Sepal.Length.mediana Sepal.Width.media
1     setosa              5.006                5.000             3.428
2 versicolor              5.936                5.900             2.770
3  virginica              6.588                6.500             2.974
  Sepal.Width.mediana
1               3.400
2               2.800
3               3.000

Es necesario emplear la función cbind, de lo contrario, nos sumará los resultados de las variables numéricas:

aggregate(Sepal.Length + Sepal.Width~ Species, data = iris, 
          FUN = function(x) c(media = mean(x), mediana = median(x)))

     Species Sepal.Length + Sepal.Width.media Sepal.Length + Sepal.Width.mediana
1     setosa                            8.434                              8.450
2 versicolor                            8.706                              8.600
3  virginica                            9.562                              9.600

Entradas relacionadas

Subconjunto de estadísticas descriptivas en R

2015-06-24

Estadísticas descriptivas en R

Title

Problema

Deseamos calcular estadísticas descriptivas a nuestros datos.

Solución

Vamos a ver algunas de las fórmulas que hemos visto de manera dispersa en otras entradas.

Función summary

# Para una sola variable
summary(iris$Sepal.Length)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  4.300   5.100   5.800   5.843   6.400   7.900

# Para una tabla (data frame)
summary(iris)

  Sepal.Length    Sepal.Width     Petal.Length    Petal.Width          Species  
 Min.   :4.300   Min.   :2.000   Min.   :1.000   Min.   :0.100   setosa    :50  
 1st Qu.:5.100   1st Qu.:2.800   1st Qu.:1.600   1st Qu.:0.300   versicolor:50  
 Median :5.800   Median :3.000   Median :4.350   Median :1.300   virginica :50  
 Mean   :5.843   Mean   :3.057   Mean   :3.758   Mean   :1.199                  
 3rd Qu.:6.400   3rd Qu.:3.300   3rd Qu.:5.100   3rd Qu.:1.800                  
 Max.   :7.900   Max.   :4.400   Max.   :6.900   Max.   :2.500

Función fivenum

Resumen con los 5 números de Tukey empleados en los diagramas de caja: mínimo, bigote inferior, mediana, bigote superior, máximo

# Para una variable solamente
fivenum(iris$Sepal.Width)

[1] 2.0 2.8 3.0 3.3 4.4

Función boxplot.stats

boxplot.stats(iris$Sepal.Width)

$stats
[1] 2.2 2.8 3.0 3.3 4.0

$n
[1] 150

$conf
[1] 2.935497 3.064503

$out
[1] 4.4 4.1 4.2 2.0

Podemos acceder a los elementos de la lista anterior, con el símbolo $ seguido del elemento de la lista:

$stats - vector con los 5 números de Tukey.
$n - número de observaciones.
$conf - intervalo de confianza para la media.
$out- los valores de los valores atípicos (outliers).

Con el paquete psych

Para una tabla

install.packages("psych")
require("psych")
describe(iris)

             vars   n mean   sd median trimmed  mad
Sepal.Length    1 150 5.84 0.83   5.80    5.81 1.04
Sepal.Width     2 150 3.06 0.44   3.00    3.04 0.44
Petal.Length    3 150 3.76 1.77   4.35    3.76 1.85
Petal.Width     4 150 1.20 0.76   1.30    1.18 1.04
Species*        5 150  NaN   NA     NA     NaN   NA
             min  max range  skew kurtosis   se
Sepal.Length 4.3  7.9   3.6  0.31    -0.61 0.07
Sepal.Width  2.0  4.4   2.4  0.31     0.14 0.04
Petal.Length 1.0  6.9   5.9 -0.27    -1.42 0.14
Petal.Width  0.1  2.5   2.4 -0.10    -1.36 0.06
Species*     Inf -Inf  -Inf    NA       NA   NA

Estadísticas por grupo

describeBy(iris, group = iris$Species)

group: setosa
             vars  n mean   sd median trimmed  mad
Sepal.Length    1 50 5.01 0.35    5.0    5.00 0.30
Sepal.Width     2 50 3.43 0.38    3.4    3.42 0.37
Petal.Length    3 50 1.46 0.17    1.5    1.46 0.15
Petal.Width     4 50 0.25 0.11    0.2    0.24 0.00
Species*        5 50  NaN   NA     NA     NaN   NA
             min  max range skew kurtosis   se
Sepal.Length 4.3  5.8   1.5 0.11    -0.45 0.05
Sepal.Width  2.3  4.4   2.1 0.04     0.60 0.05
Petal.Length 1.0  1.9   0.9 0.10     0.65 0.02
Petal.Width  0.1  0.6   0.5 1.18     1.26 0.01
Species*     Inf -Inf  -Inf   NA       NA   NA
--------------------------------------- 
group: versicolor
             vars  n mean   sd median trimmed  mad
Sepal.Length    1 50 5.94 0.52   5.90    5.94 0.52
Sepal.Width     2 50 2.77 0.31   2.80    2.78 0.30
Petal.Length    3 50 4.26 0.47   4.35    4.29 0.52
Petal.Width     4 50 1.33 0.20   1.30    1.32 0.22
Species*        5 50  NaN   NA     NA     NaN   NA
             min  max range  skew kurtosis   se
Sepal.Length 4.9  7.0   2.1  0.10    -0.69 0.07
Sepal.Width  2.0  3.4   1.4 -0.34    -0.55 0.04
Petal.Length 3.0  5.1   2.1 -0.57    -0.19 0.07
Petal.Width  1.0  1.8   0.8 -0.03    -0.59 0.03
Species*     Inf -Inf  -Inf    NA       NA   NA
--------------------------------------- 
group: virginica
             vars  n mean   sd median trimmed  mad
Sepal.Length    1 50 6.59 0.64   6.50    6.57 0.59
Sepal.Width     2 50 2.97 0.32   3.00    2.96 0.30
Petal.Length    3 50 5.55 0.55   5.55    5.51 0.67
Petal.Width     4 50 2.03 0.27   2.00    2.03 0.30
Species*        5 50  NaN   NA     NA     NaN   NA
             min  max range  skew kurtosis   se
Sepal.Length 4.9  7.9   3.0  0.11    -0.20 0.09
Sepal.Width  2.2  3.8   1.6  0.34     0.38 0.05
Petal.Length 4.5  6.9   2.4  0.52    -0.37 0.08
Petal.Width  1.4  2.5   1.1 -0.12    -0.75 0.04
Species*     Inf -Inf  -Inf    NA       NA   NA

Nube de datos

2020-03-05

Descriptive statistics by group in R

Problem

Solution

References

2020-03-04

Descriptive statistics in R

Problem

Solution

The psych package

2019-12-28

How to convert a continuous variable to discrete in R?

Problem

Solution

References

Related posts

2018-12-08

Discretización de variables en R

Problema

Solución

Entradas relacionadas

2015-08-13

Analizar subgrupos de un data frame con la función aggregate en R

Problema

Solución

Resultado

Más niveles de agrupación

Más variables numéricas

Entradas relacionadas

2015-06-24

Estadísticas descriptivas en R

Problema

Solución

Con el paquete psych

Referencias