These notes largely follow the book, and I don’t have time to do all the exercises. In some cases I’ve compared Simon Wood’s “by hand” solutions in the text to other packages or methods.

Ch. 1: Linear Models

Chapter 1 is about linear models and their theory.

Exercises

Exercise 4: Write out factor models with model matrices

Write out the following models in form \(\mathbf{y} = \mathbf{X}\mathbf{\beta} + \mathbf{\epsilon}\).

  1. ‘balanced one-way ANOVA model’: \(y_{ij} = \alpha + \beta_{i} + \epsilon_{ij}\) where \(i=1,2,3\) and \(j=1,2\).

It has one factor, with 3 levels, and each level has 2 observations. R can make this with model.matrix.

b <- factor(rep(1:3,each=2))
model.matrix(~ b, data.frame(b=b))
##   (Intercept) b2 b3
## 1           1  0  0
## 2           1  0  0
## 3           1  1  0
## 4           1  1  0
## 5           1  0  1
## 6           1  0  1
## attr(,"assign")
## [1] 0 1 1
## attr(,"contrasts")
## attr(,"contrasts")$b
## [1] "contr.treatment"

So we write it as: \[ \begin{bmatrix} y_{11} \\ y_{12} \\ y_{21} \\ y_{22} \\ y_{31} \\ y_{32} \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ 1 & 0 & 0 \\ 1 & 1 & 0 \\ 1 & 1 & 0 \\ 1 & 0 & 1 \\ 1 & 0 & 1 \end{bmatrix} \begin{bmatrix} \alpha \\ \beta_{2} \\ \beta_{3} \end{bmatrix} + \begin{bmatrix} \epsilon_{11} \\ \epsilon_{12} \\ \epsilon_{21} \\ \epsilon_{22} \\ \epsilon_{31} \\ \epsilon_{32} \end{bmatrix} \]

  1. A model with 2 factor variables (3 and 4 levels, respectively) and 1 observation per combination: \(y_{ij} = \alpha + \beta_{i} + \gamma_{j} + \epsilon_{ij}\).

We do it in R, note that we use expand.grid to generate all unique combinations. There’s only 1 observation per stratum, so that’s sufficient.

beta <- factor(1:3)
gamma <- factor(1:4)
model.matrix(~ beta + gamma, expand.grid(beta=beta,gamma=gamma))
##    (Intercept) beta2 beta3 gamma2 gamma3 gamma4
## 1            1     0     0      0      0      0
## 2            1     1     0      0      0      0
## 3            1     0     1      0      0      0
## 4            1     0     0      1      0      0
## 5            1     1     0      1      0      0
## 6            1     0     1      1      0      0
## 7            1     0     0      0      1      0
## 8            1     1     0      0      1      0
## 9            1     0     1      0      1      0
## 10           1     0     0      0      0      1
## 11           1     1     0      0      0      1
## 12           1     0     1      0      0      1
## attr(,"assign")
## [1] 0 1 1 2 2 2
## attr(,"contrasts")
## attr(,"contrasts")$beta
## [1] "contr.treatment"
## 
## attr(,"contrasts")$gamma
## [1] "contr.treatment"

It’s written as: \[ \begin{bmatrix} y_{11} \\ y_{21} \\ y_{31} \\ y_{12} \\ y_{22} \\ y_{32} \\ y_{13} \\ y_{23} \\ y_{33} \\ y_{14} \\ y_{24} \\ y_{34} \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 & 0 & 0 & 0 \\ 1 & 1 & 0 & 0 & 0 & 0 \\ 1 & 0 & 1 & 0 & 0 & 0 \\ 1 & 0 & 0 & 1 & 0 & 0 \\ 1 & 1 & 0 & 1 & 0 & 0 \\ 1 & 0 & 1 & 1 & 0 & 0 \\ 1 & 0 & 0 & 0 & 1 & 0 \\ 1 & 1 & 0 & 0 & 1 & 0 \\ 1 & 0 & 1 & 0 & 1 & 0 \\ 1 & 0 & 0 & 0 & 0 & 1 \\ 1 & 1 & 0 & 0 & 0 & 1 \\ 1 & 0 & 1 & 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} \alpha \\ \beta_{2} \\ \beta_{3} \\ \gamma_{2} \\ \gamma_{3} \\ \gamma_{4} \end{bmatrix} + \begin{bmatrix} \epsilon_{11} \\ \epsilon_{21} \\ \epsilon_{31} \\ \epsilon_{12} \\ \epsilon_{22} \\ \epsilon_{32} \\ \epsilon_{13} \\ \epsilon_{23} \\ \epsilon_{33} \\ \epsilon_{14} \\ \epsilon_{24} \\ \epsilon_{34} \end{bmatrix} \]

  1. A model with a factor variable (2 levels) and a continuous variable: \(y_{i} = \alpha + \beta_{j} + \gamma x_{i} + \epsilon_{i}\)
b <- factor(rep(1:2,times=c(2,4)))
x <- c(0.1,0.4,0.5,0.3,0.4,0.7)
model.matrix(~ beta + x, data.frame(beta = b, x = x))
##   (Intercept) beta2   x
## 1           1     0 0.1
## 2           1     0 0.4
## 3           1     1 0.5
## 4           1     1 0.3
## 5           1     1 0.4
## 6           1     1 0.7
## attr(,"assign")
## [1] 0 1 2
## attr(,"contrasts")
## attr(,"contrasts")$beta
## [1] "contr.treatment"

Written out: \[ \begin{bmatrix} y_{1} \\ y_{2} \\ y_{3} \\ y_{4} \\ y_{5} \\ y_{6} \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0.1 \\ 1 & 0 & 0.4 \\ 1 & 1 & 0.5 \\ 1 & 1 & 0.3 \\ 1 & 1 & 0.4 \\ 1 & 1 & 0.7 \end{bmatrix} \begin{bmatrix} \alpha \\ \beta_{2} \\ \gamma \end{bmatrix} + \begin{bmatrix} \epsilon_{1} \\ \epsilon_{2} \\ \epsilon_{3} \\ \epsilon_{4} \\ \epsilon_{5} \\ \epsilon_{6} \end{bmatrix} \]

Exercise 11: Replicate lm