These notes largely follow the book, and I don’t have time to do all the exercises. In some cases I’ve compared Simon Wood’s “by hand” solutions in the text to other packages or methods.

# Ch. 1: Linear Models

Chapter 1 is about linear models and their theory.

## Exercises

### Exercise 4: Write out factor models with model matrices

Write out the following models in form $$\mathbf{y} = \mathbf{X}\mathbf{\beta} + \mathbf{\epsilon}$$.

1. ‘balanced one-way ANOVA model’: $$y_{ij} = \alpha + \beta_{i} + \epsilon_{ij}$$ where $$i=1,2,3$$ and $$j=1,2$$.

It has one factor, with 3 levels, and each level has 2 observations. R can make this with model.matrix.

b <- factor(rep(1:3,each=2))
model.matrix(~ b, data.frame(b=b))
##   (Intercept) b2 b3
## 1           1  0  0
## 2           1  0  0
## 3           1  1  0
## 4           1  1  0
## 5           1  0  1
## 6           1  0  1
## attr(,"assign")
## [1] 0 1 1
## attr(,"contrasts")
## attr(,"contrasts")$b ## [1] "contr.treatment" So we write it as: $\begin{bmatrix} y_{11} \\ y_{12} \\ y_{21} \\ y_{22} \\ y_{31} \\ y_{32} \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ 1 & 0 & 0 \\ 1 & 1 & 0 \\ 1 & 1 & 0 \\ 1 & 0 & 1 \\ 1 & 0 & 1 \end{bmatrix} \begin{bmatrix} \alpha \\ \beta_{2} \\ \beta_{3} \end{bmatrix} + \begin{bmatrix} \epsilon_{11} \\ \epsilon_{12} \\ \epsilon_{21} \\ \epsilon_{22} \\ \epsilon_{31} \\ \epsilon_{32} \end{bmatrix}$ 1. A model with 2 factor variables (3 and 4 levels, respectively) and 1 observation per combination: $$y_{ij} = \alpha + \beta_{i} + \gamma_{j} + \epsilon_{ij}$$. We do it in R, note that we use expand.grid to generate all unique combinations. There’s only 1 observation per stratum, so that’s sufficient. beta <- factor(1:3) gamma <- factor(1:4) model.matrix(~ beta + gamma, expand.grid(beta=beta,gamma=gamma)) ## (Intercept) beta2 beta3 gamma2 gamma3 gamma4 ## 1 1 0 0 0 0 0 ## 2 1 1 0 0 0 0 ## 3 1 0 1 0 0 0 ## 4 1 0 0 1 0 0 ## 5 1 1 0 1 0 0 ## 6 1 0 1 1 0 0 ## 7 1 0 0 0 1 0 ## 8 1 1 0 0 1 0 ## 9 1 0 1 0 1 0 ## 10 1 0 0 0 0 1 ## 11 1 1 0 0 0 1 ## 12 1 0 1 0 0 1 ## attr(,"assign") ## [1] 0 1 1 2 2 2 ## attr(,"contrasts") ## attr(,"contrasts")$beta
## [1] "contr.treatment"
##
## attr(,"contrasts")$gamma ## [1] "contr.treatment" It’s written as: $\begin{bmatrix} y_{11} \\ y_{21} \\ y_{31} \\ y_{12} \\ y_{22} \\ y_{32} \\ y_{13} \\ y_{23} \\ y_{33} \\ y_{14} \\ y_{24} \\ y_{34} \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 & 0 & 0 & 0 \\ 1 & 1 & 0 & 0 & 0 & 0 \\ 1 & 0 & 1 & 0 & 0 & 0 \\ 1 & 0 & 0 & 1 & 0 & 0 \\ 1 & 1 & 0 & 1 & 0 & 0 \\ 1 & 0 & 1 & 1 & 0 & 0 \\ 1 & 0 & 0 & 0 & 1 & 0 \\ 1 & 1 & 0 & 0 & 1 & 0 \\ 1 & 0 & 1 & 0 & 1 & 0 \\ 1 & 0 & 0 & 0 & 0 & 1 \\ 1 & 1 & 0 & 0 & 0 & 1 \\ 1 & 0 & 1 & 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} \alpha \\ \beta_{2} \\ \beta_{3} \\ \gamma_{2} \\ \gamma_{3} \\ \gamma_{4} \end{bmatrix} + \begin{bmatrix} \epsilon_{11} \\ \epsilon_{21} \\ \epsilon_{31} \\ \epsilon_{12} \\ \epsilon_{22} \\ \epsilon_{32} \\ \epsilon_{13} \\ \epsilon_{23} \\ \epsilon_{33} \\ \epsilon_{14} \\ \epsilon_{24} \\ \epsilon_{34} \end{bmatrix}$ 1. A model with a factor variable (2 levels) and a continuous variable: $$y_{i} = \alpha + \beta_{j} + \gamma x_{i} + \epsilon_{i}$$ b <- factor(rep(1:2,times=c(2,4))) x <- c(0.1,0.4,0.5,0.3,0.4,0.7) model.matrix(~ beta + x, data.frame(beta = b, x = x)) ## (Intercept) beta2 x ## 1 1 0 0.1 ## 2 1 0 0.4 ## 3 1 1 0.5 ## 4 1 1 0.3 ## 5 1 1 0.4 ## 6 1 1 0.7 ## attr(,"assign") ## [1] 0 1 2 ## attr(,"contrasts") ## attr(,"contrasts")$beta
## [1] "contr.treatment"

Written out: $\begin{bmatrix} y_{1} \\ y_{2} \\ y_{3} \\ y_{4} \\ y_{5} \\ y_{6} \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0.1 \\ 1 & 0 & 0.4 \\ 1 & 1 & 0.5 \\ 1 & 1 & 0.3 \\ 1 & 1 & 0.4 \\ 1 & 1 & 0.7 \end{bmatrix} \begin{bmatrix} \alpha \\ \beta_{2} \\ \gamma \end{bmatrix} + \begin{bmatrix} \epsilon_{1} \\ \epsilon_{2} \\ \epsilon_{3} \\ \epsilon_{4} \\ \epsilon_{5} \\ \epsilon_{6} \end{bmatrix}$