Linear regression

Applied Statistics for Life Sciences

Least squares estimation

Meadowfoam flowering

Mean flowering depends on light intensity.

the “response” variable is flowers per plant
the “explanatory” variable is light intensity

In an ANOVA model, the explanatory variable is treated as categorical.

one mean estimated per unique value of the explanatory variable
values fixed in advance by researcher for designed experiments

But what if we had observational data instead?

Could we estimate mean flowering as a continuous function of intensity?

casenr	age	rfft
126	37	136
33	36	80
145	37	102
146	37	85

casenr	age	rfft
126	37	136
33	36	80
145	37	102
146	37	85

Least squares estimates in R

According to the model, a one-unit increment in $x$ corresponds to a $β_{1}$ -unit change in mean $Y$ :

# fit model
fit <- lm(formula = rfft ~ age, data = prevend)
fit


Call:
lm(formula = rfft ~ age, data = prevend)

Coefficients:
(Intercept)          age  
    134.098       -1.191

With each additional year of age, mean RFFT score decreases by an estimated 1.191 points.

formula = <RESPONSE> ~ <EXPLANATORY> specifies the model
data = <DATAFRAME> specifies the observations

Error variability and model fit

The residual standard deviation provides an estimate of error variability:

$\hat{σ} = \sqrt{\frac{1}{n - 2} \sum_{i} e_{i}^{2}} (estimated error variability)$

The proportion of variability explained by the model is: $R^{2} = 1 - \frac{(n - 2) {\hat{σ}}^{2}}{(n - 1) s_{y}^{2}} (1 - \frac{error variability}{total variability})$

1 - (n - 2)*sigma(fit)^2/((n - 1)*var(rfft))

[1] 0.4043103

Age explains 40.43% of variability in RFFT.

An alternative model

$\log (RFFT) = β_{0} + β_{1} \times age + ϵ$

The model now implies that the mean RFFT score is a nonlinear function of age:

$RFFT \propto e^{β_{1} age}$

And we can still fit it using least squares:

lm(log(rfft) ~ age, data = prevend)


Call:
lm(formula = log(rfft) ~ age, data = prevend)

Coefficients:
(Intercept)          age  
    5.21739     -0.01951

Linear models can be used to capture more than just linear relationships!

Kleiber’s law

Kleiber’s law refers to the relationship between metabolic rate and body mass.

We can estimate it via the SLR model: $\log (metabolism) = β_{0} + β_{1} \log (mass) + ϵ$

Fitted model: $\log (metabolism) = 5.64 + 0.74 \times \log (mass)$

fit <- lm(log.metab ~ log.mass, data = kleiber)
fit


Call:
lm(formula = log.metab ~ log.mass, data = kleiber)

Coefficients:
(Intercept)     log.mass  
     5.6383       0.7387

Kleiber’s law: model interpretation

Exponentiating both sides of the fitted SLR model equation:

$\underset{e^{\log (metabolism)}}{\underset{⏟}{metabolism}} = \underset{e^{5.64}}{\underset{⏟}{280.99}} \times \underset{e^{0.74 \log (mass)}}{\underset{⏟}{{mass}^{0.74}}}$

So we’ve really estimated what’s known as a power law relationship: $y = a x^{b}$ .

multiplicative, not additive, relationship
doubling $x$ corresponds to changing $y$ by a factor of $2^{b}$

The estimate and interval for $β_{1}$ in the SLR model can be transformed appropriately for a more direct interpretation:

Every doubling of body mass is associated with an estimated 66.87% increase in median metabolism.

Inference in regression

Review

How much does RFFT decline with age?

Simple linear regression (SLR) model: $RFFT = β_{0} + β_{1} age + ϵ$

# least squares estimates
fit <- lm(formula = rfft ~ age, data = prevend)
fit


Call:
lm(formula = rfft ~ age, data = prevend)

Coefficients:
(Intercept)          age  
    134.098       -1.191

Interpretation:

With each additional year of age, mean RFFT score decreases by an estimated 1.191 points.

Inference for the slope parameter

If the errors are symmetric and unimodal, then the sampling distribution of $T = \frac{{\hat{β}}_{1} - β_{1}}{S E (β_{1})}$ is well-approximated by a $t_{n - 2}$ model.

Significance test: ${\begin{cases} H_{0} : β_{1} = 0 \\ H_{A} : β_{1} \neq 0 \end{cases}$
Confidence interval: ${\hat{β}}_{1} \pm c \times S E ({\hat{β}}_{1})$

$P (T > | T_{obs} |) \approx 0$ : evidence of an association (true slope is not zero)
confidence interval using $t_{206}$ critical value: (-1.389, -0.992)

Inference for the intercept is analogous, but not very common.

Model summary

The model summary shows most quantities of interest, except CIs.

# model summary
summary(fit)


Call:
lm(formula = rfft ~ age, data = prevend)

Residuals:
    Min      1Q  Median      3Q     Max 
-56.085 -14.690  -2.937  12.744  77.975 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 134.0981     6.0701   22.09   <2e-16 ***
age          -1.1908     0.1007  -11.82   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 20.52 on 206 degrees of freedom
Multiple R-squared:  0.4043,    Adjusted R-squared:  0.4014 
F-statistic: 139.8 on 1 and 206 DF,  p-value: < 2.2e-16

${\hat{β}}_{0} = 134.10, {\hat{β}}_{1} = - 1.19, \hat{σ} = 20.52$
Age explains an estimated 40.43% of variation in RFFT.
With each year of age mean RFFT declines by an estimated 1.19 points (SE 0.10).
There is a significant association between age and mean RFFT score (T = -11.82 on 206 degrees of freedom, p < 0.0001).

Take a moment to locate the quantities that support the conclusions listed at right.

Kleiber’s law

fit <- lm(log.metab ~ log.mass, data = kleiber)
summary(fit)


Call:
lm(formula = log.metab ~ log.mass, data = kleiber)

Residuals:
     Min       1Q   Median       3Q      Max 
-1.14216 -0.26466 -0.04889  0.25308  1.37616 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  5.63833    0.04709  119.73   <2e-16 ***
log.mass     0.73874    0.01462   50.53   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.4572 on 93 degrees of freedom
Multiple R-squared:  0.9649,    Adjusted R-squared:  0.9645 
F-statistic:  2553 on 1 and 93 DF,  p-value: < 2.2e-16

${\hat{β}}_{0} = 5.638, {\hat{β}}_{1} = 0.739, \hat{σ} = 0.457$

There is a significant association between body mass and metabolism (p < 0.0001): body mass explains 96.49% of variation in metabolism; with 95% confidence, a unit increment in log mass is associated with an estimated increase in mean log metabolism between 0.7097 and 0.7678.

Kleiber’s law

How much energy do we consume on a daily basis?

Conversions:

110lb $\approx$ 50kg
180lb $\approx$ 80kg
1 kJ $\approx$ 0.239 kcal

# range of body mass
mass <- seq(50, 80)

# interval for mean log energy
pred <- predict(fit, 
                newdata = data.frame(log.mass = log(mass)), 
                interval = 'confidence')
# convert
estimates <- 0.239*exp(pred)

Using the SLR model, estimated resting energy consumption is:

$\hat{y} = 281 \times {mass}^{0.74}$

Left, prediction curve with 95% confidence interval.

Kleiber’s law

How much energy do you consume on a daily basis?

Conversions:

110lb $\approx$ 50kg
180lb $\approx$ 80kg
1 kJ $\approx$ 0.239 kcal

# range of body mass
mass <- seq(50, 80)

# interval for mean log energy
pred <- predict(fit, 
                newdata = data.frame(log.mass = log(mass)), 
                interval = 'prediction')
# convert
estimates <- 0.239*exp(pred)

Using the SLR model, estimated resting energy consumption is:

$\hat{y} = 281 \times {mass}^{0.74}$

Left, prediction curve with 95% prediction interval.

Hubble constant

The Hubble constant $H$ relates a galaxy’s relative distance and velocity as $H = \frac{v}{d}$ .

# fit SLR model without an intercept
fit <- lm(distance ~ velocity - 1, 
          data = hubble)

Least squares estimate of $β = \frac{1}{H}$ :

$\hat{β} = 0.0123$

90% CI for the age of the universe:

# interval for age of universe in bn yr
km.mpc <- 3.09e19
yr.sec <- 1/(60*60*24*365)
confint(fit, level = 0.9)*km.mpc*yr.sec/1e9

              5 %     95 %
velocity 10.98235 13.12108

With 90% confidence, the universe is estimated to be between 10.98 and 13.12 billion years old.

1 / 37

Linear regression Applied Statistics for Life Sciences

Linear regression
Least squares estimation
Meadowfoam flowering
PREVEND data
PREVEND data
Lines
Linear trends in data
Linear trends in data
Correlation
Fitting lines to data
Fitting lines to data
Fitting lines to data
Measuring quality of fit
Measuring quality of fit
Least squares line
The SLR model
Least squares estimates in R
Error variability and model fit
Model specification
An alternative model
Kleiber’s law
Kleiber’s law: model interpretation
Inference in regression
Review
Review
Standard errors for the coefficients
Inference for the slope parameter
Model summary
Confidence intervals
Prediction in SLR
Intervals for the mean
Intervals for predicted values
Uncertainty bands
Kleiber’s law
Kleiber’s law
Kleiber’s law
Hubble constant