Confidence intervals

Constructing interval estimates to attain a specified coverage rate

Today’s agenda

[lecture] \(t\) confidence intervals for the mean
[lab] computing and interpreting confidence intervals

From last time

Under simple random sampling:

the sample mean \(\bar{x}\) provides a good point estimate of the population mean \(\mu\)
its estimated sampling variability is given by the standard error \(SE(\bar{x}) = \frac{s_x}{\sqrt{n}} = \frac{\text{sample SD}}{\sqrt{\text{sample size}}}\)

mean	sd	n	se
5.043	1.075	3179	0.01906

The mean total HDL cholesterol among the U.S. adult population is estimated to be 5.043 mmol/L (SE 0.0191).

Interval estimation

A common interval estimate for the population mean is: \[\bar{x} \pm 2\times SE(\bar{x}) \qquad\text{where}\quad SE(\bar{x}) = \left(\frac{s_x}{\sqrt{n}}\right)\]

The mean total cholesterol among U.S. adults is estimated to be between 5.005 and 5.081 mmol/L.

Two related questions:

In what sense are these values “plausible”?
Where did the number 2 come from?

The \(t\) model

Consider the statistic:

\[ T = \frac{\bar{x} - \mu}{s_x/\sqrt{n}} \qquad\left(\frac{\text{estimation error}}{\text{standard error}}\right) \]

The sampling distribution of \(T\) is well-approximated by a \(t_{n - 1}\) model whenever either:

the population distribution is symmetric and unimodal

the sample size is not too small

\(t\) model interpretation

The area under the density curve between any two values \((a, b)\) gives the proportion of random samples for which \(a < T < b\).

\[(\text{proportion of area between } a, b) = (\text{proportion of samples where } a < T < b)\]

For example:

for 50% of samples, \(T < 0\)

# area less than 0
pt(0, df = 20 - 1)

[1] 0.5

written as \(P(T < 0) = 0.5\)

\(t\) model interpretation

The area under the density curve between any two values \((a, b)\) gives the proportion of random samples for which \(a < T < b\).

\[(\text{proportion of area between } a, b) = (\text{proportion of samples where } a < T < b)\]

For example:

for 83.5% of samples, \(T < 1\)

# area less than 1
pt(1, df = 20 - 1)

[1] 0.8350616

written as \(P(T < 1) = 0.835\)

\(t\) model interpretation

The area under the density curve between any two values \((a, b)\) gives the proportion of random samples for which \(a < T < b\).

\[(\text{proportion of area between } a, b) = (\text{proportion of samples where } a < T < b)\]

For example:

for 97% of samples, \(T < 2\)

# area less than 2
pt(2, df = 20 - 1)

[1] 0.969999

written as \(P(T < 2) = 0.97\)

\(t\) model interpretation

The area under the density curve between any two values \((a, b)\) gives the proportion of random samples for which \(a < T < b\).

\[(\text{proportion of area between } a, b) = (\text{proportion of samples where } a < T < b)\]

For example:

for 3% of samples, \(T > 2\)

# area greater than 2
pt(2, df = 20 - 1, lower.tail = F)

[1] 0.03000102

notice: \[ \begin{align*} P(T > 2) &= 1 - P(T < 2) \\ (0.03) &= 1 - (0.97) \end{align*} \]

\(t\) model interpretation

The area under the density curve between any two values \((a, b)\) gives the proportion of random samples for which \(a < T < b\).

\[(\text{proportion of area between } a, b) = (\text{proportion of samples where } a < T < b)\]

For example:

for 13.5% of samples, \(1 < T < 2\)

# area between 1 and 2
pt(2, df = 20 - 1) - pt(1, df = 20 - 1)

[1] 0.1349374

notice: \[ \begin{align*} P(1 < T < 2) &= P(T < 2) - P(T < 1) \\ (0.135) &= (0.97) - (0.835) \end{align*} \]

\(t\) model interpretation

The area under the density curve between any two values \((a, b)\) gives the proportion of random samples for which \(a < T < b\).

\[(\text{proportion of area between } a, b) = (\text{proportion of samples where } a < T < b)\]

For example:

for 94% of samples, \(-2 < T < 2\)

# area between 1 and 2
pt(2, df = 20 - 1) - pt(-2, df = 20 - 1)

[1] 0.939998

written \(P(-2 < T < 2) = 0.94\)

A closer look at interval construction

So where did that 2 come from in the margin of error for our interval estimate?

\[ \bar{x} \pm \color{blue}{2}\times SE(\bar{x}) \]

Well:

\[ \begin{align*} 0.94 &= P(-\color{blue}{2} < T < \color{blue}{2}) \\ &= P\left(-\color{blue}{2} < \frac{\bar{x} - \mu}{s_x/\sqrt{n}} < \color{blue}{2}\right) \\ &= P(\underbrace{\bar{x} - \color{blue}{2}\times SE(\bar{x}) < \mu < \bar{x} + \color{blue}{2}\times SE(\bar{x})}_{\text{interval covers population mean}}) \end{align*} \]

For 94% of all random samples, the interval covers the population mean.

So the number 2 determines the proportion of samples for which the interval covers the mean, known as its coverage.

Effect of sample size

The sample size determines the exact shape of the \(t\) model through its ‘degrees of freedom’ \(n - 1\). This changes the areas slightly.

The exact coverage quickly converges to just over 95% as the sample size increases.

n	coverage
4	0.8607
8	0.9144
16	0.9361
32	0.9457
64	0.9502
128	0.9524
256	0.9534

Changing the coverage

Consider a slightly more general expression for an interval for the mean:

\[ \bar{x} \pm c\times SE(\bar{x}) \]

The number \(c\) is called a critical value. It determines the coverage.

larger \(c\) \(\longrightarrow\) higher coverage
smaller \(c\) \(\longrightarrow\) lower coverage

The so-called “empirical rule” is that:

\(c = 1 \longrightarrow\) approximately 68% coverage
\(c = 2 \longrightarrow\) approximately 95% coverage
\(c = 3 \longrightarrow\) approximately 99.7% coverage

Interpreting critical values

\[ P(\color{#FF6459}{-2 < T < 2}) = 1 - 2\times P(\color{blue}{T > 2}) \]

Look at how the areas add up so that: \[ P(\color{blue}{T > 2}) = 0.03 \] Moreover: \[ P(T < 2) = 1 - 0.03 = 0.97 \]

So the critical value 2 is actually the 97th percentile of the sampling distribution of \(T\).

also called the 0.97 “quantile”
(percentiles expressed in proportions are called quantiles)

Exact coverage using \(t\) quantiles

To engineer an interval with a specific coverage, use the \(p\)th quantile where:

\[p = \left[1 - \left(\frac{1 - \text{coverage}}{2}\right)\right]\] In R:

# coverage 95% using t quantile
coverage <- 0.95
q.val <- 1 - (1 - coverage)/2
crit.val <- qt(q.val, df = 20 - 1)
crit.val

[1] 2.093024

The effect of increasing/decreasing coverage on the quantile is:

increase coverage \(\longrightarrow\) larger quantile \(\longrightarrow\) wider interval
decrease coverage \(\longrightarrow\) smaller quantile \(\longrightarrow\) narrower interval

Contrasting coverage with precision

Precision refers to how wide or narrow the interval is.

Precision depends on every component of the margin of error:

critical value used
sample size
variability of values

By contrast, coverage depends only on the critical value used.

Confidence intervals

Interval estimates constructed to achieve a specified coverage are called “confidence intervals”; the coverage is interpreted and reported as a “confidence level”.

# ingredients
cholesterol.mean <- mean(cholesterol)
cholesterol.sd <- sd(cholesterol)
cholesterol.n <- length(cholesterol)
cholesterol.se <- cholesterol.sd/sqrt(cholesterol.n)
crit.val <- qt(1 - (1 - 0.95)/2, df = cholesterol.n - 1)

# interval
cholesterol.mean + c(-1, 1)*crit.val*cholesterol.se

[1] 5.005566 5.080310

With 95% confidence, the mean total cholesterol among U.S. adults is estimated to be between 5.0056 and 5.0803 mmol/L.

The general formula for a confidence interval for the population mean is

\[ \bar{x} \pm c\times SE(\bar{x}) \]

where \(c\) is a critical value, obtained as a quantile of the \(t_{n - 1}\) model and chosen to ensure a specific coverage.

Recap

The “common” interval estimate for the mean is actually an approximate 95% confidence interval:

\[ \bar{x} \pm 2 \times SE(\bar{x}) \]

captures the population mean \(\mu\) for roughly 95% of random samples
replacing 2 with a \(t_{n - 1}\) quantile allows the analyst to adjust coverage
the \(t_{n - 1}\) model is an approximation for the sampling distribution of \(\frac{\bar{x} - \mu}{SE(\bar{x})}\)
- approximation improves with increasing sample size or symmetry
- usually good quality except in “extreme” situations

Interval interpretation:

With [XX]% confidence, the mean [population parameter] is estimated to be between [lower bound] and [upper bound] [units].

Extras

Simulation of coverage

Artificially simulating a large number of intervals provides an empirical approximation of coverage.

at right, 200 intervals
94% cover the population mean (vertical dashed line)
pretty close to nominal coverage level 95%

This is also a handy way to remember the proper interpretation:

If I made a lot of intervals from independent samples, 95% of them would ‘get it right’.