• explores a possible linear relationship between two quantitative variables

Model

In R: lm(Response ~ Explanatory)

We assume $Y={\beta }_0+{\beta }_{1}X+ϵ$ using unknown statistics

Where:

• Y = response variable

• ${\beta }_0=$ true intercept

• ${\beta }_1=$ true slope

• X = explanatory variable

• $ϵ=$ random error term (measure of the variability in the relationship)

• $E(Y|X)={\beta }_0+{\beta }_{1}X$

  • expectation of Y at a value of X
  • theoretical mean of Y at X

The model performs 3 tests to estimate the parameters:

1. Intercept

• tests $H_0:{\beta }_0=0$ vs $H_1:{\beta }_0\ne 0$ with a t-test

2. Slope

• tests $H_0:{\beta }_1=0$ vs $H_1:{\beta }_1\ne 0$ with a t-test (does it have a slope)
• measures how strong the linear relationship between the two variables is

3. ANOVA F-test

• also tests $H_0:{\beta }_1=0$ vs $H_1:{\beta }_1\ne 0$
• same hypothesis as the second test, p-value is the same, $t^2=F$

Estimating ${\beta }_0$ and ${\beta }_1$

$\hat{Y}={\hat{\beta }}_0+\widehat{{\beta }_1}X$ (parameters/estimations)

• least squares
  • $\sum e_i^2=\sum (Y_i-\widehat{Y_i}{)}^2$
    • goal is to find the estimators that minimize $\sum e_i^2$ (residual)
  • $e_i=Y_i-{\hat{Y}}_i$
    • e = residual
  • $\widehat{{\beta }_0}=\overline{Y}-\widehat{{\beta }_1}\overline{X}$
  • ${\hat{\beta }}_1=\frac{\sum (X_i-\overline{X})(Y_i-\overline{Y})}{\sum (X_i-\overline{X}{)}^2}$

Confidence intervals for the two parameters

Inference

we assume that $ϵ\sim N(0,{\sigma }^2)$ and that the $ϵ$ terms are independent

Testing $H_0:{\beta }_1=0$

i.e. there is not relationship between X and Y

• true slope = 0, Y not related to X then
  If we assume $ϵ\sim N(0,{\sigma }^2)$ and $H_0:{\beta }_1=0$
• $\frac{{\hat{\beta }}_1-0}{SE({\hat{\beta }}_1)}\sim t(n-2)$ (t statistic)
  • $SE(\widehat{{\beta }_1})=\frac{S}{\sqrt{\sum (x_i-\overline{x}{)}^2}}$
  • $S^2=\frac{\sum (y_i-{\hat{y}}_i{)}^2}{n-2}$
    • n-2 because two parameters are being estimated(${\beta }_0$ and ${\beta }_1$)

Measuring the correlation

• measure of the strength of the linear relationship
• SLR is not made to measure correlation between non-linear relationships
  • 
  • last two use different techniques
  • (always plot your data beforehand)
    r = Pearson Correlation Coefficient
    $R^2$ = Coefficient of Determination