Polson–Sokolov Lab · Simple linear regression

Regression Model Lab

Each point is drawn from its own normal distribution. The center of that normal sits on the line a·x + b, and its spread σ is the same at every x. The red curves show those distributions at a few values of x.

← All labs

Based on Chapter 12, Linear Regression, of Polson and Sokolov, Bayes, Artificial Intelligence, and Deep Learning (vsokolov.org/html/_book).

Data point (xᵢ, yᵢ) True mean line a·x + b N(a·x + b, σ²) at that x ±2σ band (95%) Least-squares fit

The line is a line of means

For each x, the model says y is random, and its average is a·x + b. The line does not pass through the points. It passes through the centers of the red curves.

E[y | x] = a·x + b

The noise is normal with one σ

Every red curve has the same width. That is the constant-variance assumption (homoscedasticity). The errors are also independent of each other and of x.

εᵢ = yᵢ − (a·xᵢ + b) ~ N(0, σ²), iid

Three ways to write the same model

These all say the same thing. The last form is the one the picture shows: a normal distribution for each y, centered on the line.

yᵢ = a·xᵢ + b + εᵢ
yᵢ | xᵢ ~ N(a·xᵢ + b, σ²)

Try this in class

  1. Drag σ down to 0.1. The points collapse onto the line and the red curves become spikes. Drag it up to 4 and the same points spread out, but the line of centers does not move.
  2. Look at the highlighted points inside one vertical slice. Their spread matches the red curve next to them. Each slice is its own small normal sample.
  3. Set n = 20 and press "New sample" several times. The green fitted line wobbles around the blue true line. Set n = 1,000 and it barely moves.
  4. Count the points outside the shaded ±2σ band. About 1 in 20 should fall outside it. Check the bottom rows of the table.
  5. Change the slope to a negative value. The curves move with the line, and their shape stays the same.