Polson–Sokolov Lab · Logistic regression

Logistic Regression Lab

Each dot is a simulated NBA game: the Vegas point spread on the x-axis, and whether the favorite won (1) or lost (0). The red S-curve is the reference model with β = 0.156. Fit the model by maximum likelihood, read off win probabilities, and move the cutoff to see the confusion matrix and the ROC curve change.

← All labs

Based on Chapter 13, Logistic Regression and Generalized Linear Models, of Polson and Sokolov, Bayes, Artificial Intelligence, and Deep Learning (vsokolov.org/html/_book).

Simulated game (jittered) Observed win rate, 2-point bins Reference model, β = 0.156 MLE fit to these games Cutoff t Chosen spread
ROC curve · fitted p̂ as the score
Confusion matrix · predict a win if p̂ > 0.50
Predicted 1Predicted 0
Actual 1
(fav. won)		–TP–FN
Actual 0
(fav. lost)		–FP–TN
Accuracy (TP + TN) / n–
Sensitivity TP / (TP + FN)–
Specificity TN / (TN + FP)–
Precision TP / (TP + FP)–
Area under the ROC curve–
From log-odds to a decision

One coefficient, three scales: log-odds, odds, probability

The model is linear on the log-odds scale. Exponentiate to get odds, then divide odds by one plus odds to get a probability. These buttons load the worked cases.

  1. Linear regression fails for a yes/no outcome

    A straight line through 0/1 data predicts values below 0 and above 1 once the spread is large enough. We want P(Y = 1 | X), which must stay in (0, 1). The logistic curve does that for any value of β₀ + β₁X.

  2. Read β on the log-odds scale

    Each extra point of spread adds 0.156 to the log-odds that the favorite wins. Equivalently, it multiplies the odds by e0.156 = 1.17. The change in probability per point is not constant: the curve is steepest at 50% and flattens near 100%.

  3. Maximum likelihood picks β

    There is no least-squares formula here. R's glm, and this page, climb the log-likelihood with Newton–Raphson steps (iteratively reweighted least squares). The standard error comes from the curvature of the log-likelihood at the top.

  4. A cutoff turns probabilities into decisions

    With no intercept and a positive spread, every p̂ is above 0.5, so the 0.5 cutoff always picks the favorite. Sensitivity is 100% and specificity is 0%. Raise the cutoff and the model starts calling upsets in close games. The ROC curve shows every cutoff at once, and its area (AUC) is the chance that a random win gets a higher p̂ than a random loss.

The logistic model
log( p / (1 − p) ) = β₀ + β₁X₁ + … + βₚXₚ
p = e^(β₀+β₁X₁+…) / (1 + e^(β₀+β₁X₁+…))
NBA, glm(favwin ~ spread − 1)
Spread 4: 0.156 × 4 = 0.62 → e^0.62 / (1 + e^0.62) = 65%
Spread 8: 0.156 × 8 = 1.25 → e^1.25 / (1 + e^1.25) = 77%
Newton–Raphson step, with p = logit⁻¹(Xβ), W = diag(p(1 − p))
β ← β + (XᵀWX)⁻¹ Xᵀ(y − p)
SE(β̂) = √ diag (XᵀWX)⁻¹
Confusion-matrix metrics
Accuracy = (TP + TN) / n
Sensitivity = TP / (TP + FN)
Specificity = TN / (TN + FP)
Precision = TP / (TP + FP)

Takeaway for students. The reference 65% and 77% come straight from plugging the spread into the fitted curve. The market looks calibrated because the curve passes through the observed win rates. Whether to call a game for the underdog is a separate business decision, set by the cutoff and the cost of each kind of mistake.

Two predictors, a straight-line boundary

Tennis: break points won by each player

Each dot is one simulated Grand Slam match, placed at the break points won by player 1 (x) and player 2 (y). The reference fit glm(Result ~ BPW.1 + BPW.2 − 1) on 943 real matches from 2013. With two predictors, the set of matches where p̂ = 0.5 is a straight line.

Player 1 won (simulated) Player 2 won (simulated) Reference boundary, p̂ = 0.5 p̂ = 0.25 and 0.75 MLE boundary Chosen match

Symmetry in the tennis coefficients

β̂₁ = 0.402 and β̂₂ = −0.418 are close to equal and opposite. Winning a break point helps player 1 about as much as losing one hurts. So the boundary is close to the 45° line BPW.2 = BPW.1, tilted slightly by the ratio 0.402 / 0.418 = 0.96.

boundary: BPW.2 = (0.402 / 0.418) × BPW.1

Effect size: a three-break gap

The reference fit approximates both coefficients by 0.40. Then a gap of three break points gives 0.40 × 3 = 1.2 on the log-odds scale, and p̂ = 77%, up from 50% at parity. That is 27 percentage points.

logit⁻¹(1.2) = e^1.2 / (1 + e^1.2) = 0.77

LinkedIn: coefficients as odds multipliers

In the LinkedIn example a top MBA has β̂ = +1.16, so it multiplies the odds of becoming VP, CXO or partner by e1.16 = 3.19. A year of experience multiplies them by e0.09 = 1.09. The ratio 1.16 / 0.09 ≈ 13 is the "worth 13 years" result.

Featureβ̂eβ̂
Top MBA+1.163.19
Function count+0.261.30
Years of experience+0.091.09
Industry count−0.220.80

Try this in class

  1. Load "Spread = 4" and walk the chain in the panel: 0.624 on the log-odds scale, odds of 1.87, probability 65%. Ask students to do spread 8 by hand before pressing the second button.
  2. Set n = 100 and press "New season" a few times. Watch β̂ bounce around 0.156 and its SE grow. Set n = 2,000 and the SE shrinks to about √(100/2000) ≈ 0.22 of its n = 100 size.
  3. At the 0.5 cutoff the model always picks the favorite. Ask why specificity is 0%. Then slide the cutoff to 0.75 and read the new sensitivity and specificity off the matrix and the ROC point.
  4. Tick "Fit an intercept too". β̂₀ lands near 0, which is what a calibrated market implies: a spread of 0 should mean a 50/50 game.
  5. In the tennis plot, click a match at 8 vs 5 and another at 3 vs 0. Both have a three-break gap. Compare their probabilities and explain the difference using β̂₁ ≠ −β̂₂.