AII 600 midterm prep

Problem bank. Weeks 1–6. Fall 2026

Week 8, 14 October, 2.5 hours, pen and paper. One double-sided cheat sheet, printed or handwritten. A calculator is fine. No laptop, phone, watch, or assistant.

The exam checks that you can use the basic concepts without an assistant, and that you can think about the modelling assumptions those concepts rest on.

This prep document covers Foundations, Estimation, and linear and logistic regression from Patterns. The problems are from the textbook and Labs 1–7. Arithmetic stays small enough for paper. An assumption item may be a printed plot. Name the pattern, name the assumption that failed, and say which way a number you already computed would move.

Week 7 (testing, predictive values, expected value) is also on the midterm. That section will be added after 7 October.

Reading guide

The textbook is Bayes, Artificial Intelligence, and Deep Learning (Polson and Sokolov). Read the sections named below. The exam is pen and paper, so skip every Computing block and any derivation whose only job is to justify a formula you will not write. The labs and this bank are how you check that the reading stuck.

You do not need every example. You need the move in the table in the next section: name the concept, write the one-line mechanism, finish the arithmetic, say what it means, and say which way it moves if an assumption changes.

Week 7 is on the exam. Those problems will be added after 7 October. The reading for it is listed now so you can start.

Weeks 1–3. Foundations

Slides: Foundations. Labs 1, 2, and 3. Lecture through week 3 stops at the Normal.

Chapter 1. Probability and Uncertainty. Read from the opening through Normal: odds as a probability, the hitting streak, random variables, expectation and variance, covariance and linear combinations (Tortoise and Hare), joint / marginal / conditional, independence, Dutch Book, binomial / Poisson / Normal including the 1987 crash. Skip CDFs, the inverse-CDF sampling section, Kolmogorov’s strong law, and the Computing exercises.

Chapter 2. Bayes’ Rule. Read Bayes’ rule and the law of total probability, the odds form, the prosecutor’s fallacy, the island, Nakamura, Sally Clark, Wald, sensitivity and specificity, Alice’s mammogram. Skip Naive Bayes as a spam classifier, the Obama election simulation, and the Computing exercises.

Week 4. Estimation

Slides: Estimation. Lab 4.

Chapter 1, CDFs (discrete and continuous), the weak law, and the idea of a sampling distribution.

Chapter 3. Bayesian Learning, only the conjugate stories that show up on paper: the coin-toss posterior, beta-binomial, Poisson for counts, Normal with unknown mean. Skip Poisson-gamma, exchangeability as a theorem, mixtures, MCMC, the computational bridge, and Multivariate Normal.

Chapter 5. A/B Testing, through confidence intervals (including the \(z\)-score and the 1987 crash as a tail, which you already met in Chapter 1). Stop before multiple testing, RCTs, and split-plot designs.

Chapter 11. Pattern Matching, Model Estimation through MSE vs MAE: likelihood, MLE for a Bernoulli, square loss versus absolute loss. Stop at penalized likelihood.

Weeks 5–6. Patterns

Slides: Patterns. Lab 5.

Chapter 11, the rest through Prediction Accuracy (regression and classification metrics). Skip Complex Functions if it is only neural-net notation.

Chapter 12. Linear Regression, through residual analysis, log and power transforms (mammal brain/body, Apple), categorical variables and dummy encoding, and the golf earnings example. Stop at quantile regression and Bayes regression. Skip Orange Juice and Newfood as computing.

Chapter 13. Logistic Regression, through model fitting, log-odds, and the confusion matrix. Skim the ROC if you want a picture of TPR against FPR; you will not be asked to draw one. Stop before the kernel trick, GLMs, and Polya-Gamma.

The Patterns lecture also points at Chapters 14 and 16. Those are not on the midterm.

Week 7. Testing, predictive values, expected value

Slides: the first half of Decisions, 7 October. Labs 6 and 7.

You already have the arithmetic from Chapter 2 (sensitivity, specificity, \(P(D\mid T)\)). Revisit it. Week 7 names \(P(D\mid T)\) a predictive value.

Chapter 5, Hypothesis Testing and Confidence Intervals, now as a decision: a significant difference is not yet a launch. Still stop before multiple testing.

Expected value as a decision is Chapter 1, Expectation and Variance, including Expectation and Strategy in the exercises (the oil drills). Full utility, paradoxes, and Kelly are Chapter 4, week 9, after the midterm.

Not on the midterm

Do not spend midterm time on Chapter 4 (utility, Ellsberg, Allais, Kelly), Chapter 6, Chapters 7–10, trees, ridge, or anything in Part 3 (deep learning). Chapter 3 from Poisson-gamma, Chapter 12 from quantile regression, and Chapter 13 from the kernel trick are the same: later.

What you have to be able to do

Each row is a concept and the use of it. If you cannot do the use on paper, on a problem you have not seen, you do not have the concept yet.

Concept What using it looks like
Complement, mutually exclusive, independence \(P(\text{at least one})\) without adding; \(P(A\text{ or }B)=P(A)+P(B)-P(A\text{ and }B)\); when multiplication is good enough and when it is not
Joint, marginal, conditional Finish a table; \(P(Y\mid X)\) vs \(P(X\mid Y)\); independence as \(P(Y\mid X)=P(Y)\)
Bayes Prior, likelihood, posterior. \(P(D\mid T)\) is not \(P(T\mid D)\)
Linear combination \(E(aX+bY)\) always; variance needs the covariance
Binomial vs Poisson Name the count from whether there is a list of trials; write \(P(X=k)\) for small \(n\)
Normal \(z=(x-\mu)/\sigma\); a quantile; a left-tail probability
Estimate Method of moments and MLE for Bernoulli, Poisson, and Normal (the mean) are all \(\bar y\); what that number is
CLT \(\mathrm{sd}(\bar X)=\sigma/\sqrt{n}\); a 95% interval for a proportion
OLS Slope as a rate; \(\hat y\) at a new \(x\); a residual plot as the check, not \(R^2\)
Dummy variables \(k\) levels, \(k-1\) columns; the reference is a choice
Logistic \(p=\sigma(x^T\beta)\); \(\beta\) is log-odds, not a change in \(p\); a threshold is a loss
Modelling assumption Name it; why it was used; what would make it false; which way the number moves

An assumption item is those four sentences. “Independence might not hold” is not an answer.

  1. Name the assumption (independence, iid, Poisson mean \(=\) variance, linear mean, constant variance, Normal tails, \(P(\text{data}\mid H)\) known, 0.5 threshold, who is in the sample).
  2. Why it was used (independence makes the computation easier; iid gives \(\hat\lambda=\bar y\); Normal errors make OLS the MLE; the estimate is whatever this sample produced).
  3. What in this problem would make it false. For an estimate: is this sample the population you are solving for?
  4. Which way the reported number moves, or whether it is still good enough for the purpose. A larger \(n\) of the same sample does not fix bias: the centre is in the wrong place. Sally Clark vs a clean shift is the other template: same assumption, two purposes, two verdicts.

1. Probability, independence, expectation

From Foundations and Lab 1. Book Chapter 1.

A probability is a number you will bet at, and it has to add to one when exactly one of the events happens. Independence lets you multiply; that is a modelling choice, not a fact about the world. Expectation always plugs in linearly; variance does not, unless the covariance is zero.

You should be able to: write \(P(\text{at least one})\) from the complement (do not add the single-trial probabilities); write \(P(A\text{ or }B)=P(A)+P(B)-P(A\text{ and }B)\), and drop the last term only when the events are mutually exclusive; say when the product is good enough (a clean shift) and when it is not (Sally Clark); compute \(E(aX+bY)\) and \(\mathrm{Var}(aX+bY)\) from means, variances, and one covariance.

Problem 1. A clean shift

A warehouse scanner misses a package with probability \(0.08\). Eight packages pass in a row. Call the shift clean if none are missed.

  1. Assuming misses are independent, what is \(P(\text{clean shift})\)?

  2. \(P(\text{at least one miss})\) from (a). Do not add \(0.08\) eight times.

  3. Give one reason consecutive scans are not independent.

  4. Why might you still use the independence model for this probability, and name one example from class where multiplying as if independent is not good enough.

  1. \(P(\text{clean})=(0.92)^8\approx 0.513\).

  2. \(1-(0.92)^8\approx 0.487\). Adding \(8\times 0.08=0.64\) double-counts shifts with two or more misses.

  3. A jam, a bad label batch, or a lighting change hits several packages in a row. One miss can also mean the next packages are rushed.

  4. Independence gives a number we can compute from one rate, and for “how unusual is a clean shift?” that number is close enough. Sally Clark is the counterexample: treating two SIDS deaths as independent turns \(1/850{,}000\) into \(1/73{,}000{,}000\) and makes the mother look guilty.

Problem 1B. \(P(A\text{ or }B)\)

Tonight’s email goes to people who opened last week (\(P(A)=0.12\)) or who clicked last week (\(P(B)=0.05\)). Both: \(P(A\text{ and }B)=0.03\).

  1. Are \(A\) and \(B\) independent? Compare \(P(A\text{ and }B)\) to \(P(A)P(B)\).

  2. \(P(A\text{ or }B)\).

  3. If \(A\) and \(B\) were mutually exclusive, what would \(P(A\text{ or }B)\) be? Why is that the wrong number here?

  1. No. \(0.12\cdot 0.05=0.006\), not \(0.03\). Independence is the product.

  2. \(P(A\text{ or }B)=0.12+0.05-0.03=0.14\).

  3. \(0.12+0.05=0.17\). Mutually exclusive means both cannot happen, so you would add and stop. Here 3% did both, so you have to subtract.

Problem 2. Dutch Book

Book Chapter 1, Dutch Book and Coherence.

A racetrack analyst quotes \(P(\text{Alpha})=0.50\), \(P(\text{Beta})=0.35\), \(P(\text{Gamma})=0.25\) for a three-horse race. Exactly one horse wins.

  1. Which axiom do the quotes violate?

  2. Should you buy or sell equal contracts on all three, and what is the sure profit on a \(\pm 100\) stake?

  3. She revises Gamma to \(0.15\). Is the book now coherent?

  1. Exactly one horse wins, so the three quotes have to add to \(1\) (coherence; no Dutch book). They sum to \(1.10\).

  2. Sell. Take \(S=-100\): collect \(110\) up front, pay \(100\) on whichever horse wins, profit \(\$10\) in every world.

  3. Yes. The prices now sum to \(1\), so there is no sure profit from buying or selling equal amounts on all three.

Problem 3. A mix of two salespeople

Lab 2 Exercise 2; in-class stop.

You split Saturday’s leads: half to Tortoise, half to Hare. \(P=\tfrac12 T+\tfrac12 H\) is the average per rep. \(E(T)=E(H)=1.5\), \(\mathrm{Var}(T)=0.25\), \(\mathrm{Var}(H)=2.25\), \(\mathrm{Cov}(T,H)=-0.75\).

Compute \(E(P)\) and \(\mathrm{Var}(P)\). Compare \(\mathrm{Var}(P)\) to \(\mathrm{Var}(T)\). What did the covariance do?

\(E(P)=1.5\). \[ \mathrm{Var}(P)=\tfrac14\cdot 0.25+\tfrac14\cdot 2.25+2\cdot\tfrac12\cdot\tfrac12\cdot(-0.75)=0.0625+0.5625-0.375=0.25. \] Same mean and same variance as Tortoise alone. Without the covariance you would have \(0.625\), worse than Tortoise. Perfect negative correlation is what cancels Hare’s noise.

Problem 3B. Oil: expected payoff is not the decision

Book Chapter 1, Expectation and Strategy.

An oil company can use a standard drill or a more expensive horizontal drill. Chance of a small / moderate / large find: \(0.2\), \(0.5\), \(0.3\). Payoffs in $millions:

small moderate large
Standard 20 30 40
Horizontal \(-20\) 40 80

  1. Expected payoff and variance for each strategy. Which maximises expected payoff?

  2. If a \(\$20\) million loss is a problem for the firm, which do you choose, and what assumption about “maximise \(E(X)\)” just failed?

  3. A geological study would reveal the size of the find before you choose the drill. What is the most you should pay for that study? What did you assume about the study?

  1. Standard: \(E=31\), \(\mathrm{Var}=49\). Horizontal: \(E=40\), \(\mathrm{Var}=1200\). Horizontal maximises expected payoff.

  2. Standard. Maximising \(E(X)\) ranks a sure 31 and a 40-with-a-possible-loss-of-20 only by the mean. A \(\$20\) million loss is a different cost.

  3. With the study you pick standard on small (\(20\) beats \(-20\)), horizontal otherwise: \(0.2\cdot 20+0.5\cdot 40+0.3\cdot 80=48\). Willing to pay \(48-40=8\) million. You assumed the study is perfect, and that you will actually switch the drill once you see the result.

Problem 4. True or false

Book Chapter 1, True/False Variance and Expectation. Answer each in one sentence.

  1. If \(\mathrm{Cov}(X,Y)=0\), then \(X\) and \(Y\) are independent.

  2. If \(X\) and \(Y\) are independent, \(\mathrm{Var}(2X-Y)=2\mathrm{Var}(X)-\mathrm{Var}(Y)\).

  3. \(E(X+Y)=E(X)+E(Y)\) only if \(X\) and \(Y\) are independent.

  4. \(E(\bar X)=\mu\).

  5. Correlation lies in \([0,1]\).

  1. False. Uncorrelated does not imply independent, except in special cases (joint Normal).

  2. False. \(\mathrm{Var}(2X-Y)=4\mathrm{Var}(X)+\mathrm{Var}(Y)\) under independence.

  3. False. Linearity of expectation does not need independence.

  4. True.

  5. False. Correlation is in \([-1,1]\).

2. Joint tables, Bayes, evidence

Lab 2 and the second half of Foundations. Book Chapter 2.

A joint table is three distributions in one grid: joint, row/column sums (marginals), and ratios (conditionals). Bayes is the same ratio with names: prior, likelihood, posterior. \(P(D\mid T)\) is not \(P(T\mid D)\). The prosecutor’s fallacy is reporting the second and calling it the first.

You should be able to: finish a table and read \(P(Y\mid X)\) and \(P(X\mid Y)\) from different denominators; check independence as \(P(Y\mid X)=P(Y)\); update a two-type prior with a binomial likelihood; turn a 2-by-2 of counts into TPR and into \(P(D\mid T)\), and say which number Alice needs after a positive.

Problem 5. Joint table

Book Chapter 1, Joint Distributions. Same move as Lab 2 Exercise 1.

A credit-card book of \(10{,}000\) customers:

bal  def 0 1
1 (low) 8,940 64
2 651 136
3 (high) 76 133

  1. Marginal distribution of def.

  2. Conditional distribution of bal given def \(=1\).

  3. Predict status for a customer with a high balance. Compare to low balance.

  4. A product team wants to pitch only to bal \(=3\) because “that is where the defaults are.” Using the numbers: who does that mix up, and what would you check before restricting the book?

  5. Is def independent of bal? Compare \(P(\mathrm{def}=1\mid\mathrm{bal}=1)\) to \(P(\mathrm{def}=1)\).

  1. \(P(\mathrm{def}=0)=0.9667\), \(P(\mathrm{def}=1)=0.0333\).

  2. Given default: \(64/333\approx 0.192\), \(136/333\approx 0.408\), \(133/333\approx 0.399\).

  3. \(P(\mathrm{def}=1\mid\mathrm{bal}=3)=133/209\approx 0.636\). \(P(\mathrm{def}=1\mid\mathrm{bal}=1)=64/9004\approx 0.007\). About ninety times more likely at high balance.

  4. The team is mixing \(P(\mathrm{bal}\mid\mathrm{def})\) with \(P(\mathrm{def}\mid\mathrm{bal})\). Most defaults are not in the high-balance cell in count terms (64 and 136 sit elsewhere); the rate is high at bal \(=3\). Restricting the pitch to high balance misses current customers in bands 1 and 2 and bothers a slice that is only 64% default, not “the defaulters.” Check the purpose: collections vs a wellbeing product are different losses.

  5. No. \(0.007\) is not \(0.033\). Independence would be \(P(\mathrm{def}=1\mid\mathrm{bal})=P(\mathrm{def}=1)\) in every row.

Problem 6. Manchester

Book Chapter 2, Manchester.

Someone in a bar is supporting Manchester United. \(P(\text{born within 20 miles})=1/20\). If born there, they support United with probability \(7/10\); otherwise \(1/10\). What is \(P(\text{born there}\mid\text{supports United})\)?

\[ P(B\mid U)=\frac{(7/10)(1/20)}{(7/10)(1/20)+(1/10)(19/20)}=\frac{0.035}{0.130}\approx 0.269. \] Most United shirts in a typical bar were not born in Manchester.

Problem 7. Tesla chip

Book Chapter 2, Tesla Chip.

Thirty percent of chips from Matsushita (3% defective), 20% Philips (5%), 50% Hitachi (4%). A chip is defective. Posterior on each manufacturer?

\(P(D)=0.30\cdot 0.03+0.20\cdot 0.05+0.50\cdot 0.04=0.039\).

\(P(\text{Matsushita}\mid D)=9/39\), \(P(\text{Philips}\mid D)=10/39\), \(P(\text{Hitachi}\mid D)=20/39\).

Problem 7B. You did not need the incidence

Book Chapter 2, Lung Cancer.

Women who smoke are about 13 times more likely to develop lung cancer than women who do not. In 2016, \(17.9\%\) of women smoked.

  1. \(P(\text{smoker}\mid\text{lung cancer})\). Let \(y=0.179\) be the smoking rate and \(x\) the rate of lung cancer among nonsmokers. Show that \(x\) cancels.

  2. Most people guess a number well below \(74\%\). What did they treat as the answer instead of \(P(\text{smoker}\mid\text{cancer})\)?

  3. The \(13\) is a relative risk from a population. What would make it the wrong likelihood ratio for this woman?

  1. Smokers with cancer are proportional to \(13xy\), nonsmokers with cancer to \(x(1-y)\). Then \[ P(S\mid C)=\frac{13y}{13y+(1-y)}=\frac{13\cdot 0.179}{13\cdot 0.179+0.821}\approx 0.74. \] The absolute risk \(x\) cancels. You needed the relative risk and the smoking rate, not \(P(\text{cancer})\).

  2. They answered \(P(\text{cancer}\mid\text{smoker})\), or the smoking rate \(17.9\%\), or “\(13\) times more likely so it must be nearly certain.” The question is the other conditional.

  3. This woman is not a draw from the CDC table: age, asbestos, a family history, or a clinic that already selected for respiratory disease. Then the \(13\) is someone else’s likelihood ratio, and \(74\%\) is not her posterior.

Problem 8. The island

Lab 2 Exercise 4; Book Chapter 2, Island Problem.

A man is arrested on an island of \(1{,}001\) people. The crime-scene trait occurs in \(1\) in \(10{,}000\) people; it matches him. Uniform prior. \(P(\text{match}\mid\text{guilty})=1\).

The prosecutor tells the jury: “The match probability is 1 in 10,000, so there is a 99.99% chance he is guilty.”

  1. Prior odds he is innocent.

  2. Posterior odds of innocence, and \(P(\text{guilty}\mid\text{match})\).

  3. One sentence to the jury.

  1. \(O(I)=1000/1=1000\).

  2. Bayes factor for innocence is \(10^{-4}\), so \(O(I\mid E)=0.1\). Then \(P(I\mid E)=0.1/1.1\approx 0.091\), so \(P(G\mid E)\approx 0.91\).

  3. The prosecutor reported \(1-P(\text{match}\mid\text{innocent})\), not \(P(\text{guilty}\mid\text{match})\). Expected matches among the other 1,000 people: \(0.1\). That \(0.1\) is why the posterior is \(0.91\), not \(0.9999\), and not a conviction.

Problem 8B. A match is not a conviction

Book Chapter 2, Forensic Match.

\(P(E\mid G)=0.90\), \(P(E\mid\text{innocent})=0.02\). Prior \(P(G)=1/200\).

  1. Bayes factor for a match, posterior odds, and \(P(G\mid E)\).

  2. A second, conditionally independent eyewitness has Bayes factor \(3\). Posterior probability after both?

  3. Is \(40\%\) enough to convict “beyond a reasonable doubt”? What assumption about the eyewitness would make the \(3\) too large?

  1. \(\mathrm{BF}=0.90/0.02=45\). Prior odds \(1/199\). Posterior odds \(45/199\approx 0.226\), so \(P(G\mid E)=45/244\approx 0.184\).

  2. \(O=3\times 45/199=135/199\), \(P=135/334\approx 0.404\).

  3. No. A \(45\times\) match and a \(3\times\) witness still leave the prior in charge. If the eyewitness is more likely to pick the person who already matched (not conditionally independent given guilt), you multiplied a Bayes factor you did not have. Same machine as the island and Nakamura.

Problem 9. A closer

Lab 2 Exercise 6.

Half the bootcamp convert 70% of qualified leads, half convert 30%. First week: 3 closes out of 4. Equal priors.

  1. Likelihood ratio \(P(\text{3 of 4}\mid 70\%)/P(\text{3 of 4}\mid 30\%)\).

  2. \(P(\text{70\% closer}\mid\text{3 of 4})\).

  3. One sentence to the sales manager: put them on the enterprise book, or is four leads still “too small to say”?

  1. The \(\binom{4}{3}\) cancels: \(0.7^2/0.3^2\approx 5.44\).

  2. \(5.44/(1+5.44)\approx 0.845\).

  3. Four leads moved you from 50% to 85%. That is not “too small to say.” The enterprise book is a 15% chance you just promoted the 30% closer. Say that, then decide.

Problem 9B. Likelihoods you treated as known

Lab 2 Exercise 7, written part. The arithmetic is in the lab; this is the assumption.

New products have high sales 8% of the time. If sales are high, a focus group is positive 70% of the time; if low, 15%. You treat those two numbers as \(P(\text{Pos}\mid H)\) and \(P(\text{Pos}\mid L)\), then report \(P(H\mid\text{Pos})\approx 0.29\).

  1. What two distinct things can go wrong when you take focus-group reactions as \(P(\text{Pos}\mid H)\)? Who is in the room is one class of problem; what people say versus what they do is another.

  2. If the people who show up are already more enthusiastic than the market, which of \(P(\text{Pos}\mid H)\) and \(P(\text{Pos}\mid L)\) moves, and what happens to \(P(H\mid\text{Pos})\)?

  3. The 8% is the number to beat. Is 29% a launch? What would you need to know about the cost of a failed launch versus a missed hit?

  1. The room is not the market: volunteers, staff, or a city that already likes the category. And a stated “I would buy” is not a sale. Either way you have estimated a different likelihood than the one the posterior needs.

  2. Both likelihoods of a positive reaction go up if the room is more enthusiastic than the market. The likelihood ratio \(P(\text{Pos}\mid H)/P(\text{Pos}\mid L)\) can rise or fall; if the low-sales group becomes almost as positive as the high-sales group, the Bayes factor shrinks and \(P(H\mid\text{Pos})\) falls toward the 8% prior. The usual failure is that a positive stops distinguishing.

  3. 29% is not a launch on its own. A failed launch that is costly relative to a missed hit wants a higher posterior than 29%; a cheap test with a large upside can live with 29%. The 8% is the prior, not the decision.

Problem 10. Mammogram

Lab 2 Exercise 8; Alice in lecture.

Prevalence 1%, detection rate 80%, false-positive rate 10%. In 1,000 people like Alice: 10 have cancer, 8 true positives, 99 false positives, 2 false negatives, 891 true negatives.

  1. TPR, FNR, FPR, TNR from the 2-by-2.

  2. \(P(\text{cancer}\mid\text{positive})\).

  3. A radiologist says “the test is 80% accurate.” Which of your four rates is that, and why is it not the number Alice needs after a positive?

  1. \(\mathrm{TPR}=8/10=0.8\), \(\mathrm{FNR}=0.2\), \(\mathrm{FPR}=99/990=0.1\), \(\mathrm{TNR}=0.9\).

  2. \(8/(8+99)=8/107\approx 0.075\).

  3. “80% accurate” is sensitivity, \(P(T\mid D)\). Alice needs \(P(D\mid T)\approx 7.5\%\). Most positives are the 99 false alarms from the 990 without cancer.

Problem 10B. Should Nvidia use the test?

Book Chapter 2, Sensitivity and Specificity.

A chip is defective with probability \(0.1\%\). The test always flags a defective chip, and flags a good chip with probability \(1\%\).

  1. Sensitivity and specificity.

  2. \(P(\text{defective}\mid\text{flagged})\) and \(P(\text{flagged})\).

  3. Advise Nvidia whether to scrap every flagged chip. What loss did you just use, and what would flip the advice?

  1. Sensitivity \(1\), specificity \(0.99\).

  2. \(P(D\mid T)=0.001/(0.001+0.01\cdot 0.999)\approx 0.091\). About \(1.1\%\) of chips are flagged.

  3. Do not scrap them all: \(91\%\) of flags are good chips. That advice assumes a good chip is costly to throw away relative to a defective one that ships. Flip it if a defective chip in the wild is catastrophic (a recall, a car) compared with the cost of a spare: then flag-and-retest, or a second test on the \(1.1\%\), rather than “the sensitivity is 100% so ship only the negatives.”

Problem 10C. Eight tests, one rider

Book Chapter 2, Floyd Landis.

Landis gave eight urine samples. Specificity \(95\%\). Treat the eight tests as independent given innocence.

  1. \(P(\text{all eight negative}\mid\text{innocent})\). Then \(P(\text{at least one positive}\mid\text{innocent})\).

  2. What did independence do, and why might consecutive samples from one rider not be independent?

  3. In court, is “at least one positive” under independence a probability of guilt? What else do you need?

  1. \(0.95^8\approx 0.663\), so \(P(\text{at least one FP}\mid\text{innocent})=1-0.95^8\approx 0.337\).

  2. Independence lets you raise \(0.95\) to the 8th. One lab, one rider, one stage: a contaminated kit or a lab shift hits several samples together. Then \(P(\text{at least one FP})\) is closer to \(0.05\) than to \(0.34\).

  3. No. That is \(P(E\mid\text{innocent})\), the prosecutor’s number. You still need a prior and \(P(E\mid\text{guilty})\). Eight opportunities inflate the chance of a flag under innocence; they do not, by themselves, convict.

Problem 10D. The prior moved

Book Chapter 2, Breathalyzer Test.

Sensitivity \(99\%\), specificity \(90\%\), prior that the driver is over the limit \(10\%\).

  1. \(P(\text{over}\mid\text{positive})\).

  2. At Christmas the prior is \(20\%\). Same posterior?

  3. The device did not change. What did, and for a checkpoint that tests everyone, which prior is the right one?

  1. \(0.99\cdot 0.10/(0.99\cdot 0.10+0.10\cdot 0.90)=0.099/0.189\approx 0.524\).

  2. \(0.99\cdot 0.20/(0.198+0.10\cdot 0.80)=0.198/0.278\approx 0.712\).

  3. The base rate. A random checkpoint uses the population prior (10% on an ordinary night, 20% at Christmas). A stop after weaving is a different prior, and the \(52\%\) is the wrong number for that stop.

Problem 11. Alarm, explaining away

Lab 3 Exercise 1.

\(p(b)=2\cdot 10^{-4}\), \(p(e)=10^{-2}\), \(p(r=1\mid e=1)=0.5\), \(p(r=1\mid e=0)=0\), and

\(p(a=1\mid b,e)\) \(b\) \(e\)
0 0 0
0.1 0 1
1 1 0
1 1 1

  1. \(P(a=1\mid b=0)\) and \(P(b=1\mid a=1)\). The second is Bayes with two causes.

  2. Same posterior if \(p(b)=10^{-3}\). What did raising the prior do?

  3. The radio reports a small earthquake. Does \(P(b=1\mid a=1,r=1)\) go up or down from \(P(b=1\mid a=1)\)? Why, in two sentences, and why is that not “the alarm was a false positive”?

  1. \(P(a=1\mid b=0)=0.1\cdot 0.01=0.001\). Then \[ P(b=1\mid a=1)=\frac{2\cdot 10^{-4}}{2\cdot 10^{-4}+0.001\cdot(1-2\cdot 10^{-4})}\approx 0.167. \]

  2. About \(0.50\). A larger prior on burglary, with the same likelihood, raises the posterior. That is Bayes, not a new alarm model.

  3. Down. The radio is an alternative cause of the same alarm (explaining away). The sensor still sounded; burglary is just no longer the remaining explanation.

Problem 12. Holes you can see

Lab 2 Exercise 3; Wald.

316 planes returned. Cockpit: 36 returned, 21 down. Engine: 29 returned, 31 down. Someone counted 36 cockpit holes on the planes that came back and wants cockpit armor.

  1. Among the planes you can see, how common is cockpit damage?

  2. Among planes that took a cockpit hit, how often did the plane come back? Same question for engine.

  3. Which of those numbers goes in the brief? One sentence on what \(36/316\) is actually describing.

  1. \(36/316\approx 0.114\).

  2. Cockpit: \(36/57\approx 0.63\) came back. Engine: \(29/60\approx 0.48\).

  3. Armor goes where returning was rare given the hit. The 11% is holes on the planes you can see. The 63% vs 48% is the decision.

3. Binomial, Poisson, Normal

Labs 3 and 4, and the last block of Foundations. Book Chapter 1, named distributions.

A binomial is a list of independent yes/no trials. A Poisson is a count with a rate and no such list. The Normal is a location and a scale; a \(z\)-score is a change of units. \(P(X\le x)=\Phi((x-\mu)/\sigma)\). A large count can borrow the bell.

You should be able to: name Binomial or Poisson from the story; write \(P(X=k)\) for \(n=4\) by hand; standardise a Normal and read a tail or a quantile; say which way a tail probability moves if the left tail is heavier than the bell; read a histogram against a fitted density and say where the model is lying.

Problem 13. Name the count

Lab 3 Exercise 2.

For each, say Binomial or Poisson, and why.

  1. 80 independently tested circuit boards. \(X=\) number that fail.

  2. Cardiac-arrest calls to 911 between midnight and 6am.

  3. A subscriber is shown 12 thumbnails. \(X=\) number they click. Each click independent with the same \(p\).

  4. Goals Hull scores against Man U on Saturday.

Then one calculation: overnight calls \(\mathrm{Poisson}(\lambda=2.2)\). Write \(P(X=0)\) and compute it. Why is that the right model rather than \(\mathrm{Binomial}(n,p)\) for some \(n\)?

  1. Binomial: a list of 80 independent yes/no trials.

  2. Poisson: a rate, no list of trials.

  3. Binomial: 12 independent clicks.

  4. Poisson: a rate of goals, not 90 independent minute-trials with a fixed \(p\).

\(P(X=0)=e^{-2.2}\approx 0.111\). There is no roster of “potential calls” with a success probability. The Poisson is the count that remains when you only have a rate.

Problem 14. Four at-bats

Lab 3 Exercise 3, first number. Lecture self-check.

Joe DiMaggio, \(p=0.325\) per at-bat. Treat four at-bats as independent Bernoulli trials.

  1. Name the distribution of \(X=\) hits in a game.

  2. Write \(P(X>2)\) as two binomial terms, then compute it. Do not paste \(0.104\) from a slide.

  3. What did independence do for you, and what would a hit raising the next \(p\) do to this probability?

  1. \(X\sim\mathrm{Binomial}(4,0.325)\).

  2. \(P(X=3)+P(X=4)=\binom{4}{3}(0.325)^3(0.675)+(0.325)^4\approx 0.0927+0.0112=0.104\).

  3. Independence lets the pmf factor. A hit raising the next \(p\) raises \(P(X>2)\): the mass on 3 and 4 hits moves up relative to the independent calculation. The 56-game streak is this 10% stacked, not a typical night.

Problem 15. Two independent Poissons

Lab 3 Exercise 4, the one term you can write by hand.

Hull (home) \(X\sim\mathrm{Poisson}(0.71)\), Man U (away) \(Y\sim\mathrm{Poisson}(2.41)\), independent.

  1. Why is independence of \(X\) and \(Y\) a modelling choice here?

  2. \(P(X=0,Y=1)\), the \(0\)–\(1\) that actually happened. Write it as a product of two Poisson pmfs.

  3. If the two teams’ scores were positively dependent (a high-scoring night for both), would \(P(0\text{--}1)\) go up or down?

  1. Goals by one side can affect the other (red cards, sitting on a lead). Independence is what lets \(P(X=x,Y=y)=P(X=x)P(Y=y)\).

  2. \(P(0,1)=e^{-0.71}\cdot e^{-2.41}\cdot 2.41\approx 0.106\).

  3. Down. Positive dependence puts more mass on \((0,0)\) and on high–high, and less on mixed scores such as \(0\)–\(1\).

Problem 16. Warranty

Lab 4 Exercise 1.

Life \(L\sim N(10,3^2)\) years. The warranty pays if \(L<5\). A shipment has 80 independent components.

  1. \(p=P(L<5)\). Write the \(z\)-score. Name the distribution of \(X=\) number of claims in the shipment, and write \(E(X)\).

  2. If lives have a heavier left tail than this Normal, which way does \(E(X)\) move? Which of the two models did you change, and which did you keep?

  1. \(z=(5-10)/3=-5/3\approx -1.67\), so \(p=\Phi(-1.67)\approx 0.048\). \(X\sim\mathrm{Binomial}(80,p)\), \(E(X)=80p\approx 3.8\).

  2. A heavier left tail raises \(p\), so \(E(X)\) rises. You changed the model for \(L\). The binomial layer that turns \(p\) into a count is the same.

Problem 17. A month in the book

Lab 4 Exercise 2.

Monthly return \(X\sim N(0.008,0.035^2)\).

  1. \(P(X<-0.08)\).

  2. \(z\)-score of a \(-12\%\) month.

  3. The \(1\%\) quantile of \(X\).

  4. The officer wants \(P(X<-0.08)=0.05\) without changing the mean. What \(\sigma\) would that take? Use \(\Phi^{-1}(0.05)\approx -1.645\), not \(1.96\).

  5. The 1987 crash was a 5-sigma month under a Normal fitted to earlier data. Is \(P(X<-0.08)\) under the Normal too small, too large, or could it go either way?

  1. \(z=(-0.08-0.008)/0.035=-2.51\), so \(P\approx 0.006\).

  2. \(z=(-0.12-0.008)/0.035\approx -3.66\).

  3. \(q=0.008+0.035\cdot\Phi^{-1}(0.01)\approx -0.073\).

  4. \((-0.08-0.008)/\sigma=-1.645\) gives \(\sigma\approx 0.0535\). The book’s \(0.035\) is too tight if a 5% month is the risk the officer actually wants to price.

  5. A heavier-tailed histogram than the bell makes far-left probabilities larger than the Normal says, so \(0.006\) is too small. That is the crash lesson, not a maybe.

Problem 17B. The histogram is not a bell

Lecture: Normal tails; Lab 4 Exercise 2. Same monthly returns as Problem 17, now as a picture.

The histogram is 80 months. The dashed curve is \(N(0.008, 0.035^2)\), the model from Problem 17, scaled to the same bins.

Monthly returns. Histogram of 80 months; dashed curve is the Normal used in Problem 17, scaled to the same bins.

  1. Where does the histogram disagree with the dashed curve? (The dotted vertical line is \(-8\%\), the event in Problem 17a.)

  2. Is \(P(X<-0.08)\approx 0.006\) from the Normal too small, too large, or could it go either way?

  3. Collecting 800 months instead of 80, still from this world, would make the histogram look more like the dashed curve. True or false, and what did you just assume?

  1. The centre is a bit too peaked, and both tails have mass the Normal put at nearly zero. Several months sit left of \(-8\%\), where the dashed curve is already on the floor.

  2. Too small. The left tail of the histogram is heavier than \(\Phi(-2.51)\).

  3. False. A larger \(n\) makes the histogram closer to the true density, which here is not Normal. The dashed curve is the wrong model, not a noisy estimate of the right one. The 1987 lesson is the same: more months will not manufacture a bell. The eight months left of \(-8\%\) are already a larger left tail than \(\Phi(-2.51)\approx 0.006\) allows.

Problem 18. A large count

Lab 4 Exercise 3.

\(n=400\) people, each buys independently with \(p=0.12\). \(X=\) number who buy.

  1. Name the distribution of \(X\). Write \(E(X)\) and \(\mathrm{sd}(X)\).

  2. Using the Normal with that mean and sd, \(P(X\ge 60)\) is \(P(Z\ge ?)\). What is the \(z\)? You may leave the tail as \(1-\Phi(z)\).

  3. Why is a Normal even in the conversation? The \(X_i\) are 0–1.

  1. \(X\sim\mathrm{Binomial}(400,0.12)\). \(E(X)=48\), \(\mathrm{sd}(X)=\sqrt{400\cdot 0.12\cdot 0.88}=\sqrt{42.24}\approx 6.50\).

  2. \(z=(60-48)/6.50\approx 1.85\). \(P\approx 1-\Phi(1.85)\).

  3. CLT: a sum of many independent Bernoulli trials is approximately Normal. The binomial is already that sum. You are not claiming each person is Normal.

4. Estimation: CDF, moments, likelihood, CLT

Estimation. Book Chapters 1.4, 1.6, 3, 5 (through CIs), 11.4.

An estimate is a function of the data, hence random. Method of moments matches \(E(X^k)\) to the sample moment. Maximum likelihood picks the \(\theta\) that makes the observed sample most probable. For Bernoulli, Poisson, and the Normal mean, both methods return \(\bar y\). The CLT is why \(\bar X\) looks Normal even when \(X_i\) does not, and why the standard error falls as \(1/\sqrt{n}\).

You should be able to: read \(P(X=x)\) from a CDF; write \(\hat\lambda=\bar X\) and say whether that is method of moments, MLE, or both; turn \(\hat p\) and \(n\) into a 95% interval; say what happens to the interval if \(n\) quadruples; compare a bar of observed counts to the Poisson (or binomial) those counts were supposed to follow.

Problem 19. CDF and a leftover delivery

Book Chapter 1, Conditional Probability (delivery times).

Delivery day \(X\in\{1,\ldots,7\}\) with CDF

\(x\) 1 2 3 4 5 6 7
\(P(X\le x)\) 0.10 0.20 0.70 0.75 0.80 0.90 1

  1. Fill \(P(X=x)\).

  2. \(P(X=4\mid X\ge 4)\).

  1. Differences of the CDF: \(0.10,0.10,0.50,0.05,0.05,0.10,0.10\).

  2. \(P(X=4\mid X\ge 4)=0.05/(1-0.70)=0.05/0.30=1/6\).

Problem 20. Method of moments

Lecture: Poisson alpha particles; Normal body temperature.

  1. \(X_1,\ldots,X_n\) iid \(\mathrm{Poisson}(\lambda)\). Write the method of moments estimator of \(\lambda\).

  2. \(Y_i\) iid \(N(\mu,\sigma^2)\). Write the method of moments estimators of \(\mu\) and \(\sigma^2\). How does \(\hat\sigma^2\) differ from R’s var()?

  3. Five counts: \(2,0,3,1,4\). Compute \(\hat\lambda_{\mathrm{MoM}}\). Is this also the MLE? Why, in one sentence?

  1. \(E(X)=\lambda\), so \(\hat\lambda=\bar X\).

  2. \(\hat\mu=\bar Y\), \(\hat\sigma^2=\frac1n\sum(Y_i-\bar Y)^2\). R’s var() uses \(n-1\). Method of moments uses \(n\). For large \(n\) they agree; the MLE for \(\sigma^2\) is also the \(n\) version.

  3. \(\hat\lambda=10/5=2\). Yes: for Poisson, MLE and method of moments are both the sample mean, because \(E(X)=\lambda\) and the derivative of \(\ell(\lambda)=\sum y_i\log\lambda-n\lambda\) both give \(\bar y\).

Problem 21. Bernoulli MLE

Book Chapter 11, MLE for Bernoulli.

\(y_1,\ldots,y_n\in\{0,1\}\), \(P(Y=1)=\theta\).

  1. Likelihood \(L(\theta)\).

  2. Log-likelihood \(\ell(\theta)\).

  3. MLE.

  4. Second derivative: why is it a maximum?

  1. \(L(\theta)=\theta^{\sum y_i}(1-\theta)^{n-\sum y_i}\).

  2. \(\ell(\theta)=\sum y_i\log\theta+(n-\sum y_i)\log(1-\theta)\).

  3. \(\hat\theta=\bar y\).

  4. \(\ell''(\theta)=-\sum y_i/\theta^2-(n-\sum y_i)/(1-\theta)^2<0\) wherever both counts are positive.

Problem 22. Poisson MLE

Book Chapter 3, Poisson MLE, paper parts.

Clicks per minute, \(n=20\), \(\sum y_i=79\) (so \(\bar y=3.95\)). Poisson \(\lambda\).

  1. Write \(\ell(\lambda)\) and show \(\hat\lambda_{\mathrm{MLE}}=\bar y\).

  2. After a few days, about 20% of minutes are zero. The Poisson with \(\lambda\approx 4\) has \(P(X=0)=e^{-4}\approx 0.018\). What did the 20% do to the model, and would you keep a single Poisson?

  1. \(\ell(\lambda)=\sum y_i\log\lambda-n\lambda\), derivative \(\sum y_i/\lambda-n=0\), so \(\hat\lambda=\bar y=3.95\).

  2. Too many zeros for a Poisson with that mean. A single Poisson cannot match a high mean and \(P(X=0)\approx 0.20\). Two rates is the alternative. The MLE did not “fail at arithmetic”; the Poisson model is wrong.

Problem 22B. iid, then a flash sale

The Poisson MLE \(\hat\lambda=\bar y\) used 20 consecutive minutes as iid draws from one \(\lambda\).

  1. What does iid do for that estimator?

  2. Minutes 16–20 were a flash sale. The first 15 minutes averaged 3.2 clicks; the last five averaged 6.2. What is \(\hat\lambda\) if you pool, and what is it if you keep only the ordinary minutes?

  3. For staffing an ordinary minute tomorrow, which number do you take, and what assumption would have to be true for the pooled 3.95 to be the right forecast?

  1. iid is what lets the likelihood factor as \(\prod e^{-\lambda}\lambda^{y_i}/y_i!\) and what makes the MLE a single mean. Without it you do not have one \(\lambda\).

  2. Pooled: \((15\cdot 3.2+5\cdot 6.2)/20=3.95\). Ordinary only: \(3.2\).

  3. Take \(3.2\) for an ordinary minute. The pooled 3.95 is the right forecast only if tomorrow is a 1-in-4 flash-sale world, or if you do not know whether a sale is on. iid was false for the purpose of “ordinary tomorrow.”

Problem 22C. Observed versus the Poisson you fitted

Lecture: alpha-particle observed vs expected. Same move as Problem 22b, as a picture.

100 minutes of click counts. The bars are the observed frequencies. The points are \(100\times\mathrm{Poisson}(\hat\lambda)\) with \(\hat\lambda=\bar y\), the MLE.

Clicks per minute. Bars: observed. Points: expected under Poisson(mean).

  1. What did \(\mathrm{Poisson}(\bar y)\) get wrong, and where?

  2. The MLE is still \(\bar y\). Why is that not a defence of the model?

  3. Name one data-generating story that would produce this plot. Would you keep a single Poisson for staffing?

  1. Too many zeros: the bar at 0 is far above the point. The middle counts (3–6) are a bit low relative to the Poisson, which spent its mean on a shape that the extra zeros forced \(\bar y\) down from.

  2. \(\hat\lambda=\bar y\) is the best Poisson. The plot is a check of the model. A good MLE of a wrong model still misses the extra zeros.

  3. A flash sale mixed with idle minutes, or two rates, or minutes when the site is down. For staffing an ordinary minute you want the busy minutes, not one Poisson whose mean is a compromise between idle and busy. Same verdict as 22b and 22B, now from the picture.

Problem 23. CLT and a confidence interval

Lab 4 Exercise 4; Book Chapter 5, Voter: CI.

  1. \(X_1,\ldots,X_{36}\) iid \(\mathrm{Poisson}(3)\). \(\mathrm{sd}(\bar X)\)? Approximate \(P(\bar X>3.5)\) by a Normal with that sd.

  2. A poll of \(n=1000\) voters, \(400\) say they will vote for the candidate. True or false: a 95% CI for the proportion includes \(45\%\).

  3. How large a sample do you need so that a 95% interval for a mean, with \(\sigma=8\) known, has half-width \(1\)?

  4. The poll in (b) was 1,000 landline respondents on a weekday afternoon. Name the sampling assumption the interval needs, and one way that sample would miss the population of actual voters. Does the interval get too narrow, or is the centre in the wrong place?

  1. \(\mathrm{Var}(X_i)=3\), \(\mathrm{sd}(\bar X)=\sqrt{3/36}=1/\sqrt{12}\approx 0.289\). \(P(\bar X>3.5)\approx 1-\Phi((3.5-3)/0.289)=1-\Phi(1.73)\approx 0.042\).

  2. False. \(\hat p=0.40\), \(\mathrm{se}=\sqrt{0.4\cdot 0.6/1000}\approx 0.0155\), interval \(0.40\pm 1.96\cdot 0.0155=(0.370,0.430)\). \(0.45\) is outside.

  3. \(1.96\cdot 8/\sqrt{n}=1\) gives \(\sqrt{n}=15.68\), \(n\approx 246\).

  4. The interval assumes a random sample from the population of voters. A weekday landline sample misses mobile-only and working-hour voters. That is bias in the centre, not a too-narrow interval. A larger \(n\) of the same sample would make you more sure of the wrong number.

5. Linear regression

Patterns, through log transforms and OLS. Book Chapter 12, up to residual analysis and log-log.

The model is \(E(y\mid x)=\beta_0+\beta_1 x\) (or more columns). OLS picks \(\beta\) to minimise squared residuals. The slope is a rate: the change in predicted \(y\) for a one-unit change in \(x\). A log-log fit is the same model after a log-log transform, \(y=ax^b\). Dummy variables: \(k\) levels need \(k-1\) columns; the omitted level is the reference, and that is a choice.

\(R^2\) answers a narrow question. It is not “is this model good.” A residual plot is the check.

You should be able to: interpret \(\hat\beta_1\) and form \(\hat y\) at a new \(x\); say what a curved mean, non-constant variance, or an outlier in \(e\) vs \(\hat y\) means; turn \(y=ax^b\) into a linear equation in logs and interpret \(b\); say what happens to fitted values if you change the dummy reference; look at a scatter of two groups and say what a single line is averaging.

Problem 24. OLS and a residual plot

Book Chapter 12, Residual Analysis and OLS Derivation (d).

lm(price ~ sqft)     # price in $1000s

(Intercept)  50.23    (se 12.35)
sqft          0.1234  (se 0.0089)
Residual se  45.67 on 98 df
R-squared     0.662

  1. Interpret the slope in dollars per square foot.

  2. Predicted price for a 2,000 sq ft house.

  3. For simple regression, \(\hat\beta_1=S_{xy}/S_{xx}\). What does that say \(\hat\beta_1\) is made of?

  4. A residual plot against fitted values shows a clear parabola. What would you do, and why is \(R^2=0.66\) not an argument to keep the linear model?

  1. Each additional square foot is associated with \(\$123.40\) in predicted price.

  2. \(50.23+0.1234\cdot 2000=297.03\) thousand, about \(\$297{,}000\).

  3. It is the sample covariance of \(x\) and \(y\) over the sample variance of \(x\). The slope is a scaled association, not a causal claim.

  4. A parabola in the residuals means the linear mean is wrong. Add a square, or log \(y\), or log-log, and look at the new residuals. \(R^2=0.66\) can be large while the mean is still bent; \(R^2\) does not see the pattern in \(e\) vs \(\hat y\).

Problem 24B. Who is in the sample

The same price ~ sqft fit is on 100 houses that sold last year.

  1. Name two modelling assumptions besides “the mean is linear in sqft.”

  2. Houses that did not sell are missing. If unsold houses are larger and overpriced, which way is \(\hat\beta_1\) pulled relative to the slope for listed houses?

  3. You will use \(\hat y\) to advise a seller on asking price. What would have to be true for that use, and what would make you refuse?

  1. The 100 sales are like draws from the population you care about (iid). Errors have constant variance. sqft is not standing in for omitted size-correlates (lot, school, renovation) that you will not have at prediction time.

  2. Large unsold houses sit above the sold cloud in asking price but never enter \(y\). The sold line is fitted only on houses whose price was accepted. That usually flattens or tilts the sold slope relative to listed asking prices: you see completed trades, not the full asking curve. The direction to write: \(\hat\beta_1\) is a slope among accepted prices, so it is not the slope of list price on sqft.

  3. The advice is for a house that will sell on the same market as last year’s sales. Refuse if the house is in a different segment (new build, distressed, a size outside the 100) or if the residual plot already said the linear mean is wrong. The \(R^2\) does not settle this.

Problem 24C. MSE or MAE

Book Chapter 11, MSE vs MAE.

Predictions \(\hat y=(2,3,4,5,20)\), actual \(y=(2,4,4,6,6)\).

  1. MSE and MAE.

  2. Which metric did the \(20\) dominate, and what assumption about the cost of error is MSE making?

  3. For house prices, when would you report MAE rather than OLS?

  1. Errors \(0,-1,0,-1,-14\). \(\mathrm{MSE}=(0+1+0+1+196)/5=39.6\). \(\mathrm{MAE}=(0+1+0+1+14)/5=3.2\).

  2. MSE. Squaring treats a $14 miss as 196 times a $1 miss. That is the right loss if big misses are disproportionately costly (and if you believe the Normal likelihood, which is the same square).

  3. When a few mansions or data errors should not own the fit: listing prices, a median asking price, or when the residual plot already shows a heavy tail. OLS/MSE is the default only if that square is the loss you actually have.

Problem 24D. Four residual plots

Book Chapter 12, Residual Analysis (d); lecture PGA and mammal residual checks.

Each panel is residuals against fitted values from lm(y ~ x) on \(n=80\) points. The same axes labels on all four.

Residuals versus fitted values. Four different data-generating stories, one linear fit each.

  1. For each panel, name the pattern in one phrase (none / curved mean / non-constant variance / one outlier).

  2. Which modelling assumption of OLS failed in B, in C, and in D? (A is the control.)

  3. You need a price at \(x=8\). In which panel do you refuse the linear \(\hat y\), and in which do you keep the line but change the loss or how you talk about uncertainty?

  1. A none (random scatter around 0). B curved mean (a U or parabola). C non-constant variance (spread grows with fitted). D one large positive residual.

  2. B: \(E(y\mid x)\) is not linear in \(x\). C: constant variance. D: least squares / MSE treats that point as an ordinary miss; it is not. A is the case where those three are still plausible.

  3. Refuse B: the line is the wrong mean, so \(\hat y\) at \(x=8\) is systematically off. Keep the line in C and D only with a change of use: in C report a spread that grows with \(x\), or transform; in D use MAE, or drop a data error after you have named it, rather than let one point own \(\hat\beta\). \(R^2\) is not in any of the four pictures, and would not have told B from A.

Problem 25. Log-log as a power law

Book Chapter 12, Log Transformations. Same move as mammal brain/body and Apple growth in lecture.

Advertising \(x\) and revenue \(y\) (both in $millions) follow \(y=ax^b\). Data: \((x,y)=(1,5),(2,8),(4,14),(8,25),(16,45)\).

A log-log fit gives \(\log y=1.564+0.798\log x\) (natural logs).

  1. Transform the power law to a linear model.

  2. What are \(\hat a\) and \(\hat b\)? Interpret \(b\).

  3. Predict revenue at \(x=10\).

  4. Why might you prefer this to OLS of \(y\) on \(x\) in the original scale, even if the raw \(R^2\) looks fine?

  1. \(\log y=\log a+b\log x\).

  2. \(\hat b=0.798\), \(\hat a=e^{1.564}\approx 4.78\). A 1% increase in advertising is associated with about a 0.80% increase in revenue: diminishing returns. Doubling \(x\) multiplies \(y\) by \(2^{0.798}\approx 1.74\), not \(2\).

  3. \(\hat y=4.78\cdot 10^{0.798}\approx 30.0\) million.

  4. The original scale is multiplicative and the residual spread usually grows with \(x\). Log-log makes the mean linear and often gives constant variance. \(R^2\) on the raw scale is not comparable to \(R^2\) on the log scale, and a high raw \(R^2\) can hide the non-constant variance.

Problem 25B. Two scatters, same \(n\)

Book Chapter 12, Log Transformations; lecture mammal brain/body and Apple.

Advertising \(x\) and revenue \(y\), 60 campaigns. Left: \(y\) against \(x\) with the OLS line. Right: \(\log y\) against \(\log x\) with its OLS line.

Left: revenue against advertising, linear fit. Right: the same campaigns after a log-log transform.

  1. What does the left plot say about \(E(y\mid x)\) and about \(\mathrm{Var}(y\mid x)\)?

  2. Why is the right plot the same power-law model after a log-log transform, not a different theory of revenue?

  3. The left-hand \(R^2\) is large. Is that an argument to keep the linear model?

  1. The mean is bent (diminishing returns) and the vertical spread grows with \(x\) (non-constant variance). Both are the signature of a multiplicative \(y=ax^b\varepsilon\). The spread is the easier one to see; the curve is why the left-hand line overshoots the middle campaigns.

  2. \(\log y=\log a+b\log x+\log\varepsilon\) is \(y=ax^b\) written so that OLS can see a straight mean. The slope on the right is \(b\). You did not switch from “revenue depends on advertising” to a new mechanism; you took logs until the mean was linear.

  3. No. A large \(R^2\) can sit on a bent mean with non-constant variance. The residual plot (or the left scatter itself) is the check. Same lesson as Problem 24d and as the mammal plot in lecture.

Problem 26. Dummy variables

Lecture: \(k\) levels, \(k-1\) columns.

A house has style \(\in\{\text{colonial}, \text{ranch}, \text{split}\}\). You fit \[ \mathrm{price}=\beta_0+\beta_1\mathrm{sqft}+\beta_2\mathbf{1}_{\mathrm{ranch}}+\beta_3\mathbf{1}_{\mathrm{split}}+\varepsilon. \]

  1. Why not three style dummies plus an intercept?

  2. What is \(\beta_2\)? What house is \(\beta_0+\beta_1\mathrm{sqft}\) describing?

  3. You recode and drop ranch instead of colonial. Which fitted values change? Which coefficients change?

  1. You cannot put three style columns and an intercept into a sum: every house is colonial or ranch or split, so the three 0/1 columns add to 1. That is why you need \(k-1\) columns. The omitted style is the reference.

  2. \(\beta_2\) is the difference in expected price between ranch and colonial, at the same sqft. The intercept line is colonial.

  3. Fitted values do not change. If you change the reference, every coefficient changes and the fitted values do not. You cannot compare \(\beta_2\) across the two fits without translating.

Problem 26B. Two clouds, one line

Lecture dummy variables; same house-price setting as Problem 26.

Left: ranch and colonial on the same scatter, one OLS line through all 80 houses. Right: the same points, two parallel lines (one intercept per style, common slope on sqft).

Price against living area. Grey: colonial. Black: ranch. Left: one line. Right: two intercepts, common slope.

  1. What is the single line on the left averaging, and why is that slope the wrong rate for a ranch?

  2. What did the dummy do on the right that the left plot could not?

  3. You will quote \(\hat\beta_{\mathrm{sqft}}\) to a seller. Which plot’s slope do you send, and what would make you refuse even the right-hand picture?

  1. It is a compromise between two clouds that sit in different parts of the sqft range. Ranch is larger and more expensive, so the pooled line climbs from colonial on the left to ranch on the right. The dollars-per-square-foot on the left is partly a style mix, not a size effect.

  2. It let the intercept jump for ranch and kept a common slope. The vertical gap is \(\beta_{\mathrm{ranch}}\), the number Problem 26 asked for.

  3. Send the right-hand slope, with the style named. Refuse if the two clouds are not parallel (style changes the rate, not just the intercept), if “ranch” is standing in for lot or school, or if this house is not in the same market as these 80 sales. Same sample warning as 24B.

Problem 27. True or false, linear regression

Book Chapter 12, True/False: Linear Regression. Also Book Chapter 11, MSE vs MAE (c)–(d).

  1. OLS minimises the sum of absolute residuals.

  2. Adding a predictor cannot decrease \(R^2\).

  3. Multicollinearity biases OLS.

  4. If \(y_i=x_i^T\beta+\varepsilon_i\) with iid Normal errors, MLE for \(\beta\) is OLS.

  5. MAE is more affected by a single huge residual than MSE is.

  1. False. OLS is squared residuals. Absolute residuals are MAE.

  2. True. In-sample \(R^2\) cannot fall when you add a column. That is why it is not a model-selection criterion.

  3. False. OLS stays unbiased. The variance of \(\hat\beta\) blows up.

  4. True. The Normal log-likelihood is \(-\frac{1}{2\sigma^2}\sum(y_i-x_i^T\beta)^2\) plus a constant.

  5. False. MSE squares the residual; the outlier dominates MSE more than MAE.

6. Logistic regression

Patterns, logistic block. Book Chapter 13, pen-and-pencil. Labs 5–7 when posted: rework those on paper too.

\(p=\sigma(x^T\beta)\) is a probability in \((0,1)\). The coefficient is a change in log-odds, not in \(p\). A threshold of \(0.5\) treats the two errors as equally costly. That is a decision, whether or not anyone wrote down the loss.

You should be able to: turn \(x^T\beta\) into \(p\) with \(\sigma\); interpret \(e^{\beta}\) as an odds ratio; fill TPR, FPR, precision from a 2-by-2; say why accuracy is the wrong summary when the base rate is small; move the threshold when the two errors have different costs.

Problem 28. From log-odds to a probability

Book Chapter 13, Sigmoid. You need the function, not a derivative.

\(\sigma(z)=1/(1+e^{-z})\). A fitted log-odds is \(z=-0.5+0.4x\).

  1. What is \(p\) at \(x=0\)? At \(x=0\), what does \(z=0\) mean when it occurs?

  2. Show, in one line, that \(\sigma(-z)=1-\sigma(z)\). Why does that matter for a “no event” probability?

  3. If \(x\) increases by 1, what happens to the odds? To \(p\)? (The second answer is not a constant.)

  1. \(z=-0.5\), \(p=\sigma(-0.5)\approx 0.378\). \(z=0\) means \(p=1/2\), a coin flip under the model, which here is at \(x=0.5/0.4=1.25\), not at \(x=0\).

  2. \(1-\sigma(z)=e^{-z}/(1+e^{-z})=\sigma(-z)\). \(P(\text{no event})=1-p=\sigma(-z)\).

  3. Odds are multiplied by \(e^{0.4}\approx 1.49\). Probability follows the sigmoid, so the same \(+0.4\) in log-odds moves \(p\) a lot near \(0.5\) and almost not at all near \(0\) or \(1\).

Problem 29. Log-odds

Book Chapter 13, Log-Odds Interpretation.

\[ \log\frac{p}{1-p}=-1.5+0.8\,\mathrm{usage}-2.0\,\mathrm{annual}. \] annual \(=1\) for an annual contract, \(0\) for month-to-month. Usage in hours.

  1. \(P(\text{churn})\) for 5 hours, month-to-month.

  2. Odds ratio for switching to annual, usage held fixed.

  3. How much would usage need to rise to double the odds of churning?

  4. A product manager hears “annual cuts churn by 2.0.” What did they mix up, and what sentence would you send instead?

  1. Log-odds \(= -1.5+4=2.5\), \(p=\sigma(2.5)\approx 0.924\).

  2. \(e^{-2}\approx 0.135\). Annual multiplies the odds of churn by \(0.135\), an 86% reduction in odds, not a 2-point drop in probability.

  3. \(e^{0.8\Delta}=2\) gives \(\Delta=\log 2/0.8\approx 0.87\) hours.

  4. \(-2.0\) is a log-odds coefficient. The probability drop depends on where you start on the sigmoid. Send: “holding usage fixed, an annual contract multiplies churn odds by \(0.14\).” Then give \(p\) at a usage the manager actually has.

Problem 30. Confusion matrix and a threshold

Book Chapter 13, Confusion Matrix and Metrics. Same accounting as Lab 2’s 1,000 mammograms, now with a cost.

Predicted fraud Predicted legit
Actual fraud 85 15
Actual legit 50 4,850

  1. Accuracy, precision, recall.

  2. Why is accuracy a poor metric here?

  3. Missing a fraud costs \(\$500\); a false alarm costs \(\$10\). Which error should the threshold lean toward, and which way do you move it from \(0.5\)?

  1. Accuracy \(=4935/5000=0.987\). Precision \(=85/135\approx 0.630\). Recall \(=85/100=0.85\).

  2. Predicting “all legit” is already \(0.98\) accurate and catches no fraud. The base rate is 2%.

  3. FN is 50 times costlier than FP, so raise recall: lower the threshold. You will flag more legitimate transactions. That is the loss, not a defect in the probability model.

Problem 30B. What the coefficient assumed

A logistic default model on last year’s book: \[ \log\frac{p}{1-p}=-3.0+0.02\,\mathrm{balance}-0.8\,\mathrm{student}. \] student \(=1\) if the borrower is a student.

  1. Holding balance fixed, what did the model assume the \(-0.8\) is?

  2. Students have higher balances on average. If you drop balance and refit with student alone, which way does the student coefficient typically move, and what assumption just failed?

  3. A 0.5 threshold on \(p\) is used to deny credit. What loss did that assume, and who is hurt if false denials are costlier than missed defaults?

  1. It assumed that, at the same balance, students have lower log-odds of default than non-students. That is not “being a student causes lower default.”

  2. The student coefficient typically rises (becomes less negative, or positive). Students look riskier in the margin because they carry more balance. The assumption that failed is that the student comparison is at the same balance.

  3. Equal cost of false deny and missed default. If false denials are costlier, raise the threshold (deny only at higher \(p\)). Students with high balances are the group the no-balance version would punish.

Problem 31. True or false, logistic

Book Chapter 13, True/False: Logistic Regression.

  1. A logistic coefficient is the change in probability for a one-unit increase in the predictor.

  2. The decision boundary \(\{x:p(x)=1/2\}\) is a line in the features that enter \(x^T\beta\).

  3. If the model gives \(P(Y=1)=0.7\), changing the classification threshold does not change that probability.

  4. The odds ratio \(e^{\beta}\) can be negative.

  1. False. It is a change in log-odds. Probability is the sigmoid of that.

  2. True. \(\{x:x^T\beta=0\}\) is the classification line from lecture (where \(p=1/2\)).

  3. True. The probability is the model. The threshold is a later decision.

  4. False. \(e^{\beta}>0\) always. It can be less than 1.