Ball
Observed count
Expected count (exact)
Normal with the same mean and SD
Ball using coin A
Ball using coin B
Expected count (exact mixture)
Normal with the same mean and SD
Each ball hits one peg per row and bounces right with probability p. The bin it lands in counts its right bounces, so the pile builds a binomial distribution. Switch to a mixture coin to see when that pile stops looking like a bell curve.
← All labsBased on Chapter 1, Probability and Uncertainty, of Polson and Sokolov, Bayes, Artificial Intelligence, and Deep Learning (vsokolov.org/html/_book).
The first two demos use the same coins: coin A (p = 0.7) with probability 0.3 and coin B (p = 0.5) otherwise. Only the moment of choosing the coin differs. Each sets n = 100 and drops 5,000 balls. The third demo spreads the coins apart so two humps show up with only 20 rows.
The CLT is a statement about large n, and its failure is too. At n = 12 the two coins' piles are centered at 8.4 and 6.0 with SDs of about 1.6 and 1.7, so they overlap almost completely. Their mixture is only slightly wider than a binomial (SD 2.02 instead of 1.72) and has almost no skew (0.02). A normal curve fits it well. The difference grows with n: the centers separate like 0.2n, while each pile's width grows only like √n. Around n = 50 a second hump appears, and by n = 100 the pile clearly has two.
K is a sum of n bounces, K = X1 + … + Xn. The classic CLT says this sum looks normal when the Xi are independent with the same distribution. If the coin is picked fresh at each peg, every bounce is an independent Bernoulli(0.56). The mixture collapses to one coin and the bell curve appears.
Given the coin, bounces are independent. Without knowing it, a right bounce is evidence the ball has coin A, which makes the next right bounce more likely. So any two bounces of the same ball are positively correlated, with Cov(Xi, Xj) = Var(p). That correlation never fades, however many pegs there are.
A sum of n terms has n variances and n(n−1) covariances. With independence the covariances vanish and Var(K) grows like n. Here they add up to n(n−1)·Var(p), which grows like n² and eventually dominates. The inflation factor 1 + (n−1)ρ keeps rising with n.
The law of large numbers still works, but inside each ball: K/n converges to that ball's coin, 0.5 or 0.7, never to 0.56. So K/n has a two-point limit, and the standardized K converges to a two-point shape instead of N(0,1). The CLT does hold within each coin, so the pile becomes two bells weighted 70/30. The normal curve with the correct mean and SD still misses both humps.
| n | Per peg | Per ball | Inflation |
|---|---|---|---|
| 12 | 1.72 | 2.02 | 1.38× |
| 50 | 3.51 | 5.74 | 2.67× |
| 100 | 4.96 | 10.4 | 4.38× |
| 400 | 9.93 | 37.9 | 14.6× |
Takeaway for students. Sample size helps only when observations are independent. Students in the same classroom, repeated measurements on the same patient, and households in the same survey cluster all share a hidden "coin". Averaging more of them does not wash that out. This is the reason for random-effects models, cluster-robust standard errors, and the Beta-Binomial model, and the variance inflation 1 + (n−1)ρ is the survey statistician's design effect.
At every peg the ball goes right with probability p or left with probability 1 − p. Each bounce is a Bernoulli trial, independent of the others.
After n rows, the bin number equals the number of right bounces. Many different paths lead to the middle bins, and only one path leads to each edge.
K is a sum of n independent trials, so by the Central Limit Theorem its shape approaches a normal curve as n grows.