Are they different?
Does the drug group really differ from the placebo group, or is it luck?
Each set is 15 points on a circle. Two follow a fixed rule, one comes from chance. Can your eye tell?
Which circle is chance?
Your turn: can you make up random numbers?
Click 15 points on the circle, as randomly as you can.
Optional: how the gap score works (you can skip this)
Walk around the circle and measure each gap between neighboring points, as a share of the whole circle. Square each gap, add them up, and multiply by the number of points: \(s = T \sum (\text{gap}/L)^2\), with \(T = 15\) points and \(L = 100\) the circle's length.
Perfectly even points have every gap equal to 1/15, so \(s = 1\). All points at one spot leave one gap equal to the whole circle, so \(s = 15\).
Random points score \(2T/(T+1) = 1.875\) on average, and most land between 1.4 and 2.6.
Scores of 10,000 random sets. Yours is marked.
Idea from V. I. Arnold, Euler Groups and Arithmetics of Geometric Progressions, paragraphs 9 and 10: powers of 2 modulo n look random, but their gaps are a little too even.
Same or different?
Each round is a small trial of a drug against a placebo (a dummy pill). The drug's true response rate (the share of all patients who would improve on it) is hidden; in half the trials it equals the placebo's.
1. Patients per group
2. Your call
filled = improved outline = did not.
bright = what the trial recorded pale = what would have happened on the other treatment
Randomization
In physics, if you repeat an experiment the same way, you get almost the same result. So if you change one thing and the result changes, that change caused it.
Patients are not like that. Two patients on the same drug can do very differently, because of health, age, habits and things we cannot control.
Sicker patients tend to do worse no matter what they take. If more of them end up on placebo, the placebo group does worse because of who is in it, and the drug looks better even if it does nothing.
So we flip a coin for each patient: heads gets the drug, tails gets placebo. Then, on average, sicker and less sick patients end up in both groups in the same numbers, and in a large trial the split is close. The coin also splits things we never measured, like sleep or diet. What is left is luck, and the randomization test below shows how big luck can be.
Idea from D. R. Cox, Planning of Experiments (1958).
Each column is one of 40 patients: top icon = result on the drug, bottom = on placebo.
The colored bar is each patient's health at the start: orange sicker, teal less sick.
Who decides who gets the drug? Try both. When patients choose, who takes the drug depends on their health. In this demo the less sick pick it (healthy-user bias); in practice doctors often give new drugs to sicker patients, which biases the other way. Watch how far the trial's answer lands from the true effect.
Why the gap is not the drug
The gap you see is the drug's effect on the people who took it, plus how different the groups already were.
Do the groups match?
A fair comparison needs groups alike before the drug, including in what nobody measured.
| Trait | Drug | Placebo | Match? |
|---|
The randomization test
In a trial, a random schedule (think of a coin flip) decides who gets the drug. Suppose the drug does nothing, so each patient's result stays fixed. Re-randomizing flips the coins again and asks how big a gap luck makes.
Observed
After re-randomizing
Press "Re-randomize once".
Predict, then check
Examples
Each card re-randomizes 2,000 times in simulation. Orange bars are gaps at least as big as the real one.
Takeaway
- Chance makes differences too.
- A test asks: is the gap bigger than chance usually makes?
- Small trials fool everyone. Big trials let real gaps stand out.
This is why trials randomize: the coin flip itself tells us what gaps chance makes. The share of re-randomizations with a gap at least as big as the real one is the two-sided p-value (module 2 counts only gaps favoring the drug, the one-sided version).