A team ships a change, watches the dashboard for two days, sees conversion up 3%, and declares victory. The number was noise, and the next release quietly gives it back.
Assignment has to be deterministic — hash the user ID together with the experiment name, so the same person gets the same variant on every device and in every session. Anything less and the two groups blur into each other.
Assignment also isn't exposure. A user can be bucketed into variant B and never reach the screen that shows it. Log the exposure where the variant actually renders; counting the other 40% as treated dilutes whatever effect exists.
The analysis decisions belong before the start — primary metric, sample size, minimum detectable change, duration, stopping rule. Fixed in advance, they make a measurement. Chosen afterwards, they make a search.
Which is what peeking is. Check daily and stop when it looks good, and with enough looks random noise will eventually cross any threshold we care to name. The test in the chart found no effect at all. The stopping rule manufactured one.
A result we were free to stop looking for at any moment isn't evidence. It's the shape of our own impatience.