What “Fair” Means for a Random Generator

A fair random object generator gives every matching object the same chance of being picked, every time. Over many picks, that means each object should come up about the same number of times — not exactly the same, because chance always produces some variation, but within the range that chance alone would explain.

That last point is important. A generator that produced perfectly equal counts would itself be suspicious. Fairness is about the pattern of variation, which is what a statistical test checks.

The Test Setup

  1. We used the generator’s own code — the same engine that runs the random object generator — not a simplified copy.
  2. We ran it 100,000 times on 2026-11-03, picking one object at a time from all 1,207 objects.
  3. We counted how many times each object was picked.
  4. We compared the counts with what pure chance predicts, using a chi-square goodness-of-fit test.
  5. Separately, we generated 20,000 lists of 5 to check for duplicates, and ran No repeats until every object had appeared.

The Results

With 100,000 picks and 1,207 objects, a fair generator should pick each object about 82.85 times. Here is what happened.

How often each object was picked in 100,000 picks
  • Observed
  • Expected by chance
05010015020025030055–59: 4 (expected by chance: 3.8)55–59: 4 (expected by chance: 3.8)5560–64: 20 (expected by chance: 18.3)60–64: 20 (expected by chance: 18.3)6065–69: 56 (expected by chance: 59.5)65–69: 56 (expected by chance: 59.5)6570–74: 121 (expected by chance: 135.2)70–74: 121 (expected by chance: 135.2)7075–79: 244 (expected by chance: 220)75–79: 244 (expected by chance: 220)7580–84: 253 (expected by chance: 261.3)80–84: 253 (expected by chance: 261.3)8085–89: 225 (expected by chance: 230.9)85–89: 225 (expected by chance: 230.9)8590–94: 170 (expected by chance: 154.2)90–94: 170 (expected by chance: 154.2)9095–99: 81 (expected by chance: 79)95–99: 81 (expected by chance: 79)95100–104: 21 (expected by chance: 31.4)100–104: 21 (expected by chance: 31.4)100105–109: 10 (expected by chance: 9.8)105–109: 10 (expected by chance: 9.8)105110–114: 2 (expected by chance: 2.4)110–114: 2 (expected by chance: 2.4)110253Times an object was pickedNumber of objects

Each bar counts the objects that were picked a given number of times. The dashed line is the distribution pure chance predicts; the closer the bars follow it, the fairer the generator.

Show the data as a table
Times pickedObjects (observed)Objects (expected)
55–5943.8
60–642018.3
65–695659.5
70–74121135.2
75–79244220
80–84253261.3
85–89225230.9
90–94170154.2
95–998179
100–1042131.4
105–109109.8
110–11422.4
  • Every object was picked at least once. The least-picked came up 58 times; the most-picked, 111 times.
  • The standard deviation of the counts was 8.85, against 9.1 predicted by chance.
  • The chi-square statistic was 1,142.2 with 1,206 degrees of freedom, giving p = 0.905.
  • None of the 20,000 lists contained a duplicate.
  • No repeats showed all 1,207 objects in 242 rolls of 5, the minimum possible, with 0 repeats.

Reading the Chi-Square Result

The chi-square test asks: if the generator were perfectly fair, how likely is it that the counts would vary at least as much as they did? That probability is the p-value.

A very small p-value — below 0.0001, say — would mean the variation is too large to be chance, which would suggest bias. Our p-value of 0.905 is nowhere near that. In plain terms, the counts look exactly like what a fair generator produces.

Testing Lists and No Repeats

Single picks are only part of the story. Lists of several objects must never contain the same object twice, and No repeats must work through every object before any comes back. Both behaved exactly as designed: no duplicates in any list, and the minimum possible number of rolls to see everything.

A few of the objects in the test
  • Teapot
  • Kite
  • Bicycle
  • Pinecone
  • Trumpet
  • Backpack

All 1,207 objects, from every category, were in the pool. Which ones happen to land at the top or bottom of the counts changes with every run — that is what chance looks like.

What the Test Can’t Tell You

  • It tests this run, not every run. A different 100,000 picks would give slightly different counts. That is why we rerun the test whenever the code or the list changes.
  • It tests the full list, not every filter. Filters only narrow the pool; the same picking code runs on whatever remains.
  • It can’t prove perfection. Statistics can show there is no evidence of bias, not that bias is impossible. Using a cryptographically secure source and rejection sampling is what makes bias unlikely in the first place.

Check It Yourself

You don’t need code to see fairness in action. Set the random object generator to a small category, generate one object 50 times and tally the results. With a small pool, every object should come up several times, and no object should dominate. The test above is the same idea at a much larger scale, and the method is documented on how we build our lists.