Reference class forecasting: where the estimate sits in the class you chose

Scale

The four fixed panels share one scale, 150 to 650 days, and hold the estimate at 210 days. The live chart above rescales with what you type.

Basis

Every figure is counted against the class in the rows below and against nothing else.

Assumptions

The class is the visitor's own; no project in it is checked for comparability; a duration equal to the estimate counts as half; nothing is fitted, binned or smoothed.

Model version

1.0

The formula, in words

• Sort the class, then count what is under the estimate The position is the number of comparable projects shorter than the estimate, divided by how many projects are in the class, as a percentage. A project exactly as long as the estimate is neither above nor below it, so it counts as half, and the readout shows the half rather than resolving it quietly in one direction. In statistics this number is called a percentile, and the word is used here once and then dropped in favour of saying what it is: the share of the class that came in under the estimate.
• The median is the middle, and with an even class the middle is between two projects Sort the class and take the middle value. Ten projects have no middle one, so the median is the midpoint of the fifth and sixth, $\frac{255 + 275}{2} = 265$ days on the class this page opens with, and the staircase draws that as a flat run rather than a point because no project sits at 265. The page measures the gap from the median rather than from the average for one reason, and it is demonstrable rather than a matter of taste: add one project of 900 days to this class and the average moves 55.9 days while the median moves 10. A class assembled by hand is exactly where one very long project turns up.
• The class size sets how fine the answer can be, and no arithmetic improves it With ten projects every step is ten points and there are eleven positions the estimate can occupy, or twenty one once ties are allowed. No project lies between 200 and 215 days to change the count. That is the resolution of the class, printed on the page rather than smoothed over, and it is why the position carries a decimal place that would otherwise be false precision: with eight projects the steps are 12.5 points, and 37.5% rounded to 38% would hide that the figure is exactly three eighths.
• The page reports a position and stops there The class median is 1.3 times the estimate on the shipped values. That figure is not an instruction, and the page does not offer it as one. Multiplying an estimate by a class ratio, or planning to a chosen percentile of a class, is a decision that needs a target, a risk appetite and somebody willing to own it, and none of the three can be typed into this page. What the arithmetic can do is put a number where it can be argued with, and what it cannot do is settle the argument.

A worked example

Shipped defaults: ten comparable projects and an estimate of 210 days.

The class, sorted. 180, 195, 220, 240, 255, 275, 300, 330, 390, 465.

The mean would have told a different story, and worse. The ten durations total 2,850, so the mean is 285 days against a median of 265. Now add one project that ran to 900 days. The class is eleven, the total is 3,750, and the mean jumps to 340.9 days, a move of 55.9 days on one addition. The median moves from 265 to the sixth of eleven, which is 275 days, a move of 10 days. One project moved the mean five and a half times as far as it moved the median, and a class somebody assembled by hand is exactly the sort of set one very long project arrives in.

And the class decides the answer. Hold the estimate at 210 days and change nothing else.

The estimate did not move by one day. Its position nearly doubled between the widest reading and the narrowest, and the median moved 62.5 days across the four. The class chose the answer, and somebody chose the class.

Check it yourself

Add an eleventh project of 900 days and read the median against the average. The median moves from 265 to 275 days, ten days. The average moves from 285 to 340.9, nearly fifty six. One project, one class, and the two summaries disagree about what happened by a factor of five and a half.

What this does not model

Whether any project in the class resembles yours in scope, size, technology, interfaces, contract or country, which nothing on this page can check; how the class was assembled and what it turned away; projects that were cancelled, abandoned or never allowed to finish, which are missing from every class assembled from projects that finished; any measure of how uncertain the position itself is, which a class of ten cannot support and which more arithmetic on the same ten cannot manufacture; anything about the tail beyond the longest project in the class, since the class stops there and the world does not; what any project in the class cost or what it was worth; the difference between a project that ran long and a project that was started before it was understood; the passage of time, so a class mixing work from very different years is treated as one class; any correlation between the projects, so a class of ten from one programme is treated as ten independent readings when it may be closer to one; and what to do about any of it.