Length bias: what is open reads longer than what is closed

Assumptions

Each band stands at its midpoint; no spread inside a band; the ledger is closed and complete; opening rate and duration mix steady across the window the ledger covers.

Model version

1.0

The formula, in words

• The two means divide by different denominators. The ledger mean is the total file days divided by the number of files, so every file gets one vote. The glance mean is the count weighted sum of squared durations divided by the total file days, so every day gets one vote.
• The gap is the spread divided by the mean. The distance between the two means is exactly the variance of the ledger divided by its mean, and nothing else. That is why the gap disappears when every duration is the same, widens as the spread widens, and never brings anything in from outside your ledger.
• A band's share changes when what is counted changes. A band's share of the files is its file count over the total count. Its share of the glance is its file count times its duration, over the total file days. A long band gains in the second exactly what it gave up in the first.
• A band conceals its own internal spread. Each band is one number standing for a group of files, and the variation inside it does not appear in these inputs. The gap shown is therefore a floor, and the true figure sits above it rather than below.

A worked example

The ledger, as it loads

Four bands, 280 files.

The ledger mean

$\frac{6,000}{280} = 21.4$ days. Every file counted once.

The glance mean

$\frac{297,750}{6,000} = 49.6$ days. Every day counted once.

The gap

$49.6 - 21.4 = 28.2$ days, a ratio of $2.3\times$.

The identity, checked by hand

The variance of the ledger is $\frac{297,750}{280} - 21.4286^{2} = 1,063.39 - 459.18 = 604.21$. Divide by the mean: $\frac{604.21}{21.4286} = 28.2$. The gap is the ledger's own spread divided by its own mean, and nothing was introduced from outside.

The two weightings, band by band

The 90 day band is the least common thing in the ledger and the most likely thing to be sitting in front of somebody at a randomly chosen moment. Rarity in a ledger of finished work is not rarity in a view of open work, and that sentence is the whole page.

The same mean, and a different glance

Chip 1 holds 15, 20, 25 and 30 day bands at 70, 100, 70 and 40 files. That is 280 files and $1,050 + 2,000 + 1,750 + 1,200 = 6,000$ file days, so the ledger mean is $\frac{6,000}{280} = 21.4$ days, identical to the default. Its second moment is $15,750 + 40,000 + 43,750 + 36,000 = 135,500$, so the glance mean is $\frac{135,500}{6,000} = 22.6$ days and the gap is 1.2 days against the default's 28.2. Two ledgers, the same file count, the same mean, and a glance that reads 49.6 days in one and 22.6 in the other. The mean carries no information at all about how far the glance will sit from it.

No spread, no gap

Chip 2 puts all 280 files in the 20 day band. $M_1=5{,}600$, $M_2=112{,}000$, where $M_1$ is total file days and $M_2$ the count weighted sum of squared durations. Ledger mean 20.0, glance mean $\frac{112,000}{5,600} = 20.0$, gap 0.0 days, ratio $1.0\times$. Both weightings are the same weighting. The variance is zero and the identity in §4.1 returns zero without a special case.

The tail does not have to be common

Chip 3 cuts the 90 day band from 25 files to 12. The ledger falls to 267 files, dayMass to 4,830 and sqMass to 192,450. The ledger mean falls to $\frac{4,830}{267} = 18.1$ days and the glance mean to $\frac{192,450}{4,830} = 39.8$ days, a gap of 21.8 days, which is 77 per cent of the gap that 25 files produced. The 90 day band now holds $\frac{12}{267} = 4.5$ per cent of the ledger and $\frac{1,080}{4,830} = 22.4$ per cent of the glance, a factor of five. Halving the long files removed less than a quarter of the gap.

Check it yourself

The ledger still holds 280 files and the ledger mean is still 21.4 days, while the glance mean falls from 49.6 to 22.6. The mean is the same and the glance is not.

What this does not model

The spread inside each band, which carries the same bias and which no arithmetic here can recover from a midpoint; any open file at all, whose duration this page never estimates and whose source it never judges; a ledger whose opening rate or duration mix changed across the window it covers, which is the steady state the length biased mean assumes and which this page cannot test; the difference between a renewal process, where one interval ends as the next begins, and a queue of files that overlap, which is close enough to teach the mechanism and is not identical to it; censoring, meaning the long files still open when the ledger was closed and therefore missing from it, which would push both means up and the ledger mean further than the glance mean; any cause of a long duration, of which the page names none and can name none; and any real queue, office, process or person, none of which this ledger describes.