The Probabilistic Delivery Forecaster

Story-point estimation asks a human to guess the size of a work item before it is built. Monte Carlo Forecasting asks a different question: given how fast this team has actually finished items lately, when does a pile of them empty out? The first method degrades as AI changes what a “work item” costs. The second stays valid as the work gets noisier, because it never looks at the item at all; it just reports a wider distribution. This note covers how the throughput method works, the 2026 evidence that per-item estimation is breaking under AI-assisted development, and where the existing tools leave a gap.

The method throws away the estimate on purpose

A Monte Carlo delivery forecast needs one input: a history of throughput, the count of items finished per day or per week. No sizes, no points. It simulates the future by resampling that history. To forecast a 40-item backlog, draw a random past week’s throughput, subtract it from 40, draw again, and count weeks until the remainder hits zero. That is one trial. Run it ten thousand or more times and the finish dates form a distribution. You read a commitment off a percentile: the 85th is the date you beat 85% of the time. One worked example from Nave’s explainer: a 40-task backlog starting March 1 comes back as an 85% probability of completion by July 5, for any 40 items, not these specific ones.

The reason this substitutes for estimation is that a team’s own delivery variance already encodes the thing story points try to guess. Big items and small items, blocked work and smooth work, all of it is baked into the historical spread. This is the empirical core of the Little’s Law family of flow thinking, but where the law relates only long-run averages and is silent on spread, the simulation keeps the whole distribution. The Cycle Time Scatterplot is the same data viewed per-item; Monte Carlo is the same data projected forward.

Two assumptions carry the forecast. First, items are roughly similar in size, so counting them is meaningful rather than a fiction that hides one six-month epic among fifty one-day tickets. Second, the recent past resembles the near future, the stationarity assumption. When a disruptive change introduces a new source of variation, throughput becomes non-stationary and the forecast inherits the error. The fix is operational, not statistical: keep items right-sized, and re-run the forecast whenever team size or work mix shifts rather than trusting a stale distribution. This is the method’s sharpest vulnerability, and rapid AI adoption is the regime that strains it most. Fast change is exactly when the recent past stops resembling the near future, so the window of still-representative history shrinks and the forecast has to lean on a shorter, fresher slice of data.

AI is breaking the per-item estimate, not the throughput

The case that this method matters more now rests on measured 2026 data, not speculation, though it is a single proprietary dataset from Faros AI, a metrics vendor with an interest in the conclusion that raw throughput misleads. Its 2026 report on AI-assisted engineering found task throughput per developer up 33.7% and epics per developer up 66%, while PR merge rate rose only 16.2%. The same dataset shows code churn up 861% under high AI adoption, the incidents-to-PR ratio up 242.7%, and median time in review up 441.5%. Their summary line is the one that matters for forecasting: “throughput measures what was shipped, not what survived.”

Read those numbers as a variance story. AI collapses the cost of generating a change and inflates the cost and delay of confirming it, the Verification Gap made quantitative. A single item might take an agent two minutes to draft and then three days to review, test, and refactor. Per-item estimation was always a guess; under that spread it becomes noise. But throughput measured at the point of done, after review and merge, still captures the real rate, review bottleneck included. The method that samples finished work absorbs the AI-induced variance instead of choking on it, which is the argument for treating Monte Carlo as the estimation approach that survives the transition. This is the practical companion to the measurement questions in SDLC Delivery Metrics for AI-Assisted Engineering.

There is a caveat the throughput number hides. If AI raises the variance of cycle time faster than it raises the mean, the forecast distribution widens, and an 85th-percentile date drifts far from the median. The rising churn and review-delay figures point that way rather than at a simple slower mean, because rework and stalled reviews land on some items and not others, stretching the spread rather than shifting every item equally. Monte Carlo will still be correct, it will just be correct about a less comforting spread, and it will surface the tail that a velocity average conceals. That tail is the Flow Debt the method is built to expose.

The tool landscape and the gap

The math is not the moat; it is a spreadsheet loop. The friction is getting clean throughput data out of a tracker and rendering an honest distribution. ActionableAgile Analytics, from Daniel Vacanti, who introduced these techniques to Kanban practice, pulls cycle-time and throughput data from Jira or Azure DevOps and runs the simulations. Nave and 55degrees sell adjacent flow-metrics products. Most live as paid Jira add-ons priced and shaped for enterprise Kanban programs.

That leaves observable gaps rather than a proven market. Nothing widely used is framed around the AI-transition thesis, where the pitch is not “better Agile metrics” but “estimation from actuals is the version that still works when agents write the code.” The self-serve, low-ceremony end is thin: a tool that eats a CSV export or reads the GitHub API directly, renders a scatterplot with 50/85/95 percentile lines and a forecast histogram, and flags aging work items, without a Jira license or a consultant. Whether that gap is a market or just a gap is the open question. Teams that would benefit most are often the ones least likely to have adopted flow discipline, and the failure mode is throughput becoming a target the moment it goes on a dashboard, which corrupts the very history the forecast depends on.

Try it

Forecast a real backlog from a CSV (2-3 hours, any language with a random number generator, or a spreadsheet). Export closed issues from a GitHub or Jira project with their close dates. Bucket them into weekly throughput counts. Write the resampling loop: for a backlog of N items, repeatedly draw a random historical week, subtract, count weeks to zero, and store the result. Run 10,000 trials, sort the finish dates, and read off the 50th, 85th, and 95th percentiles. What you are looking for is the width between the 50th and 95th. A wide gap means your throughput is volatile and any single-date promise is a lie; a tight one means the team is predictable enough to commit. Compare the 85th-percentile date against what the team would have estimated by hand.

Make the AI-variance effect visible (an afternoon, on top of the above). Split the same history into a pre-AI-adoption period and a recent period, and run the forecast on each. If the recent period shows a higher median throughput but a wider percentile spread, you have reproduced the Faros finding on your own data: faster, but less predictable. That divergence is the case for forecasting from actuals rather than estimating per item.

Sources

Nave, Monte Carlo Simulation Explained — step-by-step mechanism and the two forecast questions
Faros AI, The AI Acceleration Whiplash — 2026 metrics on throughput gains against churn, incidents, and review delay
ActionableAgile Metrics for Predictability — Vacanti’s foundational treatment of flow metrics and forecasting
Panaxeo, Monte Carlo Velocity Forecasting — the stationarity requirement and its failure modes