2009-05-11

Story point inflation

My teams have begun estimating the size of product backlog items in story points. We estimate individual user stories and tasks in story points, we sum the individual estimates to give us the total amount of product backlog left, and we track product burndown in story points on a sprint-by-sprint basis. Tracking product burndown yields our sprint-by-sprint velocity, which we use to forecast our project done dates. At the micro level of each sprint, we continue to re-estimate user stories and tasks in man-hours of team effort, and we burn down the man-hours as we complete each task. As we gain more experience and confidence with story points, we may drop man-hours estimation and work entirely in story points.

We usually can't estimate the size of all the user stories up front, all at once. We want our meetings to be short--we'd rather spend our time building useful product--so we don't have enough time to estimate all the stories in one estimation meeting. We want to get started implementing the high value stories even though we haven't yet thoroughly estimated the lower value stories, so we begin the first sprint without having estimated everything up front. The product backlog changes over time, so there are always new stories to estimate after we get started. The team's understanding of the backlog deepens with each sprint, and they want to re-estimate stories that they previously estimated. All of these factors contribute to the reality that backlog estimation continues throughout the life of the project.

An unintended consequence of ongoing backlog estimation is story point inflation. Estimates for new stories are larger than for similar previously estimated stories, and previously estimated stories are re-estimated larger. Story A and Story B might be similar, but, after having completed Story A and learning that it was more difficult than they thought, the team estimates (or re-estimates) Story B larger. Are they sandbagging? Are they being honest? Are they conflating size and complexity, measured in story points, with effort, measured in man-hours?

Story point inflation is a problem. If you have estimated all the user stories in the product backlog, and if you haven't added any new stories, then the aggregate size of the product backlog shouldn't change--it is a constant. If you know your team's sprint-by-sprint velocity, then you can forecast your project done date. But if the size of the product backlog keeps growing due to re-estimation, then you have no way to gauge your team's true velocity, and no way to forecast your project done date.

In the future, we'll try to mitigate the problem of story point inflation by standardizing our size estimates for user stories. We'll post a table showing benchmark user stories and their estimated sizes across various projects, and we'll review all estimates against the benchmarks. This should help us avoid story point inflation. We'll have better, more stable estimates, a more accurate gauge of team velocity, and more accurate forecasts of our project done dates.

Related reading:
  • http://blog.mountaingoatsoftware.com/is-it-a-good-idea-to-establish-a-common-baseline-for-story-points and the comment http://blog.mountaingoatsoftware.com/is-it-a-good-idea-to-establish-a-common-baseline-for-story-points#comment-21036
  • http://jamesshore.com/Blog/Estimate-Inflation-A-Cautionary-Tale.html

No comments:

LinkWithin

Related Posts with Thumbnails