Skip to main content

Estimation: Precision and Accuracy (and Economics)

I was listening to a episode of the NPR show Planet Money on the accuracy of economic forecasts and how economic forecasts often have precision, but are often inaccurate. Consider this exchange between two of the show's hosts:

Alex Blumberg: On average, the leading forecasters, like Prakken's Macroeconomic Advisors, have around a one percentage point margin of error, which might sound pretty good. But let's say you forecast a two percent growth rate for the year. That means you're saying the actual rate will be somewhere between one percent and three percent.

Adam Davisdon: Now those are two totally different economies. One percent, that doesn't even keep up with population growth. That's a bad year. Unemployment will go up. It'll feel bad in the U.S. Three percent is the opposite, a pretty good year.

In one of the interviews, Simon Johnson, (Economist, MIT, Peterson Institute for International Economics) said this about economic estimates:
I would call it a necessary evil. There's so much imprecision. There's so much, you know, lagging in terms of our updating, that in some sense, we'd be better off without forecasts. We'd better off, you know, making up our minds afresh every day. But the problem is the businesses, the institutions, all involve thinking about the future and planning for the future. And you can't do that without taking a view of the future.
Which sounds quite analogous to estimates on software projects. One of the reasons that Planning Poker estimation uses a non-linear scale (for example, a Fibonacci sequence)) is that, in general, as tasks get larger, you are less accurate. While it might be meaningful to talk about a 1 hour task as opposed to a 2 hour task,  it's less likely to be useful to estimate 6 hours as opposed to 5 hours for a task. (A Planning Poker desk might have 1,2,3,5,8, 13, etc, hours as possible values.)

Planning Poker is intuitive, and often matches experience, yet many people insist on estimating a task that takes more than 3 hours as 4 rather than 5. This is probably because everyone forgets that estimates are just that: "estimates." Estimation gets better as a team works together, but the estimates are only one part of the planning process. If the goal of the team is to make a commitment to deliver functionality, then the non-linear jump in estimates is a measure of risk. The best way to reduce risk is to work in smaller pieces. When faced with a task that is bigger than an 8, and 13 seems too big, see if you can decompose the task in to smaller tasks, thus improving your ability to estimate accurately.

Estimation is difficult, not just for software teams, but we need something to help us plan, so just be mindful to give estimates the appropriate precision. And reestimate work remaining as you go.

Comments

Popular posts from this blog

Continuous Integration of Python Code with Unit Tests and Maven

My main development language is Java, but I also some work in Python for deployment and related tools. Being a big fan of unit testing I write unit tests in Python using PyUnit. Being a big fan of Maven and Continuous Integration, I really want the  Python unit tests to run as part of the build. I wanted to have a solution that met the following criteria:
Used commonly available pluginsKeep the maven structure of test and src files in the appropriate directories.Have the tests run in the test phase and fail the build when the tests fail.
The simplest approach I came up with to do this was to use the Exec Maven Plugin by adding the following configuration to your (python) project's POM.

<plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>exec-maven-plugin</artifactId> <executions> <execution> <configuration> <executable>python</executable> <workingDirectory>src/test/python</workingDirect…

Displaying Build Numbers in Grails Apps

Being a fan of Continuous Delivery, identifiable builds, and Continuous Integration: I like to deploy web apps with a visible build number, or some other way of identifying the version. For example, having the build number on the login screen for example. In the Maven/Java world, this is straightforward. Or at least I know the idioms. I struggled with this a bit while working on a Grails app,  and wanted to share my solution. There may be other, better, solutions, but the ones I found approaches that didn't quite work they way that I'd hoped.

My requirements were:
To display a build number from my CI tool, where the number was passed in on the command line. In Bamboo, for example you might configure a grails build as
-Dbuild.number=${bamboo.buildNumber} warTo only change build artifacts and not any source files.To not misuse the app version, or change the names of any artifacts.To be simple and idiomatic.I realized that that Grails itself changes the application metadata (appl…

Motivation Visibility, and Unit Testing

I've always been interested in organizational patterns (such as those in Organizational Patterns of Agile Software Development). I've recently found myself thinking a lot about motivation. I'm now reading Drive: The Surprising Truth About What Motivates Us and just finished Rob Austin's book on performance measurement. Being the parent of a three year old, I'm finding more and more that "because I said so, and I'm right" isn't too effective at home. My interests in motivation are closely related to my interest in writing software effectively. Writing software is partially a technical problem about frameworks, coding, and the like, but the harder (and perhaps more interesting) problem is how to get a group of people working together towards a common goal. Agile practices, both technical and organizational, build a framework which makes having the right amount of collaboration and feedback possible. But there's a bootstrapping process: How do yo…