Measure Intangibles

Combine Responses from your Crowd


In Using the Wisdom of Your Crowd I explained how you have a crowd at your fingertips, and why studies show using their collective knowledge can generate much better business decisions than just delegating the decision to an expert. I laid out three simple steps to use the wisdom of your crowd:

1. Decompose the problem (combining estimates of several sub-problems gives better accuracy than a single estimate of the top-level problem)
2. Identify your Crowd (hint: they are already around you)
3. Pose the Questions (so they are answered with numbers)

Step 4, Aggregate the Responses from your crowd is still simple in principle. A spreadsheet can give all the information you need to communicate the best choice, how it was arrived at and the remaining uncertainty.

The responses to any question you asked can be represented as a statistical distribution curve, like the familiar Normal Distribution bell curve except that it is not symmetrical. In a range of cost or time estimates, the optimistic and likely points are usually closer and the likely and pessimistic points are further apart.

Triangle V PERT Distributions

You need at least three estimates but the more, the better precision you will get. A good way to get more estimates is to ask each member of your crowd to give you their own pessimistic, likely and optimistic estimates. That also gets your crowd members to think a bit harder about their estimates. There are ways to calibrate your crowd to get even better estimates.

A statistical distribution is a much richer way to represent your data than the single guess you might have put up with in the past. When you draw the curve, it becomes obvious how well you collectively understand the problem.

My lightbulb moment came when I realised that you can add, subtract, multiply or divide distributions in a spreadsheet as easily as if they were single numbers.

For example, you could build distributions for each of your company's product ranges, and simply add them together to get a distribution of next year's overall revenue that still encapsulates all the crowd estimates and the level of uncertainty they represent. The sales team might still work towards a single target, but corporate management now have a much richer view, especially when they do the same for overhead and variable costs.

Next you can use it for estimates of

  • how much revenue we will earn next year if we just continue as is, vs
  • the cost of a project to make our business perform better, and
  • how much revenue we will earn next year if we do the project

Now we can get a view of how likely the proposed project is to be the best use of time and money, compared to competing project proposals.

Or turn this on its head and add together the distributions representing the cost of mitigating a risk and the residual risk after mitigation. If you subtract that result from the distribution of the estimated risk today, you get another distribution, the value of mitigating the risk that way. Now you can compare different ways to mitigate risk, even though they may have quite different results.

Finally you can now compare business improvement projects on the same objective, numeric scale with mitigating business risk without over-simplifying; something that has largely escaped us until now.

There is a printable PDF infographic depicting AcuteIP.com's approach to using the Wisdom of your Crowd

With enough preparation, you can do this yourself. AcuteIP.com offers a set of services to make it easy, from workshops and technical services to managing the whole project.
Contact Graham.Harris at AcuteIP.com to explore possibilities.

Is this post useful to you, or could it be useful to someone you know? Please do us both a favour- spread the word by sharing it through the colourful social media buttons at left on PC or below on mobile.

Have you used techniques like this to evaluate risk or communicate uncertainty? Tell us your experience with a comment.

© 2018 Graham.Harris at AcuteIP.com