2

I'm just beginning, and despite my search, I cannot find any simple formula that would return a number between 0 and 100%, i.e. I do not care whether there was over or underestimation.

The simplest thing I could think of would be:
Actual Effort - |Estimation Delta| / Actual Effort.

To me, using simply Estimation Effort / Actual Effort can give more than 100% in case of overestimation, and an Accuracy of 125% sounds weird.

  • 1
    What is your input to the "formula" you're looking for. – Alexey R. Feb 8 '18 at 10:56
  • 1
  • @AlexeyR. The inputs would be both Actual Effort and Estimated Effort. – John V Feb 8 '18 at 11:10
  • 1
    Are you really able to measure "effort" this way? It would seem like measuring time or cost would be more suitable. In any case, why not just truncate at your floor/ceiling, of you don't care about mis-estimates? – Todd A. Jacobs Oct 15 at 18:33
3

Estimation accuracy formula

You said:

I cannot find any simple (estimation accuracy) formula that would return a number between 0 and 100%

  1. When the actual is the same as estimation, we want the estimation accuracy to be 100%.
  2. When the actual is 0 we want the estimation accuracy to be 0%.
  3. When the actual is twice the estimation (or more) we want the estimation accuracy to be 0%.

Given these, the formula would be: (Estimated - ABS(Estimated - Actual or 2*Estimated whichever is less)) / Estimated

2

If you're looking for any function that return the value inside 0-100 it makes sense to check Sigmoid function. It produces values from 0 to 1, which are easily converted to 0 - 100;

This is my example on how to use Sigmoid approach.

Here is the normal Sigmoid function:

enter image description here

You have to choose the acceptance range start point starting from which you consider your accuracy value equal to 100%. Let it be 4. Zero value would be -4. However your estimation is not 4 but 10 as you point out in your comment below. Thus you need to convert the coordinates where your 10 would become 4 for Sigmoid and 0 would become -4. This is the simple linear expression. It would look like:

d(x) = 4/5 * x - 4, where x - your estimation. Here is it on the chart.

enter image description here

Now what you only need is the absolute value of deviation from the estimated value. Say it would be diff and according to your comment below the actual work is measured as 200 units of whatever. Then you calculate:

Accuracy = Sigmoid(d(10 - |diff|)) * 100,

hence if your diff = +/-5, your accuracy would be 50%, if your diff = +/-190, your accuracy would be zero.

And here is the resulting graph for your case:

enter image description here

  • But this formula does not really make sense: if you estimate 10, and observed 200, the formula result is accuracy of 95%! And if your estimate is entirely correct, then VA-VO=0 and the whole result will be 0. – John V Feb 8 '18 at 11:06
  • @user970696 check my update on Sigmoid approach – Alexey R. Feb 8 '18 at 13:09
1

First we need to define 'Accuracy'.

If we overerestimated, then it's simple - if we spent 1/3 as much time, our accuracy was 1/3. 1/N x estimate = 1/N accuracy.

Underestimating, though, is the complicated part. What I'm going to assume is that if we spent 3x as much as we estimated, then our accuracy is 1/3. N x estimate = 1/N accuracy.

Assuming you want accuracy as a formula of how much the actual differed from the estimate, then the simplest is:

Code: return (act<=est) ? act/est : est/act;

Excel: =IF([@ACT]<=[@EST],[@ACT]/[@EST],[@EST]/[@ACT])

Case examples:

  1. Est = 5, Act = 0; Acc = 0
  2. Est = 5, Act = 1; Acc = 0.2
  3. Est = 5, Act = 3; Acc = 0.6
  4. Est = 5, Act = 5; Acc = 1
  5. Est = 5, Act = 10; Acc = 0.5
  6. Est = 5, Act = 16; Acc = 0.3125
  7. Est = 5, Act = 2000; Acc = 0.0025
  8. Est = 5, Act = infinite; Acc = 0 (Hypothetical)
0

1-MIN(Actual,ABS(Estimation-Actual))/Actual

...To show how this would work with examples:

Case 1: Estimation 5, Actual 3:

1 - MIN(3, ABS(5-3))/3
= 1 - MIN(3,2)/3
= 1 - 2/3
= 1/3

Case 2: Estimation 3, Actual 5:

1 - MIN(5, ABS(3-5))/5
= 1 - MIN(5,2)/5
= 1 - 2/5
= 3/5

Case 3: Estimation 3, Actual 3:

1 - MIN(3, ABS(3-3))/3
= 1 - MIN(3,0)/3
= 1 - 0/3
= 1
  • 1
    Hi, welcome to PM.SE! Although your formula may be applicable in some cases, it lacks some explanation. Some people may not understand your usage of MIN function, for instance. Could you please explain how your formula works to avoid flags and downvotes? Thanks! – Tiago Cardoso Oct 13 at 9:47
0

For accuracy the way you seem to think of it (as a number from 0 to 100), the simplest is probably

100 * MIN(actual, estimated) / MAX(actual, estimated)

where MIN(A,B) returns the smallest of A or B; and MAX(A,B) returns the largest of A or B. This is equivalent to Sarov's answer.

The problem with this formula is that it isn't linear. For example, given an actual of 10, if the estimate was under by 5 then the accuracy is 50%, but if it's over by 5 then the accuracy is 66%. And if the overestimate amount doubles from 5 to 10, the accuracy only changes from 66% to 50%.

You're going to have this problem no matter what formula you choose. It's inherent in the fact that you're requiring accuracy to be a number between 0 and 100.

To avoid that problem, I would consider using something different, such as the error in the estimate. So something like

Estimation Error = |(Estimated Effort)/(Actual Effort) - 1|

Multiply by 100 to convert to percentages (for example, 1.0 becomes 100%). So instead of wanting an accuracy of 100%, you want an estimation error of 0.

Some examples with this formula, keeping the same example of actual=10:

  • An estimate that is off by 10 (either way) gives an error of 1.0 (or 100%). In other words, the error is 100% of the actual.
  • An estimate that is off by 20 (so the estimate was 30 but the actual was 10), gives an error of 2.0 or 200%. In other words, the error is 200% of the actual, and by the way it is double the error of an estimate that was only off by 10.
0

I'm just beginning, and despite my search, I cannot find any simple formula that would return a number between 0 and 100%

Let's look at the meaning of the numbers first, because I would instinctively not expect a 0-100% range to be the most valuable metric here.


My formula

What you're interested in is accuracy, i.e. how far you were off target. The important thing to note here is that overestimation vs underestimation is irrelevant in terms of accuracy.

  • If you end up with 125% (overestimation), that's 25% over.
  • If you end up with 75% (underestimation), that's 25% under.

Whether over or under, what matters is that you were off 25%.

As you already found, Estimation Effort / Actual Effort gives you the 125%/75% value. To reduce this to the 25% value, you need to subtract the formula is:

| (Estimation Effort / Actual Effort) - 100% |

Note that you could also use | 100% - (Estimation Effort / Actual Effort) | since the absolute values are equal. It yields the same result.

This tells you how far you were off, where 0% is the best result, and there is no upper bound on how wrong you were. This is important to remember!


Your formula

The formula you posted:

Actual Effort - |Estimation Delta| / Actual Effort

can algebraically be reduced to

Actual Effort * ( 1 - |Estimation Delta|)   

This is very close to the formula I just mentioned, but it seems like you've either made an algebraic mistake somewhere or are taking a roundabout way to come to a similar conclusion.


Expecting a 0%-100% range

any simple formula that would return a number between 0 and 100%

While 100% makes sense (perfect estimation), 0% does not. Where is the lower boundary of 0%? If I estimate 1 day of work:

  • and it actually takes 1.5 days, you might say that is an estimation accuracy of 50% since I was 50% off.
  • and it actually takes 2 days, you might say that is an estimation accuracy of 0% since I was 100% off.
  • and it actually takes 366 days, what percentage would you say I was off?

Since I was 36,500% off, any formula that would net you the first two values would now net you a result of -36,400%.

If you were to cap this to 0%, that would mean that based on your metric, you would be unable to differentiate between any actual effort that is more than double of the estimated effort. Especially when talking about small tasks, the likelihood of this happening increases dramatically. For a very short task, any bug or obstacle is likely to take a large chunk (percentage-wise) of the planned work time. That same bug would take a smaller chunk (percentage-wise) of the planned work time of a large task.

There is no upper boundary on accuracy. That is to say that when you overestimate, there is an upper boundary (since actual effort cannot be below 0, you invariably can never be off by more than 100%). But when you underestimate, it's possible in extreme cases that you were off by several orders of magnitude.

The number system you are proposing obfuscates those cases and makes out as if the last two examples I mentioned (taking 1 extra day or taking 366 extra days) are equally wrong, which they most definitely are not.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.