When discussing reviews and inspections, one of the first questions will often be “What will it save me?” Next to time and money, an important part of the savings are in aspects of cooperation, comprehension and acceptance. This warm handover will definitely provide a smoother flow within your project and between demand-supply, if applicable.
Naturally, the numbers are important as well. That’s why you need an ROI-calculation. In this calculation, the amount of time saved is calculated based on the amount of fixed Majors and ROI-factor (the amount of hours saved per fixed Major).
The consequence of choosing
Choosing a reference for the ROI-factor, determines the outcome of your calculation of the amount of hours saved. However, what is the impact of your choice?
In the following examples, we assume that finding and fixing a Major in the phase where it was created, costs about 1 hour (1).
First, lets assume fixing a Major issue in the phase it was created takes half the time compared to fixing the same Major in the next phase. Each phase a Major escapes, the costs of fixing doubles. With (pre)production as a reference, this leads to the following savings in hours per fixed Major per document type.
When clear to everybody, this way of calculating the ROI is perfectly valid. Maybe you should use the testing phase as a reference and not the (pre)production phase since the testing phase is thé phase dedicated to finding defects.
However, not all escaped Majors will escape to the last phase of the project. Part of it will be found and fixed in one of the phases following the phase in which it was injected. The ROI-factor per Major will be somewhat like below when we assume all Majors will be found and fixed equally spread over all following phases.
The Major issue injected in the REQ-phase will be found and fixed for 20% in the REQ-phase (32 hours saved), 20% in the FD-phase (16 hours saved), 20% in the TD-phase (8 hours saved), 20% in the Code-phase (4 hours saved) and 20% in the TC-phase (2 hours saved). The average amount of time saved is (32+16 + 8 + 4 + 2)/5 = 12.4 hours.
For FD’s it’s (16+8+4+2)/4 = 7.5 hours. Etc.
In both situations, with a fixed reference or with a dynamic reference, it’s clear that fixing a Major as soon as possible will lead to the highest ROI.
Dynamic ROI-calculation, different variations
In the previous example of dynamic ROI-calculation, we assumed an equal distribution over the phases for finding and fixing the Major issue.
Now, let us assume that it’s not thát easy. Probably the distribution of finding and fixing is not equally over all phases. Probably there is some percentage distribution.
The following examples are based on a Major issue injected in the REQ-phase. The columns show the hours saved based on the cumulative percentage of escaping. Like in above examples, the ROI-factor is 2.
- The first column (0%) shows the ROI in hours when 0% of the injected Majors escapes the injection phase.
- The second column (25%) shows the ROI in hours when 25% of the Majors escape to the following phase . 25% Escapes the first phase, 25% of 25% escapes the second phase, 25% of 25% of 25% escapes the third phase etc.
- The third and fourth columns (50% and 75%) show the ROI in hours when 50% (resp. 75%) of the Majors escape to the following phase.
- The fifth column shows that the ROI in hours gets very low if a Majors escapes to the last phase. After this phase, the ROI will turn zero since no efforts are put into finding and fixing Majors.
Regardless of the percentage of Majors that escape in a phase, there is a positive ROI compared to letting the Major escape one more phase. However, the difference in ROI get’s lower with each phase. (32 – 18.3 = 13.7) (18.3 – 10 = 8.3) (10 – 5.8 = 4.2) (5.8 – 4 = 1.8)
So, depending on the “escape percentage”, the ROI in hours obviously changes.
An ROI-factor of 2. Really?
Then we move on to the situations in which it costs less than a factor 2 to fix an escaped Major. This ROI-factor 2 caused a saving of 2^5 = 32 hours for a Major injected and fixed in the REQ-phase. Maybe the project works in small iterations so finding and fixing a Major costs less time. Maybe the project decides to fix the issue in the current phase without reworking the previous phases. Maybe the projects works in large waterfall steps where finding and fixing a Major costs much more than a factor 2.
Let’s see what happens with some hypothetical but probable ROI-factors. The graphs show, again, the ROI-factor depending on the percentage of Majors that escape to the next phase.
Factor 1.3 average to fix a Major (1.3^5 = 3.7)
Factor 1.5 average to fix a Major (1.5^5 = 7.6)
Factor 2.5 average to fix a Major (2.5^5 = 97.7)
Factor 3 to fix a Major (3^5 = 243)
Most likely the numbers now dazzle you. Unless you’ve skipped the above and scrolled to the conclusion directly 🙂 What you have to remember is that there are different ways to look at the ROI-factor. In the end it does not really matter which method you use, as long as everybody understands what your reference is.
Personally, I prefer the most simple one. Take the last phase of your project and double the time needed to find and fix a Major for each following phase. Of course this does not match reality since Majors will almost never escape to the production phase. However, if nobody understands your ROI-calculation, probably nobody will take the time to understand it at all. And it takes you a lot of time to explain 🙂
Unless you experience an ROI-factor <1, all measures to find and fix Majors will help to save money. Structured reviewing is one very effective measure to take.
(1) Based on my own metrics calculated over >300 business requirement documents. Calculated over áll document types (>1000 documents), the average is 0:42