Category Archives: Blog

Walkthrough | hguorhtklaW

Boring walkthroughDo you recognize the situation in which a walkthrough is organized where the author talks a group of attendees through a document, turning pages and explaining what already is written down? Where the attendees are more focused on their smart phones and laptops instead of trying to concentrate on the topic at hand? Maybe it’s time to reverse roles instead of turning pages.

A walkthrough is defined by IEEE (1028-1997, IEEE Standard for Software Reviews)  as a form of review “in which a designer or programmer leads members of the development team and other interested parties through a software product, and the participants ask questions and make comments about possible errors, violation of development standards, and other problems“.

When well-organized and well-prepared, this form of review certainly can be effective. The reality however is that only the author is partly prepared and the attendees play a passive role or just start thinking about the subject at the time the author mentions it. This leads to a situation in which a passive attitude overrules the active.

What would happen when we reverse roles? The reviewers explain the author what he wrote in the document. This requires an active participation of the reviewers (in preparation and presentation) and leaves no option for the author who needs to be active as well at least during the presentations.

Reverse WalkthroughThis method is called a “reverse walkthrough”. The definition is not described by IEEE, so I took the liberty to define it as follows: “A reverse walkthrough is a form of review in which the development team and other interested parties lead the designer through it’s own product, and the designer asks questions, makes comments and provides clarification on misinterpretations and ambiguities.”

A reverse walkthrough should be performed when the product has a high risk of being misunderstood. For example when it’s written in an other language, when a lot of slang is used, when it’s placed within a very specific context or when it concerns a very complex matter. One or more reviewers summarize their view on the document in a short presentation, as seen from their own role. The moderator makes sure reviewers get time to give their view, keeping discussions short but allowing short explanations and exchanges of ideas.

I will spend some time on writing down the process and guidelines for the reversed walkthrough on this website, so you can try it out yourself!

Update 06-10-2016: the description of the process and guidelines can be found here.

Reviewing in SCRUM

reviewinscrum“In our SCRUM team, we do not need to perform reviews.” is one of the first remarks when talking about reviewing. I think more reviews are done than you might be aware of. A first thought below. Can you think of more situations in which you review in a SCRUM situation?

It starts with the items on the backlog. Are these clear to everyone concerned? What does ‘clear’ mean? Can you build, test and accept the items, or is any extra information needed to do this? Asking yourself this question is already static testing the backlog items. Why not do this in a structured way?

  • Pay attention in the retrospective to see if there are any learning points mentioned that can be prevented at the start of a sprint by clarifying the backlog items.
  • Evaluate for yourself: did you ask for clarification on the same subject for more than one backlog item? It might be good to add this subject to your review-checklist at the beginning of each sprint.
  • Are clarifications, if necessary, always directly updated into the backlog item descriptions? If not, consider the option to perform the review (using a checklist) during the planning poker. Update the items directly when a clarification was needed.

Then static testing during the sprint. Are static tests done on code, (website)content or regression test cases? These are reviews, too! Depending on the situation you can discuss your findings directly with the author, or you can group you findings and deliver it in a batch. In all situations, a strategy can help you in structuring your review and making it as effective as possible. Use checklists (which you maintain during the sprint retrospective) to be sure not to forget specific subjects during your reviews.

  • Static code review: sometimes peers or testers are asked to review a piece of code a developer just created. Using a checklist based on previous experiences,  it becomes easy to maintain grip on the review. It also makes the review more predictable when other team members are asked to perform the review. Most of the time, findings can be fixed directly.
  • Review content: in some situations textual content is part of a deliverable. For example in websites, news letters, communication to customers, billing etc. it might be needed to review the contents before the item it Done. In some situations, findings can be fixed directly. In other situations it might be needed that more people, even from outside the SCRUM team, need to take a look. A more structured approach of the review might come in handy. The process as described on this website is a good starting point. Use whatever suits your situation.
  • Review (regression) test cases: test cases in SCRUM sometimes are considers ‘throw away test cases’. A peer review at the coffee machine might be useful enough. On regression test cases, which probably will exist for a longer time and which are quite important during the whole project to prove stability of the business value, might require a more intense review. If more than one or two reviewers are involved, especially if these people are not all part of the SCRUM team: a more strict process will be useful to keep the review within it’s time boundaries and to make sure all participants focus on their personal area of interest.

At the end of the sprint, during the sprint review, a demo is given of the items that are ready to be delivered. This demo also is kind of a review; in a live setting a product is shown to one or more reviewers who can comment on what they see. It can be organised as any review or inspection meeting. Time-boxed, some form of comment administration and a moderator (i.e. the scrum master) who leads the session.

In SCRUM, reviews are part of the daily activities. Without having a review process, the review activity itself can be structured in a way that suits the situation. The structure, be it in the form of checklists or in the form of agreements (mentioned in the DoD!) provides the team a guide to catch as much as possible during the reviews.

— update 12-01-2017 —
A slightly adjusted version of this blog is published (in Dutch):
– Tijdschrift voor IT Management 01-2017 (

Structured Reviewing Academy opens its doors

SR_AcademyLogoToday the Structured Reviewing Academy has opened its doors. In this online e-learning environment you can enroll into courses and get certified on subjects related to structured reviewing.

The first course, Reviewer Foundation, is now available. Registration to the Structured Reviewing Academy is completely FREE! If you pass the exam at the end of the course, you will receive a digital certificate. (Which you can show your manager, connect to your LinkedIn-account or print and put above your bed)

Check it out! Building Blocks

Yesterday the Structured Reviewing Building Block went live on This is the first step in which a generic description of Structured Reviewing is available. More details will be added later in the WIKI that’s being build on the background of these Building Blocks. Of course, a lot of details can already be found on! Building Block

The website is part of the TMap Suite which consists of the well known TMap NEXT test method, the new TMap HD (Human Driven) quality driven approach and the website including the Building Blocks. is an initiative of Sogeti (and .com) and forms a free knowledge base sharing test and quality knowledge.


Wikipedia – Sinterklaas

Hi <name>,

It is my current impression,
Reading the comments there are,
There’s no need for a session,
On <author> his BR.

So please take your time,
Use your human cognition,
Let me know in a rhyme,
If you’d approve the revision.

I’d like to receive your reaction,
Before the 9th of next week,
Reply on this mail,
So I don’t have to seek.

Enjoy coming weekend,
And let me be clear,
It’s all ’bout the contents,
I’ll see you next year.

Sinterklaas and ReviewPiet

And this are some of the reactions:
As I still not know enough
Judging this revision is tough
Therefore I will let this through
And reply this poem to you
Heb het commentaar gezien
Met een reject die ik niet verdien
Toch maar de auteur gespaard,
Het punt is geen verdere discussie waard.
Verder ben ik meer een man van peace
Zodat ik deze versie release.
I went through my comments,
And the reactions too,
If you ask me for approval, then I say,
I do, I do, I do
Dear Piet, Approved
There is not much that I can say
I had no comments anyway

Looking at the comments in the sheet
And the responses of <author>
The updates done can be agreed
So the BR will be planned soon.

Summary of this epistle :
For as far as I’m concerned
You can blow the whistle
The approval of the BR is earned.

A matter of perspective

When reviewing, everyone is asked to judge the document under review from his own perspective. More eyes can spot more errors, inconsistencies and omissions. I’ve gathered some video’s from YouTube, illustrating the need for multiple roles, but also illustrating the limits that exist because of your role and focus.

Based on this illustration, some lessons can be learned.
– More roles involved means more focus, which might lead to logging more issues.
– More than two reviewers per role will, in most cases, not lead to logging more issues.
– Everyone has it’s own focus, based on role, experience, knowledge etc. One reviewer will not, and does not have to, find all issues.

First, check out this movie until 0:55 and see how many changes you can spot. Note them down. Then, gather some colleagues and divide the attention. (Back wall, left wall, floor, people in the back, attributes on the table) How many changes did you spot now? Watch the rest of the movie to see what you’ve missed!

How many passes does the team in white make?
This video also illustrates that your role, your experience, your focus can affect your reviewing capabilities.

“We already discussed it, so I don’t have review comments”

Right! If I’d be paid for each time I’ve heard this reaction…

The positive side of this reaction is that communication apparently is existing. It’s good to have discussed a design with different stake holders. But, as communication can lead to misunderstanding, all meetings and discussions still do not guarantee that all parties have the same interpretation.

To illustrate this, I’ve collected some YouTube movies. Probably you recognize the situations and understand that having had a discussion or meeting does not imply a correct design. Reviewing is still needed!

Check out Wikipedia for the history and description of the game “Chinese Whispers” or “Telephone”.

Chinese Whispers

The telephone game

Stille post (German)

What does reviewing save you?

When discussing reviews and inspections, one of the first questions will often be “What will it save me?” Next to time and money, an important part of the savings are in aspects of cooperation, comprehension and acceptance. This warm handover will definitely provide a smoother flow within your project and between demand-supply, if applicable.

Naturally, the numbers are important as well. That’s why you need an ROI-calculation. In this calculation, the amount of time saved is calculated based on the amount of fixed Majors and ROI-factor (the amount of hours saved per fixed Major).

The consequence of choosing
Choosing a reference for the ROI-factor, determines the outcome of your calculation of the amount of hours saved. However, what is the impact of your choice?

In the following examples, we assume that finding and fixing a Major in the phase where it was created, costs about 1 hour (1).

First, lets assume fixing a Major issue in the phase it was created takes half the time compared to fixing the same Major in the next phase. Each phase a Major escapes, the costs of fixing doubles. With (pre)production as a reference, this leads to the following savings in hours per fixed Major per document type.

When clear to everybody, this way of calculating the ROI is perfectly valid. Maybe you should use the testing phase as a reference and not the (pre)production phase since the testing phase is thé phase dedicated to finding defects.

However, not all escaped Majors will escape to the last phase of the project. Part of it will be found and fixed in one of the phases following the phase in which it was injected. The ROI-factor per Major will be somewhat like below when we assume all Majors will be found and fixed equally spread over all following phases.

savingMajNextThe Major issue injected in the REQ-phase will be found and fixed for 20% in the REQ-phase (32 hours saved), 20% in the FD-phase (16 hours saved), 20% in the TD-phase (8 hours saved), 20% in the Code-phase (4 hours saved) and 20% in the TC-phase (2 hours saved). The average amount of time saved is (32+16 + 8 + 4 + 2)/5  = 12.4 hours.

For FD’s it’s (16+8+4+2)/4 = 7.5 hours. Etc.

In both situations, with a fixed reference or with a dynamic reference, it’s clear that fixing a Major as soon as possible will lead to the highest ROI.

Dynamic ROI-calculation, different variations
In the previous example of dynamic ROI-calculation, we assumed an equal distribution over the phases for finding and fixing the Major issue.

Now, let us assume that it’s not thát easy. Probably the distribution of finding and fixing is not equally over all phases. Probably there is some percentage distribution.

The following examples are based on a Major issue injected in the REQ-phase. The columns show the hours saved based on the cumulative percentage of escaping. Like in above examples, the ROI-factor is 2.

  • The first column (0%) shows the ROI in hours when 0% of the injected Majors escapes the injection phase.
  • The second column (25%) shows the ROI in hours when 25% of the Majors escape to the following phase . 25% Escapes the first phase, 25% of 25% escapes the second phase, 25% of 25% of 25% escapes the third phase etc.
  • The third and fourth columns (50% and 75%) show the ROI in hours when 50% (resp. 75%) of the Majors escape to the following phase.
  • The fifth column shows that the ROI in hours gets very low if a Majors escapes to the last phase. After this phase, the ROI will turn zero since no efforts are put into finding and fixing Majors.


Regardless of the percentage of Majors that escape in a phase, there is a positive ROI compared to letting the Major escape one more phase. However, the difference in ROI get’s lower with each phase. (32 – 18.3 = 13.7) (18.3 – 10 = 8.3) (10 – 5.8 = 4.2) (5.8 – 4 = 1.8)

So, depending on the “escape percentage”, the ROI in hours obviously changes.

An ROI-factor of 2. Really?
Then we move on to the situations in which it costs less than a factor 2 to fix an escaped Major. This ROI-factor 2 caused a saving of 2^5 = 32 hours for a Major injected and fixed in the REQ-phase. Maybe the project works in small iterations so finding and fixing a Major costs less time. Maybe the project decides to fix the issue in the current phase without reworking the previous phases. Maybe the projects works in large waterfall steps where finding and fixing a Major costs much more than a factor 2.

Let’s see what happens with some hypothetical but probable ROI-factors. The graphs show, again, the ROI-factor depending on the percentage of Majors that escape to the next phase.

Factor 1.3 average to fix a Major (1.3^5 = 3.7)savingMajPhases_1_3

Factor 1.5 average to fix a Major (1.5^5 = 7.6)  savingMajPhases_1_5

Factor 2.5 average to fix a Major (2.5^5 = 97.7) savingMajPhases_2_5

Factor 3 to fix a Major (3^5 = 243) savingMajPhases_3

In conclusion
Most likely the numbers now dazzle you. Unless you’ve skipped the above and scrolled to the conclusion directly 🙂 What you have to remember is that there are different ways to look at the ROI-factor. In the end it does not really matter which method you use, as long as everybody understands what your reference is.

Personally, I prefer the most simple one. Take the last phase of your project and double the time needed to find and fix a Major for each following phase. Of course this does not match reality since Majors will almost never escape to the production phase. However, if nobody understands your ROI-calculation, probably nobody will take the time to understand it at all. And it takes you a lot of time to explain 🙂

Unless you experience an ROI-factor <1, all measures to find and fix Majors will help to save money. Structured reviewing is one very effective measure to take.



(1) Based on my own metrics calculated over >300 business requirement documents. Calculated over áll document types (>1000 documents), the average is 0:42