The Worst and Best Review Scenarios for Technical Documents

peer review
Review process is where most technical documents either sink or swim: either they improve as a result of to-the-point and clear comments or they become muddled and actually deteriorate as a result of overlapping and conflicting feedback comments.
I believe the quality and outcome of a review process has a lot more to do with the way it is structured than the individual characteristics or the qualifications of the review team since more often than not the reviewers are almost all SMEs who know their stuff inside out. But despite that, if the process is not structured well it’s bound to fail and disappoint.

(1) Here is a worst case scenario:

You send out your document to multiple reviewers at the same time, and in turn, they also distribute it to pretty much everybody they know. There are no deadlines to the reviews.
So the comments start to trickle in from one then the other reviewers.
For a while there might be a pause. Nothing comes in. Then, this might be followed by an avalanche of notes from people you don’t even know (assuming you work for a big company with thousands of employees).
As you wade your way through layers of varying or even contradictory comments, you start to get second and third draft comments from the earlier reviewers who begin to react to each other’s comments.
Soon it turns into a free-for-all commenting tornado where you start to get hit from all directions and you don’t know which comment to give priority to. And in the meanwhile your deadline comes and goes and you are left there standing as the fall-guy and taking one on the chin for the team, even though that was not your original intention. There is no clean and rational closure to the process.

(2) And here is a much better scenario:

You select a Principal Reviewer (PR) to coordinate all reviews and approve the final draft of all comments.
Three deadlines are set for the first, second, and final draft of comments.
You send the document only to the PR, with the deadlines for all three cycles. You make it clear that you will accept the feedback comments only from the PR and not directly from individual reviewers. Thus the PR becomes the gatekeeper for qualifying and funneling the comments.
The PR does not send out the document simultaneously to all the reviewers.
First one reviewer goes through the document and records his or her comments. The document that is passed on to the second reviewer who has a chance to view the comments made earlier by the first reviewer. This way the second reviewer does not end up repeating the comments by the first one. And if there is a disagreement, the second one can take it directly with the first one instead of leaving it to you to resolve the conflicts.
The process is repeated as many times as there are reviewers.
In this serial fashion, the reviewers finish the comments and the document goes back to PR who puts the final seal of approval before returning the document back to you for editing.
This cycle is repeated for one or two more times, if necessary. And once all the cycles are completed and once you secure the PR’s final approval, the review process is done. Period. There is a rational, documented, and mutually-agreed-upon closure to the process.

Software Help

There are many software programs out there that lock the checked out document so that no two reviewers can edit a document at the same time. When the first user checks in the document, the program allows another user to check out the document. MS SourceSafe, for example, is one such program. Adobe Acrobat X (http://www.pcmag.com/article2/0,2817,2370981,00.asp) and integrated documentation platforms like Author-it offer a similar capability as well.
These programs work best when there is a central review authority assigned to reconcile all the different comments and relay them to the writer as the “official set” of comments.
When such a Principal Reviewer does not exist, the technical writers end up spending a lot of time and effort not only to incorporate the comments back into the source document but also to justify their choices (why Comment A was accepted for Item X but Comment B was declined) and sometimes backpedaling and adopting a position rejected earlier (deleting Comment A for Item X and incorporating Comment B).

2 Comments

  1. Pidge Perry on March 5, 2012 at 11:14 am

    This is a good idea.
    I was wondering, though – what do you think about having all the reviewers (and the tech writer) in a meeting at the same time? This way the reviewers come in with their edited document, and everyone goes through the document at the same time – thereby hashing out any discrepancies right then and there.
    What do you think?



    • admin on March 5, 2012 at 11:35 am

      Pidge, thanks for writing. That’s of course a great idea — in theory. In practice, everybody is “busy, busy, busy…” Since in most engineering organizations documentation is delegated a low priority, project managers usually focus on development issues first. The priority is usually on getting the product to the market on time in good working order. That’s why the reviews are done in a piece-meal fashion. If you can get all your reviewers (assuming there is a fixed list of names) attend regular “review meetings”, coordinate the effort and iron out the discrepancies at those meetings, power be to you. That would really help the review process.