The Rules of CBR (Can't Be Right)

I gave up using a red pen while reviewing work several years ago because someone in our department said she felt as if she were being graded in school. Regardless of your pen color though, a critical part of performing & reviewing analysis is the ability to quickly spot mistakes. It helps to have a sixth sense regarding CBR—items that obviously “Can’t Be Right.” In case you don’t have that special power, here are rules you can use to help spot mistakes – whether they’re yours or someone else’s:

• Before you work on or review analysis, think about what the answer should or will likely be. If the results aren’t in the ballpark, and there’s no apparent reason, do some digging.
• When reviewing work, start with a “skeptical” attitude - the expectation that something’s wrong - and look specifically for mistakes.
• Assume things typically won’t change dramatically (or at least outside a typical range). If changes look like big deviations from the norm, investigate why.
• Try to “break” things—when testing a spreadsheet or program or reading a document, look for ways to make it not work or look for passages that don’t make sense.
• With a spreadsheet, do the unexpected—put in numbers that you wouldn’t normally expect (i.e., a negative number where it should be positive, change the order of magnitude of important numbers, etc.). As a double check, if a spreadsheet uses lots of formulas, dramatically change some numbers that should make the results change.
• Do things or read sections out of the natural sequence. This often makes irregularities more recognizable.
• When reading, repeatedly ask the questions, “Why would I know that? Does it tell me that somewhere else in the document? Is the point consistent within the document?” Be skeptical when you think you have satisfactorily answered these questions.

While it might feel a little better to not use a red pen while marking up analysis, it feels tremendously better to catch a mistake before your boss or client does. Use these rules to help increase that likelihood. And if you have rules that you use successfully, let me know, and we’ll put them in a follow-up post.