How To Analyse Specifications - Part 1

Specifications are the foundation of a project. It tells Developers what to develop, and it tells Testers what to test. If the Specifications are of a low quality, then the development work and the testing work will be of a low quality - unless the Specifications are improved.

It should be no surprise then, that the very first job of a Tester is to test the specifications. This is often called Static Testing. Some Testers have a tendency to leap into creating test scripts, but this can (and often does) become a waste of time. So it is testing the specifications against themselves that is the Tester's first priority.

But what do you test them against? What are the criteria that determine whether the Specifications are doing their job correctly?

The answer is in the word specifications, and runs from the first letter to the eighth letter - it is the word "specific". This is the first test of the Specifications - they should be specific.

They should not be vague - a Business Analyst once told me that he had "deliberately left the specifications vague to cover a range of possibilities". I was unimpressed.

Specifications should not use words that don't describe specifically should happen.

Words and phrases such as "handle appropriately", "correct", and "as expected" are very often used in Specifications. I call them "fluffy" words or phrases, because they describe approximately what should happen, but they do not clearly describe what should happen. They are not specific.

The First Test
So this is the first test of the Specifications.

How specific is it? Does it use fluffy words? Does it use fluffy phrases?

These questions need to be raised by the Tester, and clarifications sought wherever the Specification fails the test, e.g "What do you mean when you say that the data output should be handled appropriately? How exactly should it be handled?"

The Second Test

The second test of the Specifications is accuracy.

Now evidently, the Tester is not, at least in the beginning, in a position to say that the Specifications are inaccurate. They are based on discussions between the Business Analyst and one or more people in the business, who we shall refer to as the Product Owner(s). Since the Tester was not a part of those discussions, they are not in a position to say whether or not the Specifications accurately reflect what was said.

But as a Tester, you need to look out for functionality being described differently in different parts of the Specifications - you may even find contradictions within the Specifications!

This can often involve getting out pen and paper and rearranging various points of the Specifications. You can do this in written form, or table form. In the process of this rearrangement, you will see new patterns emerging, which will highlight to you any differences.

This is testing the documentation, or Static Testing.