Carnegie Quality

The company I work for recently finished a software project. I was hired right before testing began and we had engaged a systems integrator to do the analysis, design, and development. For reasons still unknown to me, the SI did not document any software requirements. It should come as no surprise that testing didn’t go very well – we often had difficulty validating pieces of functionality (what is this function supposed to do?) and there was a lot of re-work as the project identified missing or mis-implemented requirements.

It goes without saying that the project would have gone a lot smoother, completed closer to the planned delivery date, and produced a higher quality product if requirements had been done. But requirements should be done correctly. A best practice within requirements analysis is the use of the Requirements Traceability Matrix (RTM). In this post, I’ll briefly describe the RTM and explain why you as a tester should be leveraging it.

The Requirements Traceability Matrix

In the broadest terms the RTM is simply a matrix of requirements showing each requirement’s relation to something. It could be the source for the requirement or the design component that describes the implementation of the requirement. Or it could be the test case(s) that covers the requirement. Just think of a spreadsheet with a list of requirements, a unique ID for each requirement, and the corresponding relationship.

Advantages of Using the RTM

The tool itself is very straight forward – it’s the proper use of the tool that provides benefit to your project. The RTM is not something that is used solely by the analysts or testers – it is something that the entire team utilizes.

Ensure Complete Test Coverage

How do you approach defining your test cases? More importantly, how do you know when you’re done? If you use a RTM, you always know the answer to this. As you plan out each test case, decide which requirements each test case will cover – then map those in the RTM. When all requirements are matched against a test case, you’re done (though you’ll probably still want to look at special cases, alternative tests, etc…).

Help Focus Testing

Along with knowing when your test coverage is complete, a RTM can help reduce unnecessary focus on the same set of requirements. You can quickly see in the matrix when a requirement is covered and avoid continuing to assign more test cases to it.

Validation of Requirements Consistency

I’m not suggesting that testers are responsible for validating requirements – we’re not. This should be done during requirements reviews and walkthroughs, and if problems are not caught there, they should certainly be found during technical design. But the reality of life is that these things are often not done as well as they should be, and requirements sometimes conflict with each other. You may have one requirement state that all numeric input fields are integer, and then another one that states that input field #5 on screen #2 is a numeric field with precision to 2 decimals. Using a RTM can help find these kind of problems.

Permit Prioritization

Another question for you – you’ve just been given the system by the development team and have been asked by management to do a quick assessment of quality. How would you approach this? There are probably numerous ways to attack this, but one way is to look at the RTM and see which set of test cases would cover the largest set of requirements and do those first.

Provide Proof of Testing

Your sponsor asks you “How do you know you’re done with testing? Did you test everything?” With a RTM it’s easy to provide the proof.

Help With Future Projects

Finally, if you have a system to collect the right kind of data, you can provide valuable insight to management for future work. Which requirements resulted in the most defects? Which requirements had the most Change Requests performed? Data like this allows management to assess high risk areas of the application and plan against those risks in the future.

For More Information

I hope you’ve found this introduction to the RTM helpful. Most good books on testing will have a word or two to say about the RTM. There is also a good collection of links at this site.

Use Cases have been around for quite awhile now and are considered by many to be best practices in software development. I continue to be surprised at how few organizations actually use them though. Why is this? Too difficult, lacking in the right skill set, perceived to be too much work for not enough benefit, or just plain organizational inertia? Not sure myself, but I do know that if you’re not using Use Cases, you should be.

I’m working on polishing up a Use Case template and will publish it soon – of course you can get a zillion off the web yourself. I’d suggest starting at the Wikipedia entry and going from there. If you haven’t already, you should really own a copy of Bruce Cockburn’s Writing Effective Use Cases, the only book on the subject you’re going to need, unless you’re doing some really in-depth stuff.

If you need some ammo within your organization on why you should be working with Use Cases, here you go:

Functional Requirements Presented in Text Form

Use Cases are a great way to present functional requirements in an easy-to-understand text form. End users and sponsors in particular appreciate having requirements in a format they can quickly interpret. Process flow diagrams are often used to capture business processes, but they are not as easy to understand by those not trained in diagramming processes.

A Common Language and Understanding

It’s easy for everyone involved on the project to refer to Use Cases as a common understanding about the project. Sponsors can easily view the range of functionality and comment on things left out, or items that don’t need to be present. Developers can refer to Use Cases for a quick way to come up-to-speed on how the business operates and how the various functional requirements fit together. Use Cases tell end users exactly how they will interact with the finished application.

Get Your Requirements Reviewed

Because Use Cases are in straight-forward text, they are much easier to review. This will greatly improve the odds that at least some of the project’s documentation is reviewed and commented upon. You always want to prevent hearing sponsors or end users say “That’s not what we wanted” at the end of the project!

Who Does What – The Actors

Use Cases have actors – which are the stakeholders of the system. Actors can be primary end users, secondary users, organizations, or other systems. By listing out the actors, you get a clear picture of everyone involved in the system.

And Why – The Goals

Each actor within the system has a goal, or something that they need to achieve. Documenting what you need the system to do seems like a simple thing to do, but you’d be amazed at how it focuses the project on the things that really matter.

Goals Fail

If you’re writing Use Cases properly, you’re thinking about failure scenarios. What happens when a particular goal is not met? Documenting failure scenarios and conditions early in the process helps create a more robust and complete design.

Scope

Take all of your Use Cases together and you have your total system scope. Use Cases help you form the boundary between what the project is doing, and what it’s not doing.

Test Scenarios

Use Cases make a great starting point for test scenario and test cases. They define users, roles, responsibilities, goals, steps, and error conditions – a large chunk of what you need for your test scenarios and test cases.