In this post, I’ll discuss some common techniques for analyzing and identifying solutions – and also provide some links to additional information.

Assumption Analysis

List out all of your assumptions about the problem, and then analyze each of them. Are they all necessary? If it were possible to change one or more assumptions, would a solution to the problem exist? Check out Virtual Salt for more details on this approach.

Cause and Effect Diagrams

Cause and effect diagrams – also called Ishikawa diagrams (after Kaoru Ishikawa) or fishbone diagrams (due to their appearance) – allow you to analyze the causes of a particular problem. You begin by drawing a horizontal line that represents the problem. Then you draw a “spine”, or diagonal line to the left off from the main problem line, that represent factors of the problem. These factors might be systems, people, processes, policies, materials, etc… Then for each identified factor, drill down and draw a vertical line for each possible cause of that factor. Asking “why” is a great technique to uncover possible causes. You can then use additional techniques, like a Pareto chart, to get additional insight into the problem. There’s a good explanation and example at MindTools.

SWOT

SWOT analysis is a strategic analysis approach that analyzes strengths, weaknesses, opportunities, and threats. First – a business objective/desired end state is defined. Then, key strengths (assets/characteristics that can help meet the objective), weaknesses (attributes of the situation that hurt the objective), opportunities (external conditions that can help meet the objective), and threats (external conditions that can hurt the objective) are identified. These are usually visually represented in four quadrants on a single page. The key to using SWOT analysis effectively is to start with a business objective, and then make sure you properly segment out internal conditions (strengths and weaknesses) from external ones (opportunities and threats).
From a testing perspective, SWOT analysis can assist when making strategic decisions, such as whether or not to outsource a portion of the testing effort. There’s a very good article at Wikipedia and some templates over at Businessballs.com.

Affinity Diagrams

Affinity diagrams were developed by Jiro Kawakita and are also known by the KJ method. Affinity diagrams are a good method for organizing the problem space. This method is also good for driving consensus. The process involves writing down ideas on sticky notes during a brainstorming session. Then, without anyone talking, the group organizes all of the notes into related areas. After that, the group can talk again and should discuss the patterns that emerged and develop headings for each collection of notes. Essentially, affinity diagrams help you “chunk” the problem into more manageable pieces.
There is a very good description at ASQ and an Excel add-on available (for purchase) at BaRaN Systems (I haven’t tried this, but it looks interesting).

Risk Analysis

I’ve previously posted about risk identification from a testing perspective and I also have a template available for developing a risk management plan. Risk analysis is a great tool when trying to spend limited resources, since it forces you to identify, prioritize, and plan for risks. You can use this approach in conjunction with decision trees to help, for example, predict how long a test phase might be.
Check out MindTools for more information.

Brainstorming

No doubt you’ve heard of brainstorming – a technique where a group generates a large number of ideas to solve a problem. There are lots of different ways to do brainstorming, but most of the methods share four common traits:

1. A focus on quantity
2. No criticism
3. Encouragement of “out of the box”/unusual ideas
4. Combining and improving on generated ideas

I should note that many people argue that brainstorming generates lower quality ideas that a group working independently. You can head over to the University of North Carolina or Wikipedia for more reading.

Decision Matrix

A decision matrix can come in handy when trying to select from two or more alternative solutions. First – come up with dimensions that you want to use to assess the solution (e.g. cost, ease of implementation, effectiveness, etc…). Then weight each factor – it’s easier if you start off with a given total of points to use (say 100), and then allocate those across each factor based on their importance. Next, assess each solution against each factor – use a numeric scale (such as 1-10). Finally, produce a weighted score (solution’s score for a given factor * that factor’s weight) and sum the results. The solution with the higher score is the better solution. Look at this article on the ASQ website for more details.

There are plenty of other techniques, but hopefully this list will get you started. Check out MindTools and the American Society for Quality for more.

Why I am posting a risk management plan on a site about testing? Well, for a couple of reasons. First, I’m trying not to limit information on my blog to testing only – I’d like to incorporate information on quality assurance, which I feel is much broader than just testing. Second, while I agree that risk management is ultimately the project manager’s responsibility, I believe that quality assurance/testers have unique insight into risk, and therefore should play a key role within risk management. And finally, I believe that many project managers don’t do risk management (or don’t do it well), and if you find yourself in this situation, it’s better to step up and make sure that risk management is done rather than just shrugging your shoulders and saying “it’s not my job”.

A couple of thoughts after I read this article:

• In my experience, it’s the role of the developer to assess the reason for the defect, not the tester. I would never go through someone’s code – I wouldn’t know what to look for! Though I understand that others are much more technical in nature and enjoy this.
• While the activities that are described in the article are spot on, I think there is too much emphasis placed on the Quality Control (testing) team. The real owner of this function is Quality Assurance. QA is responsible for making sure there are lessons learned – doesn’t mean they will do all the work, but they need to be driving this.
• I’ve had mixed experiences doing these things myself. Many times I’ve done defect analysis and root cause analysis; documented my findings with charts and graphs; and presented them to project management/development to discover not much enthusiasm. Maybe it’s me, but I hope not! What have others experienced?

Quality Control (QC)

Let’s start with Quality Control – if you have a testing background, this is probably what you’ve been doing all along. ASQ describes QC as “the operational techniques and activities used to fulfill requirements for quality.” The QC function is a gate-keeper – it prevents defective products or services from being released. In the software arena, QC is a set of validation and verification (or V&V) activities. Don’t fall into the trap of thinking that this just means software testing – it doesn’t. Verification takes place (or should, at least) at every stage of the software development life-cycle and should include reviews, inspections, and walkthroughs.

Quality Assurance (QA)

QA is a bit harder for some to understand, and is frequently incorrectly used to refer to QC activities. ASQ describes QA as “all the planned and systematic activities implemented within the quality system that can be demonstrated to provide confidence that a product or service will fulfill requirements for quality.” Some key words to consider in that definition are planned and systematic. QA takes place throughout the software development life-cycle, and is not limited to verification activities. QA involves consideration of risk, budget, resource plans, and schedules. The primary objectives of QA are: 1) reduce the rate of deliverables that do not meet quality targets, and 2) reduce the costs of meeting quality objectives. Remember, validation and verification costs are high because they normally require re-work. It is much cheaper to identify problems early in the process, improve your overall processes to reduce non-conformity, and/or consider alternative approaches that reduce risk.

General Testing

How to Break Software: A Practical Guide to Testing By James A. Whittaker
A quick read (just 174 pages with the appendices), but lots of good info. If you’re looking for something to help you develop a tester’s mentality, this is it. Not everyone can be a good tester – you need the right kind of perspective. Most developers when they do unit testing make sure that their code is working. Most good testers I know want to find out how they can break the code. The book is mostly focused on PC applications – there are sections on UI testing and system interface testing. The book also comes with some software, which to be honest I’ve never tried.

Effective Software Testing: 50 Specific Ways to Improve Your Testing By Elfriede Dustin
This is a reference that is essentially 50 fully documented suggestions/best practices. While some of them are obvious (like #2 “Verify the Requirements”) the author does a good job at explaining the whys and hows behind each tip. The suggestions are grouped together logically (there’s a section on the requirements phase, test planning, test design, test management, etc…), so it’s easy to pick up this book over the life of your project and get specific hints tailored for where you are in the project life-cycle.

Software Testing By Ron Patton
There’s a second edition out now, though I haven’t yet picked it up. This is a great introductory book to software testing. It covers the different forms of testing, how to review documentation and code, test planning, and test case writing. One thing that Ron Patton really focuses on is the role of the tester in software quality: “The goal of a software tester is to find bugs, find them as early as possible, and make sure they get fixed.” Couldn’t say it better myself!

Test Management

Managing the Testing Process: Practical Tools and Techniques for Managing Hardware and Software Testing By Rex Black
Second Edition. Chapters include planning, mapping out the test coverage, defect tracking, managing test cases, staffing, and organizational considerations. I enjoy Rex Black’s writing style – it is very down-to-earth with a focus on practical advice. Each chapter ends with a real-life case study and exercise questions. While I think there’s good material for any test manager in this book, it would be particularly helpful for those either new to test management or are setting up a new test system/organization.

Critical Testing Processes: Plan, Prepare, Perform, Perfect By Rex Black
Builds on Black’s previous book. As the title suggests, the book’s focus is on testing processes. You can think of this book as a method for implementing continuous improvement in your testing process. Like his previous book, this one is full of practical advice and tips. I particularly like the planning chapters, and the chapters on reporting defects and producing test metrics. Throughout the book the author illuminates many of the chapters with a hypothetical case study – so you can see how to apply suggestions from the start of test planning all the way through post-test execution process improvement. Very highly recommended.

Quality Software Project Management By Robert T. Futrell, Donald F. Shafer, and Linda I. Shafer
The main audience for this book is Project Managers. Each chapter is well documented with footnotes and has it’s own bibliography, which is quite handy. There are a large number of templates and checklists. The book ties in very well with PMI standards, but is probably a bit more practical than your average PMI project management book (and since it’s on just software, more relevant to the software PM). The book’s strength is tying basic software engineering and software quality assurance concepts in with the process of planning, executing, and controlling a project. The book weighs in at over 1,500 pages, so it may not be exactly what you need – but if your PM doesn’t have a copy, they should.

Quality Assurance and Certification

Software Quality Assurance : From Theory to Implementation By Daniel Galin
This is a college textbook, but has a lot of very practical information. The book does a good job at laying a foundation about quality testing, but I think it’s stronger points are the sections on quality assurance. Looking at establishing programs for quality improvement, software/quality metrics, certification, or standards? Then this is a great addition to your library. There is also a section on organizational alignment for quality assurance.

The Certified Quality Manager Handbook Edited By Duke Okes and Russell T. Westcott
This book is a must if you’re getting certified by ASQ in Quality Management. It’s a handbook, so don’t expect in depth treatment on any single topic, but it’s a great overview of quality management. Use this as a reference to help identify areas that you want to research/learn more about.

General Skills

Flawless Consulting: A Guide to Getting Your Expertise Used By Peter Block
So what’s this book doing on a test manager’s bookshelf, you may ask. Testing is a funny activity in software development, if you think about. We don’t produce the product and in many cases we don’t have any real say in whether the software goes out the door or not. We should be honest, impartial, and knowledgeable producers of information that others (such as project management and sponsors) utilize – in other words, we are consultants. This book, more than any other on the subject IMO, will help you hone skills so that the information you produce is valuable, pertinent, and used by others.