Rapid Software Testing

Length: 3 days

Leader: James Bach

Overview: Developed and taught by James Bach, this 3-day, hands-on class introduces you to rapid software testing, a complete testing methodology designed for a world of barely sufficient resources, information, and time. Based on the principles in the book Lessons Learned in Software Testing: a Context-Driven Approach, this class presents an approach to testing that begins with personal skill development and extends to the ultimate mission of software testing: lighting the way of the project by evaluating the product.

The philosophy of rapid testing presented in this class is not like traditional approaches to testing, which ignore the thinking part of testing and instead advocate neverending paperwork. Products have become too complex for that, and testers are too expensive. Rapid testing uses a cyclic approach and heuristic methods to constantly re-optimize testing to fit the needs of your clients. Rapid testing isn't just testing with a sense of urgency, it's mission-focused testing that eliminates unnecessary work, assures that everything necessary gets done, and constantly asks what testing can do to speed the project as a whole.

One important tool of rapid testing you will cover is the discipline of exploratory testing - essentially a testing martial art. Exploratory testing combines test design and test execution into one process that finds a lot of problems quickly. If you are an experienced tester, you'll find out how to articulate those intellectual processes of testing that you already practice intuitively. If you're a new tester, hands-on testing exercises help you gain critical experience.

If you outsource development or testing...

This class has been taught at outsource firms in India on behalf of their clients so that they can do a better job of testing without needing detailed test procedures. But more importantly, the rapid testing methodology is about getting a lot of value for the testing dollar (value that simply can't be reproduced by throwing untrained bodies at the problem) so that your top management won't see testing as a commodity activity that any stranger will do as well as you. Even if you outsource, you may want to have a core team of testers back at headquarters who can rapidly test products to check the "testing" done by outsource firms.

If you are burdened with clerical requirements...

This class has been taught in organizations pursuing the CMM and organizations subject to FDA and other regulatory requirements. Rapid testing is about thinking. As long as your organization wants you to think well and find important problems quickly, this is a class that applies to you. However, a lean form of test documentation is advocated in the class, to the extent you can possibly lean it. Session-based test management is also taught, which allows you to measure and document exploratory testing in a manner compatible with more "formal" process cultures.

Workshop Goals: In this course, you will learn:

• Concise, universal heuristics and models for instant test design
• How to tackle any product or product idea instantly
• How to analyze a test heuristic or practice
• How to test despite ambiguous or missing specifications
• How to deal with overwhelming complexity or confusion
• How to know when to stop or suspend the test process
• How to prepare and deliver an impromptu test report

Intended Audience: The ideal student is anyone who feels driven to be an excellent software tester or software test manager.

The class is useful to all levels of tester, but seems to be most appreciated by experienced testers who want to become expert testers. The most fun is had when strong-minded and skeptical students attend the class. James Bach notes, "They challenge me and make the class better, just like testers should. I try to make the class the most stimulating intellectual experience you can handle."

Another ideal student is the tester whose job is to check the work done by offshore outsourcing firms. You don't have time to do a full-blown test project. So, learn how to make a brief test project work.

Outline: What is Rapidity and How Does it Relate to Thoroughness and Rigor?

Key Idea: Think Like a Scientist

• Epistemology, The Study of Knowledge
• Technique: Abductive Inference
• Technique: Conjecture and Refutation
• Testing is About Asking Questions
• Testers Distinguish Inferences from Observations
• Testers Use Heuristics

Key Idea: Know Your Coverage and Oracles

• The Universal Test Procedure
• Rapid Modeling
• A Universal Heuristic Testing Model
• Seven Big Problems of Testing
• Rapid Oracles

Key Idea: Use Exploratory Testing to Find Bugs Fast

• The Internal Structure of Exploratory Testing
• Blending Exploratory and Scripted Testing
• Notetaking and Test Documentation
• High Accountability ET with Session-Based Test Management
• The Plunge in and Quit Heuristic
• The No Questions Heuristic

Key Idea: Focus on the Bugs that Matter

• Quick Testing vs. Coverage-Based Testing vs. Risk-Based Testing
• Risk-Based Test Management vs. Risk-Based Test Design
• Heuristic Risk Analysis

Key Idea: Run Crisp Test Cycles

• Test Cycle Heuristics: "test all scopes" and "test right now"
• How to Work with Developers So They Go Faster and Support Testing Better
• Test Cycle Convergence and Stopping Heuristics
• Rapid Bug Investigation
• Reporting Your Status Responsibly

Key Idea: Use a Diversified Test Strategy

• How to Evolve a Test Strategy
• Test Strategy Heuristics
• Contrasting Test Techniques
• Rapid Test Automation

Key Idea: Make Sure Your Testing Fits the Project

• Context-Driven Test Methodology
• The "Good Enough" model
• Good Enough Testing with the Context Model
• The Missions of Testing
• Testability

Exercises (distributed throughout the class)

• Test the Mysterious Sphere
• Wason Selection Task
• Test the Famous Triangle
• Use Exploratory Modeling on a Small App
• Find an Oracle for Font Size
• Discover the Role of Repetition in Test Strategy
• Report the Completeness of Testing
• Exploratory Testing with Playing Cards
• Produce a List of Testing Issues for a Disk Management Application
• Test a Product You Work with Every Day

Audiovisual and classroom setup needs: One computer is needed for every two students.

Additional Information

When someone tosses you a program and says "you have one hour to test this" can you do it? Are you confident in your work? Can you explain what you did and why? This unique 3-day course taught by James Bach introduces you to Rapid Software Testing, the skill of testing any software, any time, under any conditions, such that your work stands up to scrutiny. This is the closest thing in the business to a martial art of software testing. Geared more for experienced testers, but useful for new testers, too, this seminar provides hands-on demonstrations and drills as well as portable heuristics that help you create tests on the run.

Are you looking for a practical class?

In this class you'll test real software, under time pressure. You'll practice cutting applications down to size with rapid idea generation techniques. You will practice critical reasoning on your feet, by yourself and in small teams.

James Bach developed this class from his own experiences at Apple Computer, Microsoft, Borland, and several software testing companies. Every technique he presents is one he personally uses, on a daily basis, on test projects.

Why rapid testing?

Most testing classes try to teach you how to test thoroughly. The problem is that you're not given the time and resources you need to properly execute a thorough test process from beginning to end. Rapid testing is a way to scale thorough testing methods to fit an arbitrarily compressed schedule. Rapid testing doesn't mean "not thorough", it means "as thorough as is reasonable, given the constraints on your time." A good rapid tester is a skilled practitioner who can test productively under a wider variety of conditions than conventionally trained (or untrained) testers.

The other reason to study rapid testing is respect. Historically, testers have had trouble gaining the respect of developers and other people on a software project. After all, from the outside, the testing activity doesn't look like much. Most of the value of testing comes from how testers think, but even excellent testers struggle to articulate or justify their ideas about how to test. Rapid testing is a personal discipline, teachable and learnable, that allows you to think and talk about testing with confidence. By contrast, a conventionally trained tester generally is limited to complaining about how the requirements aren't fully documented, or about how some other condition necessary for arbitrarily thorough testing has not been met. That behavior rarely inspires respect.

The rapid testing techniques are indispensable when you are asked to test something on short notice, off the top of your head, early in the development process, or when you're in the process of developing test cases and procedures for future use. These techniques are also useful even when you're called upon to test more thoroughly, and given the time and resources to do so.

How does rapid testing work?

Instead of explicit algorithms and instructions, skill and heuristics are emphasized. A core skill is the ability to think critically. Thus, you'll discuss and practice the art of being skeptical and of separating observations from inferences. This is a thread that runs throughout the class. You will report bugs and be challenged to explain the relationship between your conjecture that something is amiss and the observations you made. A good tester thinks like a scientist or a detective.

Rapid test design is an organized process, driven by a set of concise heuristics (think of them as guidelines) designed to assure that you don't forget to test anything important. For new testers, the heuristics provide basic guidance. For experienced testers, the heuristics help you organize and access your experience, so that even under pressure, you perform like an expert and feel like one, too. With practice, you get better and better at testing rapidly while still being fully accountable for your work.

Another element emphasized in the class is exploratory testing, which is the opposite of pre-scripted testing. There are often good reasons to pre-script tests, but there are also many situations where defining and recording tests in advance of executing them would take far too long and find far too few bugs. In exploratory testing, the tester designs and executes tests at the same time.

Where did the rapid testing ideas come from?

From 1987 to 1995, James Bach has worked mostly alone to develop a systematic heuristic-based test methodology that applied to commercial mass-market software projects. Traditional test methodology didn't work well for market-driven test project. Starting in 1995, he began to collaborate with other thinkers and writers in the field, who helped find and fix errors in his work, and helped extend it beyond the scope of market-driven projects. "What began, for me," he says, "as my own vision of testing, merged with other ideas to become a community vision. Some of my colleagues eventually came together to identify themselves as the Context-Driven School of software test methodology. Others helped create the Agile Alliance, which published the Agile Manifesto. Three of us published the first context-driven testing textbook Lessons Learned in Software Testing. I and my colleagues speak and write regularly about testing, doing our best to advance the state of the art."

The ideas in this class are drawn not only from experience, but are also grounded in epistemology, cognitive psychology, decision theory, and other fields. Testing is a far more interesting field than most people realize. We're at the crossroads of many other traditions.

The original motivation for all this was James Bach's personal quest to be a truly expert software tester. "It is an ongoing journey, and this class represents the best I have to show for it, at any given moment. My goal with the class is to propel each student forward on his or her own quest for expertise and self-confidence."