Chapter 1.1: Testing strategies: Scientific and manual testing
Manual forms of testing are also the most expensive ones. Still, they can be a sharp tool in our testing strategy toolbox.
Scientific testing 🤑🤑🤑🤑🤑
The question often arises of how close our testing strategy should be to reality. Do we need to run manual tests on the production system? Or should our staging system be an exact copy of the production system? Is it necessary for our testing framework to drive a real browser or even be capable of simulating all the browsers and devices we want to support? Taken to the extreme, this would amount to a scientific approach to testing. When introducing this category, my initial intention was to wink at the fact that this is too much of a good thing in most cases. But looking closer, it turns out that it may be appropriate to rely on an (almost) scientific approach in some instances.
I chose the term scientific testing to describe a type of testing that is as close as possible to the scientific approach. In this specific case, this means: testing a system under conditions that mirror real-world usage. Preferably we test the production system itself and choose a representative set of users to observe. Ideally, the users do not know that we watch them. In practice, we have to honor the privacy of our users, though.
Scientific testing can also be used to find bugs. Typically, however, these tests are conducted primarily to uncover usability problems. For this purpose, they can prove vital, justifying the high cost in many cases.
However, this type of testing is not suitable as a systematic approach to identifying defects in our system. Therefore, scientific testing will not be on the menu in the further course of this book.
I value your advice!
What approach works for you and your team?
Let us know in the comments!
Manual testing 🤑🤑🤑🤑
Systematic manual testing can lead to excellent results when it comes to preventing deploying bugs. At least, the QA people I have had the pleasure of working with are excellent at finding every conceivable type of bug, including those that often only occur in particular edge cases. So this type of testing often unearths bugs that would otherwise go undetected for a long time.