Chapter 1.0: Testing strategies: Introduction
Chapter 1.0: Testing strategies: Introduction
When it comes to automated testing, we know many different terms for different types of tests. So let's find out what strategies there are and their strengths and weaknesses.
When it comes to automated testing, we know of many different terms for different types of tests: E2E tests, acceptance tests, integration tests, unit tests, ... to name a few. Unfortunately, as far as I know, no official authority can determine and decide once and for all what is a real E2E test or what is an integration test. Therefore, I will distinguish between the following three types of automated tests in this book:
E2E tests: are used to test a system, including its entire infrastructure (data persistence, third-party services, etc.).
Acceptance tests: test the fulfillment of acceptance criteria from the users' point of view; we replace the supporting infrastructure with mocks.
Unit tests: validate the functionality of a smaller unit of the system from the consumer's (developer, user, or other parts of the system) point of view.
This chapter is available for free to all subscribers! The next chapter about scientific and manual testing strategies will be available to paid subscribers only. Upgrade to the paid subscription if you don’t want to miss it!
Especially the naming of the second category acceptance tests is controversial. Often these tests are called integration tests. However, I have decided to use the term acceptance tests. In my opinion, this form of testing is primarily about checking whether we meet the requirements for our application (acceptance criteria) from the user's point of view. While this implies that the integration of the individual parts of our application (or even the entire system) works, explicitly testing this aspect is not our primary goal.
There is also no agreement on the definition of acceptance testing. The description used in this book is not universally accepted! Some sources state that acceptance testing, like E2E testing, should be performed on a system similar to the production system. According to this definition, acceptance tests are merely a variant of E2E tests.
A key aspect of my definition is that I am talking about automated acceptance testing, designed to test the acceptance criteria of a specific application (which is part of a larger system). This book is about testing Vue.js-based Single Page Applications. All user interaction with our system happens using the Vue application. This means that we write all acceptance criteria from the user's point of view regarding the interaction with the Vue application. Those parts of our system that exist outside our Vue application (e.g., databases, backend services) are invisible to the users. Therefore, to simplify things, when writing tests, we assume they work.
However, I would like to explicitly point out that this interpretation is not universally accepted and that there are diametrically opposed definitions! Because of the vagueness of the terminology, I consider it essential that everyone within a team or a company has a shared understanding of what kind of test does what. Therefore, I would like my three categories to be understood as a suggestion; teams can adapt the naming to their needs.
In addition to these three flavors of automated testing, there are various forms of manual testing, which we will divide into scientific and manual testing.
When we think about choosing the proper testing strategy, we should be aware that every development team is doing testing in some way or another, even if there is no explicitly defined testing strategy. Usually, in this case, one of the most expensive types of testing is used: chaotic, manual testing by the developers and other stakeholders. But what are more reasonable and subsequently cheaper alternatives to this?
On the following pages, we will look at potential testing strategies. We start with the most expensive strategy and work our way to the less expensive testing variants. Expensive in this case usually means that over the entire lifecycle of a system, we need to spend a lot of time on testing. However, one must remember that the cheaper variants can sometimes mean a higher initial effort. In some circumstances, the more expensive manual tests are more accurate and do not deliver false positives. But beware: the other way around, when testing our application manually, we may also miss bugs that automated tests would have detected.
Upgrade to the paid subscription if you don’t want to miss the next chapter about scientific and manual testing strategies