Tuesday, 21 February 2023

Part 1: Testing Strategies for Microservice Architecture

Christoph Ebeling

Founder & Managing Director

Microservices make many things easier, however with them come new challenges. Over the next three week's we will be posting a series of blog posts targeted towards individuals as well as teams that intend to use or already are using microservices and who would like to improve continuous integration and deployment. We will be sharing different approaches to deal with automated testing and how to deploy microservices quickly and independently.

Blog Series: Overview of testing strategies

In this blog we will be reviewing various testing types and strategies in order to evaluate their eligibility for cloud microservices deployments. One key challenge in particular is that it is becoming far more difficult to test applications and achieve continuous deployment. We want to understand why this is the case and provide recommendations that can be useful for developer teams.

Manual vs Automated Testing

We often see that years after an engineer has written software, teams ask themselves how the existing software actually works and what it was set up to do. The scenario that often arises is that part of the software starts to malfunction and the root cause is unknown, since one does not know the actual specifications. The first important step in this case, is to fully understand if an application is working well by testing it. For this we can either use manual testing or automated testing.

Comparing modes of testing.jpg

In certain situations manual testing may be well suited as it is faster than automated testing in the short term. Problems faced with manual testing however are that we do not cover regression, we do not have documentation and we do not have the assurance of code quality. The lack of documentation in particular is highly underrated.

Automated testing on the other hand can be more advantageous, as it will ensure you keep up to speed with your software in the long term. For example, when you do tests using a Behaviour Driven Development (BDD) approach, the benefit is that the test describes the functionality of a software and can be automated.

Challenges with testing software.

Writing tests are commonly understood to be highly beneficial for all stakeholders, the question that arises is why automated testing is not done in the first place. When working with various companies in the last years we often find that developer teams agree that automated testing is the right approach and an aspiration to implement it exists. Unfortunately developer teams sometimes struggle with it, the practice may not have been properly adopted or has not fully taken off in their organisations.

We have collected some of the main reasons why automated testing has not been readily adopted in organisations vis-à-vis developer teams.

Teams are missing the right tooling.

Writing automated tests on a daily basis can be time intensive and take up to 50% of the daily workload capacity of developers. This is a significant amount of time and allocation of resources. Thus, if incorrect or unsuited tooling is used, developers will inevitably become discouraged, or be reluctant to implement it from the start. It is essential that you make sure you have the best suited and up-to-date tooling, catered towards your software needs and technical capacities.

Slow Continuous Integration Server.

If you have a slow Continuous Integration (CI) server you will have significant challenges getting all the builds that you need while at the same time the entire testing architecture will hold back your developers. In other words a slow CI server means all team members that are working on tasks, need to wait for the CI server for a period of time, sometimes up to 15 minutes or more. This will have a direct impact on your team's workflow and motivation. They may get frustrated, feel their time is wasted, eventually not want to continue working or try to move on to other tasks and responsibilities.

Lack of knowledge regarding testing frameworks.

In order for your developer teams to write effective tests and leverage the full potential of using testing frameworks it is important to invest the time to gauge relevant knowledge gaps and address these in forms of L&D sessions or peer-to-peer collaboration work. Using a testing framework that does not match the needs or the technical capacities of your developers will have a negative effect on their workflow and output.

Addressing these three factors can have a significant impact on solving ongoing issues. From our experience they are relatively easy to fix yet there are two important elements to address that may make it challenging for teams to fully adopt the fixes. These are a low return of investment and good software architecture from the start.

Both elements are intrinsically related and are not necessarily easy to solve. In the case of bad software architecture, writing tests will be inefficient and results may be insufficient. The intention of writing tests is so that you are able to automatically test if an application is working, which eventually will enable you to do continuous delivery. The benefits of it are multiple i.e. good tests enable you to know if you can deploy without breaking the system. Unfortunately, all too often we see teams start writing automated tests, spend a lot of time on them with a low return of valuable insights and metrics, relevant to the invested time.

Setting new goals through a mindset shift.

It is very important to understand the underlying goal behind automated testing. The goal is to achieve continuous deployment. We want to deliver, test and deploy our application in a continuous cycle. Most teams are already aiming for this goal, however we still see, even in experienced teams, that this cycle does not work coherently.

goals of testing.jpg

Once we are doing continuous deployment the main objective of writing tests is to establish if we can deploy without breaking the systems.

The answer to this question might seem straightforward, because yes, this is the core purpose of continuous integration. However, when you look further and take the actual test sign into account, it is not that straightforward. Many developers write code not for new software, but in order to check if their code is working,

If we have a distributed architecture with microservices and have an application with a lot of edge cases, unfortunately testing code does not work that easy anymore. We might end up testing if a certain part of your application is working or fail to consider certain edge-cases. One of the key findings that we have made is that in this case a paradigm shift is needed in our approach, in other words a mindset change. You need to think about all potential parts of the application that could fail in your deployments.

We need to start writing tests considering everything that could fail, adopting a holistic approach, not only taking the most stable parts into account but rather all the potentially weaker areas as well. Writing tests with this approach actually gets us far more information and will put us into a better position to find out where a system is failing, making sure that we find the root cause of the problem.

In microservice architecture, this approach is especially important since in a highly distributed and highly independent microservice architecture, the root cause analysis gets significantly harder.

Let us take a look at a simple example to better elaborate on the possible issues that might happen while testing a functionality dependent on an external service.

In this example from the source code presented we want to find out the monthly Net spending. In order to do this the code would need to call the auto service. Let us say we want to find the autoservice by user ID and then sum them up. The issue here is that we are using microservice architecture, so in order to write a test we need to write and add a mock.

At this point we do not know if the functionality getAmount is Net or Gross. Above we have our net spendings. When mocking up this functionality we assume that this functionality will get us the net amount, however there is a lot of uncertainty on if it will correctly do this. The test will not give us the exact answer to this functionality and even if it does, somebody may change the order of getAmount functionality at a later stage whereby we would lose this information. In order for us to deal with the aforementioned problem we need a different test type.

Evaluating different Test Types

Three common test types that can be considered are Unit Tests, End-to-End Functional Tests and Integration Testing

Unit Test.

Unit tests could be an option in this case as they are fast on execution and are well suited for complex systems. However, unfortunately Unit Tests are insufficient to ensure that software is working and require a high effort to maintain. This reflects what we initially mentioned, in that many teams spend a lot of time on testing, the problem is that most of them do not have high cyclomatic–complexity. This is particularly evident with smaller projects and within some start-up environments. Teams end up spending a lot of time on adjusting unit tests and do not really get valuable insights from them.

Another problem related to unit tests, is that they are only valuable if you are 100% certain about how you want your software architecture to be set up. If you have an architecture that is clear and will not change much, then unit testing can be valuable for you. However, if you are a startup that is rapidly prototyping a unit test, to put it simply, it is useless and a waste of time, since the architecture of your entire application can change at any stage.

End-To-End Function test.

An alternative option is to use end-to-end (E2E) function tests. E2E function test, test the entire application, which enables us to deploy the software with confidence. This would be a good solution however there is some room for concern, as end-to-end tests have a significant disadvantage. They are hard to maintain and become even harder to maintain the bigger your application gets.

Some developers may have experience from back in the day using Selenium for E2E tests. With Selenium, you could automate anything a real user would do with your application. The structure has not been designed to properly handle modern apps with forms and tables, thus it lacks a stable and reliable way to refer to elements. It works slightly better today however it still has its problems such as debugging a test. A common problem with E2E. So in this case end-to-end tests cannot be the best solution.

Integration tests as a solution.

Integration tests are pretty straight forward. Usually you test from a certain point in the application and test until you reach the database then give the results back. In other words you test the integration of different service components in your new application. However you do not test the frontend.

The integration test will solve the mocking task for us and will also solve the questions on if we have net or gross in our application, since we test all these functionalities in combination. In this case the most suitable solution.

About a decade ago in a 2013-Java-world, integration tests would have been the final solution. However, even though integrated tests are able to test large parts of the application and are still fast enough for modern computers, they are not quite sufficient for testing microservice architectures.

In the next blog post we will discuss why integration tests are not sufficient for microservice architecture as well as present the first of two specific strategies we recommend for microservice testing. Stay tuned for next week's blog!

Share this article!

Nexode

Solutions

Content

Social

NEXODE CONSULTING GmbH

OBERWALLSTRAßE 6

10117 BERLIN