The importance of performance testing for a modern product

A retail site measuring shopping cart purchases sees a loss of revenue due to fewer purchases; a hotel reservation system sees a drop in the number of reservations; a bank may see fewer transactions made.

What is performance testing in software engineering?

Performance testing is used for testing scalability, reliability, speed/response time, and resource usage of a solution under particular workload. It is a test that shows that the system performs right and does the right things.

Performance testing aims at achieving not one, but several goals due to the fact that many types of testing are similar and even combine with each other to solve one problem.

Subtypes of performance testing

Listed below are the subtypes of performance testing.

Load testing is a subtype of performance testing during which indicators are collected and performance and response time of a software and hardware system or device are determined in response to an external request. The main goal of QA load testing is to establish compliance with the requirements for the system or device. Load test results can help make design and capacity decisions in order to ensure reliability of the solution and to imitate the behavior of real users.

Stress test, also known as ramp-up test or capacity test, is a test which finds the limited capacity of the system. Virtual users are steadily incremented until performance saturation occurs (adding more virtual users results in response time growth rather than the growth of transactions processed per second).

Stress testing qa example

Stress testing in software testing allows evaluating the behavior of the software product at an increased intensity of operations and sets the maximum level of solution performance. It is carried out until one of the criteria reaches a test stop. Below are the main criteria of stress testing:

• Hardware reaches maximum performance (CPU>80%, memory>90%).
• HTTP and HTTPs requests get an error response.
• The program crashes.

Often, this type of testing is carried out if the customer expects a sharp increase in the load on the system. For example, for e-commerce it may be sales days.

Fixed-load test is a test which runs with a fixed number of virtual users for a specific period of time to measure response times under different load conditions determined from the capacity test. This test measures how fast the system is.

Load testing definition with examples

Longevity test is a very long high load test used to verify the system stability. 
The minimum duration of the test is 12 hours (up to a week or more is even better). The duration depends both on the level of load and requirements for the tested application. Therefore, a test duration shorter than 12 hours results in a higher level of load. And on the contrary, if the test duration is longer than 12 hours, then the level of load should be decreased.

QA performance testing

Performance testing metrics

Below are the metrics that can be collected during performance testing.

• Response time is the amount of time the system takes to react to a user request. This is a measure of how fast the system is.
• Significant increase in response times or decrease in stability. This is a measure of how powerful the system is.
• Reliability is the property of a system which indicates its ability to maintain its level of operation.
• Scalability is the property of a system which indicates its ability to be readily enlarged. That means that the capacity of an ideally scalable system should increase as fast as the power of system resources increases.
• The number of simultaneous users is the number of users who have an active connection to the system for a particular period of time. QA specialists simulate simultaneous users to carry out load testing of the solution.
• The number of concurrent users is the total number of users who can access and use the system exactly at the same moment. Concurrent users are frequently utilized for stress testing.
• The number of operations performed by the system in a certain period of time (throughput).
• Transactions per second (TPS) refer to the number of transactions that a network is capable of processing each second. By comparing the actual transaction time with the expected one (or the number of transactions per second), you can conclude how successful the system has been under load testing.
• The percentage of errors is calculated as the ratio of invalid answers to valid ones over a period of time.

Conclusion

It is necessary to constantly analyze the software product to anticipate possible issues and find out which behavior scenarios are typical for users. Customers need to take into account what will happen without carrying out load tests and how it can affect user experience.

Performance testing does not reveal bugs but helps to find out what the system is capable of. The clients can be sure that their websites or applications will not fall when there are more users or the database grows.

At EffectiveSoft, our seasoned QA engineers are ready to provide any type of software testing. Just contact us for more information about software testing services.