If you have followed our previous series of articles and done your homework, you have already realised the importance of attracting visitors to your site by thinking omnichannel, packaging your offers with a good story, and perhaps also taken the step and internationalizing your site. All this work is quickly for nothing if the site can’t keep up with the pressure and goes down, but also a less radical scenario can have dramatic effects. A study from Akamai has shown that 40 % of the users leave an e-commerce site if the page load time is more than 3 s, and 64 % of these users select another site next time the will go shopping. There is a lot to gain by securing that the site will deliver even when the pressure is high.
When should you Perform Load Tests?
Load testing serves a purpose to give a measurement on how well the system works on a given load, i.e. when a certain amount of users perform one or several specific tasks at the site. These measurements are normally asked for before a larger campaign where you expect a significant increase of the number of visitors, or in connection for larger updates of the system. At these events, you want to make sure that the given load can be handled, or simply examine the boundaries for what the system can handle. The latter is by definition not really a load test, but a stress test, but the technology is the same.
Even smaller changes can have large consequences for the performance. This is normally not captured in function or regression tests, so it can be a good idea to conduct load tests continuously in connection to every release as a part of the system’s QoS testing.
In the two following articles, we will show more in detail how you; (i) make sure that the system can handle a specific number of users, and find bottle-necks before a campaign for example (stress test), and (ii) continously follow up the system’s performance (QoS testing).
In this introduction, we will instead look at more general ways on how to proceed if you want to set up a load test.
What should we Measure?
The most basic measure at a load test is the load time for different pages at the site, and preferably measures for different geographical places. If it turns out that the load time at full load is still within good margins below the acceptable value, there is no need to measure anything else. If it instead turns out that the load time increases more that desired, there is very little information about the reasons for this. To understand better what affect the result, you should look at the individual load time for the different parts that build the content on each page, such as HTM, CSS, javascript, json and images. It is also important to measure things as bandwidth, CPU usage, and memory usage, to determine the bottle-necks in the system.
Which Pages should be Tested?
To receive a fair result, it is most important that the load that is generated to the system actually reflects how the users move around the site, and the tasks that they usually perform. Good knowledge about your users are necessary to create a brilliant load test. Tools like Google Analytics can be a way to start.
To perform load tests on an e-commerce site, it is normally not enough to just surf on the site, but you need to simulate that products are added to the cart, that users register, or log in, and that orders are actually carried out.
How do you Perform a Load Test?
The easiest way to create a load test is to use a tool for this purpose, such as OctoPerf. These tools lets you create user scenarios, generate load from these user scenarios, and present the result of the load tests.
A user scenario is typically created from a recording of a user that performs a certain task at the site, and saves this as a script. The script can later be manipulated so it collects login information from a table or randomly generates data.
A load test is then created by selecting a load profile and sets the amount of users that should be active at every point of time, and which user scenarios that these should perform. These user scenarios are then carried out from a separate server park that calls the site that should be tested over Internet. There is also the possibility to set that different portions of the scenarios runs from different geographical places.
During the test, the measurements, such as load times, are recorded, and presented to the user to they can analyse the result of the tests.
In the next article we will go through an example on how you can set up a load test before a campaign, and how you interpret the results.
