New HP Best Practices Document: "Load Testing Scenarios Best Practices"

A new HP best practices document has just been published, called "Load Testing Scenarios Best Practices" (you'll need an HP Passport to access it).  It provides concepts and guidelines for designing and executing load testing scenarios, including scenarios involving large numbers of virtual users, large amounts of data, and long-running scenarios.

 

Let me know what you think about it in the comments box below, as well as additional tips and best practices you'd like to share.  

 

To whet your appetite, here's the introduction to the document:

 

Introduction to Load Testing Scenarios

 

Modern system architectures are complex. While they provide an unprecedented degree of power and flexibility, these systems are difficult to test. Whereas single-user testing focuses primarily on the functionality and the user interface of a system component, application testing focuses on the performance and reliability of an entire system.


For example, a typical application testing scenario might depict a thousand users that log in simultaneously to a system on Monday morning. What is the response time of the system? Does the system crash? To be able to answer these questions and more, a complete application performance testing solution must do the following:

 

  • Test a system that combines a variety of software applications and hardware platforms
  • Determine the suitability of a server for any given application
  • Test the server before the necessary client software has been developed
  • Emulate an environment where multiple clients interact with a single server application
  • Test an application under the load of tens, hundreds, or even thousands of potential users

A load testing scenario defines the events that occur during a testing session. It defines and controls the number of users to emulate (known as virtual users, or Vusers), the actions that they perform, and the machines on which they run their emulations.


When the number of Vusers reaches the thousands, the resources required to run the scenario might come close to, or even exceed, the available computing resources. For example, the computers used to simulate the load (known as the Load Generators) might start to run out of memory, or there might not be enough CPU available to serve all of the Vusers.


When running a long load test, it can be very frustrating for the test to run for hours only to have something fail – especially if it’s something other than the application under test. It is generally possible to mitigate this by planning the load test correctly. This document offers a number of strategies which can be adopted to help you achieve tests involving scenarios up to and exceeding a thousand Vusers (more than 5000 Vusers is generally considered a large test when using the Web protocol.  For other protocols, large tests involve a thousand users or more).

Comments
NaveenKumar N | ‎05-27-2013 11:01 PM

Useful resource. Thanks to HP!

foragerr | ‎06-11-2013 10:19 AM

From the document:

 

Use hard-coded think time values.

 

 

Why is this recommended?

| ‎06-11-2013 09:46 PM

Hi foragger,

 

Keeping the original think time values as they were recorded allows LoadRunner to accurately reproduce the user's session. If you need to vary the think times at playback, you can use the Run Time Settings options for think time, and have the change applied uniformly to all think times in the session.  The options allow you to:

  • Ignore the think time
  • Replay the think time as specified in the call to lr_think_time;
  • Increase or decrease the replay time uniformly by defining a multiple of the recorded think time;
  • Increase or decrease the replay time randomly by defining a random factor to be applied to the recorded think time.
  • Limit the think time to a maximum value

So the recommendation is to use the Run Time Settings to manage variations in think time, rather than applying logic in the script to manipulate the think times.

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
Showing results for 
Search instead for 
Do you mean 
About the Author
Malcolm is a functional architect, focusing on best practices and methodologies across the software development lifecycle.
Featured


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.