The Protocol Complexity Matrix and what it means for your load testing

 

As a general rule, the more complex a protocol is to script, the more the protocol relies on monitoring the network stream and the less that it needs to understand the actual application under test’s user interface. It also tends to be true that with this type of protocol, the more virtual users of that protocol type will run on any specific load generator. 

This has led to the development of the Protocol Complexity Matrix and can be visualized by the graphic below.

 

protocol complexity matrix.PNG

 

This is a representation of common results and you may find that your individual use of a protocol may provide more or less virtual users than indicated.  This diagram shows you that if you want to make the job of scripting easier, it comes at the cost of memory.  Using more memory makes the protocol less scalable up to the point that you reach the GUI protocol, which out of the box has a 1:1 ratio between virtual user and load generator.

 

The goal of any organization is to find the right balance.  Nearly any application can be scripted with the Winsock protocol, regardless of technology used by the application under test. However, working with raw socket commutations can be difficult to learn and requires a great deal more time to develop and test; thus making the solution impractical for many projects. This is why higher-level protocols have been developed, to save time.

 

What is not easily represented in the matrix is another fact; the more reliant you are to the network stream, the more susceptible your scripts are to changes in that stream.  So the application that is scripted with QTP can undergo massive changes in the background and the scripts for that application can still function without modification.  But these same changes to an application recorded with the Winsock protocol could render the entire script useless and a throw away. This fact brings me to my next question:

 

How many users can my Load Generator run?

 

In previous versions of LoadRunner and Performance Center, a protocol footprint spreadsheet was distributed by HP to provide a guideline for how much memory a script of a particular protocol would consume.  The problem was recipients of the footprint spreadsheet thought of it as definitive rather than prescriptive. It is much better to use a repeatable process and a formula to mathematically estimate the total number of users a load generator can run on a per-script basis. The process and formula outlined below should be used for every script that you intend to use in load testing, and every load generator that will participate in the test as well.

 

You can download LoadRunner here to experience it for yourself!

 

Here are the steps you should take these scripts that you intend to use in a load test:

1)      Develop your script and ensure that it will run with multiple users, multiple iterations.  This is the normal development of the script of any protocol.

2)      Create a load test using the script that you want to examine with one virtual user and execute the test. Use a delay of a few minutes in starting the script and monitor the load generators available MB of RAM and note any decrease when the virtual user starts.  This is the “First VUser Memory”.

3)      Modify the load test to use five – ten virtual users and execute the test again.  Keep the same delay for the start and have each user start one minute apart.  Monitor the load generators available MB of RAM and note the drop in RAM as each user starts.  Average the amount of RAM that the second virtual user on up uses and this is the “Each Additional VUser Memory

4)      Determine the total RAM available on the load generator This is the “TOTAL LG RAM

5)      Subtract 700 – 750 MB of RAM for the OS

6)      Determine what is 75 percent of the remaining available RAM

7)      Subtract the “First Vuser Memory”

8)      Divide what is left by the “Each Additional Vuser Memory”. 

9)      Add one and you have the “Total Virtual Users that will run on that load generator

10)  Repeat this process and formula for each script and every load generator that has a different level of memory.

The equation will look something like this:


total virtual users on a load generator.PNG
This equation will work for all but a handful of protocols.  These are the protocols that this equation may not work: Citrix, RDP, any protocol with the term GUI in it, and QTP\UFT scripts.  That is because these scripts can further be limited due to GDI resources.  These are graphical resources that are an attribute of the video card installed on the computer.

 

GDI resources are not able to be monitored and they cannot be adjusted without replacing the video card.  When GDI resources are consumed any virtual user that attempts to run on that LG will fail.

 

To learn more about LoadRunner and Performance Center and how to estimate the total number of users a load generator can run on a per-script basis visit the product homepages here.

 

You can also download HP LoadRunner here and improve your load testing today!

 

Labels: Load Testing
Comments
Pitush(anon) | ‎08-05-2014 10:56 PM

If I am getting 2.4444 as the final figure what does this mean, does my LGR support 2 vuser or 20 or 200

| ‎08-06-2014 01:33 PM

Pitush(anon),

 

Can you please provide the following:

 

  • TOTAL LG RAM
  • First Virtual User Memory
  • Each additional Virtual User Memory

 

As an example.  If you have a 4GB LG machine the TOTAL LG RAM is 4,000

If your First Virtual User Memory is 10MB and Each Additional Virutual User Memory is 3MB then the result would be:

 

4,000 - 750 = 3,250

3,250 * .75 = 2,437

2,437 - 10 = 2,427 ( 1 virtual user running)

2,427 / 3= 809

809 + 1 = 810 (Total Virtual Users)

 

Hope this makes the process a little clearer.

 

Craig

ronald sercely | ‎08-07-2014 01:13 PM

two comments.

 

GDI Objects cannot be monitored within LR, but are easy to monitor from Task Manager. 

 

Processes Tab -> then at top - View->Select Columns - GDI Objects can be selected.

 

Also, I created a case and HP R&D acknowledged - regardless of the size of the generator machine, there is an absolute limit of 140 TruClient Vusers on a host

Loadrunnersam(anon) | ‎08-28-2014 10:20 AM

Hi Craig,

 

Whats your intention in adding +1 at last. 

 

Thanks,

Sam.

 

Loadrunnersam(anon) | ‎08-28-2014 10:22 AM

I got it..the one first you have subtracted right...Thanks..Craig for your formula....

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
Showing results for 
Search instead for 
Do you mean 
About the Author
I have been working in the computer software industry since 1989. I started out in customer support then software testing where I was a ver...
Featured


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.