Re: Load Generators - Virtual vs Physical (2795 Views)
Reply
Occasional Advisor
avpi
Posts: 10
Registered: ‎04-23-2012
Message 1 of 13 (2,828 Views)

Load Generators - Virtual vs Physical

Would it be a good practice to use one physical server machine with the capacity of several virtual server machines instead of using several dedicated server machines as load generators? For example, if we plan on using 4 virtual servers as load generators for 2,000 concurrent users and then try to replace it with one physical server with the capacity of these 4 virtual servers, would we encounter any issues when running our scenarios? Is this a good practice?

Please use plain text.
HP Expert
Shane_Evans
Posts: 152
Registered: ‎04-08-2010
Message 2 of 13 (2,823 Views)

Re: Load Generators - Virtual vs Physical

If you can get a 64-bit physical machine with the resources to match 4 virtual machines, I think most would agree to do that versus running 4 VMs of similar combined capacity. I think you will find the I/O requirements of a large number of virtual users can be quite high, and you may end up with more capacity on the one physical machine.
Product Manager - HP LoadRunner
www.hp.com/go/loadrunner
Please use plain text.
Occasional Advisor
avpi
Posts: 10
Registered: ‎04-23-2012
Message 3 of 13 (2,812 Views)

Re: Load Generators - Virtual vs Physical

Thanks for your reply.

 

In the HP LoadRunner Installation Guide for v11, I see a note stating that VuGen recording is not supported on 64-bit operating systems. So, if we get a 64-bit machine, would we encounter any issues with VuGen?

 

Also, could you please list the system requirements for installing LoadRunner if we use a physical machine versus a virtual machine? 

Please use plain text.
Occasional Advisor
avpi
Posts: 10
Registered: ‎04-23-2012
Message 4 of 13 (2,811 Views)

Re: Load Generators - Virtual vs Physical

Sorry, just adding one more question to this post....

 

Would it be best to run all the scenarios from this single load generator (physical) machine?

 

I'm just confirming because I've always used several different virual machines in the past and ran the scenarios by choosing to run from each of these LGs and wanted to make sure if there was any difference in changing it this way.

Please use plain text.
Senior Member
TheMadHatter
Posts: 3
Registered: ‎06-29-2011
Message 5 of 13 (2,799 Views)

Re: Load Generators - Virtual vs Physical

If you are only using that machine for a load generator, it should not matter if it can record or not. 

 

But if it helps, from the LR11 Patch 03 ReadMe:



Note: VuGen recording is not supported on 64-bit applications. You can record 32-bit applications on 64-bit operating systems.

 

 

Please use plain text.
Occasional Advisor
avpi
Posts: 10
Registered: ‎04-23-2012
Message 6 of 13 (2,795 Views)

Re: Load Generators - Virtual vs Physical

[ Edited ]

Thanks for sharing the information on the LR11 patch 03 readme file!

 

So, just to confirm, would you recommend running scenarios for upto 2,000 concurrent users from a single physical 64-bit load generator machine that would actually quadruple the spec of 1 virtual server (i.e. 16GB of RAM instead of 4GB each)?

Please use plain text.
Frequent Advisor
SteveO_44
Posts: 44
Registered: ‎05-08-2012
Message 7 of 13 (2,792 Views)

Re: Load Generators - Virtual vs Physical

You basically need to see if your server is capable of handling your anticipated users (load.)

 

There's no standard answer for "How many VUs can I run on X machine?"  The memory footprint of each VU, the hardware specs of your loadgen, etc.

 

You get to try.  :-)  If it works, you have your answer.

Please use plain text.
Occasional Advisor
avpi
Posts: 10
Registered: ‎04-23-2012
Message 8 of 13 (2,780 Views)

Re: Load Generators - Virtual vs Physical

Thanks for your response!

 

Since we're still in the initial planning stages for performance testing, I wanted to reach out to the community to verify what the best practice would be in choosing our load generator machines. Based on our estimations, we initially requested for 4 dedicated load generator (virtual) machines for running up to 2,000 concurrent users sharing the load between these 4 servers. But, now, we've been asked if we can use 1 physical machine which has 4 times the capacity of a single virtual machine, instead of the 4 virtual machines. Once we make a decision on the hardware, that will be our setup for performance testing and cannot be altered. I've always used several different load generator machines to share the load and haven't tried running all from one machine. So, I wanted to make the best decision so we don't have any issues later during testing.

 

I understand there's no standard for choosing the number of load generators based on VU count. But, based on past experiences and general practice, any suggestions in what the best recommended solution in this case would be, 4 virtual servers or 1 physical server?

Please use plain text.
Frequent Advisor
SteveO_44
Posts: 44
Registered: ‎05-08-2012
Message 9 of 13 (2,763 Views)

Re: Load Generators - Virtual vs Physical

We push back REALLY hard whenever someone even mentions a VM for loadgens or controllers.

 

There's too many factors at play that will taint your results when you venture into the VM space to execute performance tests.

Please use plain text.
Advisor
Rajesh_Sahu
Posts: 31
Registered: ‎08-03-2011
Message 10 of 13 (2,739 Views)

Re: Load Generators - Virtual vs Physical

Hi

I follow this general practice of using physical LG machine.

 

50 users per load generator for machine with 512 mb memory.The primary factors affecting the script execution are Memory and CPU Utilization.Here is not definative figures as such for this.The way to work it out is baselining. You could run your users say 50 on a generator and monitor the stats such as CPU, memory. Say if the CPU consumes 35% resources for running 50 users then gauge the figure of the maximum capacity

 

Regards,

Rajesh Sahu

 

Please use plain text.
Frequent Advisor
AnSan
Posts: 68
Registered: ‎10-03-2011
Message 11 of 13 (2,714 Views)

Re: Load Generators - Virtual vs Physical

The following was why we stay away from VM's for the generator, unless someone can point to a way to fix this issue:

 

VM Clock drift.  The system clock is also virtualized and occasionally has to resync with the physical system clock.   This system clock is tied to your virtual user timing directly and the amount of drift is unpredictable and ungovernable.

Please use plain text.
Advisor
Shawn Wales
Posts: 24
Registered: ‎05-22-2008
Message 12 of 13 (2,701 Views)

Re: Load Generators - Virtual vs Physical

Using VM's used to be a viable option... and may still be 'if' you are using anything but TruClient. 

 

Once you move to TruClient your major contention will be CPU and Memory.  For example I had Web HTTP/HTML script where I could run 200 users on an old 2.8ghz dual core machine with 2gb ram.  That same configuration could only run about 6 users in TruClient.  Moving to a new Quad Core i5 processor with 16gb ram got me back to 50 users or so.

 

TruClient is definitely the future direction for Web apps as it provides many benefits.  It is still early in it's maturity so I expect many changes.  Right now I would not recommend a virtual environment for it, but if you do decide to go that way there are a few tips that will make your experience better.  Make sure your RunTime settings for Interstep Interval are set to at least 2000.  Basically the more 'think time' you can give it between steps the better.  It allows the CPU to multitask better.  This will help support more users and help resolve the clock issues you may experience when running in a VM.  You can lookup the clock issues as others have described it in more depth.  Also be sure to monitor the CPU.  Keep it below about 85% and your contention issues will be less.

 

Also consider your cost.  A mutli-processor VM server will cost much higher than a group of high end workstations.  Right now the best cost/performance option might be something in the line of an Intel i5/i7 processor with lots of ram and Windows 7 64bit.  I know many will note that LR11 does not officially support 64 bit... however that is only for scripting as the latest patch notes it is supported for running the scripts.  Hopefully full support will arrive with LR 11.5.

 

 

 

 

Please use plain text.
Advisor
greyhelm
Posts: 13
Registered: ‎08-09-2013
Message 13 of 13 (1,414 Views)

Re: Load Generators - Virtual vs. Physical

[ Edited ]

I have (sort of) similar issue:  1 controller, 4 generators; now all running Loadrunner version 11.52, on Windows server 2008 R2 SP1 virtual machines.  For 100-, 300-, 500- virtual user tests, which we traditionally have run on our customer-facing web site prior to releasing updates to production, we now constantly get "generator using over 80% CPU" errors every time scenarios run.  This did not occur prior to upgrading from 11.0 to 11.52; or moving from Windows 2003 to Windows 2008 R2.  Other than being VM's, all machines are well within HP recommended specs for memory, CPU, paging, etc.  However, Windows PerfMon output bears out the error message, when executed on the generators during load tests:  the generators constantly get pegged at 100% CPU for practically the whole execution of the scenario.  We've even seen this "generator >80% CPU" error with as few as FIVE (5) virtual users; though certainly more prevalent with larger number of simultaneous users.  We run virtual users as threads; this is to HTTP/HTML app, not a new app, never had this issue previously.  See attachment for more info / screen captures.  Any advice?  That is, other than loading up generators with multiple and/or multi-core CPU's?  <--Or, is that the only solution??  Anyone else seen this CPU issue on load generators, with either LR 11.52, or with VM generators on Windows server 2008 R2?  Note:  load gen servers each have multiple IP addresses assigned to them, and performance test all invoke "IP Spoofing."  Has no one else experienced this particular "CPU >80%" issue with load generators?  Thank you.

Please use plain text.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation