HP LoadRunner and Performance Center Blog

Displaying articles for: March 2009

LoadRunner Cloud: Get out your Umbrellas and Rubber Boots

So - today I read about the new HP annoucement for "HP Cloud Assure" and how we are going to help customers to gain more confidence in cloud-based applications. HP Cloud Assure delivers industry-leading IT management solutions as a managed service to help companies gain the benefits of cloud services without sacrificing control and increasing business risk." (just to quote the website). What this really means for quality's sake, is to ensure we don't become too casual about the responsibility for application performance in the cloud. "Don't look at me, man - performance is something the cloud guys do" should not be a common excuse for cutting performance-corners from your test plan.


As a performance tester, I'm reminded of a very large load test I worked years ago where we were running almost 1,000,000 browser connections from load generators all over the world. It was nearly impossible back then to get our quite simple website to scale - the infrastructure had issues, the firewalls had issues, the server-to-server clustering had issues and the database honestly was the last of our bottlenecks to be resolved. The scale of that test is dwarfed by comparison to today's cloud infrastructures, especially when you consider the almost exponential increases in density and computing power. I think that most engineers should be very concerned about the challenges of testing cloud-based applications with LoadRunner.





  • the topology for running an application in the cloud has transformed dramatically...it's not the same ole' web architectures you already know





  • monitoring the system-under-test will be a challenge if you don't have local access to servers and/or proper permissions




  • your capability to conduct root-cause analysis and hunt down a bottleneck in the servers will be limited, sometimes dramatically



So, if you have applications that are hosted out in the cloud or have dependencies on web services running remotely across the internet, you would probably realize a lot of benefit from working with HP SaaS. They have more than 9 years experience conducting performance testing "in the cloud" and by the way, they have some of the best LoadRunner guys I've ever known working there.



Check out the live webcast


Labels: LoadRunner

Taking visibility for granted?

Testers are naturally curious people. We enjoy the creation of questions for the purpose of finding the truth, or better still for the purpose of creating new truths. For centuries, the true professional testers have been scientists who more often than not were the determiners of new truths and thus we recall them as inventors. Back in the mid-1700's a theologian and scientist Joseph Priestley conducted several experiments to determine what types of gasses were generated by plants. It started with a curiosity about the observed behavior of a wax candle burning within a glass jar. And it was truly a curious thing because the wax candle burned out long before it exhausted the fuel supply of wax or wick. The flame had consumed all of the fuel supply of oxygen from the sealed environment, proven by the fact that when he tried to re-ignite the candle inside the jar using a simple magnifying glass to converge intense rays of sunlight on the candle's wick, it failed.


When Priestley attempted the same experiment but this time adding a sprig of mint into the glass jar with the candle the result was similar at first: the candle burns, oxygen is consumed, candle goes out, can't re-ignite it with sunbeams. But after nearly a month with the candle left isolated in the jar with the sprig of mint, Priestley then re-attempted to ignite the candle with the magnifying glass and rays of sunshine. And of course it worked. The candle was lit and once again consumed the oxygen until it burned out. Priestley deduced that the plant was somehow producing a gas that allowed the candle's flame to burn once again. What he could not have predicted, is that he would be the first to discover a new truth about the role of oxygen in photosynthesis.


Priestley's methods really got me thinking, especially about his test tools and techniques. All he used were rudimentary equipment and simple deductive reasoning for analysis. What if we were to attempt this same experiment today using contemporary scientific inventions? We would probably use a oxygen sensor to measure the amount of oxygen in the jar. This would make all the other implements for the experiment obsolete: the wax candle, the magnifying glass, the need for bright sunshine. The creation of the lambda probe oxygen sensor in the late 1960's was a response to the demand for measurement of oxygen in an experiment, machine or system. The design of the sensor allowed for visible measurement of an invisible gas. Even Priestly himself would have appreciated the sensor not simply for the new visibility it provided, but for the ease of use and accuracy in measuring his experiment.


For Priestley in the mid-1700's photosynthesis was unknown and oxygen was both invisible and immeasurable. Boris Beizer more than 200 years later was challenged to measure the known workings of the computer which were, for all intents and purposes, inside an invisible black box. The discoveries and solutions that resulted from their work show us how inappropriate it is to take any prior science for granted and also provide for us a new baseline for how to test. Simply, it is easier to correlate a visible test measurement to the tests objectives or pass/fail criteria. As a result, testing tools today already make test measurements visible, actionable and automatically correlate* the results back to pass/fail criteria. Today we take for granted that nearly every testing tool comes with mechanisms for "making visible" the performance metrics from the system under test. Just as we take for granted that every modern automobile now uses oxygen sensors and an onboard computer as essential components to improve fuel efficiency.


But just making something visible isn't enough. Consider LoadRunner's monitoring and diagnostics capabilities. Could you imagine today having to monitor CPU resource utilization without having the test tool automatically make the measurement visible for you? In the 1980's Boris Beizer shared stories about his counting CPU ticks with an AM radio next to the machine. That sounds like such an old solution - almost like having to measure oxygen with a wax candle in a jar. My point is that visibility should be understood as a means to improving measurability. Measurability is what truly accelerates the testing process. Innovation in performance testing should improve and extend the visibility and measurability we have today. What more can we make visible? What new methods of measuring, arranging and correlating test data can we create? Can we automate the capabilities we have today or build intelligence to aggregate or parse this new data?


And we don't have to start with a wax candle in a jar or an AM radio.


*- see lr_end_transaction("login", LR_FAIL);

ALM webcast with Brad and Paul next week

Next week Paul Ashwood and Bradd Hipps will be giving a webcast over on theserverside.com, presenting a whitepaper and case study Application Lifecycle Management (ALM). There will be some interesting details on how performance testing can be integrated into the lifecycle for application development and delivery to production. If your company or testing organization includes performance as part of the SDLC or Lifecycle processes, this might be very useful information and I'm sure Brad would love to hear some questions from you.


Here's the official abstract:


"Effective companies are riding the latest waves in application modernization. These waves touch nearly all of IT from technology and staffing to application architectures and release strategies.


HP Application Lifecycle Management (ALM) solutions help your IT organization make the most of these trends and avoid being swamped in the process.  ALM from HP is an integrated suite of leading solutions that enables your IT leaders to answer comprehensively the key questions business stakeholders have regarding application modernization.


You are cordially invited to view HPs unique perspective on what ALM is and how our solution renders better business outcomes. All attendees will receive our new white paper, “Redefining the application lifecycle: Looking beyond the application to align with business goals.”


Click here to register

Labels: LoadRunner

Tomorrow: a webinar with Nationwide

 

















HP


Webinar with Nationwide
























HP Software recently released the 9.5 versions of HP LoadRunner software and HP Performance Center software. With these new releases, HP is addressing today's top of mind application challenges around rapid technology change and adoption of new processes. In this challenging economic environment, companies have to be as lean and agile as possible. Please join HP and Nationwide for an informative Webinar where you will hear the highlights of the latest release as well as get a preview of an actual implementation of HP Performance Center 9.5 from John Seling of Nationwide. You will hear:



  • What new capabilities are included in the 9.5 release

  • How Nationwide leveraged the new features in 9.5 to shorten their test timeframes and make their tests more realistic

  • More about the new integration with Shunra Software

Join us to find out how HP's performance validation solutions can enable you to achieve legendary QA projects. All attendees will receive our new white paper, "Innovations in enterprise-scale requirements management, quality planning, and performance testing".

















Register Now »


















Take control of application performance and scalability challenges

























DATE:


March 17, 2008







TIME:


10:00 a.m. PT / 1:00 p.m. ET







SPEAKERS:


John Seling, Performance & Data Engineering Manager, Nationwide Priya Kothari, Product Marketing Manager (LoadRunner & Performance Center), HP Software







DURATION:


60 minutes with Q&A







REGISTER:


Click here to register for the webinar











Labels: LoadRunner

Explained: Virtual User Days, ("a day in the life of a virtual user")

In the complex world of LoadRunner licensing we have a mechanism that allows you to have a special pool of virtual user licenses that are called Virtual User Days.  The textbook description of a Virtual User Day (a.k.a. VUD) is:














Virtual User Days are the licensing mechanism that allows Virtual Users to be executed in an unlimited number of runs against a single AUT within a single 24-hour period of time.


 


As you can see, that description isn’t very clear about describing how the VUD licenses actually get implemented in the LoadRunner Controller.  I’ve answered a few emails over the last few months to describe this, so I thought it might be good to share the answers and explanations here.  Virtual User Days are sold in a quantity of Virtual Users that can be executed within a 24-hour period, where once the vuser license is consumed in the first test run it can continue to be used for multiple test runs for the next 24-hours. 



For instance, the same 1000 VUDs can be used for executing tests as follows:




- unlimited test runs that run a maximum of 100 virtual users…for 10 days

- unlimited test runs that run a maximum of 500 virtual users…for 2 days


- unlimited test runs that run a maximum of 1000 virtual users…for 1 day




Keep in mind, that the 24-hour timer starts from the first time you run a test for that day.  You can request that HP Support give you a specific time-of-day to start your testing, like 9 AM.  The proper understanding here is that VUDs are only decremented from the pool when the vuser thread first runs and it will only decrement more vusers from the pool if:




A. It is 24-hours later.

        -or-


B. Another test run needs more vusers than have already been decremented in the last 24 hours.




Example:




  • Customer runs test #1 for 1000 virtual users for 4 hours  (1000 VUDs are decremented from the pool)


  • Customer runs test #2 for 1500 virtual users for 2 hours (500 more VUDs are decremented from the pool)


  • Customer runs test #3 for 400 virtual users for 6 hours (0 VUDs are decremented from the pool)


  • Customer goes to bed after a 12 hour day (while they are dreaming about advanced LoadRunner correlation…VUDs expire at 12:00am)


  • Customer wakes up and runs test #4 for 300 virtual users for 2 hours (300 VUDs are decremented from the pool)



For a real customer there was some confusion about how the VUDs get used, or “activated” was the language they used.  The hypothetical situation is that they had 500 Virtual User Day licenses and proposed 3 scenarios:




Scenario #1:  We activate 100 VUDs at the start of the day. We run a load test for 100 VUDs, then follow this with another load test for 100 VUDs on the same day. Our understanding is that we have used up 100 VUDs at end of the day. We have exhausted all the 100 VUDs at the end of the day. We are left with 400 unused VUDs.



Answer:  that is correct…if the second run goes to 150 virtual users, they will only have 350 VUDs remaining for the next 24-hour cycle.



Scenario #2: We activate 100 VUDs at the start of the day. We run a load test for 50 VUDs, then follow this with another load test for 50 VUDs on the same day. Our understanding is that we have used up 50 VUDs at end of the day and that there are about 50 unused VUDs. We are now left with 450 VUDs, inspite of activating 100 VUDs at the start of that day.



Answer:  that is incorrect – there is no way to “activate” a virtual user manually.  The only way to “activate” a VUD is to run a virtual user and decrement a license from the VUD pool.  So, the VUDs are decremented from the pool when they first enter the running status.  The customer’s first run in 24 hours sets the bar for VUD consumption for the 24-hour period.  They will still have 450 VUDs remaining at the end of the day.



Scenario #3: We activate 100 VUDs at the start of the day. Due to some unforeseen issue, no test runs could be initiated. Our understanding is that at the end of the day, we are left with 500 VUDs.



Answer:  that is only partially correct – again, there is no way to manually “activate” the vusers, so if they never got any vusers running then it wouldn’t decrement any VUDs from the license pool.  They would still have 500 VUDs remaining at the end of the day.


Labels: LoadRunner

Steve's blogging about Performance Center

My colleague Steve Feloney is the Product Manager for HP Performance Center and he has recently started a new blog on PC over here:




 




 




Performance Center has been around for a few years, but I think you'll find that the new version has some significant improvements that are really cool for performance center of excellence (CoE) and centralized management of performance testing projects.


 



Steve and I will bring you information about LoadRunner and Performance Center and hopefully we can describe the very cooperative vision for how these two products evolve together.


 



Please welcome Steve to the blogosphere and be a pal - subscribe! Thanks!!

Labels: LoadRunner

Everything you wanted to find about LoadRunner, but didn't know the URL

Recently I've had some inquiries about finding all the information available in the entire universe about HP LoadRunner, specifically stating that it's very common to get "not found" as a return to results. So, I thought I'd take a few minutes of my generous free time to list the main links to find the official (and some unofficial) links to LoadRunner information on the web.







Let's start with the Hewlett-Packard website - which can just seem like the deepest jungles in the Amazon where you will find that Google does a better job searching and indexing our site that we do.

The main concern about the URLs on the HP website is that they appear to be very long and cryptic and often look to be dynamic strings.

So, with special thanks to our really good friends at Google, here are some very handy links to find LoadRunner information in the www.hp.com website:
















Of course, there's all kinds of external sites - ranging from interactive forums, to FAQ's and answers to every possible interviewing question that you might get asked (I remind you it is unethical to cheat on interviews) and there are some quite large communities of LoadRunner gurus out there. Here's a few forums and blogs that I know, love and use daily:












And a posting like this should also include some of the key individual contributors to the LoadRunner performance testing and performance engineering discipline - our leaders and true gurus:








Although it may seem obvious that I'm just posting a huge bunch of links to get click-throughs on this blog, I honestly must tell you that was not my intention. I have heard from numerous customers that there just isn't the same community of LoadRunner users and engineers out there - like we used to have in the Mercury knowledge base. Hopefully you'll find these links helpful and lasting - people and places you can trust will always have great information about LoadRunner and performance testing.





(note:  updated on 3/9/2009…to accurately indicate affiliation for the individuals listed.)




Labels: LoadRunner
Search
Showing results for 
Search instead for 
Do you mean 
About the Author(s)
  • I have been working in the computer software industry since 1989. I started out in customer support then software testing where I was a very early adopter of automation, first functional test automation and them performance test automation. I worked in professional services for 8 years before returning to my roots in customer support where I have been a Technical Account Manger for HP's Premier Support department for the past 4 years. I have been using HP LoadRunner since 1998 and HP Performance Center since 2004. I also have strong technical understanding of HP Application Lifecycle Management (Quality Center) and HP SiteScope.
  • Malcolm is a functional architect, focusing on best practices and methodologies across the software development lifecycle.
  • Michael Deady is a Pr. Consultant & Solution Architect for HP Professional Service and HP's ALM Evangelist for IT Experts Community. He specializes in software development, testing, and security. He also loves science fiction movies and anything to do with Texas.
  • Mukulika is Product Manager for HP Performance Center, a core part of the HP Software Performance Validation Suite, addressing the Enterprise performance testing COE market. She has 14 years experience in IT Consulting, Software development, Architecture definition and SaaS. She is responsible for driving future strategy, roadmap, optimal solution usage and best practices and serves as primary liaison for customers and the worldwide field community.
  • HP IT Distinguished Technologist. Tooling HP's R&D and IT for product development processes and tools.
  • WW Sr Product Marketing Manager for HP ITPS VP of Apps & HP Load Runner
Follow Us


HP Blog

HP Software Solutions Blog

Labels
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation