Times, timing and timeouts in TruClient

(This post was co-written with Geng-Cheng Shen (Victor), from the TruClient R&D Team)

 

TruClient is asynchronous by design. This is due to the asynchronous nature of the modern web applications that TruClient is intended to test, and the technology that is used to implement them. It is crucial to understanding this important concept to get the most out of the technology. It also implies that no matter the test structure, or whether the test itself is designed to pause or halt, there may still be some JavaScript or network activity taking place behind the scenes.

 

TruClient has several concepts related to time, timing and timeouts. This article explains these concepts, how to configure them and how they work together. If you are not familiar with TruClient and its functionality in LoadRunner, meet the browser-based testing technology in this blog post. 

 

Timeouts

 

Because TruClient is asynchronous, a step does not have to wait for a previous step to complete in order to be executed. This can be problematic if the next step depends on the previous step completing.  For example, if a step creates an object which is used by a subsequent step, the subsequent step mustn’t execute until the object has been fully constructed. The End Event concept was introduced into TruClient to solve this challenge.

 

End Event

 

The End Event is a trigger which is set when a step completes its execution. TruClient will wait for the End Event to be triggered running subsequent steps. Let’s refer to the time from when a step’s execution starts till the time that the End Event is triggered as the Step Execution Time.

 

1.png

 

Step Execution Time=the time at which the end event is reached - the time at which the step starts running

 

But what happens if the End Event is not reached? Or if it takes a long time till it’s reached? The test execution is stuck.

Step Timeout to the rescue!

 

Step Timeout

 

The Step Timeout is the time that TruClient’s engine waits for the End Event to be triggered. If the End Event is not reached during the Step Timeout interval, the step fails and returns an error, meaning that the execution has failed.  After TruClient executes a step (which, since TruClient is asynchronous, it will return immediately), the TruClient engine will wait until either the End Event is reached before the timeout, or the timeout expires, in which case it returns an error. The Step Timeout is measured in seconds, defaulting to 180 seconds.

 

So, now we know when the next step starts running. But what happens if a step starts running, but its object hasn’t been constructed?  The TruClient engine can wait, but it cannot wait indefinitely. What if, for some reason, the object never appears (just like the situation when the End Event is never reached)? The solution is to use the Object Timeout to handle it.

 

Object Timeout

 

The Object Timeout is the time interval by which a step’s object must appear.  If the object doesn’t appear by the end of the Object Timeout, the step returns an error (meaning that the execution fails). In the Run-Time Settings, the Object Timeout is configured in the Replay section, as “Maximum time for object-not-found”, and is measured in seconds, defaulting to 20 seconds.

 

2.png

 

End-Of-Network identification timeout

 

The End Event can be set to ‘Network Completed’.  The ‘End-of-Network identification timeout’ is a helper timeout for this value of the End Event mechanism, and is triggered when the TruClient engine identifies a “network silence”. This occurs in a time period in which no related synchronous network activity was detected on the wire. The ‘End-of-Network identification timeout’ value is measured in milliseconds and defaults to 150 milliseconds.

 

Timing

 

We defined End Event to mark the completion of the step’s execution. But if we start running the next step right after the End Event of the previous step is triggered, it still might be too early! This is due to synchronization aspects that might interfere with the execution. To reduce the probability of this happening, we introduced the concept of Pacing.

 

Pacing 

 

Pacing is the time that TruClient engine needs to wait after the End Event of the previous step is triggered before starting to run the next step. In the Run-Time settings, Pacing is referred to as the “Inter-step interval”.  It’s measured in milliseconds, and defaults to 500 milliseconds.

 

You can set the pacing in the Run-Time Settings. Once you set it, it will affect all steps. Note that this setting is used to calibrate the TruClient engine to work better. A high value will provide better synchronization but can harm the “real user” actions, while a low value might results in more steps failing.

 

We don’t recommend that Pacingbe used as part of the business process logic. The same functionality can be achieved using the Wait Step instead.

 

Wait Step

 

A Wait Step is a step that does nothing (i.e. waits) for the specified time interval. It’s a better solution than pacing for business process synchronization (e.g. taking into account rendering time, a server that is known to be slow, etc…).

 

3.png

 

4.png

 

Typing interval

 

On typing steps (steps that simulate a user typing data into edit boxes etc.) the typing speed can be controlled by the ‘Typing interval’ setting in the Run-Time Settings dialog. It is measured in milliseconds (from keystroke to keystroke), and defaults to 350 milliseconds.

 

Times

 

Minimum Time

 

Other LoadRunner protocols use the notion of Think Time to better simulate a real user experience. It is typical for users to not execute a step immediately after finishing the previous one – instead they often pause to think in between. In LoadRunner, the Think Time actually stops the test execution.  But TruClient can’t stop the test execution, because of its asynchronous nature. TruClient introduces a similar notion called Minimum Time, which provides the same functionality as Think Time.

 

The Minimum Time determines the least amount of time that a step’s execution will take. The TruClient engine will not run the next step until at least the Minimum Time has passed since starting the current step.

5.png

 

In TruClient, there are three places where you can set the Minimum Time:

  1. In the step editor, you can set the Minimum Time explicitly, and this value can be different for each step. You can set it to a specific number, but if the recording time is two seconds or longer, you can also select the special value “As Recorded(*)”, which means that it will use the recorded duration as the Minimum Time (in the same way as Think Timedoes).
  2. In the Run-Time Settings, you can check the "Replay using recorded duration for steps". This is equivalent to setting each of the recorded steps’ durations to ‘As Recorded(*)’ in the step editor. Once you check this option, it will apply to all steps, unless you explicitly set a different value for that step in the step editor.
  3. In the Run-Time settings, you can further configure the Minimum Time by using a random time (this mechanism is identical to LoadRunner’s think time notion). The randomizing setting only takes effects when you configure the Minimum Time as described in step one or two. If you also check "Apply minimum time settings on wait steps", then the wait time will also be randomized.

Now, since both Pacing and Minimum Time affect the time when the next step starts running, how do they affect it when they are both configured together?

 

If the Minimum Time is 0 (the default value), the next step starts running when the end event of the previous step is reached plus the pacing time:

 

6.png

 

If Minimum Time is greater than or equal to the execution time plus the pacing time, the next step starts running at the time at which the previous step starts running plus the Minimum Time.

 

 

7.png

 

If the Minimum Time is less than the execution time plus the pacing time, the next step starts running when the end event of the previous step happens plus the pacing time.

 

8.png

 

Wasted Time

 

From the explanation above, you can see that in order to improve the chance of a step executing successfully, or to better simulate the real user experience, the TruClient engine actually wastes some time. This time is critical when looking at the response time of a transaction.

This time is called Wasted Time, and users can decide whether to include or exclude this time when they view the report of transaction.

 

For normal steps (i.e. steps other than Wait Steps), the formula for wasted time is:

 

If minimum time is 0, or it’s less than (execution time + pacing), then wasted time = pacing;

 

Otherwise, wasted time = minimum time - execution time.

 

For Wait Steps, the execution time is the wait time. But since the purpose of wait steps is also to postpone the running of next step, we treat Wait Steps as Wasted Time.

 

So for wait steps, wasted time = wait time + pacing.

 

__________________________________________________________________________________________________

 

I hope you found this explanation of how TruClient deals with time-related issues useful.  Feel free to leave us a comment in the box below.

 

Thanks to Victor for providing this article!

 

You can download LoadRunner here

 

Learn more about HP Loadrunner

 

Join TruClient LinkedIn group

 

TC icon_blue-05.png

Comments
ronald sercely | ‎03-13-2014 05:27 AM

Overall - great article.

 

One comment however - I do find it unfortunate that you use the term "pacing" in this article, as "pacing" has been present as a run time setting in VuGen since its inception, and means something very different than what is discussed here. When a reader of this article discusses this with another engineer - how are they to distinguish which of the two definitions of "pacing" are being referred to?

 

I guess we could say "iteration pacing" vs "step pacing" ?

HP Expert | ‎03-13-2014 11:50 AM

Hi Ron,

When I meant Pacing, I meant Pacing in the world of TruClient only.

However I get your point, if you'll read carefully you will see that Pacing is actually referred to "inter-step interval" which is more accurate.

 

Thanks,

Guy

Les Murphy | ‎03-14-2014 10:31 AM

Thanks for sharing this information.  A question:

 

"TruClient is asynchronous by design. "

 

It seems to me like TruClient scripts are always synchronous as to how the steps are executed.  One step has to encounter it's end event before the next step is started.

 

Am I missing something?   I have not needed to capture timings for multiple parallel asynch events, but is it possible to do so in TruClient?  For example, to measure the response time for each individual porlet in a dashboard, where the individual portlets are each refreshed in the background via Ajax.

 

 

HP Expert | ‎03-16-2014 01:51 AM

Hi Les,

TruClient engine is JS based, as so it is working asynchronously and aimed to work like that due to the AUTs nature to be asynchronous as well. The user model is indeed synchronous, but the asynchronous notion on how the applications are working in the background is an important to utilize the technology and harness it for your needs.

 

Best,

Guy

| ‎03-31-2014 12:05 PM
Fantastic post, thanks.
Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
Showing results for 
Search instead for 
Do you mean 
About the Author
Featured


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.