September 14th, 2018

Unlock New Tools Using Selenium Scripts in Website Performance Monitoring

For anyone unfamiliar with Selenium, and unless you’re a software engineer you probably are, it’s an open-source browser automation framework. Although traditionally used in cross-browser functional testing of web applications, it has been used for other types of testing, like website load testing. Back in 2010, Neustar acquired monitor and load-testing provider BrowserMob, which was likely the first load-testing service that could launch thousands of concurrent browsers and became the basis for the Neustar Website Performance Monitoring (WPM) solution.

Selenium automates most of the major browsers and has available SDKs for many popular languages. Neustar’s Selenium bot uses JavaScript for the control interface. This means that the native functionality like the Date and Math classes are available, as well as the typical control flow that handles conditions (if then) and exceptions (try ... catch).

Selenium/WebDriver makes it much easier to automate complex web application scenarios because it interfaces with the browser’s Document Object Model (DOM). This allows to the browser’s engine (e.g. WebKit) to handle all the hard work, like parsing HTML documents, executing JavaScript, and building HTTP POST Requests from user-submitted form data.

Another major advantage of browser automation is gaining access to the built-in diagnostic tools. Chrome’s DevTools is an amazing tool, but specifications like the HAR data structure or the Navigation Timing API (Level 2) offer programmatic options for custom event timing and reporting.

WPM’s Simplest Script

You may notice some differences in this simple WPM script when compared to traditional Selenium/WebDriver scripts.

Performance monitoring scripts mostly differ with the use of breakpoints (beginStep, endStep), which are used to segment the alerting and timing events.

The full scripting API can be found at: http://docs.wpm.neustar.biz/testscript-api/index.html

WPM’s Most Interesting Selenium Script

Back when I was working for one of the largest computer manufacturers in the world, they were looking to understand performance metrics that were non-standard to our default reporting.

The custom timing event that was needed was accessible using metrics derived from the native browser’s Performance Timing API, https://developer.mozilla.org/en-US/docs/Web/API/Performance/timing

After calculating the custom timing variable, the value is inserted into the log files (i.e. HAR) of that synthetic monitoring sample.

WPM’s Most Interesting Selenium Script Implementation

Line #15 the custom timing variable is calculated and set.

Line #18 overrides the default “step” timing (based on load event) with an example calculation that takes the difference between two DOM states.

Line #19 if you do not want to override the default “step” timing, then it can be logged in the HAR as a key/value pair.

The takeaway is that Neustar’s scripting engine exposes a lot of functionality via the APIs, allowing for customized alerting and reporting.

Data Extraction

Many large organizations are using SIEM, Splunk, or other tools for log analysis because they only want the raw data, which would ideally be accessible via HTTPS APIs.

WPM’s account administration API — https://apidocs.wpm.neustar.biz/ — offers the ability to extract the HAR files generated in the previous script example. Specifically, the method “Get Raw Sample Data” would be used to retrieve the HAR.

Conclusion

Neustar’s Professional Services engineers are expert Selenium script writers and have solved a ton of interesting problems by using browser automation which many of our clients never knew were even possible.

Scripts can parse text from the HTML document, and then leverage this for other types of “health” checks. One of our e-commerce clients runs an interesting validation test, as part of its ongoing availability monitor. The client wants to validate that the number of items returned from a search will pass the minimum threshold. For example, a Google search for “Selenium” currently returns 29.9 million results. It would be trivial to extract this value and throw an exception if the scripted assertion fails.

If your organization is currently using Selenium, or you’re interested in learning more about Neustar’s load testing and external monitoring services, we should set up a time to talk. You can send me an email at ben.jones@team.neustar.

As a follow-up to this blog post, look for my next post where I will focus on data extraction using the WPM Account Administration APIs. I’ll also provide a quick example of how the custom key/value pair is extracted from the HAR file.

References

HAR — http://www.softwareishard.com/blog/har-12-spec/

Navigation Timing Level 2 — https://w3c.github.io/navigation-timing/

Navigation Timing API — https://developer.mozilla.org/en-US/docs/Web/API/Navigation_timing_API

Performance Timeline 2 — https://www.w3.org/TR/performance-timeline-2/

Performance Timing API — https://developer.mozilla.org/en-US/docs/Web/API/Performance/timing (deprecated but used in example)

Let's Connect

Find out how Neustar can help you succeed in the connected world.

Contact Us   Give us a call 1-855-898-0036