Unlock New Tools Using Selenium Scripts in Website Performance Monitoring
For anyone unfamiliar with Selenium, and unless you’re a software engineer you probably are, it’s an open-source browser automation framework. Although traditionally used in cross-browser functional testing of web applications, it has been used for other types of testing, like website load testing. Back in 2010, Neustar acquired monitor and load-testing provider BrowserMob, which was likely the first load-testing service that could launch thousands of concurrent browsers and became the basis for the Neustar Website Performance Monitoring (WPM) solution.
Another major advantage of browser automation is gaining access to the built-in diagnostic tools. Chrome’s DevTools is an amazing tool, but specifications like the HAR data structure or the Navigation Timing API (Level 2) offer programmatic options for custom event timing and reporting.
WPM’s Simplest Script
You may notice some differences in this simple WPM script when compared to traditional Selenium/WebDriver scripts.
Performance monitoring scripts mostly differ with the use of breakpoints (beginStep, endStep), which are used to segment the alerting and timing events.
The full scripting API can be found at: http://docs.wpm.neustar.biz/testscript-api/index.html
WPM’s Most Interesting Selenium Script
Back when I was working for one of the largest computer manufacturers in the world, they were looking to understand performance metrics that were non-standard to our default reporting.
The custom timing event that was needed was accessible using metrics derived from the native browser’s Performance Timing API, https://developer.mozilla.org/en-US/docs/Web/API/Performance/timing
After calculating the custom timing variable, the value is inserted into the log files (i.e. HAR) of that synthetic monitoring sample.
WPM’s Most Interesting Selenium Script Implementation
Line #15 the custom timing variable is calculated and set.
Line #18 overrides the default “step” timing (based on load event) with an example calculation that takes the difference between two DOM states.
Line #19 if you do not want to override the default “step” timing, then it can be logged in the HAR as a key/value pair.
The takeaway is that Neustar’s scripting engine exposes a lot of functionality via the APIs, allowing for customized alerting and reporting.
Many large organizations are using SIEM, Splunk, or other tools for log analysis because they only want the raw data, which would ideally be accessible via HTTPS APIs.
WPM’s account administration API — https://apidocs.wpm.neustar.biz/ — offers the ability to extract the HAR files generated in the previous script example. Specifically, the method “Get Raw Sample Data” would be used to retrieve the HAR.
Neustar’s Professional Services engineers are expert Selenium script writers and have solved a ton of interesting problems by using browser automation which many of our clients never knew were even possible.
Scripts can parse text from the HTML document, and then leverage this for other types of “health” checks. One of our e-commerce clients runs an interesting validation test, as part of its ongoing availability monitor. The client wants to validate that the number of items returned from a search will pass the minimum threshold. For example, a Google search for “Selenium” currently returns 29.9 million results. It would be trivial to extract this value and throw an exception if the scripted assertion fails.
If your organization is currently using Selenium, or you’re interested in learning more about Neustar’s load testing and external monitoring services, we should set up a time to talk. You can send me an email at firstname.lastname@example.org.
As a follow-up to this blog post, look for my next post where I will focus on data extraction using the WPM Account Administration APIs. I’ll also provide a quick example of how the custom key/value pair is extracted from the HAR file.
Navigation Timing Level 2 — https://w3c.github.io/navigation-timing/
Navigation Timing API — https://developer.mozilla.org/en-US/docs/Web/API/Navigation_timing_API
Performance Timeline 2 — https://www.w3.org/TR/performance-timeline-2/
Performance Timing API — https://developer.mozilla.org/en-US/docs/Web/API/Performance/timing (deprecated but used in example)