Diagnosing Load Test Errors – Where to Start for Holiday Success
Picture this: you just completed hours of internal Web services preparations with your system administrative team prior to the holidays. You discovered possible points of failure and made appropriate modifications with the expectation of a perfect load test. You take a few minutes to relax, refill the coffee mug sitting in front of you, and connect to the conference bridge where real-time discussion about the load test will occur. Things go well for the first 20 minutes of the test, no errors have presented themselves and your system administrator reports back that all systems are good and within normal operating thresholds.
Then it happens: one of the simulated users logs an error stating that it has timed out. Your eyes look at the error with a quizzical glance. The system administrator mentions that everything still looks good on the servers, however another error is logged with a similar message.
Your puzzled glance turns into a paralyzed stare as tens and hundreds of additional errors compound into the logs, abrogating your prior hopes that were facilitated from hours of now-futile preparations. Just as you slump into your chair, the full service load testing engineer announces, “Ah, this is good! It probably doesn’t seem as such right now, but we may have discovered a significant configuration problem with the database management system. Five minutes might be all that’s needed to fix it.”
Are Load Test Errors Bad? No!
Thomas Edison once said, “I have not failed. I’ve just found 10,000 ways that won’t work.” I doubt that Edison was referring to Web performance load testing, but it applies as much here as it did when he was referring to his inventions. Errors are necessary to find weak spots or breaking points of your Web system and infrastructure, and are ultimately why load testing is such an important process. Discovering errors helps to reveal problems with embedded media, sourced dependency files, bandwidth allocations, database configuration as well as many other factors that could be degrading your Web system’s continual and immediate availability to prospective customers. If errors are not discovered, then one of two situations must have occurred — either you just finished going through a series of load tests and resolved the issues that were found, or you have not performed any load testing at all. Just as failure is necessary for our personal success and resilience in life, load testing errors are necessary for the success and dependability of Web services.
Load Test Errors 101: Diagnosing
It is important to group together errors that may relate to each other, such as one error stating that an expected element was not available on the page and another stating that half of the page failed to load . Another commonly seen association between errors is data traffic timeouts and service unavailable responses from the Web server, indicating that a fully saturated point may have been reached. It is because of these relationships among errors that it is important to follow a routine for diagnosis that makes the most sense for a given situation.
When I diagnose any number of errors that occur during a load test, I start with the highest level data that is available at that time, which is usually the general error message being logged. I then glance through the frequency of occurrence as well as the general time frame for each error, make any applicable relationships for possible root causes of errors and then dig into the object-level data to uncover underlying influences that may be instigating or affecting the errors.
Converting Load Test Errors into Real Value
Many business executives and analysts equate value to not only the information learned from a situation or process, but also (and often more importantly) from the financial impact that the situation has on the organization. Proper execution of a Web performance load test and its associated error diagnosis has the potential to add significant real value to a company. The value is due, not only to discovering weaknesses and strengths in your Web services — and therefore a greater chance of success going forward — but in the beneficial cost-to-reward financial value created by such a process.
If you employ the type of personnel who can perform these load testing and related steps in a knowledgeable and informed capacity, then you are already equipped for on-demand load testing success. If not, then as mentioned above, the full service load testing professionals at Neustar have everything you need. Don’t be afraid of errors, embrace them and use the knowledge gained to create a successful and sustainable online business this holiday season.