You’ve heard the pitch before: if you fail to test your website before the holidays, you’re taking a major risk. Your site could run slowly and cause customers to bail. Or it might not be available to customers at all.
But maybe you haven’t seen some of the numbers from last year. During the 2013 holidays, top e-tailers experienced page load times as high 6.8 seconds, plus as much as 14 total hours of downtime.1 You don’t have to crunch the math to know the impact was expensive.
To help protect your bottom line and brand reputation, here are three best practices for holiday load testing. You’ll detect a theme: load testing is both a technical and a business practice. In other words, it’s all about the customer experience.
Know what to test. Check the data—and ask Marketing.
Load testing doesn’t cover every aspect of a website. Why bother? You need to test the pages and functions your customers use most. If your CEO’s bio page loads a tad slowly, chances are your call center won’t be flooded with complaints. But if your shopping cart or login jam, you’ve got real problems.
Those are obvious examples—educated guesses. The real best practice: don’t guess. Take a close look at the data. A terrific tool is Real User Measurements (RUM), used by more and more companies to see what actual customers experience. RUM tracks every page every single customer visits. It’s a rich harvest of data on the buyer journey and site performance.
Your RUM data probably will tell you to test your cart and login. But it might also suggest you fix a slow-loading video for a surprisingly popular item a few clicks from the home page, maybe something the folks in Marketing didn’t peg as a holiday hit. Or hey, maybe the data will reveal how amazingly popular your CEO is—time to put some zip in that bio page after all!
You get the point: whether engineers or marketers, we’re all surprised by something every single day. Better to find those surprises early. Let the data be your guide.
Speaking of marketers, communicating with yours—or with developers or anyone else that adds content to your site—is another way to zero in on areas to test. Say the holidays are in full swing. Marketing decides to drive sales with 50% savings on best-selling items. The sooner you know, the sooner you can decide if you need to test. If your web team isn’t in the habit of asking for a heads-up—or your counterparts across the building don’t routinely give them—it’s time to nudge the culture by communicating better.
Scale up incrementally. And again, talk to Marketing.
Neustar’s professional services engineers have done thousands of load tests. One of the most common mistakes we see: overestimating current scale. Going with your gut alone—“We can handle more load, no sweat”— often leads to crashes once the testing begins. Besides being inefficient, it’s a bit of a buzz-kill.
Here’s a better plan: start small and add load incrementally. Identify the scale you need, for instance, 3,000 concurrent users. Launch the test with 10% of that number, see what happens and, assuming all goes well, add 10% more. Repeat as necessary—until you see errors spike or the application collapse. You’ll nail down current scale more precisely and with less heartburn.
Backing up a moment, how do you compute the scale you need going forward, whether for a whole new site, a new application or Cyber Monday? Here again, you need input from other stakeholders, marketing first and foremost. What are the sales forecasts? Your marketers fuel your traffic, so look to them for answers, supplemented by your own historical data and analysis. It’s not always an exact science, but it beats hunches.
When the party’s over... actually, the party’s never over.
Once you’ve completed holiday testing, you’ve got a lot of useful data—and not only for understanding and optimizing current performance. Imagine if you could use your load test scripts for website monitoring, gaining 24/7 value from your initial investment.
In fact, you can. Neustar, for example, uses the same platform for monitoring and testing. One dashboard lets you see it all—the full scope of your performance data.
This makes it easier to ensure your site continues to work as it did after testing. If you’re in ecommerce, there’s always another seasonal event right around the corner. As soon as you catch your breath after the holidays, there’s Valentine’s Day, St. Patrick’s Day, Mother’s Day and more. You may also want to load test for new code releases, plus major site upgrades or total redesigns. With load testing data added to your website monitoring, you get a richer performance picture and more diagnostic tools.
It’s one more way get more value from your holiday testing—and deliver the site experience your customers expect.
Questions? Speak to a Neustar load testing expert at 1-888-367-4812.
(1) Source: http://www.websitepulse.com/blog/web-performance-and-availability-during-holiday-season-2013-infographic