Starter Strategies for Testing Your Site
May 08, 2015
If you aren’t testing, you’re leaving money on the table. The problem is that it can be so hard to start testing—which tool do you use? What do you change first? What does statistical significance have to do with it? Don’t worry. By the end of this article, you’ll know everything you need to set up your first test.
First off, let’s go over the two separate types of testing: A/B and multivariate. One isn’t necessarily better than the other, but it’s important to know the differences.
If you’ve read about conversion optimization testing, you’re probably familiar with multivariate testing. This consists of changing one specific variable within the page. For example, you’d change the buy button color, or the font that the title of the page is in. Then, you’d run an equal amount of traffic to both versions, and see how the change affected user behavior (including purchases, email sign-ups, etc.).
The downside of multivariate testing is that, since you’re only changing one small variable at a time, it can take a lot of tests before you come across anything that significantly improves your conversion rate (or other metrics).
A/B testing, on the other hand, involves testing version A of a page vs. version B of a page. Sometimes, a few things are changed (for example, the buy button size, color, and placement). Other times, the entire layout of the page is changed and two drastically different versions are tested against each other. For example, a shorter page with a large image slider at the top vs. a long page with more text and less graphic elements.
Either way, it’s important to set expectations when you start the testing process. Especially for sites that don’t already have a huge audience, expecting to test one thing and end up with a massive increase in conversions is unrealistic. The process is more about gradual improvements over time. Even if you’re a small-to-medium sized business, testing can help you, especially if you test larger variables at first and narrow it down to what works best for you.
Further reading: Statistical significance & other A/B pitfalls
Pop-ups (or pop-downs, pop-overs, etc.)
Pop-ups used to be relegated to only the spammiest sites on the internet, but are now a fairly common practice. There are plenty of options to choose from, and you should test a few different types to see what performs best with your audience:
Whichever type you choose, make sure it performs just as well on mobile as it does on a computer. More and more people are shopping with their phones or tablets, and a frustrating mobile experience can drive visitors away.
Don’t just test different types of pop ups, test different variations of the same one. Once you know that pop-down style tends to get the biggest increase in results, test different colors, sizes, the time delay before showing up, and text.
It’s counterintuitive, but it’s worth testing a color that contrasts with your main site color scheme. For example, in this case study, the online shop used a purple pop-down even though they were initially worried about it clashing with the main color scheme. That slight clash worked in their favor, making it more noticeable and getting the best results out of all colors tested.
It’s worth noting that you also have to decide what to put in the pop-up. If your goal is to increase conversions, try testing a promotion for free shipping vs. a promotion for $10 off. If you want to build an email list, you’ll be testing email opt-ins. And so on.
Some people will tell you long pages are always better, others insist that shorter copy is for the best. In reality, it can go either way (and often does). Which is why you need to test drastically different page lengths for your product pages, landing pages (for collecting email address or lead information), and homepage, at least.
When it comes to testing the graphics on a site, you want to look at the big picture first. More dramatic changes will give you an idea of what’s likely to work (or not work) with your audience. Then, you can hone in on the optimal design for your site over time.
With that in mind, it’s a good idea to test things like:
The call to action
Testing the call to action is always a favorite, and for good reason: it can cause dramatic changes. In case you aren’t familiar, the call to action (often abbreviated as “CTA”), is essentially whatever you’re trying to get the visitor to do on that page, whether that’s sign up for your email list or buy a product.
For example, if you’re an e-commerce store owner, you’d be testing the buy button wording, color, and size, on your product pages. If you have a landing page that you’re driving people to for an email marketing campaign, you’d be testing the wording around the email form and the button that they click to sign up (“Subscribe” vs. “I want in,” as an example).