Ecommerce Expertise / Ecommerce Marketing / How To Sell Online

3 Proven Ecommerce A/B Testing Steps to Increase Revenue 22%

/ 4 min read


Most Popular Reads

You’ve been measuring your online store analytics and have been actively working to improve your site’s conversion rate using varying creative, product photos and more. Yet, simply replacing creative, copy and the like, then checking your analytics dashboard for any upticks in sales isn’t the most ideal nor productive way to go about making a real conversion improvement on your site.

Want to know what is? A/B testing.

A/B testing for ecommerce websites continues to get more user-friendly, and does the exact same thing as you’re currently doing, with one simple, but necessary difference: A/B testing occurs in real-time, comparing different versions of your site against one another to prove actual conversion improvement.

A/B testing occurs in real-time, comparing different versions of your site against one another to prove actual conversion improvement.

See, there’s a difference between actual conversion improvement and theoretical conversion improvement. Web traffic varies by day of week, month of year, and a million other random factors. Judging whether or not your site improved conversions based on a change in copy, creative, layout or what have you is a correlative guess at best, and just plain wrong at worst.

Instead, A/B testing occurs simultaneously, with some customers getting one view and others getting another. This allows for the timing of the test to be a control, rather than the creative or change in layout itself.

Sound confusing? Don’t worry, it isn’t, and plenty of online small businesses are already using it to increase conversions and sales. In fact, we tapped into one of those stores, of which used A/B testing to increase revenue by 22%. Here, get step-by-step details to how online shop SwingSetMall, which sells thousands of swing sets, parts and accessories, used A/B testing and the resulting data to ensure their site created a workable sales funnel from their homepage to their product to their checkout.

SwingSetMall’s Secret to A/B Testing Success


Founded as a family business in 1989, SwingSetMall now runs a growing ecommerce site with over a thousand swing sets, parts and accessories. Selling directly from their Washington warehouse and vetted manufacturers, they are the largest online provider of their kind. SwingSetMall’s primary focus is on quality over quantity, and they offer a low price guarantee, a 30-day money back guarantee and superior customer service.

Step 1: Test multiple offers

In seeking to find a promotional offer for customers that would truly influence the buying decision, SwingSetMall tested several different options:

  • A percentage off discount
  • Free bonus items
  • A specific dollar amount discount

As part of their testing process, SwingSetMall wanted to see if making an existing offer more enticing was something that could improve average order cost and conversion rates.

To the company’s surprise, the answer was a resounding “yes.” SwingSetMall already advertised free shipping on orders over $99 across the site, but found that by adding the “No coupon code necessary” copy, which makes the process more convenient for the customer, the company created a measurable increase in conversion rates and revenue, at 10% and 22% respectively.

Step 2: Visibility is key

A large part of SwingSetMall’s success was due to the method of the company’s promotion. In general, it’s easy for customers to miss or ignore promotions promoted via a banner. Why? Because ads are so incredibly common that consumers’ eyes just skip over the parts of the website where a banner typically is. In fact, banner blindness is a recognized phenomenon, with one study saying that up to 86% of online consumers are banner blind.

Banner blindness is a recognized phenomenon, with one study saying that up to 86% of online consumers are banner blind.

To get around this, SwingSetMall used SpringMetrics, a platform that allowed the company to create different kinds of active promotions across the site and then A/B test them. For their free shipping offer, the promotion tactic that worked best was a “pop-down” style banner that appears after the visitor has been on the site for at least 5 seconds.

“We might miss a few visitors this way, but I feel customers are more receptive to the ‘pop-down’ style banner vs. an in-your-face ad that hits you immediately upon visiting a site,” said the product manager Andrew St. Hilaire.

Step 3: Measure your results

The SwingSetMall team understands the importance of ecommerce analytics and testing. For each promotion, they A/B tested the offer with visitors to the site. In other words, half of the website visitors saw the new offer type while the other half saw the original. From there, it was easy for the SwingSetMall team to determine whether the visitors that saw the promotion bought more or converted at higher rates.

Of course, the company didn’t stop there. Once SwingSetMall stumbled upon a promotion that converted well, they also A/B tested colors and themes. In their case, a purple background color proved to increase in the promotion’s effectiveness, even though it clashed slightly with the online store’s color palette. To theorize a bit, it’s possible the clashing promotion was more effective because it stood out from other offers on the site.

Your next steps

Think all of this sounds like a lot of work? Sure, it takes time, but less than you’d think once your tools are set up and ready to go. Plus, A/B testing legitimately pays off. For SwingSetMall, a 22% increase in revenue and a 10% improvement in overall conversion rate for that particular offer was more than enough incentive to begin using A/B testing on a regular basis.

A/B testing takes some time and effort, but it legitimately pays off.

If you want to create your own A/B testing success story, here’s how to start:

  1. Set up an app to run tests for your site design, product pages, or promotional offers. You can find these apps under the Split Testing or Analytics categories of our app directory.
  2. Test at least three different offers or variations before you declare a winner, and make sure that your sample sizes aren’t so small they’re skewing your numbers. As a best practice, you’ll want at least 20 people to try each version of the site (40 customers overall).
  3. Once you find an offer, a site design change, or the like that creates an increase in revenue or conversions, refine it. Try changing smaller elements (like the background or text color) and track results.
  4. Once you’re confident you’ve optimized that element of your site, it’s time to test something else. Headlines, copy, CTAs, page layouts and shipping are all great options.
  5. Enjoy growing your store’s sales based on data-driven tactics!

Want more information on A/B testing and how you can implement it right away? Check out this post on the 10 best A/B tests to try today.

Leave a Comment
  • Hey! I actually haven’t heard of an A/A split test, but will definitely look into it. Toss over the link to one of those forums if you get a chance. Thanks!

  • jimrodz

    Hi Michelle!

    You’ve got a really good discussion about A/B testing here. I assumed you have split tested a lot of websites already! :)

    By the way, are you familiar with A/A split test? Just heard about it and found a few forum sites discussing that method, but I can’t really decipher whether its good or not.

    Also, just in case your readers will be looking for a comprehensive list of split testing tools including pricing as a Google spreadsheet, this list I found might be helpful:

  • Andrei Baklinau

    How do you think…what is the most best service for making A/B tests?

Less Development. More Marketing.

Let us future-proof your backend. You focus on building your brand.