This article is meant to explain what A/B testing is and why it’s important. GoldTree will cover its practical applications as well as how I believe it’s changing the way businesses small and large approach data. At its core, A/B testing leverages data to understand how to improve a website, service, application, or e-commerce storefront. The idea of harnessing data goes far beyond increasing conversion rate, but can also invoke in its users a more “data-driven mindset” when making decisions.
What Is A/B Testing?
A/B testing is the act of splitting your web or application traffic into different “variations” that you have created. Other names for A/B testing include:
- Split testing
- Bucket testing
- Multi-variant testing (type of A/B testing)
- Redirect testing
The primary goal of A/B testing is to determine which of these variants performs the best when analyzing certain goals, such as increasing the number of clicks, downloads, or purchases. This allows you to confidently decide on changes to your website or application based on data. Which version of the site is garnering more traffic? You can then proceed with changes knowing that they will benefit your business. If your primary focus is increasing revenue or leads, then it would be your “conversion goal” that informs the focus of your A/B tests.
Now you might be wondering which exact A/B testing tool is going to provide you with the most seamless experience.
Let’s get into the pros and cons of the most popular A/B testing tools.
A/B Testing Tools
There are a variety of A/B testing platforms available on the market today. In essence, they are the tools that allow you to run an experiment in which two or more variants of a webpage are shown to visitors at random. Then, statistical analysis is used to determine which variation performs better for a given conversion goal.
Yout tool should tell you:
- How much traffic has been sent into each variation of your test
- When a test has reached “statistical significance” (confirmed the learnings)
- How different segments reacted to the test (ex: mobile vs. desktop)
- Who is seeing the test (ex: only paid ads users)
Like any competing brands with similar products, each A/B testing tool each AB testing tool will vary in how efficiently and effectively they accomplish the above tasks. Some can provide granular levels of data, others very high level data. Some are more affordable, and some are expensive. When deciding on an A/B testing tool to use, you should consider where your business, e-commerce storefront, application, or team is situated in their A/B testing and conversion rate optimization (CRO) journey.
Below are some of the most popular A/B testing tools and what I believe to be their most significant characteristics:
- Google Optimize
- Free
- Limited goal setting
- Best for analyzing data out of Google Analytics (seamless integration)
- VWO
- Mid-tier costs
- Great goal setting capabilities
- Additional features (heatmaps)
- Analytics are harder to setup
- Optimizely
- High cost
- Most sophisticated features
- Analytics are relatively simple to setup but not easily integrated with GA
- Unbounce
- Free to Paid membership options
- Simple to use
- Limited goal setting within the tool
- Analytics are limited but can be setup with GA
Realistically speaking, if you and your organization are just starting out on your A/B testing and CRO journey, Google Optimize might be the most feasible option. You most likely have Google Analytics installed and can easily set up Google Tag Manager.
To install Google Optimize you can reference Google’s documentation.
What to Do to Begin A/B Testing
Now that you have installed your Google Optimize account or any other A/B testing tool, the fun can finally begin. It’s time to start experimenting! But wait. Before you launch your first experiment, consider the following:
- What is your process for A/B testing?
- What are your goals?
- Which team members will be driving you down the A/B testing road?
- How do we complement A/B testing above and below the funnel?
Keep reading to find out exactly how to tackle each of these questions
What Is Your Process for A/B Testing?
A normal A/B testing process will normally look like this within about a month’s time. Depending on your traffic, you may be able to run a single experiment or multiple experiments within this time period.
- Data review: Review the data you currently have that can guide your A/B testing experiments. Look at Google Analytics to identify trends and behaviors. Leverage customer segments you’ve identified through interviews or surveys. Use everything and anything!
- Ideation: Based on the data, start building 10 to 15 ideas and prioritizing them on a ratio of “level of effort : potential ROI.” This will help you prioritize low hanging fruit and build a backlog based on data.
- Design: Get your design team working on methods for best incorporating these new experiments. Every member of your growth team should collaborate on the design. Gathering multiple perspectives always leads to the best experiments!
- Development: Build your experience in your A/B testing tool or using your website CMS.
- Deployment: At this point you know there are no dependencies or potential problems because you’ve worked with development, design, business stakeholders, and the product team. So now you’re ready to launch!
- Analyze Results: You’ve run your tests and gathered plenty of sessions for your experiment. You have seen a 5% increase in your goal (conversion rate, revenue, leads). WHY!? Look into the data via Google Analytics or whichever analytics tools you’re using to measure the exact changes in your user’s experience.
- Ideation Based on Winners and Losers: You now understand why user behaviors changed based on your experiment because you analyzed the results! Maybe users navigated to one of your product pages more often than others which led to the increased conversion rate. What is this indicating to you? That this best selling product needs to be highlighted front and center. From here you can develop more test ideas around this hypothesis. Boom! You have new test experiments based on your A/B tests. It’s a positive feedback loop that has the potential to make your site better every time you run a test.
What are our goals?
So now that you know what you need to do from steps A-Z for A/B testing, let’s focus on one of the most important ports of A/B testing – what you are tracking!
Your business is bound to have goals that differ from any other business goals, so you need to clearly define these. I like to use a tier system to rank goals based on their level of importance. For example:
- Tier 1 Goals: Revenue, Conversion rate (Leads or Sales)
- Tier 2 Goals: PDF Downloads, AOV
- Tier 3 Goals: Videos watched, Instagram follow
Yes, you should be tracking any goals that are relevant to your business but keep in mind you also need to prioritize them so that when you A/B test, you can clearly call out the benefits or downsides of an experiment.
Who Is the Team Driving the A/B Testing Roadmap?
This is a key consideration when defining your A/B testing roadmap. You need a team that is going to dedicate time and resources to improve your website, application, or service. Having a team that can bring diverse ideas to the table is key to gaining different perspectives for A/B testing experiments.
What I normally recommend for a team breakout is one major stakeholder from each product sub-team and one business stakeholder.
- Product team member – Leader and organizer
- Marketing team member – Top of funnel communicator
- Engineer team member – Technical expert
- Design team member – User focused and keeping tests on brand
- Business stakeholder – Voice of the customer
With a team like this, you can effectively understand any and all dependencies for A/B tests on your website while also drawing many different perspectives.
How Do We Compliment A/B Testing Above and Below the Funnel?
So you’ve built your team, define your goals and processes, and now you have results showing awesome improvements. How do you keep doubling down on your success? Get more team members involved by building A/B tests above and below the funnel you are testing on.
Let’s take a second to define these:
- Above the funnel = Ads (images, videos, and posts)
- Below the funnel = Emails, texts, post purchase communication
Every step of the way we want to be making sure your A/B tests align with the above and below funnel initiatives. How could you do this?
- Create a custom landing page that matches the ad creative above the funnel
- Create a custom landing page that matches the email communication below the funnel
Customization like this can drastically increase your conversion rate. Very few businesses implement above and below the funnel optimizations, so when users come across this level of personalization on your website, they are sure to be impacted by it.
When you think outside the box with A/B testing, the tools can most definitely support it!
Why Is A/B Testing Important?
A/B testing allows businesses to make more educated decisions about their online presence. It provides information on how to continuously evolve and improve based on the user’s preferences. During the A/B testing process, you build hypotheses, understand user behaviors, and clarify what will work best.
If you adopt a mindset that includes testing everything, learning more about your users, and settling disagreements with data, it will permeate down to all parts of your business and guide you down the right path.
If you need examples of this, just look at any major company with an online presence. Facebook, Snapchat, Lionsgate, Google, Tesla, Amazon, Stripe, Twitter, Netflix, and General Motors all leverage A/B testing in one way or another. They are data-backed, data-focused companies that continue their success by maintaining a data mindset.
For your business to achieve its goals, you need to use A/B testing because that’s what your competitor is most likely doing.
Conclusion
As you begin your A/B testing journey or look to improve it, remember to follow the steps laid out in this guide. Also, consider how you can get more stakeholders involved to increase the investment in this critical process for your business.
