How does A/B testing work?

We’ve said it before, a website is only as good as its content. But have you considered whether you’re broadcasting your content in the right way?  Your site can have the best content on the web, but users won’t see it if they can’t find it via headlines or landing pages. Part of improving this can be done through a process called A/B testing. This process isn’t unique to web development, but tech companies utilize it frequently to quickly find weaknesses across their content or interface. We’ll explain what it is, how to employ it, and how we use it within the Collective to improve our own processes.

What is A/B testing?

A/B testing is the most common way to refer to it, but the process is known by another name, split testing. In essence, it’s a test done by creating at least two independent factors to test, then showcasing each factor to a percentage of your audience. After testing, you should get a result as to which factor performs the best based on audience feedback or interaction.

It’s often used to test things like headlines, formatting of landing pages, or advertising copy. By using A/B testing to find which content an audience is more receptive to, you’ll see better engagement than if you had simply chosen content without A/B testing. It can also teach you more about what your audience is interested in.

How can I run an A/B test?

Before you even begin thinking about the application of an A/B test, decide what you’re going to be testing. Are you checking the receptiveness of different headlines? Considering moving around some of the tabs on your homepage? Once you’ve decided what you want to test (we would recommend keeping the number of testing factors to a minimum, no more than three or four), set a time to run the test. The general recommendation is about two weeks. You’re going to want all the factors of the test to be close to the day-to-day traffic of your site, so setting the test for two weeks will provide more accurate results over time.

So you’ve set the factors for testing, you’ve set the time the test will run. Now it’s time to sit back and wait for results. Don’t change anything about the test while it’s running, as this could skew results. Once you’ve received the results of the test, don’t be discouraged if there’s not a clear preference between the factors. Running a test multiple times or narrowing the factors could provide more informative results.

How does Publisher Collective use A/B testing?

When we work with advertisers to run campaigns on our partner sites, oftentimes we’ll run A/B testing on the ads to see how they might perform. This could include testing the creative content or copy within an ad, the ad placement on the page, or the ad format itself. We want to ensure that advertisers are getting the best ROI for their campaigns, and that our audiences are seeing the ads that are most interesting or relevant to them.

We also use A/B testing to test ad stack compatibility on sites. The results from these tests give us greater insight on which ad formats work well on different sites, resulting in higher revenue overall, and a better UX for our partners’ audiences.

We’re always looking for new partners to join the Collective. If you think you’ve got what it takes, apply here. We post weekly blogs on everything from industry news to programmatic explainers, and more! Check out our newest posts here.