Data-Powered B2B Video Marketing: A/B Test Your Videos and Supercharge Your Results

WinWin Videos, January 2020


What is A/B Testing?

Put simply, A/B testing is deciding which version of a piece of content or marketing tactic yields you the best results. 

For instance, when you run two nearly identical ads with different color schemes to see which gets more clicks. Or when you put your Signup Forms on two different places on your website to see which attracts more new subscribers.

When Amazon tested the one-click system, decreasing the amount of clicks needed to purchase a product, they found it drastically increased sales.

A single test can help your business, but it’s not the end of the process. You should always look for new ideas to test, using the data to maximize conversions and retention.

A/B tests let you optimize your content for the results you want, so you can continuously improve your marketing strategy.

While you can run these tests organically, running them on an ad platform like YouTube, Google, Facebook, or LinkedIn will give you more reliable and actionable data. The more tests you can run, the more reliable your data will be. Or as Thomas J. Watson puts it:

“If you want to increase your success rate, double your failure rate.” – Thomas J. Watson, Former Chairman & CEO of IBM (Click to Tweet)Winwin_videos_AB_testing.png?w=960&ssl=1A/B testing is when you test two different versions of a variable to maximize your results. Doing this regularly can lead to massive boosts in revenue for your business.


Before you start, decide on your Key Performance Metrics (KPIs)

Before you start testing, you need to know which results you want to optimize with your tests.

Do you want more clicks, or more video views? Do you want more traffic to your website, more subscriptions to your email list? Or do you want people to watch this video and make a purchase immediately?

All of these data points can provide key insights, but when you test, you’re trying to optimize for a certain metric. 

Say you want to optimize your video for website traffic. If you test two versions of a video, and one gets you three times more video views, you might be tempted to say this version won the test. 

But what if the other video got 20 percent more traffic to your site? Even though the difference is less impressive, it’s more in line with your goal for the video, so this one is the real winner.

“A mistake that some companies make is to start moving a bunch of levers around without clear planning upfront for what they're trying to optimize—and what will be impacted by those changes.” -Dan Siroker (Click to Tweet)


Decide which variable you want to A/B test

Variables, as opposed to KPIs, are the aspects of your content you can adjust before running tests. Consider some of these major variables for both your video and its corresponding ad:

  • Titles: a good title sparks curiosity and provides information
  • Captions and descriptions: describe what the video is about and why it’s worth watching
  • Thumbnails: a well-chosen thumbnail can catch much more attention
  • Color scheme of ad: presentation makes a difference
  • Call To Action button text in ad: different phrasing can be more intriguing

But it’s not just about these basics. Sometimes you need to broaden your test criteria. For example, other variables to examine include:

  • Timing: which days of the week do your ads receive the best engagement, and which times of day?
  • Placement: If the video is on your website, on which page? Where on the page? Try playing around with this, but make sure it’s on a page people frequently visit.
  • CTA: do your videos end with a clear Call To Action, such as a number to call or site to visit? Test with and without a CTA, but also test different versions of your CTA and see which converts more customers.

Remember, this often requires many tests before you get actionable results, so make your tweaks and don’t rush into any conclusions. If you haven’t used Facebook Ads yet, much of this is done for you, and with great efficiency.

Check out our article on how to run a Facebook Ad for video to learn more about this.


If you’re creating your own videos too, consider these:

  • Video resolution and quality: phone camera or professional camera? Your audience may respond differently depending on the social platform you upload it to.
  • Video Length: how long is the video? Sites like YouTube offer data on when people click away from your video, so you can see if they lose interest. 
  • Embedded Branding, such as placement and size for graphics and logos
  • Voiceover: who does it, and when? When do you have supplementary text?

Pro tip: when you hire a video production agency like WinWin Videos, we’ve done this kind of testing for you.


Only focus on one variable at a time!

The more variables you add to your testing, the more difficult it is to tell which variable is making the difference. This makes it harder to obtain valid, actionable data.

For instance, say you’re on YouTube and you decide to test two versions of a video. You want to see if a new thumbnail will get you more views, but you also want to see if a new title will help. 

You test Video A with Title 1 and Thumbnail 1, and test Video B with Title 2 and Thumbnail 2. Video B outperforms Video B. So should you use the new thumbnail, or the new title, or both?

To do this correctly, you would actually need to create 4 versions of the video.

  • Version 1 = Title 1 + Thumbnail 1
  • Version 2 = Title 1 + Thumbnail 2
  • Version 3 = Title 2 + Thumbnail 1
  • Version 4 = Title 2 + Thumbnail 2

The version that performs best will tell you how to move forward without guesswork.

Pro tip: not sure what to test? Start with the variables you think might be lacking. What do you think would have the biggest impact if it were improved?

Pro tip: If you have time or budget constraints, postpone tests for anything that seems too resource-intensive. It’s easier to change a video title than it is to redo the camera angle of a scene.


Once you’ve made these decisions, it’s time to run your tests, collect your data, and analyze which versions performed best.

So let’s overview the process:

  1. Decide on your goal for the test, and which KPIs will guide your analysis later 
  2. Decide on the variables you want to test (thumbnails, titles, color, logos).
  3. Create two versions of your content, with only one key variable changed
  4. Run each ad version separately and simultaneously. Compare your KPIs. Which performed better for the metric you wanted to measure?
  5. Learn from your results and make updates to optimize for your marketing goals.
  6. Rinse and repeat: once you’ve scrapped the low-performing version, test something else! Whether it’s a new potential title, or another variable entirely. Remember, you don’t always need to think on small variations of a theme. Sometimes a bold new hypothesis from a different direction can yield impressive results. Challenge your own assumptions.



A/B testing can be a time-consuming process, and sometimes expensive. However, optimizing your content for the best performance will lead to much higher conversions over time. The results may not always be striking or immediate, but each test leads to content and ads that will turn web users into video viewers into product purchasers.

With our WinWin Videos program, Media272 provides you with a nationwide, large-scale video production service, and we produce as many videos as you need.

More videos means more opportunities to test and improve your video marketing strategy with real-world data.

To learn more about how the WinWin Videos team works with you to create video testimonials and more, click here.