Guide To A/B Testing With Google Website Optimizer

By Jake Rocheleau. in Toolkit. Updated on February 22, 2019.

To generate more conversions on your website you’ll have to look into traffic statistics. Google Analytics is a fantastic service, but it can only go so far. The concepts behind A/B testing are sound and have been used within professional Internet marketing for years.

Basically you would create 2 pages with slightly different content: headers, images, and other stuff you want to test out. Google Web Optimizer will tally all your total visitors and split traffic between both pages evenly. Stats are provided regarding the total number of conversions onto a landing page, so you are able to compare which changes are leading to more signups. The service actually ties directly into Analytics, which is a huge bonus.

To get started check out website optimizer’s official page and sign into your account. We will be taking a detailed look at the GWO interface and how you should go about setting up a new A/B experiment.

Website Optimizer – Layout Overview

Once you first land on the GWO home screen you’ll be presented with a table of current experiments. If this is your first visit it’ll likely be empty, for now. Check in the top right corner for your account e-mail address to confirm you’re properly logged in.

Google Web Optimizer Home screen - blank tests

Directly beneath this is a drop-down menu of all your Google Analytics website profiles. If you have verified yourself on each one they will be accepted into the website optimizer software. Simply click and select the profile you want to create tests for and GWO will refresh the page data automatically.

In the sidebar you’ll also notice some potentially helpful guides and FAQs for beginners. These can be helpful if you have some time for reading. But for now let’s get started and jump right into the process and we can learn as we go. Click on Create a New Experiment to proceed.

A/B vs Multivariate Tests

When you first go to create an experiment Google will ask you to choose between A/B and multivariate testing. These terms are likely to scare you off but I promise they’re not difficult to understand.

With an A/B setup you can test 2 different versions of the same page. This option is best attributed to when you are testing totally different layouts for the same page. Multivariate is a bit softer where you just want to update a few key areas. Maybe you want to change the header text and color of your download button, or rearrange some advertisement banners.

Step 1 - select A/B-Style or Multivariate Testing

From the visitor’s perspective it won’t appear as anything strange has happened. With either method Google will automatically calculate traffic throughout the day to split visitors as evenly as possible between your variable pages and the original.

What are your Goals?

You should pause for a bit to think about your goals before moving onto the next step. Most webmasters going through A/B tests are looking to increase conversions from their web traffic. This could be gaining signups for a newsletter, mailing list, user registrations, Twitter/Facebook followers, or any handful of other user actions. But you should have your own idea of “success” which pertain to your own web goals.

The Google team has actually published a large number of tutorial videos onto YouTube. These are fairly extensive yet also lengthy. I recommend you check out the tutorials list just to see if anything jumps out. The important thing is you take your time and plan out decisions logically.

Realistically the multivariate testing experiment is a foolproof method for demoing small changes. We should choose this option for our current experiment setup, but if you’d rather test out A/B the menus are very similar.

Creating the New Experiment

First off we need to enter a name for this new experiment. Then directly underneath this form input the URL leading to the page you want to run tests on. In many cases this would just be your home page. But you can get more creative, such as checking how many visitors onto your contact page will actually send a message.

GWO Input test URL and conversion URL for new experiment

The third field requires a conversion page URL where you hope to move your visitors after they take some action on the original page. This pertains to the goals discussed earlier since you’ll need some method for testing the success or failure of the new design. The conversion link could even be a page off the navigation. Just stick to something easy for now. To get a few ideas check this Google support article focused on dynamic content within WSO experiments.

JavaScript and Page Sections

With this page all set hit Next to get the JavaScript code for your new experiment. These two blocks need to be added onto the test page and the conversion page, respectively. Google recommends you place the JS blocks within the <head> element, but anywhere in the code should work.

JavaScript blocks and additional page sections

You’ll also need to use special script tags to define each editable area or section of your testing page. This is content which will be dynamically altered through Google WO. You can even segment a few different sections within just 1 testing page. I added an example bit of code below to give you a better idea of this process.

<!-- define the featured image content section GWO -->
<script>utmx_section("FeaturedImage")</script>
<noscript>
<div id="featured-image"><img src="images/1/myimg.png" /></div>
</noscript>

And now back in GWO click the “Validate pages” button. If everything looks good Google will move you through into the next step. Otherwise you’ll be given specific error messages and cues about how to finish the validation process. Note that if you’re having trouble online try creating two local HTML files instead. GWO will let you upload an offline version of the pages just as a test-run.

Designing Variable Content

Since we are using the multivariate test I want to make just a few changes to my homepage. More specifically with a goal to increase conversions onto my videos page. To accomplish this I want to include a sidebar banner above the fold, and maybe a bold text link at the end of my navigation.

I’ve drafted a small list below with a few more ideas. You should try to implement just a few ideas at a time and let these run for 1-2 weeks. It’s always good to come back and change up the styles anyway, and you’ll receive more targeted data based on the smaller changes over time.

New Logo colors, size, position on page (grabs the users’ attention)
Removal or replacement of banner advertising for more clicks
Buttons used to convert users – signup, download, purchase, register, etc.
Web page copy (text on the page) can affect visitor mindset
Frequency, style, and colors of images and page graphics
Flashy JavaScript/jQuery effects for flashy animations or dynamic content
New icons, gradients, tab bars, or in general updated user interface elements

Studies have found that radical changes far supersede the value of minute changes. So if you are going to update parts of the layout do it big! Website Optimizer page sections are set up so that you never delete content, and all your experiment changes are temporary if you choose. But if the differences between page 1 and 2 are barely noticeable your tests will likely return sub-par results.

Replacing Sections in HTML

For each area defined via the script tags you’ll be able to check out and edit the code right from within GWO. After you pass validation the next step will display a table of all your page sections. You can now create as many alternate forms to the HTML as you feel necessary. These will be included in the next A/B test run so make sure you preview everything first!

Multivariate testing for GWO sections via HTML code

It’s also important to note how each section really works. You can make edits on this one page to many different sections, but for each duplicate variation (such as 4 different logos) website optimizer will have to break this up and evenly distribute pageviews among your visitors. So it is best to limit 1-2 edits per section for cleaner results – and remember you can add virtually unlimited sections per one(1) single variation page.

To begin making edits press the Add new variation link found to the left of the code box. This creates duplicate HTML content which you can then edit to your liking. The brilliance here is that none of your original HTML code is being edited – everything is replaced dynamically through Website Optimizer.

Final Review and Launch

After much hard work and dedication we’ve finally completed the setup! There are a number of steps in this process, so it’s easy to get lost with so many features. Double-check each of the settings on this page to ensure you’ve put everything in properly.

At the bottom you’ll find a Launch button to confirm and publish this experiment. If everything goes well you should see a success message at the top of the screen. New links also appear directly underneath your experiment headline after it publishes. You can now pause/stop the experiment at any time, or also copy the settings into a new experiment which may save you time in building a few different tests.

This process for setting up an A/B test instead of multivariate differs only slightly. Instead of editing parts on a page you simply provide an alternate URL which Google will split traffic between. Then GWO will check how many conversions each page can generate(you still need to add JavaScript). I mostly prefer to work with multivariate testing but give both setups a try to see which fits best for your conversion goals.

Juggling Multiple Experiments

Now that we’ve covered the process of creating a new experiment I want to explain the experiments view. This is also the Website Optimizer default home page view where you land initially. As you pause and stop older tests their status is changed and sorted accordingly in the experiments table.

Additionally you are given some metrics to work with for each experimental setup. Page visitors is counting the total number of unique visits between all different pages. The conversions are how many visitors follow through onto your chosen conversion page, and next to this you’ll see the percentage out of your total visitors.

completed website optimizer test w/ graphs

Website optimizer is possibly the best free tool for Internet marketers’ own curiosity. If you don’t have a reason to track conversions then A/B testing will seem utterly pointless. It’s also true that you should be generating a fair amount of traffic, at least 60k+ uniques/month to see a noticeable impact with any experiments.

But even on smaller websites, GWO is still useful to a degree. Don’t feel the requirement to base all your design intuition on traffic analytics. But if you’re even a bit curious I recommend jotting down a few ideas to brainstorm and setup later on. Below I have constructed a list of links and resources online to learn more about Google WO.

Helpful Resources

Conclusion

More advanced webmasters have most likely used a few similar A/B services before. But Google Website Optimizer really takes A/B experiments to a whole new level. The connectivity between Analytics and GWO is brilliant. And the user interface is very clean, simplistic, and practically dumb-proofed!

Hopefully this introductory guide has influenced your opinion towards multivariate testing environments. After a couple weeks of testing you’ll begin to notice patterns emerging, and using these trends you can draw heavy conclusions about the site design. Keep your eyes on the conversion numbers since these will ultimately define the success or failure of any A/B experiment.