Pretty much everyone has had at least one meeting like this: A group of co-workers sitting in an overly air conditioned conference room, arguing in circles. The argument stretches on for the foreseeable future because no one has any hard data; they're going on their gut instincts.

The good news is that marketers have a way out of this — they test. The type of test I'm going to cover in today's post is A/B testing. In an A/B test one variable is tested at a time. This is the simplest testing method, but it does take a bit of discipline.

Set Up

The place to start when setting up an A/B test is to decide what you want to track, and what counts as a success. If you're making a change to a website, are you looking for longer view times? Click throughs to other parts of the site? More sales? Identifying the goal will help identify the area to test.

In A/B split tests only one variable is tested at a time. For instance, if you're looking at optimizing an email blast you've run before, you might decide to change the headline. The control version would be the email as it is; the test version would have the new headline. The only change would be the headline.

An altered control will invalidate the results. If the test version outperforms the control, it won't be known whether the test caused a lift or the change to the control caused a depression. A decrease in response for the test compared to the control is also suspect for similar reasons.

Testing

After your test is all set up, it will need to go to the customers to see how they respond. To keep the results clean, the audience for the test and the control should be as close to identical as possible. For instance, if it's a website test, one ideal would be to alternate which version is seen by people who visit the site: The first person to visit gets the control, the second gets the test, and so on.

Results

Once enough data has been gathered to be statistically significant, look at the results and see whether the test or the control won. If the test won, that is your new control for the next round of testing. If the control won, well, you learned something.

Either way, the testing needs to go on. The more you test, the more you'll learn and the better results you'll see. Testing lets you challenge and improve on long-held assumptions about the best way to market a product. It also makes those two-hour long meetings a thing of the past — you'll have the data you need to back up your position.