One of the things marketers really excel at is testing - we A/B test and multivariate test and the most sophisticated of us test with the Taguchi method.

Then we measure, and we look at our results, and say "Ah-ha! This won. It must be the best choice."

And this is all well and good, but there's something missing.

Many marketers don't ask why the other splits didn't work.

You may argue that there's no point; it didn't work and therefore it's irrelevant. That information cannot be usefully applied except as something to be avoided in the future. This attitude misses the whole point of testing.

Testing should tell us more about the audience we're trying to reach, not just about what gives us the best numbers. If we look at what didn't work as well as what did, we can make much better judgments about what will work in the future. It's the difference between adding up numbers and understanding what they mean.