Tuesday, August 19, 2008 8/19/2008 08:23:00 AM
As many of you know, Website Optimizer divides your traffic evenly among all of your page variations for the life of your test. This strategy lets us provide more precise estimates about how each variation is performing. It also allows every variation to have an extended opportunity to reach its full potential over the test's duration.
Nevertheless, many users find that they don't gain much extra value from knowing how bad a variation is performing once they are satisfied that it won't likely be a winner. If a variation is clearly performing poorly, you may want to remove it from your test and have all subsequent traffic be allocated to your better performing variations. That way, you'll get statistically significant results quicker, while maximizing your conversion rate.
In the past, there wasn't an easy way for you to disable low performing or illogical combinations. You'd have to stop a test, make a copy, lose all your test data, and then launch a new test. That's all changing starting today. Now, you can simply select any number of page variations from your experiment report, click our new "Disable" button, and you're done. Easy as that. All your future traffic will be sent to your remaining page variations, and you'll be on the path to quicker, more actionable test results.
We've made a few other enhancements that we hope will make your testing experience even more meaningful. We've updated our reporting interface to better indicate when a page variation is actually a high confidence winner as opposed to just one that happens to be performing well at a given time. We'll also now support offline validation for A/B tests - if your test pages aren't accessible to Website Optimizer, you can instead directly upload a copy of your tagged source code and we'll make sure that everything is tagged correctly.
If you're unfamiliar with Website Optimizer, no worries. We have a wealth of resources to get you up and running in no time. Start by watching our new product tour, A/B experiment demo, and multivariate experiment demo. You'll see how quick and easy it is to create Website Optimizer tests.
We'll also be speaking at the Post-Click Optimization panel at SES San Jose on Thursday, 8/21, so feel free to stop by and say hello. Our good friend and industry expert, Bryan Eisenberg, will be at SES as well to talk about his new book, Always Be Testing. There should be lots of testing goodness in the air. We hope to see you in San Jose!
