Friday, May 16, 2008 5/16/2008 10:23:00 PM
When you think about it, websites are like mutual funds. You spend money buying them, and you expect them to bring in as much money as they can each year. But unfortunately, people seem to be more rational when selecting funds than when picking websites.
Let's say you're picking a retirement-savings mutual fund. Chances are you'll investigate the current and historical performance of each fund. It would be pretty unusual for you to take money out of an existing fund and put it into a new one without comparing how the two perform, right? In fact, if you took all your money out of one fund and put it into another one based only on gut feel or opinion, people would call you crazy.
Now let's look at how we do website redesigns today. We typically invite local agencies to razzle and dazzle us with their portfolios. We choose things that look nice and make us feel good about spending tens of thousands of dollars (if not more) on a redesign. We get enchanted by talk of Web 2.0, widgets, gadgets, video, surveys and interactive movies. If we're using a really good agency, they'll ask for or already know our current web-analytics metrics so there's a baseline against which to measure. And after months of brainstorming and discussions, we see initial concept mocks, and the new site is unveiled—perhaps a press release is issued.
The irony is, that launch day should really be the first day of a test, to see if the new site really does perform better than the old one. Sure, you think it looks better and your gut tells you it must perform better (especially since you've become emotionally invested in it). But the reality is you don't know. Even if you compare baseline web stats, you won't know if improvements are seasonal (almost any redesign in the fall outperforms a summer predecessor) or due to another external factor (like a new ad campaign).
The good news is, with Google Website Optimizer you can now pick a website as you would a mutual fund. You can run an A/B test and have half of your visitors land on your old site and half on your new site. (Here's more info on setting up a simple site bake-off. Here's a video tutorial for setting up general A/B tests in under 5 minutes.) Then you can measure which version does best. And if the new site really does do better, time to hire that agency for more work. ;-)
