Wednesday, July 02, 2008 7/02/2008 10:51:00 AM
People start drawing testing cartoons. I was speaking at a conference in Chicago about Website Optimizer and after the talk, one of the people in the audience showed me some cartoons he had doodled during my presentation. I'm quite accustomed to people doodling, IM'ing, and feed reading during my presentations and it's usually unrelated to what I'm talking about. I was quite impressed when Sean D'Souza offered to actually show me his drawings. I thought, wow, this guy has come chutzpah.
To my surprise, the drawings all related to a number of points about testing that I was trying to convey via bullet points in my presentation. Here are some of my favorites, with my reactions:
Takeaway: Online testing is enabling its adopters to outpace competition who keep sending precious traffic to poorly performing pages.
Takeaway: This is how some folks feel when seeing poorly designed landing pages. On the one hand, we feel there's an opportunity to help people dramatically improve their websites. On the other hand, it's sometimes shocking to see how sub-optimal landing pages can be.
Takeaway: People can get really, really zealous once they get into website optimization. Obviously, this idea takes testing a tad too far, but it's not the first time we've heard people take it to heart.
Takeaway: Unless you have tons of traffic, your best opportunity to boost conversion rates with statistical significance is to try some really big changes (think dramatic layout and visual changes plus text changes as opposed to pure text tweaks alone).
Takeaway: Sometimes people get so happy after they've launched their experiment that they obsessively check reports expecting one combination to pull away instantly. The goal is to be patient and understand that you want to find the long-term winner, not this hour's random winner.
Takeaway: This is one of my favorites. Sometimes people try to create as complicated and large tests as they can without realizing that a simple A/B test often gets you 80% of the gains with 20% the complexity and effort.
Thanks to Sean for sending us the pics. We heart them.
