MEC Retailer's Research Log: December 21 Clinic Notes

December 21, 2005

December 21 Clinic Notes

NOTES:

We will be adding notes here. Please refresh this page when instructed to do so.

A/B/C Split Test - Survey
	Page A	Page B	Page C
Total Votes	22	51	103
Percentage of Votes	12.50%	28.98%	58.52%

A/B/C Split Test - Actual Results
	Page A	Page B	Page C
Percent of Traffic	34%	33%	33%
New Sales	244	282	114
Change	N/A	15.57%	- 53.28%

JoAnn.com Multivariate Tests
Metric	Improvement
Average Order	137%
Revenue Per Visitor	209%
Sewing Machine Conversions	30%

Keywords Tested:

B.C.
Foxtrot
Ziggy
boondocks comic
calvin & hobbes
calvin hobbes
cartoon comic
comic art
comic page
comic store
comic strip
comics
doonesbury
free comic
funny comic
garfield
newspaper comic
online comic
sunday comic strip
web comic
webcomic
yahoo comic
"Andy Capp"
"BC Comic"
"Dog eat Doug"
"For better or for worse"
"bloom county"

Google AdWords CTR Test - Survey
Headline	Total Votes	Percentage of Votes
ReadComics Comics	270	20.80%
DailyComics Comics	790	60.86%
GoComics Comics	155	11.94%
uComics Comics	83	6.39%

Google AdWords CTR Test - Actual Results
Headline	Click-Through Rate
ReadComics Comics	0.9%
DailyComics Comics	1.2%
GoComics Comics	0.9%
uComics Comics	0.8%

Email Subject Line Test - Survey
Subject Line	Total Votes	Percentage of Votes
Test Your Marketing Intuition	268	33.38%
An MEC Invitation: Test Your Marketing Skills	138	17.31%
So You Think You're A Marketing Expert? Prove it here...	393	49.31%

Email Subject Line Test - Actual Results
Subject Line	Open Rate	CTR
Test Your Marketing Intuition	11.99%	3.34%
An MEC Invitation: Test Your Marketing Skills	12.59%	2.93%
So You Think You’re A Marketing Expert? Prove it here...	11.46%	2.90%

Intuition can be effective in determining what needs to be tested. It can provide the insights you need to decide which general direction to explore.
Intuition may fail you when you may attempt to predict, which of those tests may prove successful. Depending on intuition to select one of four possible email subject lines, for instance, it not a reliable way to maximize your chances of achieving the best open rate.
Intuition can be used, sometimes, to dig deeper down in to your test results. Sometimes it can help you glean an insight as to why the market behaved in a particular way.
Intuition typically fails miserably when applied to initial test results. In the beginning stages of the tests the conclusions may be seriously flawed. You have to wait until you have a large enough sample. The danger is particularly real when provisional results happen to match your own intuitive opinion.
A mistake many marketers make is to take the initial findings of the test, make an intuitive determination as to which is best, and stop the experiment. More often than not, this leads to a bad decision.
Intuition is often limited, in that it depends on previous experiences. It depends on pattern recognition. Because of this you may make natural assumptions that are contrary to absolute changes in the context, or to changing conditions in the marketplace.

URLs:

1. Become an MEC Research Partner

2. Email Subject Line Survey

3. Split Test - Page A

4. Split Test - Page B

5. Split Test - Page C

6. Transparent Marketing

BIO:

Dr. Flint McGlaughlin

The Marketing Experiments Journal is published by Digital Trust Inc. (DTI). Dr. Flint McGlaughlin is the Director of DTI.

Dr. McGlaughlin is a pioneer in strategy and communications. His primary research is focused on discovering the most effective means to market intellectual assets across the Internet.

Dr. McGlaughlin has advised more than 1500 companies including, AT&T, The Federal Reserve, Pitney Bowes, and Merrill Lynch. He has conducted over 2000 hours of keynote speaking. He has directed the development of more than 20,000 web pages.

Dr. McGlaughlin currently serves as the Co-Executive Producer of the new FOX Family series: "Courage" - hosted by actor Danny Glover. He also serves as a senior advisor or board member to twelve growing companies, and four charitable foundations.

His writing and television production activities have, thus far, won two Tellys, an Edwin Murrough, and an Emmy.

Dr. McGlaughlin is quick to acknowledge that the research at MEC is largely dependent on the intense, focused efforts of the Lab's Leadership Team.

Posted by Brian Alt at December 21, 2005 01:23 PM