Market research is an amazing thing. You can take the same study and reach two entirely different conclusions. Or you can achieve totally different results by the way the question is worded, what multiple choice answers are provided, what questions preceded it, or even the tone or style of the interviewer. Then there are issues with how the sample was selected, online vs. phone, whether it is representative of a national audience, whether incentives were provided, etc. etc. etc.  

That's not to say that market research should be ignored. Just that you need to be careful with it. And if you make decisions based on market research, you need to understand how and when it was collected, what the exact questions were, and who underwrote the study.

Case in point: Mobile banking demand

In the past two weeks, two reliable research companies, Jupiter Research and Compete, Inc. released research finding on whether U.S. consumers think they will want to use mobile banking when it becomes available. This type of "what if" question is even more problematic than other types of market research. Because the participant doesn't use the service in question, the interviewer first has to paint a picture of what it might look like at some future point, then ask the respondent what their level of interest is. So, the results are highly dependent on how the hypothetical service is described, and if it's a telephone interview, how enthusiastic the questioner is about it. Imagine the difference in response to these two questions:

1. How would you like to press a button on your cellphone that gave you instant, secure, free access to your bank account balance so you didn't ever bounce a check again?

or

2. At some point in the future, you might be able to download and install a Java application over the air for your mobile device that provided a subset of the functionality of online banking ported to a 2 inch screen. And, as long as you never left your phone somewhere by mistake, it should be as secure. How excited would you be about that?

Unfortunately, I haven't seen the exact questions or methodology used to produce the following press releases, so I can't say exactly how the companies reached their conclusions. However, Compete will be presenting their finding in a free webinar Thursday, so you might want to listen in. If you can't make it, I will file a followup blog post. Full disclosure: After spending much of Q1 researching and writing about mobile banking and payments, and yes, selling reports of my findings, I'm firmly in the pro-mobile banking camp (see previous coverage here). 

Finding 1: Considerable interest in Mobile Banking

Author: Compete, Inc.

Link: http://blog.compete.com/2007/05/01/mobile-banking-rebirth/

Synopsis: In an April survey of online banking users, only 19% said they would definitely not use it, while 11% said they definitely would. The vast majority (70%) between the extremes need more info before they decide. There is a measurable advantage for the negatives (38% won't/probably won't use) over the positives (29% will/probably will), but that's doesn't seem particularly negative for a service that does not yet exist.

Note: Compete will be presenting the results in a free webinar Thursday, May 3, at 2PM Eastern. Presenters: Paul Zeckser, Director Financial Services Practice & Ryan Burke, Director, Telecommunications and Media Practice

Finding 2: Little interest in Mobile Banking

 Author: Jupiter Research

Link: http://www.jupiterresearch.com/bin/item.pl/press:press_release/2007/id=07.04.23-mobile_banking.html/

Synopsis: Limited data was released to the public, but in a press release last week, with the title, JupiterResearch Finds Limited Consumer Interest in Mobile Banking, the company said only 8% of consumers were interested in mobile banking. No supporting data was provided. We will invite report author Asaf Buchner, who I respect greatly, to provide more background on Jupiter's findings.

Note: Below is the exact quote from the press release. The specific scenario here, "using mobile browsing to check account balances," may be part of the reason for the lower interest. Only about 10% of U.S mobile phone owners use mobile browsing today.  

Just eight percent of online consumers who own a cell phone are interested in using mobile browsing to check account balances.