The Research Playbook

Archive for March, 2010

Written by  Roger A. Straus    

Everyone, it seems, is currently talking about online focus groups. There is a definite sense, right or wrong, that like Web surveys, this may be the wave of the future.    

Some researchers are using the full capabilities of contemporary technology to allow synchronous, online virtual groups in which all participants can see one another and respond in real-time. The ideal would have everyone sitting around the table in a virtual reality room, interacting just as if face-to-face, without time lags or other impediments. Perhaps, at least everybody’s real-time image would be up on the screen. The technology to allow this is, however, not widely available as of 2010, nor easy to implement in practice.    

Focus Group with Moderator

Due to various limitations, most “online focus groups” continue to rely on texting. The moderator keyboards questions and probes, participants type in their responses, which then appear on everyone’s screens preceded by the writer’s name. A text-based focus group equivalent might run on smart phones, or you can show faces via Internet video cameras.    

When you manage these groups synchronously and moderate them conscientiously, the groups can offer similar dynamics as face-to-face groups. You can argue that text-based Internet groups have the additional benefit, just like online classes, of making active participation easier for individuals who are shy or find it difficult to speak in public, and reduce barriers associated with appearance (e.g., age, gender, and race).    

On the other hand, you lose the information provided by tone or pacing in Tele-groups and interactions are less spontaneous, in that one must form a response and then type it out. This may be second nature for someone who is very adept at text communication, but this applies to a limited population segment.    

Please share with me (in the comments below):
1 – A valuable take-away you had from any of our blog posts
2 – What you want us to write about next
And, please send out a tweet and tell a couple of people about this blog – Thanks!  http://researchplaybook.wordpress.com
 

	Join Our Email List

For a lengthy discussion of planning focus groups, see Part II: Designing Focus Groups for High Impact http://www.atheath.com/mrrc

Part of series on: Approaches to Data Collection

Among the most important advantages of telephone interviewing, is the researcher’s ability to control the sample. Interviewers are able to find the person in an organization most qualified to respond. They can screen respondents and use probes to find the “right” respondent. Quotas for demographic strata and unique qualifications can be controlled directly with the help of a computer-aided telephone interviewing (CATI) system used by most field houses. Broad-based lists of companies, rather than opt-in email lists, can be used, which helps with randomization.

The use of unaided response questions is easier to implement with telephone interviewing than it is with a web-based approach, which relies on a respondent to take the time to type responses. Interviewers can also probe and encourage respondents to consider more answers, using phrases such as: “Are there any other companies you can think of?”

Telephone interviewing, which is a more active recruitment method than web-based research, helps to avoid professional survey takers regardless of the incentives used. The quality of the list is clearly as important in telephone interviewing as it is in web-based data collection. However, you can detect easily when you have poor telephone lists and substitute an alternative list if one is not performing well.

Therefore, the ability to create highly stratified samples with multiple selection criteria and multiple sample quotas is a significant strength of the telephone approach. Data collection may be enhanced for some question types and, as stated, the type of questions required is an important criterion for selecting a data collection method. Not all questions are best suited to the telephone, but when the study requires probing and open-ended questions, the telephone may be the method of choice.

Please email a friend about this blog. Thanks!

Download a Free Report with more details on Data Collection  at: www.atheath.com/MRRC

	Join Our Email List

Yesterday I had the privilege of attending the IDC Directions 2010 full day conference, which I believe is the longest running conference in the IT industry. Approximately 1,000 attendees were present at the Sheraton Boston Hotel. There were four major sessions in the morning, special interest groups during lunch, three sessions each with 8 Tracks in the afternoon and a closing CIO panel discussion, which alone was worth the trip.

International Data Corporation (IDC) continues to provide leadership in several areas of market research for the IT industry. John Gantz in his usual witty and insightful manner set the tone in the morning with a talk that several other presenters referenced throughout the day.

In addition to the market sizing and forecasting data that IDC is well known for by customers and competitors alike, there were numerous sessions where customer data was presented helping to provide new insights and give direction for capitalizing on the economic recovery, which was the theme of the Directions conference this year.

I am not impressed easily, but yesterday I was impressed by the caliber of the IDC team. My only regret was that I could only attend a fraction of all the sessions offered.

One important take away is that while prospects for the future may not look wonderful today, do not let appearances fool you. Do not sit on the sidelines. The pendulum is swinging! There are signs around the global that indicate strongly the world is poised to build a smarter economy than we have ever imagined before. The warning is to move ahead and invest now or risk being left behind!

NOTE: This theme did not spring from idle hopefulness or marketing hype, the IDC team backed it up with research and analysis!

Please send out a tweet about this blog. And, sign up to be on our email list. Thanks!

	Join Our Email List

The questionnaire or research instrument (RI) is the tool that ultimately uncovers what you want to know. It is the core of the data collection process and is critical to achieving the results you need. There is much more to writing a research instrument than asking questions (See Questionnaire Design for Business Research, 2010, Tate Publishing).

It is best to write it in a way that is dynamic and engages the potential respondent. The RI design will not only impact the quality of the data collected, but factors such as the response rate (i.e., the percentage of completed interviews) and the analytic options available to the research team.

The RI must ask questions that are interactive and drill down to secure information from which to derive insights and develop action plans. However, there is a proverbial “fine line” between getting valuable information and asking too much; overly long questionnaires can be counter productive. Probing and clarification techniques, coupled with questions that allow for best guess answers, and trade-off analysis are but a few of the techniques you can use to ensure that you get the results you need to make very specific strategic and tactical business decisions.

Please tell your friends on twitter about this blog. Thanks! http://researchplaybook.wordpress.com

	Join Our Email List

It appears that Market Forecasts are on everyone’s mind these days. A sample of over N=250 purchasers of market research put Market Forecasts on the top of their shopping list this year. Interestingly, customer satisfaction ratings for Market Forecasts were not achieving the same level of enthusiasm as the demand for forecasts. This misalignment appears to be clear market opportunity.

Those of you who have been in the business of sizing and/or forecasting markets or purchasing this type of analysis know there are numerous trap doors you can fall through. As Keynes once said, “We do not know what the future will bring, except that it will be different from any future we could predict” (John Maynard Keynes)

The problems related to executing a good quality sizing and forecasting project are complex and typically require a modeling expert who collaborates closely with a domain expert to develop a sound forecast. If only one or the other type of professional tries to develop a forecast (solo) the results are often less than satisfactory. But, I digress.

This story focuses on one set of the results from a recent study, The Market Research Customer and Prospect Study conducted 3^rd quarter 2009. The intention of the study was to shed light on the direction customers want to see market research firms take now and in the coming year (2010). Evidence is strong that market forecasts are among the most desirable deliverables (Table 1).

Here is question that we asked:

Please rate the value you receive from each of the following types of deliverables. On a scale where 1 = Extremely Low Value and 7 = Extremely High Value.

Ironically, there is a lack of satisfaction with available forecasts and perhaps Box and Draper can help us understand why. They wrote, “Remember that all [forecasting] models are wrong; the practical question is how wrong do they have to be to not be useful?” (Box, George E. P.; Norman R. Draper, 1987, Empirical Model-Building and Response Surfaces. Wiley)

Twenty items were rated on satisfaction. The question asked was:

Please indicate your general level of satisfaction with products and services from MR firms you personally do business with on the following dimensions.  Please use the scale below, where 1 = Extremely Dissatisfied and 7 = Extremely Satisfied.

The item with the highest satisfaction rating was Quality of quantitative research with a mean of 5.7 and the overall satisfaction score was 5.5.  Market Forecasts received a score of 5.2 and   Syndicated services ranked last with a rating on 4.8.  Market research consumers are talking the question is, who is listening?

Please email this blog to a friend or colleague visit again soon we have more forecasting methods coming soon.  Thanks!

	Join Our Email List

Part of a series on: Approaches to Data Collection

The web has far-reaching capabilities that allow researchers to extend the coverage of their research. More geographic coverage (larger number of countries), typically at a lower cost, is an attractive combination. The lower cost and/or more coverage can help the research team to add country coverage in emerging markets or where it may be difficult and expensive to conduct telephone interviews due to the lack of native-speaking interviewers (although this may work in reverse in some countries).

Speed of data collection is often the reason a web-based approach is selected. However, the caveat is that best practices may still require several weeks (3-4) to provide time to do waves of emails to complete a random stratified sample.

For some studies the web is the most desirable approach. It is the single best way to conduct a conjoint study. It’s also the preferred method for collecting information when long lists of items or complex statements are part of the research instrument design (e.g., multiple response questions with either an “all that apply” or k of N question structure).

The k of N question structure is a ratio, where there are N items listed and k is the maximum number of items the respondent is allowed to select (e.g., “select up to four”). The k of N question type requires respondents to listen to more information than they are likely to remember. Human heuristics research shows that with lists of more than 5-7 items only the first and last items heard tend to be remembered, which could bias the results (Note: list rotation helps, but cannot overcome this problem).

Another benefit of a web-based study is it allows respondents to participate when they have time, and to start-stop-and-restart if necessary. It also allows for interesting incentive programs that can combine downloadable content with other forms of incentives. In fact, it may be highly desirable to use downloadable content as an incentive, rather than a cash-based approach to avoid attracting professional survey takers. Immediately available value-added content will attract business managers, IT professionals, and other professional groups, but will have little value to the survey takers looking for cash payouts with little or no interest in the subject matter.

Download a Free Report with details on Data Collection  at: www.atheath.com/MRRC

Please email a friend about this blog. Thanks!

	Join Our Email List