The Research Playbook

Archive for the ‘Data Collection’ Category

In our zeal to provide clean datasets by removing questionable cases, we can commit another research “sin” – the introduction of researcher bias.

More than once, I have witnessed a researcher going through a dataset case by case to try to determine if a respondent is a gamer or simply unqualified. The person started out with some basic rules or criteria, but in addition made decisions on respondents’ qualifications from a subjective position, arguing, “No one with this [attribute here] would frame answers the way this respondent did.”

Maybe the researcher was correct and the person was not a legitimate respondent, but this is a slippery slop and we should not take the task of deleting cases lightly. You should use both valid and repeatable criteria when you delete cases from a dataset.

If you are not careful you can start sliding down the slippery slope of researcher bias and not recognize it until it’s too late!

 

 

Please share your thoughts on this topic – leave a comment.

 

______________________________________

 

Sales cycle analysis is a powerful tool to help you better understand your markets. Gaining a strong appreciation for the good and bad news about your market positioning in relationship to your competition isn’t always easy to do. Moreover, if there’s bad news it’s difficult to hear. However, not knowing where you stand is, at best, dangerous and could prove fatal.

Understanding why you do or don’t get on the short list of your prospects (or stay on the short list of your customers) is critical and sales cycle analysis helps to answer this very important question.  You need to know what will facilitate or impede your progress toward becoming a preferred supplier – the position we obviously all want. If knowledge is power than sale cycle knowledge is supremacy.

Clearly, there is more than one approach to achieving insights related to sales cycle dynamics and how customers and prospects perceive a business. We won’t try to explore the options here.

However, it is worth noting that a commitment to exploring these dynamics is not a one shot deal. If you and your company are serious about sales and the factors that propel your sales, you will be well served by tracking the metrics required at least annually.

We all know markets continue to evolve quickly. A very good way to stay informed is to track market activity systematically. Creating a baseline of information and measuring against that is a great starting point and an essential part of sales cycle analysis.

You can structure sales cycle studies to help maximize your reach tactics. Knowing how to best reach your audience is a function of understanding how they search for information. More precisely it is about how customers and prospects search for information at each stage of the buying process. In addition, the new reach equation includes social networking and social media, again fast moving targets.

In addition, studies on sales cycles, almost by definition, provide competitive insights. It’s not enough to know if you’re on the short list you need to know who’s on it with you.

Combining information on brand and product positioning with a continually updated view of reach dynamics is a powerful tool in the hands of a savvy marketing professional. What are you waiting for? Get started!

 

Join Our Mailing List

Email:

For Email Newsletters you can trust

 

_________________________________________

Often when I recommend that a research team prepare a formal analysis plan the first response I hear is, “Why? The analysis isn’t due for weeks and I have too many other things to do.” Alternatively, I hear statements like, “That is too much extra work, I know what to do, I’ve done a lot of analysis work.”

An analysis plan is not extra work; it’s work that makes all the other project tasks flow efficiently. It will help you produce on-time project deliverables. Typically, you develop an analysis plan in parallel with your research instrument (RI). Like the RI the analysis plan is tied back to the goals and objectives of the study.

In addition to the obvious purpose of an analysis plan, producing a plan serves to improve the RI and manage project scope, these benefits alone will pay you for the time you devote to creating it.

The RI is referenced in an Analysis Plan (AP) and while there are no hard or fast rules and no one right way to structure an AP we can offer some guidelines. The approach presented here is as good as any and better than most.

The analysis plan approach described is specific to quantitative studies. The first step of the process will be familiar to those of you who read some of my other blog posts and publications.

Research has the greatest chance of success when the objectives are clearly stated and that is where we begin. Use these five (5) straightforward steps.

1. State the key study objectives clearly at the beginning of the analysis plan (AP) and refer to them throughout the process.

2. Describe the major comparisons for the analysis (e.g., major cross tabulations for the study such as: Customers versus Non-customers, Companies by size, Customers that are Satisfied, Neutral, or Dissatisfied).

3. State how each question is used to answer a specific objective of the study either on its own or in combination with other data points. Think through how you expect to present the results from each question. What statistics, if any, will you use in the analysis? Identify the independent and dependent variables.

4. Write a clear justification for including the information from the question in the study and perform a section by section “So what” litmus test.

5. When the analysis plan is finished, go back and make sure each key study objective has been addressed.

These five steps are the basic approach to the AP template. While it is straightforward it is not a trivial task. The key is to focus on objectives and think critically about how to execute on the primary goal of the study.

For a more detailed description of how to develop an Analysis Plan see Analysis Plans Made Easier, which is on the www.AtHeath.com Resource tab (scroll about halfway down the page).

 

Join Our Mailing List

Email:

For Email Newsletters you can trust

 

_______________________________________

Apparently many people do care about how panel recruitment affects sample development and with good reason.

Results from a poll posted on LinkedIn demonstrate that this is a serious issue. The question posed was:
“When you select a sample provider, how much importance do you place on their panel recruitment process?” Is it:

While the results derived are from a sample of convenience and hardly scientific, they are nevertheless instructional. So what did the 98 people who took time to participate tell us?

If you are a panel provider and you didn’t already know that recruitment practices play a deciding factor, it surely would be obvious now, with nearly seven out of ten (71%) prospective buyers voting “very and extremely” important.

 

Women are more likely to view recruitment as very or extremely important (72%) than men (64%) are. Differences by age were interesting. There was a very small portion of the sample (5%) in the 30-36 age group and they accounted for only 2% one of participants who voted “extremely important.” The 45+ age cohort, which was the largest age group in the sample, also had the highest proportion of votes in the extremely important and very important categories (68%). Perhaps it’s true that with age comes wisdom!

What can we learn from examining the results of this poll? I think the message is straightforward;  overall this simple polling question seems to have hit a nerve. We believe that sample development is one of the cornerstones of good research.

Market research is not an academic exercise. Real business decisions are made, or at least influenced, by the results of the research we conduct – how can you make a good business decision if the sample is faulty? Simply put, you can’t.

 

Join Our Mailing List

Email:

For Email Newsletters you can trust

 

_________________________________

No, I am not going to offer you yet another webinar, podcast, free report, or video.

We overstocked hard copies of my Book:

Questionnaire Design for Business Research

Tate Publishing (2010)

So I am making this offer to my Research Playbook readers

Purchase the book Questionnaire Design for Business Research, and you will receive a paperback copy signed by the author (my wife loves this author!)

 

My Special Offer is a signed copy [tell me who to address it to] for $16.75 with free shipping in the USA.

 

Perfect for anyone serious about:

• Raising the bar on questionnaire design in his or her organization
• Finding a cost-effective way to start designing a questionnaire
• Preparing for the next market research project
• Improving his or her research skills

IMPORTANT: To reserve your signed copy of Questionnaire Design for Business Research you must: Email me carey.azzara@atheath.com or call 508 400 6837.

 

We have less than 100 copies and my signing hand will probably give out sooner LOL. . . 

“I want to get this book in your hands.“

It is also available from the publisher’s website: http://www.tatepublishing.com/bookstore/book.php?w=978-1-61566-835-9

 

_____________________________________________

 

The questionnaire Audit, Correct, and Enhance (ACE) approach starts with the audit. The Questionnaire Quality Control (QQC) audit identifies problems and errors commonly encountered during the task of questionnaire design.

My Questionnaire seems perfectly fine to me

No one wants to hear that what she or he created is less than masterful, but there is more to Questionnaire Design than asking questions the way you do in everyday discourse. Our audit is structured to be helpful and provide actionable guidance to the author(s) of the research instrument.

We have created a Do-It-Yourself (DIY) version of this audit process and we invite you to use this process to improve your questionnaires before submitting them for fielding. The process is described in this post and we hope you find it valuable. Read the rest of this entry »

______________________________

If Sherlock Holmes were here today he’d say:

“If you’re not concerned about your social media presence you should be.” “But Holmes – why should I be concerned, does it really matter?”

“It’s elementary Watson even a “side-kick” like you must have an online presence to be successful in the marketplace today!”

If Sherlock Holmes were here today talking about social media he would have gone on to say:

“The truly important question is what does your presence or online footprint look like?” If you cannot answer this question with precision, Holmes’ deduction would be – it’s probably time to start examine your positioning.

Once you know the status of your online presence, you can begin making improvements. The question is “How do you find out about or assess your online presence?” The answer is there is no one exclusive way to proceed. There are a number of steps you can take and depending on your goals; some will be more relevant or important than others.

Here is a partial road map that might help you

Let’s call it a Web Presence Assessment (WPA), which is not an entirely new concept. It is an extension of a practice typically referred to as an audit or review and usually applied to a website. However, in this case it includes much more.

Think of it as your Ultimate Online review, which includes an assessment of your

• Website from several vantage points
• Blog(s)
• Social media marketing assets
• Content marketing approach
• Level of overall optimization [other than website]
• The development of an action plan
• And, more…

OK now let’s be a bit more specific.

Review of Your Website from Several Vantage Points

We suggest you use several approaches and software services, here are a few:

• Do a simple Google search [but don’t forget Bing] to find Rankings
• Use Yahoo’s site: URL search to explore back links
• HubSpot has a website grader and it’s free to use.
• Market Samuri (a paid service) offers valuable information on back links and several other metrics

Review your Blog(s)

The actions to review a blog are similar to those used for a website. However, you should also examine the platform and hosting you use – not all hosting is created equal!

Blog style and voice are very personal, nevertheless, it is important to be deliberate in the style you use with a particular audience and worth the time to access the fit.

Social Media Marketing Audit

Next, take an inventory of the social media assets you and your company have created. We developed a 5-10 minute assessment tools, but you can begin by simply listing what you have and where else you might invest your time.

This is an easy step and very helpful in identifying what you have and what you don’t have. Explore where your company is operating along the very large spectrum of social media activities.

Once this assessment is complete it is much easier to discuss the steps you should consider taking to improve your online presence through social media networks and how to integrate your current and future assets to maximize the marketing power they can generate.

OK this post is getting a bit long so I am going to stop here, but two other areas worth pursuing are: performing a Content Marketing Assessment and examining your Level of Overall Optimization.

Your Web Presence Assessment should be conducted as a systematic process. This will require using a set of steps, much of which you can probably accomplish as DIY approach.

However, it may require some outside help. As you begin conducting your assessment, some of you will recognize how complex this can be. In addition, some assessments require the expertise of multiple people. If you need help or simply don’t have the time to do the work hire a professional – you don’t have to go it alone – it’s OK to ask for help!

x

One of the more powerful features of LinkedIn is the Answer function. Login to your LinkedIn account and go to “More” on the main menu. The option in the dropdown is Answer, click on it and you are now in a place where questions find answers. There is nothing magical about it, this is simply a rich user-created content area that you can leverage in several ways.

Ask Questions
First, if you have a question, answers are only a few key strokes away. Simply type your question in the space provided, then sit back and wait. Of course, it would be best to take your time and construct a well thought out question, then perhaps have it reviewed by a colleague for clarity. Being thoughtful and constructing a clear question will pay dividends in the form of quality answers as well as helping you to project the best image (I’ve read some questions that were poorly written they do not play well).

Before you submit your question, it could be helpful to search for similar questions that have already been submitted. This step can help you refine your question. It could also give you the answer you are seeking without waiting for people to respond to your new question. In fact, you may find that several similar questions have been asked [include the closed questions when you search]. Answers to these questions may provide the information you want.

Use the advanced search function to find information on specific topics. You will find this search function to right of the main menu. You may type keywords here or you can click on the advanced feature and use the additional selections offered.

 

Once you have found questions and answers on your topic, exploring the information may require a bit of content analysis. Using a word processor to search one or more keywords or key phases is typically helpful. With practice you can become very good at picking out the themes that emerge across the answers and the themes from across similar questions.

Answer Questions
Another aspect of the LinkedIn Answer feature is giving answers. If you would like people to recognize you as an expert here is a perfect venue for establishing your expertise. If your answers are among the best provided in the opinion of the person asking the question, you may be recognized and achieve a “Best Answer” status.

Many people spend a good deal of time working toward that goal. Note the “This Week’s Top Experts” list on the Answer page. However, you don’t have to be top ranked as a supplier of “best answers” to gain true value from this feature.

Selectively answering questions that have attracted your target audience and adding an appropriate link as part of your answer can be a powerful tool in the quest for finding relevant connections on LinkedIn. The targeted nature of this question-answer arena is a perfect mechanism for finding people with a need you can fill.

Conclusion
While the Answer function is not a replacement for engaging on discussion groups, it is a good next step for those of you who want to find answers or people who have specific questions you can answer. Either way this is an innovative function well worth a portion of your overall LinkedIn time allocation.

We typically think of social media sites such as LinkedIn as places for networking and sharing ideas, which is certainly their primary function. Over time, however, sites like LinkedIn (and to varying degrees other networking sites) have developed an enormous global reach. The sheer size of LinkedIn’s membership has resulted in the creation of groups on nearly every topic imaginable and discussions within these groups touching on thousands and thousands of subjects.

This “user” created content has propelled networking sites and LinkedIn in particular into a realm, which transcends networking. A listening opportunity for market researchers has emerged and not just for passive listening. The opportunity to conduct active listening (a.k.a. market research studies) is possible. In fact, it is already a reality, available to anyone with the skill to extract the information.

Here are three ways to access marketing information from LinkedIn:

Discussion Groups

Discussion groups offer a wealth of content. While you may rush to start your own discussions to capture content, you might find that the information you seek is already out there. Finding the group(s) and discussions with the content you are looking for can take some time and skill. However, if you find relevant content much of the work has been done for you (assuming the topic generated a large number of comments). The trick now is to synthesize the information and pull out the themes, trends, and other pearls of wisdom.

Of course, you can start your own discussion threads within a group. The advantage is you can design the specific question or topic you want to explore. In addition, as the author of the discussion you have a little more latitude to steer the discussion, allowing you to simulate an asynchronous focus group.

Answers

LinkedIn has additional functionality such as “Answers” which allows a user to pose questions and offer them up to the LinkedIn membership (Go to the main menu click on More and select Answers). The person asking the question can grade the answers and select a “best answer.” This is the motivation for taking time to participate.

Again, you don’t necessarily need to ask a question to glean value from this function. You may find that someone, and in some cases a number of people, have already asked a question that is close enough to the question you want answered. Once again, all you have to do is synthesize the content. Well that is the hard part, but the data are often there awaiting your analysis.

Polls

The Poll function also offers a simple but potentially powerful mechanism for collecting data on an individual question. Used creatively Polls can provide market intelligence, albeit with a very limited scope. You can search existing Polls or write and launch your own. Either way there are answers to questions waiting to be discovered. For no cost or low cost you can reap a great deal of market intelligence from LinkedIn using Polls and other functions either individually or in combination with one another.

Conclusion

These and other avenues for exploring the marketplace, listening to your target audience, and performing both qualitative and quantitative market research are available to anyone who wishes to use them.

Need help with questionnaire design ? Go to http://questionnairedesign.tatepublishing.net/
 

Panels are not all created equal. Three of the most important questions you need to ask about the panels you use include (but are certainly not limited to):

How were panelist recruited?
Recruitment is typically conducted online in spite of the claims companies may make. However, the best panels use some form of verification (e.g., telephone interviews, IP address tracking, and always a double opt-in with CAPTCHA, which stands for Completely Automated Public Turing Test To Tell Computers and Humans Apart). These confirmation methods are actively managed and panelists’ profiles are updated using a systematic and scheduled process.

Are panels more than a database of emails?
All panels are housed in some form of database. However, the sophistication of the database structures and capabilities to manage the panel vary. The best of the panel management practices track panelist activity across studies and include a process for updating panelist profiles. The management process should be capable of de-duplication of panel members selected for a study when panel providers use partners (many panelists are registered with more than one panel provider). Without a de-duplication process the same person can respond twice [or more] without your knowledge.

Some panel providers use a community approach to supplying sample. The profiles and refresh cycles of names and emails is likely to vary from community to community, which presents a challenge for the providers and for you as a researcher. The Community Model is gaining momentum and reasonably sophisticated software is available to handle the duplication problem. However, the issue of how representative the communities are of a defined market remains a serious concern.

If your goal is to better understand a specific community this issue may be mitigated or not exist. If on the other hand you want to generalize beyond the community, it would be best to know the demographic parameters of the community and the target markets to which you want to generalize. Making comparisons to explore whether significant differences exist is an important part of sample quality. If there are significant differences, you may need to be cautious and perhaps consider a weighting approach to compensate for differences.  

Does the Panel Provider Profile?
The idea of profiling has become associated with terrorism and the geo-political arguments for and against practicing profiling. We are not talking about that form of profiling. Panel providers’ efforts to profile panelists have two major objectives:

1. Send panelist to complete questionnaires they are qualified to take
2. Pre-qualify panelists to improve the incidence rates for clients

These are clearly complementary efforts. Profiling is used to understand better each panelist. The more you know about a panelist or more precisely the panel as a whole the better you can target a specific group of panelists to meet the objectives of a project.

Invitations to complete a questionnaire are sent to panelists who “appear” to qualify. Screening questions are then used to identify and qualify respondents with greater specificity. However, screening questions are not a panacea, but that’s a topic for another time.