The Research Playbook

Posts Tagged ‘Questionnaire Design’

_________________________________________

Often when I recommend that a research team prepare a formal analysis plan the first response I hear is, “Why? The analysis isn’t due for weeks and I have too many other things to do.” Alternatively, I hear statements like, “That is too much extra work, I know what to do, I’ve done a lot of analysis work.”

An analysis plan is not extra work; it’s work that makes all the other project tasks flow efficiently. It will help you produce on-time project deliverables. Typically, you develop an analysis plan in parallel with your research instrument (RI). Like the RI the analysis plan is tied back to the goals and objectives of the study.

In addition to the obvious purpose of an analysis plan, producing a plan serves to improve the RI and manage project scope, these benefits alone will pay you for the time you devote to creating it.

The RI is referenced in an Analysis Plan (AP) and while there are no hard or fast rules and no one right way to structure an AP we can offer some guidelines. The approach presented here is as good as any and better than most.

The analysis plan approach described is specific to quantitative studies. The first step of the process will be familiar to those of you who read some of my other blog posts and publications.

Research has the greatest chance of success when the objectives are clearly stated and that is where we begin. Use these five (5) straightforward steps.

1. State the key study objectives clearly at the beginning of the analysis plan (AP) and refer to them throughout the process.

2. Describe the major comparisons for the analysis (e.g., major cross tabulations for the study such as: Customers versus Non-customers, Companies by size, Customers that are Satisfied, Neutral, or Dissatisfied).

3. State how each question is used to answer a specific objective of the study either on its own or in combination with other data points. Think through how you expect to present the results from each question. What statistics, if any, will you use in the analysis? Identify the independent and dependent variables.

4. Write a clear justification for including the information from the question in the study and perform a section by section “So what” litmus test.

5. When the analysis plan is finished, go back and make sure each key study objective has been addressed.

These five steps are the basic approach to the AP template. While it is straightforward it is not a trivial task. The key is to focus on objectives and think critically about how to execute on the primary goal of the study.

For a more detailed description of how to develop an Analysis Plan see Analysis Plans Made Easier, which is on the www.AtHeath.com Resource tab (scroll about halfway down the page).

For Email Newsletters you can trust

____________________________________________

Research Axiom: You can never fully recover from a poorly written questionnaire.

• No manipulation of the variables, regardless of how cleverly done
• No amount of analysis, regardless of how brilliant
• No degree of insightful interpretation, regardless of intellectual prowess

Nothing can save you from a poor research foundation. The building will collapse like a house of cards!

By Carey V. Azzara

If there is one part of the research process that I know, it is questionnaire design. It is a task repeatedly given insufficient time and attention. Clients and research professional alike often underestimate the time it will take to develop a truly well structured and concise instrument.

What amazes me most is when this task is somehow relegated to a status depicted by the attitude of:  “Once the questionnaire is done we can get on with the important stuff, like analysis and reporting.” The assumption that analysis work is the essence of the research and the expectation that interpreting the results is where the mastery of research ultimately lies is a mystery to me.

Have we not pounded the concept of garbage-in garbage-out into our heads? Can new internet tools substitute for critically thinking and the hard work of aligning the research instrument to the purpose of the study – answering the business questions that sponsors paid to learn?

If this seems like a bit of a rant, well I guess I’m guilty. My own research on research including the use of a 25 point questionnaire audit system has shown me that even well healed researchers are less diligent about quality than one would hope. Research is not only science it is a craft [perhaps an art] and if the proper fundamentals are not applied the product is less than artful.

I’ll end the ranting with an analogy [but don’t be surprised to hear more on this topic]. If you have not studied and then practiced writing poetry would you expect to publish a book of poems simply because your company’s marketing department asked you to?

Designing a good quality research instrument probably takes less talent than being a good poet, but it’s close.

For Email Newsletters you can trust

_________________________________

No, I am not going to offer you yet another webinar, podcast, free report, or video.

We overstocked hard copies of my Book:

Questionnaire Design for Business Research

Tate Publishing (2010)

So I am making this offer to my Research Playbook readers

Purchase the book Questionnaire Design for Business Research, and you will receive a paperback copy signed by the author (my wife loves this author!)

My Special Offer is a signed copy [tell me who to address it to] for $16.75 with free shipping in the USA.

Perfect for anyone serious about:

• Raising the bar on questionnaire design in his or her organization
• Finding a cost-effective way to start designing a questionnaire
• Preparing for the next market research project
• Improving his or her research skills

IMPORTANT: To reserve your signed copy of Questionnaire Design for Business Research you must: Email me carey.azzara@atheath.com or call 508 400 6837.

We have less than 100 copies and my signing hand will probably give out sooner LOL. . . 

“I want to get this book in your hands.“

It is also available from the publisher’s website: http://www.tatepublishing.com/bookstore/book.php?w=978-1-61566-835-9

_____________________________________________

The questionnaire Audit, Correct, and Enhance (ACE) approach starts with the audit. The Questionnaire Quality Control (QQC) audit identifies problems and errors commonly encountered during the task of questionnaire design.

My Questionnaire seems perfectly fine to me

No one wants to hear that what she or he created is less than masterful, but there is more to Questionnaire Design than asking questions the way you do in everyday discourse. Our audit is structured to be helpful and provide actionable guidance to the author(s) of the research instrument.

We have created a Do-It-Yourself (DIY) version of this audit process and we invite you to use this process to improve your questionnaires before submitting them for fielding. The process is described in this post and we hope you find it valuable. Read the rest of this entry »