Last Friday marked my 15th anniversary with MarketStar. Telling people I have been here longer than the furniture is NOT an understatement! As my teenagers, who were 2 and 5 on Sept., 16, 1996, are quick to point out, a lot has changed since then. “How did you survive without a cell phone?” “What is a cassette tape?” “What is an encyclopedia?” “You couldn’t use the Internet to do your homework?” The list goes on and on.

A lot has changed in market research data collection as well – from mail and telephone being basically the only survey options, to online surveys, surveys sent to mobile phones, Facebook surveys, panels, bulletin boards, etc.   But many of the research pitfalls have remained the same. For example, I learned in my first survey design class that if you write misleading, unclear or biased questions on a research survey, you will get misleading, biased results.

Biased questions are those that encourage survey participants to respond in a certain way. One way this can happen is if they contain biased terminology or are worded in a biased way.

• Biased question (simple example): Don’t you agree that the new policy is too restrictive?
• Revised question: Do you agree or disagree that the new policy is too restrictive?

Another way bias occurs is when questions lead a participant to agree or respond in a certain way.

• Biased question (simple example): You don’t like this brand of printers, do you?
• Revised question: Do you like this brand of printers?

A type of bias that is fairly easy to unintentionally introduce into a survey is what researchers often call a double-barreled question. This happens when one question actually asks for two responses.

• Biased question: Please tell me whether you would vote for or against a candidate who supports reducing federal spending on education and welfare?

The question assumes respondents logically group education and welfare together, but how would someone respond who was against cutting spending on education, but in favor of spending cuts on welfare? The best option is usually to split double-barreled questions into two separate questions. If that makes the survey too long, the higher priority question should be kept.

A less obvious way survey questions can be biased is if the answer choices are not balanced. For example, the question below has three positive answer choices and only one negative.

How satisfied are you with our product?

A. Extremely satisfied
B. Very satisfied
C. Satisfied
D. Somewhat dissatisfied

Unfortunately, it is pretty easy to innocently write a question that is biased if you aren’t experienced in survey design.  So, if an experienced market research firm is not conducting the survey, it is wise to have an experienced researcher give feedback on questions before data collection begins.

Response bias also can be intentional if the company sponsoring the research is looking for particular answers to help ensure the survey findings are favorable. This may give the company the data it wants or hopes to see. However, if accurate, unbiased data from potential customers such as why they would, or would not buy your product is not collected or is ignored, the chances of success are slim.