Attitudinal questions are common in surveys. They are often asked using an agree-disagree rating question format. The challenge is always to create statements that capture important elements of the attitudes we are trying to measure. Ideally, if the budget allows it, we should do qualitative research to gather insights into such elements and how people think and talk about them.

Even with qualitative data available, writing good attitudinal statements is not an easy task. Here are some guidelines to facilitate the process:

• Include only relevant items for the attitudes that are being measured: Adding irrelevant items or missing important ones is detrimental to the quality of the analysis.
• Write statements in present tense and avoid referencing to the past: Past tense relies on memory plus we will never know if the respondent is referring to yesterday, a week ago of ten years ago.
• Avoid giving to much information about facts or elements that can be interpreted as tales.
• Avoid ambiguity.
• Create statements that express in-favor or against opinions related to what it is being measured. Think, “I go to work every day” vs. “I love to work every day.” Do not use items that would describe different points in a continuum (“Sometimes I like to work”), as this can be confusing. If you need to do that, it means you would be better off converting the item to a separate rating question with a specific scale.
• Use direct and simple language.
•  Avoid technical jargon and use words respondents can understand.
• Make the statements short to facilitate comprehension and minimize fatigue. In long sentences, people tend to skip parts to get to the end faster, which can lead to misunderstandings. Avoid using more than one sentence.
• Use only one logical phrase per statement. Having more than one creates confusion as to what it is being evaluated.
• Avoid words like “always,” “everybody,” “nobody”. Gross generalizations are not credible. Respondents are likely to skip answering such statements or assume an artificial extreme position.
• Avoid negations and double-negations. They can lead to misinterpretations.
• Balance the number of positive and negative statements. Don’t make them all positive or negatives, which can mislead respondents in one direction or the other.

Often when I work with a client in developing a questionnaire, I get asked whether we should include a neutral point in rating questions (e.g. Very satisfied, Satisfied, Neither Satisfied or Dissatisfied, Very Dissatisfied).  A lot of research has been conducted in this realm, particularly by psychologists concerned with scale development, but not definitive answer has been found and the debate still continues. Some studies find support for excluding it while others for including it depending on the subject, audience and type of question.

Those against a neutral point argue that by including it, we give respondents an easy way out to avoid taking a position on a particular issue.  There is also the argument that equates including a neutral point to wasting research dollars, since this information would not be of much value or at worse it would distort the results. This camp advocates for avoiding the use of a neutral point and forcing respondents to tell us on which side of the issue they are.

However, we as consumers make decisions all day along and many times we find ourselves idling in neutral. A neutral point can reflect any of these scenarios:

1. We feel ambivalent about the issue and could go either way
2. We don’t have an opinion about the issue due to lack of knowledge or experience
3. We  never developed an opinion about the issue  because we find it irrelevant
4. We don’t want to give our real opinion if it is not considered socially desirable
5. We don’t remember a particular experience related to the issue that is being rated

By forcing respondents to take a stand when they don’t have a formed opinion about something, we introduce measurement error in the data since we are not capturing a plausible psychological scenario in which respondents may find themselves. If the goal of the question is to understand the variation in opinion, we should not only use a neutral point but also a “Not sure/Don’t Know/Not Applicable” option. This would allow respondents in scenarios 2 and 3 to provide an answer that is true to their experience.

For example the other day I got a customer satisfaction survey from Blackberry after a call I made to their support desk. The survey had a question in which I was asked to rate the representative, who took my call, on different aspects.  One of them was “Timely Updates: Regular status updates were provided regarding your service request.” I wouldn’t know how to answer this, since the issue I called for didn’t required regular updates. Luckily, they had a “Not applicable” option, otherwise I would have been forced to lie, and one side of the scale would be as good as the other.

An increase in non-responses and survey abandonment can also result from respondents who don’t want to air their opinion because of perceptions of low social desirability. If they are given the “Not sure/Don’t Know/Not Applicable,” they are more likely to use it than the neutral point. This would be preferable since they could be excluded from the analysis for a particular question, but information on other questions would not be lost. A better alternative yet is to provide a “Prefer not to answer” option if the question touches particularly sensitive issues.

Finally, the best antidote against having respondents gravitating towards the neutral point is to make sure that we show the questions to those who can really answer them. With the help of skip logic, we can design surveys that filter out respondents with no experience, knowledge or interest in the subject being rated. In my Blackberry example, they could have asked me first if my request needed regular updates, and if that was the case then ask me to rate my satisfaction with it. Most likely, the researcher that designed the Blackberry survey was trying to make the survey shorter, but I still could have introduced measurement error, if I hadn’t seen the “Not Applicable” option at the end of the scale, which I almost didn’t notice at first.

You may have guessed by now in which camp I am. Survey questions should be as close as possible to the way respondents would naturally answer them in real life. Sometimes we need to get there in several steps by filtering out those who can’t answer, but sometimes we just have to give them the option to be neutral.

To learn more about our consumer data service visit Consumer Shopping Behavior Insights. To request consumer shopping behavior data and insights don’t hesitate to contact us.