The recent Big (D)esign conference in Dallas, which gathered many in the graphic design industry, website developers, and gamers, dedicated a track to usability. This track included a presentation by Ryan Smith from Qualtrics. Smith discussed how survey tools are evolving to provide a better user experience and match what’s happening in other areas where technology is setting the pace. Using examples from Qualtrics, he demonstrated how survey tools are allowing us to be:

• More interactive: Different questions types have emerged in an attempt provide a more interactive experience to respondents, engage them, increase response rate and response accuracy. Examples include:
  • Rating questions using sliders, images instead of numbers of text and gauges, among others

  

  • Constant sum questions using draggable bars.

  

  • Text Highlight and Heat Maps.

  

• More intelligent: Technology allows us now to do on-the-fly analysis to personalize the experience and integrate survey results with other data sources on a timely manner. We can use advanced skip logic to present relevant questions and capture session information to improve data quality (e.g. removing speeders).

• Better subject selection: Online surveys can be distributed in different ways offer better access to the target audience. Surveys can be deployed using pop-up, pop-under, pop-over, feedback links, embedded links, and link redirects. This allow us to go where the respondents are.

• More immediate: Organizations can quickly conduct surveys with internal resources (DIY research). Survey results can be monitored since the moment the survey is launched. Online reporting capabilities allow us to get updated results continuously. The extent to which we limit or extend result sharing is under our control.

There is no doubt that technology has made it possible to create more engaging surveys, clean the data on-the-fly, facilitate access to surveys, and provide quick results. However, before you jump on the wagon of the cool question types, consider these issues:

• Need: Determine which interactive features are necessary, which ones are more of a novelty, which ones do not help respondents directly to complete the survey, and which ones increase respondents’ burden.

• Culture: When introducing graphic elements like images (e.g. smilies) or graphic representations of objects (thermometers, gauges, etc.), consider who your target audience is and what cultural barriers may influence reactions to these questions. This is especially true in international studies.

• Expectations: People are used to traditional question types, so it may be unclear what it is expected from them. Clear instructions about the scale meaning and actions the respondent need to take are recommended. This may not be a problem, once these question formats become the norm, but for now, we need to ensure respondents know how to interact with the questions.

• Impact on completion rates and data quality. The time needed to get familiar with the new question format, process the instructions (even if it takes a few seconds) and question complexity may affect completion rates and data quality. Research on this is a mixed bag, so to be on the safe side, test the questions with a few members of the target audience before a full launch.

• Reaching people with disabilities: The new question types are often not 508 compliant, making it difficult for people with disabilities to participate in surveys. If you need to reach respondents that may have disabilities, stick to the traditional formats.

Almost every day I find out about a new survey tool. As we all know, the advent of online survey tools has lowered the cost of data collection and made it easier for many internal market research departments on the client side to deploy their own online surveys.

Reduced or non-existent market research budgets continue to be the norm, and since the first publication of this post (3/30/2010) clients are still looking for free survey online tools or very cheap solutions to get the job done. When I ask what tools they use, the most common answer is still SurveyMonkey or Zoomerang.

I used SurveyMonkey a while back when I was on the client side with no budget, but it had so many limitations, when I wanted to do more advanced surveys, that I had to search for another tool. I tried Zoomerang, but I found it to be expensive given the limited features it offered. Besides, I found the interface wasn’t very intuitive. At another company, where I led the research team, the budget allowed me to use VantagePoint, which I then abandoned for Confirmit. The latter is a very advanced tool, but also quite expensive. So my quest continued and then a couple of years ago I stumbled upon SurveyGizmo, which really surprised me with a long list of features I wanted in an online survey tool at a very low cost.

Lately, I have had the chance to use Qualtrics, on behalf of several clients. As I have said below in response to some comments, Qualtrics is a terrific tool. It has tons of features, but it can only be used for free for one project. After that, you need to buy the license, which starts at a higher price point than SurveyGizmo’s equivalent option, the Enterprise edition.

There are many online survey tools that offer a free limited version so people can test the tool and have a feel for it. However, not all free versions are equal. Some give you more than others and you get the chance to really experience how useful the tool can be. This is the case of SurveyGizmo vs. SurveyMonkey. Even at the lowest subscription plan ($19/mo), SurveyGizmo offers many more features than SurveyMonkey.

Since people still ask me what online survey tools they can use for free, I decided to share the free online tool review I made of the three online survey tools that seem to be most popular, on a set of features that would allow me to create from simple to slightly more advanced surveys. By popular demand, I have reviewed the first comparison I did, published on 3/30/2010, and found a few changes and improvements, but nothing substantial.

General Overview
Number of surveys	Unlimited	Unlimited	Unlimited
Number of questions	Unlimited	10	12
Number of responses	250	100	100
Multi-page surveys
Account users	1	1	1
Question types	23	15	15
Graphic survey templates	37	15	21
Real-time reporting
Data exporting
Never delete data
Supports all languages
Price	Free	Free	Free
Question Types
Textbox – Short text
Textbox – Numeric Format Validation
Textbox – Email Format Validation
Essay – Long text
Multiple text fields
Table of text fields
Checkbox
Table of checkboxes
Radio button
Table of radio buttons
Drop-down menu
Tables of dropdown menus
Ranking – Table format
Ranking – Drag & Drop
Ranking – Top X# of items
Ranking – Stars
Constant sum
Continuous sum
Instructions/Text
Image
Video
Contact information form
Date format
Survey Formatting
Customizable font family
Customizable font size
Customizable font color
Customizable background colors
Add/Edit Options in Bulk
Add an open textbox so users can add to their own choice
Change orientation of labels and options
Change position of labels
Editable Next, Submit, and Close buttons text
Optional question numbering
Fully Customizable XHTML/CSS templates
Auto-format as currency, percentage, or a whole number
Editable general error message
Change length, height, and width of textbox or table
Editable survey width
Editable survey after deployment
Report Creation Options
Real-time online report
Graphing – Use a bar graph, pie chart or spreadsheet to view answers
Export Data to Excel, CSV
Browse All Responses Individually
Can change titles and internal report values
Clear all responses
Survey Deployment
Send email using survey tool
Password protection
Set a cutoff date and time
Set maximum response count
Enable IP blocking
Redirect browser
Link to panel providers
Incentive Management tool
Other Survey Options
Make questions required
Randomize the answers to your question
Randomize columns
Reorder Questions
Set how many checkboxes someone can check
Automatically put in a value already collected into the answer field
Upload images
Embed external images and media
Min and Max options for numerical answers
Prevent search engines from indexing survey
Thank You page
Add a “Close” button to Thank You page
Back Button Optional
Progress Bar Optional
Add a unique survey identifier to each link
JavaScript option embed surveys using JavaScript
Embed surveys using an iFrame
Embed surveys into a Webpage
Create hidden variables
Compliance with Section 508 guidelines of the Rehabilitation Act
Export survey to PDF
Export survey to Word
Manual data entry of additional records
Polls
Poll templates	37 + Custom		8 + Custom
Poll Types	Single choice, Image Choice, Ranking		Single choice, Multiple Choice
Poll result sharing
Vote protection	By IP address, by cookies, or none		On/Off
Poll Widget sizing
3rd Party Integration
Open API access
Facebook integration
Twitter integration
MailChimp integration
Affiliate program
Support
Support via email
Support – FAQ, Tutorials

Last update 6/28/2011

There are many other tools that offer a free version, but I found them ever more limited that those shown here. As you can see SurveyGizmo has many more useful features in its free version, although a few are missing. Survey deployment capabilities continue to be a weak area for Survey Gizmo, where SurveyMonkey does better. For example, in SurveyGizmo’s free version I miss the ability to send survey invitations within the tool (you can do it with a paid subscription), but that can be fixed using MailChimp or any other email distribution system. If you use an external system, be aware that in order to send reminders, you have to download the data to check who answered the survey and then extract those from the reminder list. It is a bit of work, but doable. Neither SurveyMonkey or Zoomerang let you download the data without upgrading to a paid subscription.

In this updated comparison, I added Polls, which both SurveyGizmo and Zoomerang offer as a separate survey type, given some specific features they have (voting, sharing, one-question). I find both pretty comparable.

As a professional market researcher I usually need very robust tools, so free online survey tools usually fall short on providing features needed to develop complex surveys. Notice that none of these 3 free versions offer skip logic, but if you only need to deploy linear surveys without having to skip questions or reroute respondents, and don’t want or can’t spent a dime, I recommend you give SurveyGizmo a try. If you have some change to spare, I recommend you use one of SurveyGizmo paid subscription plans, which don’t require a long term contract obligation and have many more features at a very low price.

To learn more about our consumer data service visit Consumer Shopping Behavior Insights. To request consumer shopping behavior data and insights don’t hesitate to contact us.