"Our goal was to examine how survey design encourages respondents to provide their answers in a particular format, with the underlying goal of reducing the number of respondents who might be burdened with error messages if their answers were not provided in the correct format," says Leah M. Christian, researcher with the Survey Research Unit at the university's Carl Vinson Institute of Government and lead author of "Helping Respondents Get It Right The First Time: The Influence of Words, Symbols, and Graphics in Web Surveys" in the Spring 2007 issue of Public Opinion Quarterly.
Christian along with colleagues Don A. Dillman and Jolene D. Smyth at Washington State University set out to understand how a series of verbal and visual changes influence responses to questions in a Web survey. They wondered, for instance, how the size of the answer spaces or the presence and location of symbolic instructions would affect responses.
In order to accomplish this objective, the researchers designed three consecutive Web surveys in which they focused on a simple goal: to get responders to report date answers in a particular format—a common but challenging issue in U.S. surveys as dates can be reported in a variety of ways (e.g., October 5, 2005; 10/05/05; 10-5-05). Into these surveys, the researchers embedded a series of experimental manipulations designed to influence respondents to report dates using two digits for the month and four digits for the year.
"Asking for dates is common in Web surveys and other online interfaces, so we thought it was a good type of question with wide appeal to various people interested in Web design," says Christian. "Our funding agency, the National Science Foundation, was also interested in our findings as respondents to one of their Web surveys were getting frustrated when they received error messages for date answers in incorrect formats."
Christian and her colleagues discovered that by placing letters to indicate the number of digits to use—"MM" and "YYYY"—to the left of the answer spaces and by sizing the answer spaces consistent with the instructions, they were able to increase the percentage of respondents providing a two-digit month and a four-digit year from 55 percent to 96 percent.
"We concluded that survey designers can strategically use visual design to communicate expectations and instructions to respondents about how to respond," says Christian.
Next month, Christian, Dillman, and Smyth—who frequently collaborate on similar projects and are currently working on a book together on mail and Internet surveys—will be presenting their research at the annual conference for the American Association for Public Opinion Research. In this presentation, they will provide survey designers with a conceptual framework that integrates the past ten years of visual design research. But in the meantime, Christian says she is honored that she and her colleagues have had this and several other pieces published in Public Opinion Quarterly.
"The journal has great influence in the field of public opinion and survey research, so the work we have been conducting on visual design is able to reach a large and diverse audience. We have already seen other scholars replicate and extend some of the ideas and experiments that have been published in previous issues."
