Eric M. Mindich Encounters with Authors Symposium
The program offers scholars an extended, intensive seminar with the authors of new, path breaking scholarly works. Professor Jon Krosnick of Stanford University will conduct meetings that feature a mix of lecture and discussion components. Unless otherwise noted, sessions will take place at 1737 Cambridge Street, Room N354 (map and directions).
Since the beginning of quantitative social science, a great deal of
research has been done using questionnaires, asking people to provide reports
describing their mental states, their personality dispositions, their
behavioral tendencies, their attitudes and beliefs, and much more. Thus,
asking questions and interpreting answers are core activities for social
science. It is therefore no surprise that most research methods
textbooks in psychology, sociology, political science, and many other
disciplines include a discussion of questionnaire design.
Remarkably, the structuring, wording, and ordering of questions has
traditionally been viewed as "an art, not a science", in the words of
Princeton University psychologist Hadley Cantril (1951, p. vii) over five
decades ago. And in his book, The Art of Asking Questions, Stanley Payne
(1951) cautioned that "the reader will be disappointed if he expects to find
here a set of definite rules or explicit directions. The art of asking
questions is not likely ever to be reduced to some easy formulas (p. xi)."
Thirty years later, Sudman and Bradburn (1982) agreed, saying that "no
codified rules for question asking exist (p. 2)." Sampling and data analysis
are indeed guided by such rules that are backed by elaborate theoretical
rationales. But questionnaire design has been thought of as best guided
by intuition about how to script a naturally flowing conversation between a
researcher and a respondent, even if that conversation is sometimes mediated
by an interviewer. Experienced questionnaire designers have followed
some conventions over the years, but those conventions varied enough from
individual to individual and from discipline to discipline to suggest there
are few universally-accepted principles. If a questioning approach seems
to work smoothly when respondents answer a questionnaire, then many
researchers presumed it would probably yield sufficiently useful data.
In recent years, it has become clear, though, that this is an
antiquated view that does not reflect the accumulation of knowledge throughout
the social sciences about effective question-asking. To be sure,
intuition is a useful guide for designing questions, and a good questionnaire
yields conversations that feel natural and comfortable to respondents.
However, intuition can sometimes lead us astray, so it is useful to refine our
intuitions via scientific evaluation. Fortunately, a large body of
relevant scientific studies has now accumulated, and when taken together,
their findings clearly suggest formal rules about how best to design
questions. However, this work has been scattered across the publication
outlets of numerous disciplines (e.g., psychology, sociology, and political
science, although some work has appeared in marketing, statistics,
communication, education, and the health professions), and this literature has
not yet been comprehensively and integratively reviewed in a central
place. Doing so has been a principal project for me during the past ten
years.
Because of the complexity of this literature, it does not yield
a short and efficient list of rules, each supported by a few documentary
references, and each obviously justified by all relevant studies. The
issues addressed are multifaceted, and many are still in the process of being
resolved by innovative and creative new research. But there is a great
deal of richness in the existing literature that provides useful guidance for
scholars interested in maximizing the reliability, validity, and efficiency of
the measurement instruments they employ in their research.
The
Handbook of Questionnaire Design reviews and integrates the large literature
dating back to the turn of the century illuminating the cognitive processes
involved in answering questionnaires measuring attitudes. Primarily by
experimentally manipulating question format, wording, or ordering, these
studies have illuminated how people go about answering attitude questions and
how different question formulations can produce quite different answers.
This literature has two important sets of implications. First,
it does support numerous clear and practical recommendations regarding how
best to construct questionnaires for surveys, laboratory experiments, depth
interviewing, or other forms of empirical social research. It also makes
clear recommendations about how to interpret questionnaire data and when to be
wary about its implications. The primary goal of this book is to
highlight a list of practical recommendations to questionnaire writers and
analysts of questionnaire data, and to make clear the nature of the evidence
justifying each recommendation.
The second primary goal of the
book is to reorient the entire conceptual approach questionnaire writers and
analysts bring to their tasks. Experimental studies of questionnaire
design have highlighted many insights regarding human cognition and
communication generally. Taken together, these insights constitute a way
of viewing social information processing and social behavior that is currently
gaining favor as a result of the cognitive revolution in psychology. At
its core, this view contrasts sharply with the conventional wisdom that if you
want someone's opinion, all you need to do is ask, and he or she will tell
you. The new view asserts that numerous subtle and not-so-subtle aspects
of questions can have very potent influences on responses, even among people
whose views on an issue are quite strong.
Therefore, writing effective
questions and effectively interpreting the meaning of questionnaire data
requires careful attention to numerous small details. By explaining this
new view, this book will help consumers of questionnaire data to become more
sophisticated about the meaning of those data and the degree to which their
apparent implications may be contingent upon the particular questions
asked. In addition, this new view will help questionnaire writers to
become cognitive theorists who can develop and test hypotheses about aspects
of question construction that have yet to be investigated systematically.
The third primary goal of the book is to bring together a large and
diverse set of empirical evidence scattered throughout many academic
disciplines. Questionnaire designers and analysts in any given
discipline may well be familiar with the studies in their own literature, but
they are unlikely to be informed about those in other fields. By
bringing together this wide array of studies, we hope to paint a more vivid
and compelling portrait of the cognitive processes involved in questionnaire
responding than can be provided by any single discipline's literature.
The Program on Survey Research at IQSS. 1737 Cambridge Street, Cambridge, MA. USA. (617) 496-2450
© 2003-2009 President & Fellows Harvard University. The Program on Survey Research is a program of the Institute for Quantitative Social Science. Policies.
