This course will cover the basic elements of designing and evaluating questionnaires. We will review the process of responding to questions, challenges and options for asking questions about behavioral frequencies, practical techniques for evaluating questions, mode specific questionnaire characteristics, and review methods of standardized and conversational interviewing.
Main Questionnaire: Demographic Questions - Ending
Loading...
Do curso por University of Michigan
Questionnaire Design for Social Surveys
248 classificações
University of Michigan
248 classificações
Curso 3 de 7 em Specialization Survey Data Collection and Analytics
Na lição
Unit 6: Putting It All Together
The questionnaire from start to finish; Things to put at the end; Mode Choice: Implementations for layout; Self-administered questionnaires
Conheça os instrutores
Frederick Conrad, Ph.D.Research Professor, Survey Methodology
Institute for Social Research
Frauke Kreuter, Ph.D.Professor, Joint Program in Survey Methodology
Adjunct Research Professor, Institute for Social Research
[BLANK_AUDIO]
The core is of course the main questionnaire.
And the whole course was about getting a good main questionnaire going.
But there's a little bit of structure in
the main questionnaire that we do want to talk
about, and then, of course, also talk about
what comes at the end of the questionnaire.
So the first piece to think about is the question order.
Questions should be relevent and easy when you
field them at the beginning of the questionnaire.
You get the respondent interested in participating in
the survey, because you mentioned the survey topic
and the respondent decided to participate. And you
want to start with a relevant question that's easy to
answer, even if that is not a central
question to your analysis. If you end up not
using this question, still good if you have
something that respondents can easily answer at the beginning.
It is tempting for any novice in questionnaire
design to start the questionnaire with the demographic questions.
I'm not sure what's the psychology behind that.
But without failure so far I've seen every single class
project, if students do not know about the order that a
questionnaire should be in, they start with demographics. You know,
not quite "What's your name?", because those would be confidential information,
but "What's your age?", and you know, "What's your
gender?" or "...your major?", if it's a school project.
So don't do that.
Start with the relevant and easy question.
Then the questions should be grouped into modules, and these
modules should so have a logical sequence for the respondents.
In a way, they're ordered from most salient
to least salient with respect to survey topic.
However, there are diverges from that because
sensitive questions should be moved towards the end.
So if your key research topic is on sensitive question, there
is a tension here that you have to keep in mind.
You have also to make sure that the order of
the question does not produce any bias, and we did discuss
context effects in questions throughout the course. So make sure
you review those segments if you don't remember these pieces.
When you structure your questionnaire, make
sure that you always number your questions.
It's okay to number them sequentially, it's okay to number them within a module,
but something that makes it easy to look at the questionnaire, look
at the data set, move between one and the other and give the analyst
also an idea of what is the logical order of the questions that you have.
Then you need numbers for your response options.
It is interesting how different survey
companies have different numbers as default.
So, you know, it was for a long time popular to use
to 98, 97, 99, 9999 as codes for "don't know" or refusal.
I personally prefer, if the real answer categories do not have numbers below
zero, to use negative numbers for refused and "don't know" or "does not apply."
The reason for that is, that later on, when you
analyze your data, you can assign missing value codes for
a string of variables and you don't have to look,
what is the range, the numeric range, of the possible answers.
It might be that you have some questions
in there, for example age, or weight, or, I
don't know, days since something happened, that could
hit these numbers, 98, 999 and so on.
And so then you have to be much more
careful in how you code and assign missing value codes.
It is also helpful for a coder,
so if the question is not filled in
automatically, like it would be in web surveys,
but if a coder assigns these questions after looking at
the questionnaire, that you are somewhat consistent in how "yes"
and "no" is coded, you know, just so that, whoever
puts in these numbers doesn't accidentally switch these numbers around.
So that is a thing to keep in mind, you know, when
you design the questionnaire and when you think of numbers for response options
that you match that up into an easily codeable questionaire.
There are transitions that you have to keep paying
attention to, and there's a lot of text on
this slide, but it's there for you to see
some good transitions or some transitions that are in place.
Often they are as easy as, "Now I'd like to ask you...,"
and then there's a little bit about what is going to happen now.
If there are additional items that respondents use,
like a booklet or a show card
or something, then attention can be drawn to
those, and people move on to a different topic.
But in general, it's good that you don't appear
to be jumping from topic to topic and back.
Now sometimes people over do it with these transitions.
And it is surprising how much tolerance respondents have.
Too much text is always bad.
And in particular, if you have a self- administered survey,
you probably need less transitions then you have in an interviewer
administered, just because interviewer and respondent have somewhat of a
conversation, and in a conversation you wouldn't just jump around either.
Speaking of jumping around. So sometimes
you might jump around in a questionnaire, right,
because you have a topic that doesn't apply to a respondent, and then we move on.
We talked about these filter questions.
Now, it is very important that you never leave it up
to the interviewer and respondent to decide where to go, right?
You want to have clear branching instructions,
otherwise you will have two types of error.
A so-called error of commission, which is an error where the respondent
fails to branch when he or she is instructed to do so.
And an error of omission, which is
when the respondent skips, even so they shouldn't.
The first one is frustrating for the respondent.
Okay?
They get confused, they get questions that don't apply to them.
And it's also annoying for an analyst, because they
don't necessarily know, was the filter question answered correctly
or, whatever the respondent might have ended up putting into the
follow up question, do those really belong to the respondent answers.
The errors of omission, they can't be
repaired even with a judgement call because the
respondent skipped over a section or a segment
of the questionnaire that they shouldn't have skipped.
The result is missing data, and the only thing that you
might have left is imputing the values that should be there.
In general, errors of commission are more common that errors of omission.
At least, that's the result of a research study
by Claire Redline and Don Dillman published in 2002.
Here's a snippet from that study.
You see the control condition and then two other conditions.
One in which they tried to prevent this error,
and one in which they modulated how the skipping
instruction can be enhanced, you know, with this additional
arrow here that leads the respondent to the next question.
And here are the results.
You see, in the control condition, you have
about 20% of commissions and 1.6% of omissions.
So this one here is a typical format for a self administered questionnaire.
In these other two conditions, they were able to significantly
lower the commission error, albeit at the expense of some omission.
So again a trade off decision just like we saw before.
So finally the, after the main part
of the questionnaire, you have your demographic questions.
There's often the transitional introduction to those.
"Lastly, some questions for classification purpose,"
sometimes it's called "statistical purpose" only.
You can look at all the questionnaires we pointed you to at the beginning of
this class, to see what
kind of transitions would work for you here.
Keep in mind, you do need the demographic questions, and they
should be in line with your weighting plan and your analysis plan.
So this thinking ahead is really necessary,
and you always should check that you do
have the right questions that match up with
whatever external source you want to use for weighting.
So, if you want to use your demographic questions
for poststratification for some data that's connected by the
U.S. Census Bureau, your demographic questions better translate back to
the those Census questions, otherwise you will be in trouble.
That also means that you should choose your question wording
based on those questions that you will be using in
comparison for weighting, and of course so that they match
your substantive interests, which reflects itself in the analysis plan.
And then, at the end, you should thank your respondent.
Make sure that you never forget that.
Often, it's also good to leave a little bit of open space
for additional feedback, any other things that they want to tell you.
Often at the end you'll see contact information for the project.
Again the local IRB information should either be there or in the front.
And additional contact information from the respondent is sometimes necessary, if
you plan to follow up for a quality control check, or
if you want to mail and send it to them that
you promised, a conditional incentive to participate in the survey for example.
Or if it's part of a panel study and you want to track them through them.
Remember to ask for cellphones.
Maybe ask them for cellphone numbers from family members and the like.
It can get difficult to find respondents a year or two years after your study.
Now, in the online discussion board, you submitted some questions and one of them was
pointing at this tension between salient question being in the front,
but that objectionable or sensitive question should be in the back, you know.
To balance these criteria that is a challenging piece, and one way out can
be mode decisions, and we'll come to that in one of the further segments.
Similarly, mode in which you field the survey will also affect how you design
certain question elements, and we'll come back to that in one of the next segments.
So, more choice and implication for layout will be coming up.
Explore nosso catálogo
Registre-se gratuitamente e obtenha recomendações, atualizações e ofertas personalizadas.
O Coursera proporciona acesso universal à melhor educação do mundo fazendo parcerias com as melhores universidades e organizações para oferecer cursos on-line.
© 2018 Coursera Inc. Todos os direitos reservados.
