3.1.1 Interviewer Roles, Obtaining Interviews

Welcome back, we are beginning our third lesson on interviews and interviewers.

We'll first talk about a number of tasks that interviewers engage in that

really are not about conducting interviews per se, but

they're kind of preliminary activities.

So recruiting sample members, that is placing phone calls to,

or knocking on the doors of households.

And essentially persuading sample members to become respondents.

And also, within household selection or within household sampling,

which is an additional layer of sampling that interviewers

are sometimes required to engage in to select one respondent from

the household once the interviewer has obtained agreement in general

this kind of extends the sampling design into the household.

Then we'll talk about interviews themselves, and

talk about a number of different interviewing techniques which are often,

sort of presented in counterpoint to one another, or in opposition to one another.

So, for example, we'll talk about the standardization debate,

in which on the one hand, there's a strong argument for standardizing the wording,

requiring interviewers to say essentially use the same words

in every interview they conduct versus allowing interviewers to choose

their words in a way that will essentially standardize the Interpretation.

So if it takes different words to establish the same meaning across

respondents, interviewers are able to do that under this second view.

So this is known as the standardization debate.

Another distinction is made between what's called personal style of interviewing

in which interviewers attempt basically to establish rapport with respondents, and

the formal style of interviewing, which is more business like.

And the question is as with the previous distinction

which of these produces higher quality data?

Because the personal style is, essentially, about establishing rapport,

we'll spend a little time of talking about rapport between interviewers and

respondents, and how this may help, or hurt, data quality.

And then, finally, we'll cover what are known as interviewer effects.

So, while interviewers can add value to the Enterprise in many ways, they can

also introduce error that isn't present in self-administered questionnaires.

And these interviewer effects come within two varieties.

One is biased in which interviewers,

different interviewers elicit responses in different directions, so for

example one set of interviewers might elicit more.

Pro-feminist responses then another set of interviewers question concerning feminism.

And it turns out that it's the gender of the interviewers that seems

to be responsible for this.

So female interviewers may elicit more pro-feminist responses than

male interviewers.

So this is called bias because the error is in one direction.

Or interviewers may increase the response variance by

behaving differently from one another and we'll focus on the behavior of probing,

which is the kind of one unscripted part of standardized interviews.

If interviewers do this differently the idea is that

response variance will increase as a result.

Let's spend a little bit of time talking about the different roles that

interviewers play.

As I said, they do a number of tasks besides asking questions and

recording answers, and each of these can

affect the different sources of error that we've been talking about.

So when it comes to sampling and coverage, they have an impact on coverage by

locating and contacting sample households and determining which units are eligible.

Some structures are not residential for example, and would be ineligible in

a survey that requires residential structures to be included.

And they sample within the household as I mentioned a moment ago.

And to the extent that this is successful or

not can have an impact on sampling error.

When it comes to non-response error or response rate at least interviewers do

have the task of contacting the household and gaining cooperation,

potentially persuading reluctant household members to become respondents and this can

vary between interviewers again to the extent that this is done effectively

the response rate should be high but it does vary between interviewers.

Interviewers are probably most closely associated with measurement error,

because they are asking the questions which are the measurements in

conventional surveys, where self report is the predominant measure.

So, they ask the questions, they probe for complete answers, they clarify questions,

and they record answers, so what can go wrong here?

Well interviewers can ask questions in different ways.

They can probe differently, as I mentioned a moment ago.

The questions may be misunderstood, and interviewers will be unaware of this or

unable to address misunderstanding because, for example,

the interviewing technique that they've been trained in and are required to use

requires them to stick to the script, and if departing from the script is necessary

to clarify questions that are really not able to do that.

So interviewers can contribute to these different sources of error, more or less.

So they can actually reduce these sources of error.

So for example, response rates are generally higher with interviewers than in

self-administered questionnaires.

So interviewers both add value and introduce error.

For better or worse, they can have an enormous impact on costs and timeliness.

They are usually the single largest expense in

a study that involves interviewers.

That is more money spent on interviewers than anything else when

there are interviewers in the study.

So, it's a very Important decision, whether or

not the mode involves interviews, and one that researchers should make carefully.

So, turning to recruitment, one of these tasks

that interviewers engage in that really don't involve conducting interviews.

Some interviewers are more successful than others in obtaining interviews.

If you look across studies and a system called lifetime response rate.

So what might be responsible for this?

Well it could have to do with some interviewers just being better

at recruiting perhaps because they're more extroverted or their personalities just

lend themselves to persuading strangers to take part in the study.

Another possibility particularly for telephone interviews

is that it may have to do with how they speak and interact with respondents.

So that's actually what we'll turn to next.

On the telephone, really the only information that

a household member has about the interviewer who's trying to persuade them

to take part in the study Is what they sound like and how they speak.

Really, telephone interviewing may sort of concentrate sample members

on interviewers' vocal properties and the way they interact.

In one study, Conrad and his colleagues Coded the behavioral attributes

of interviewers in 1300 telephone survey introductions.

These are audio recorded telephone introductions from

five studies that were conducted at the University of Michigan.

And the behaviours they coded included fluencies like um's and ah's.

What are known as back channels these were by the household member the sample member

not by the interviewer, back channels are for

example saying uh-huh and nodding although of course

on the telephone the interviewer would never see the household member nodding but

saying uh-huh is generally considered an acknowledgement,

a way to say to the speaker, yeah please continue I'm still listening,

and over speech and interruptions, and that's called simultaneous speech.

The idea being that these generally signal

some sort of troubled interaction if there's a lot of overspeech.

And they modeled the impact of these different behaviors

on one of three outcomes.

Agreement to take part in the study, refusal scheduled callback which

basically refers to deferring the decision until later so

the interviewer and the household member agree that someone,

maybe not the interviewer there currently talking, but somebody will call them back

in the future to talk about a possibility of taking part in the study.

So a scheduled callback is kind of In between it agreement and refusal and

so they were looking at these three outcomes on the basis of behaviors like

disfluency a household member back channels and over speech or interruptions.

So the first finding was that fillers or disfluencies like ums and

uhs, are both good and bad when it comes to participation.

So this figure shows us the filler rate, or the disfluency rate, per hundred words.

And the rates are broken into these quintiles.

The y-axis is the proportion of degrees as a function of the filler rate.

So you can see when interviewers are making zero disfluencies,

that is when they're highly fluent.

They're actually quite unsuccessful in recruiting household members.

This is actually the lowest agreement rate of all the quintiles.

As soon as they make even a small number of disfluencies,

this is more than 0 and less that 1.28 per 100 words, so not a lot of disfluency.

Agreement rate goes up substantially to 0.35.

It tends to drop off as they get more until,

when they're making the maximum number of disfluencies, fillers, greater than three

point four nine per hundred words, it's at it's lowest when there are any fillers.

However, even when interviewers are highly disfluent,

they're more successful than when they're perfectly fluent.

And the thinking was that, when they're perfectly fluent, they sound robotic.

And they don't seem to be tailoring the invitation,

the recruitment speech for the particular household member.

That seems to be the big disincentive to participate.

Another speech behavior that's involved in recruitment is, as I mentioned,

backchannels.

The household members produce more back channels uh-huh and okay or I see.

Where they ultimately agree.

This figure shows that the proportion of lose or

conversational terms with a back channel by the household member.

And as you can see, scheduled callbacks are in between agrees and refusals, but

really when they're signaling that they are paying attention and

sort of authorizing the speaker, or

in this case the interviewer to continue, they're most likely to agree.

And then, the third behavior that the authors looked at was overspeech or

simultaneous speech.

As I said before, when there is a lot of overspeech,

it generally indicates that this is an interaction that's not going well.

And as you can see, overspeech is highest when the invitation, the case,

ultimately ends in refusal.

A scheduled callback is lower, but

not as low as when the invitation led to agreement.

So the amount of over speech in an invitation

is pretty clearly related to the outcome.

However, not all overspeech works the same way.

What the authors looked at was what they called interruptions,

which are actually just what they sound like.

This is overspeech in which the household member was speaking and

then the interviewer began to speak before the household member had finished.

This type of overspeech seemed to have been used quite strategically

by the interviewers to essentially block a refusal.

So as you can see, when the invitation ended in a scheduled call back,

which again is a deferral to another time to make the decision

interviewer interruptions of household members was at its highest.

And again that the idea that the authors proposed is that they were preventing or

interrupting an utterance that would have led to a refusal

had they not intervened at that point.

So that's one set of examples of how the speech and

vocal attributes of interviewers and how they interact with household members and

how household members speak in response can effect outcomes.

Another study published at about the same time by Nora Cate Schaeffer and

her colleagues looked at a pairs of respondents

with the same propensity to participate.

So what they did was, they modeled the likelihood to participate on the basis

of a number of variables in their dataset, and this is a longitudinal dataset so

they had information about these household members going back a number of waves,

meaning previous interviews.

They found pairs who had the same likelihood,

same propensity to participate, only one member of the pair

accepted an invitation in the wave they were looking at.

And the other member of the pair declined.

So they have the same likelihood of participating but different outcomes.

What can be going on?

So they looked at the speech of both interviewers and I'm calling them

respondents here, but again they're household members because they haven't yet

agreed to do this interview.

Using conversation analytic techniques similar to, a little different than what

was used in the Conrad et al study when they coded the interactions.

They uncovered a number of Interesting provocative explanations for

this difference.

One thing that I look at it was when did at these

sample members decline the invitation and when did they accept it?

That's at what point in the invitation.

So as you can see if you look at declinations, they're occurring at various

points during the interview not exactly even numbers but with non-zero frequency.

So during the opening, during the study description, and after the request for

participation.

When you look at acceptance,

that's only happening after an explicit request for participation.

So you can see that household members,

sample members are feel no qualms about declining at any point.

It's much harder to elicit an acceptance, and

that really requires an explicit request.

Will you take part in our study?

Essentially.

They also looked at particular interviewer actions and

how they affected the odds of participating.

The odds ratio in a logistic regression.

So, if the interviewer said hello versus hi, after the respondent

answered the phone, the odds ratio was 1.44, which is marginally significant.

When the interviewer requested to speak to a sample member in a very polite way,

versus a not so polite way.

The odds also were greater that the invitation would lead to an interview,

marginally significant but, you can see, in the right direction.

And, when the interviewer introduced herself, for identified herself or

himself first versus second,

the odds were again higher that the invitation would lead to an interview.

In this case significantly higher.

Finally, they looked at a number of sample member behaviors

that seemed to indicate a willingness and interest in participating.

Or an engagement in the process that would make them more likely to participate.

So, if the sample member asked at least one question.

They were reliably more likely to participate than if they ask none.

If a sample member asked about the or

said something about the length of the interview, how long is this going to take.

Much more likely to participate than if they had no.

Nothing to say about the length as if to suggest that it would only be thinking

about how long will this take if they were considering participating.

If the sample member asks at least one W-H type question like what is this about,

the odds are less likely they'll participate than if they don't ask

such a question.

So this seems to indicate a kind of skepticism or reluctance to participate.

The effect is even stronger if they ask this question

before the interviewer requests that they participate.

But if they ask a WH type question after the interviewer asks them to participate

there's no difference, essentially, between asking that question and

not asking that question.

Because by that time the household member, sample member, has made up his or

her mind about whether they would participate or not and

so these questions have a kind of different significance at that point.

So these are some examples of how interviewers' speech and

how they interact with household members.

Can affect the outcome of their recruitment efforts,

at least on the telephone.

This is among the kind of non-interview tasks that interviewers engage in.

Next we'll turn to within household sampling, or respondent selection,

which is an interviewer activity that follows acceptance of the invitation or

agreement to participate by the household member who first spoke to the interviewer.

That person doesn't necessarily become the respondent but is considered the gate

keeper, and if that person agrees to participate another

household member may still be selected and may have to be convinced to participate.

This process of within household sampling or respondent selection

assures that any eligible member of the household can potentially be selected to

be the respondent and that's important in implementing the sample design.

Data Collection: Online, Telephone and Face-to-face

Conheça os instrutores

Explore nosso catálogo

Registre-se gratuitamente e obtenha recomendações, atualizações e ofertas personalizadas.