29. Analyzing Three Generalized Linear Models

Loading...

Do curso por University of California San Diego

Idealização, Execução e Análise de Experimentos

373 classificações

University of California San Diego

Idealização, Execução e Análise de Experimentos

373 classificações

Curso 7 de 8 em Specialization Design de Interação

You will never know whether you have an effective user experience until you have tested it with users. In this course, you’ll learn how to design experiments, how to run experiments, and how to analyze data from these experiments in order to evaluate and validate user experiences. You will work through real-world examples of experiments from the fields of IxD and HCI, understanding issues in experiment design and analysis. You will analyze multiple data sets using recipes given to you in the R statistical programming language -- no prior programming experience is assumed or required. By the end of the course, you will be able to knowledgeably design, run, and analyze your own experiments for putting empirical and statistical weight behind your designs.

Na lição

Generalizing the Response

In this module, you will learn about analyses for non-normal or non-numeric responses for between-subjects experiments using Generalized Linear Models (GLM). We will revisit three previous experiments and analyze them using generalized models. Topics include a review of response distributions, nominal logistic regression, ordinal logistic regression, and Poisson regression. This module covers lecture videos 28-29.

29. Analyzing Three Generalized Linear Models18:40

Conheça os instrutores

Scott Klemmer
Professor
Cognitive Science & Computer Science
Jacob O. Wobbrock
Professor
The Information School

So, here we are in our R terminal, our R studio and

we're looking at using generalized linear models to revisit some

of the data that we'd analyzed previously, perhaps using for

example tests of proportions or non-parametric tests.

I have a note here for you explaining the generalized linear models

which were the anovas that we've been doing.

Note that what we've done so far in the linear model actually

is incorporated in a generalized linear model as well,

it would be using the normal distribution as part of the generalized model.

Generalized linear models also have, besides their distribution

they're considering, something called the link function, and

that's how the predictors or the factors relate to the mean of the response.

Each distribution in a generalized linear model has a kind of canonical

link function that goes with it, and we'll always use the canonical one.

But you can look into this further and see that you can actually have different link

functions along with different distributions.

So the linear model is a normal distribution with

what's called the identity link function.

And you can see more about this at the Wikipedia entry for

generalized linear models.

Our first example is going to use

what I previously called multinomial logistic regression.

Sometimes it's also called nominal logistic regression.

And that's because, we are analyzing nominal variables, or

categorical variables as a response.

We did this in our previous data, prefsABCsex,

where we saw the preferences of males and females for

three different websites, a, b, or c and so this will

use the multinomial distribution with the what's called logit link function.

And you can look into that further as you like.

So let's go ahead and do our reanalysis of this data.

We'll store it in a data table that we'll call prefsABCsex and

then I'll put .2 just to highlight the fact that we are progressing

through our code file here, and we are doing this data for the second time.

Okay, so we will load that up, and as is our normal practice, we will view it,

make sure it looks correct, we have subject as a column, pref as a column, so

the preference they have for their website, and sex as a factor.

And our question is, did people's preferences vary by their sex?

Did males or females prefer different website designs?

We'll recode subject as a nominal factor, because it's a number there.

And we can do a summary over that table for each of those columns.

We can see that we have preferences for website A.

Eight people, website B, 21 people on website C, 31 people, and

we have 29 female and 31 males giving us preferences.

And we can plot those preferences here and see the distribution of A, B,

and C for males.

And what seems to be a different distribution for females.

So males again seem to have a preference for website C and females

seem to have a preference more for website B, but with some interested in C as well.

Okay, so let's go ahead and analyze these preference by sex.

And we'll use a generalized linear model for multinomial logistic regression.

Again, also sometimes called nominal logistic regression, and

we'll load in a couple of libraries to help us carry this out.

We're using a call that you'll see to anova with a capital A.

This is not to be confused with anova with a lower case a which is from

the basic stats package that comes with R.

The car libraries anova with a capital A is a little bit different and

takes a little bit different data.

You can look that up and read the help pages for those two calls.

Something that we have to be aware of is that the default in terms of how this call

works to a capital A anova is relying on contrast

tests that are not set to be the kind of the normal way we would like them to be.

So we can change how a contrast are encoded with this line here for

this X variable.

You can look into that further if you like, but it's just

one of the nuances of the capital A on nova call from the car library, so

we'll see best pattern whenever we're going to use that in nova call.

Okay, so now we'll run the multinomial logistic regression, and

we formulate our relationships here among our response

preferences by sex as we've seen before and using our new data table.

And then we will go ahead and issue the nova, command.

Notice that even though it's a nova, it's not necessarily producing,

it's not producing for us an F test, it's producing a chi squared result.

It's just using a nova as a more general term for analysis here.

Here, if we might have had only two categories of response we might have used

binomial regression.

Another generalized linear model, but

because we have three we're using multinomial, so I make that note there so

that's something you can look into if you have two response categories.

So, in our result here we have the omnibus or overall result, showing

that we have a chi-squared statistic here with two degrees of freedom.

And we see a significant p value less than .05.

So that's saying that it does seem that sex did, in fact, affect the preferences

distribution that we have in our data and we've just seen that in our plots.

Clearly by sex we see differences between these two graphs.

So because we have a significant overall test we can sort of reach in and

look a little bit more at these levels individually.

And we can do that by looking at how the proportion

of responses was within each of the sexes, within men and

then women, for the different websites and see did any

of those responses deviate significantly from what we would expect by chance.

The answer for the men's group of course is yes, and here's the graphic end.

The answer is that they really liked website C and really liked meaning

that their preference for website C was greater than we would expect by chance, so

we'll encode all of those tests In M A, M B, and

M C, and then adjust for multiple comparisons as we've done before.

And we can see the p values here show that for A,

we don't have a significant result, neither for B.

In other words, their preferences are within range of what we might expect for

this number of males by chance.

But for website C, in fact, we do have a significant, in this case, preference for

website C.

Let's see if, for

the females, any of their preferences seem to differ by chance.

Differ from chance.

And we have the answer already put here for your convenience.

They really disliked website A.

So they differed by chance in the negative direction.

Let's move to the females plot here.

Website A was so unliked, that it was significantly different from chance.

And we run the same analysis here, and

we see that that's the significant result there.

Their preference for website B, was trending towards significance,

but is not significant at the 0.05 level, and

their website, preference for C is about what we would expect by chance.

So, the short answer is, men really liked website C, whereas women really disliked

website A and that's why gender has a significant effect on website preference.

Okay, let's move now to our second generalized linear model,

where we have an ordinal response.

We'll use ordinal logistic regression for a Leichhardt type response,

a one to seven scale response.

This will actually use multinomial distribution with the cumulative

Logit link function.

Just for the under the hood those of you interested in what's under the hood here.

So, we'll read in our search, scroll, and voice data that we analyzed previously,

and since it's our second time we'll give it a .2 in the name,

remember the dot in R is not a scoping operator like in many languages,

it's just a regular character like an underscore.

We can view that data.

And we see that we have subject, technique, order, time errors in effort.

Effort's the one we're interested in.

This was previously a within subject study where they used both searching, scrolling,

and voice to find contacts and then rated their effort for those.

We're going to recode

the subjects column to have a unique identifier in each row, which

is effectively saying it was a between subject study not a within subject study.

So we do that here.

We also turn subject to a nominal factor, and

we can drop the order column because it no longer make sense.

So we set it to null and it disappears.

And we can verify the table now.

So it has a unique subject ID in each row.

It has the techniques in, the time taken errors and effort column,

which again is what we care about, the one to seven ratings.

So we can summarize over that table by column and

see our output there as we often do.

And then we can actually familiarize ourselves again with the response,

the effort rating in this case.

So we can see in the output down below here that we have

our means and medians for each of the Likert responses.

And it looks like 4.4 for scrolling.

Higher was more effort so search was 3.6 and voice was about 4.4, okay?

We can also take a look by means and

deviations by each of the levels search, scroll and voice.

And then we can get a sense for the Likert responses on the 1 to 7 scale,

which is the x axis.

So here's search, here's scroll.

Seems a few people thought it was more effort.

And then here's voice.

And of course a box plot is helpful just to see how these kind of relate.

Looks like scroll and search were somewhat similar, at least in their means.

But the ratings for voice were maybe a little bit more effortful, okay?

So then, we are going to load a couple of libraries for

the ordinal logistic regression.

And we're going to make sure that the effort column is coded

as an ordered column.

We need an ordinal response here for this call to polr,

P-O-L-R, which is the ordinal logistic regression column, to work properly.

Because we're going to use the capital anova command we'll set the contrast as we

described earlier and then we'll do the ordinal logistic regression.

We've stored that result in N that model and then we'll report the anova on that.

So our results in the end we'll see is that in rating the effort of these three

techniques we have our chi squared result with two degrees of freedom and

we see a non statistically significant result, a p value of .10.

So in this box plot we can say the over all, or

omnibus, test has not concluded that in fact we've seen significant

differences among these levels for this ordinal response.

And that somewhat makes sense I mean we have very similar results for

scroll and search and voice isn't too far away.

It was .1 so we're in the right direction perhaps but not a signficant result so

we're not warranted in following up with post hoc parawise comparisons.

For completeness I've included how we will do them.

And we have seen this pattern before with the multicon library, where we specify

two key comparisons meeting all parawise comparisons in the multiple comparisons

procedures call here adjusted for the home sequential bonfurnic procedure.

So this is how we would carry them out, but were not justified in doing them, so

I'm not actually going to comply with that and

not run the test, now if we had a significant overall test or

omnibus test then we would be justified in looking at peer wise comparisons.

Finally, let's look at our third generalized linear model Poisson

regression, which happens for rare event data and count responses.

That uses a Poisson distribution with the log link function.

So we are going to look at the error's response which was another column

in the data we were just analysing.

This is where we were counting the errors people made as they looked up

contacts with search, scroll, and voice in our contacts manager.

So we'll refamiliarize ourself with that data.

Again, where we can see the means and standard deviations of that data.

Here is the distribution of errors for, for the search approach, for

the scroll technique, and for the voice technique.

Get a kind of a feel for how things look.

Clearly not normally distributed, and

we can see, are these in fact Poisson distributed?

And we can plot and see just clearly just from the box plot we have good intuition

that there is a difference in the number of errors people are making.

But let's make that a little bit more formal, so

we will fit Poisson distribution to the search data.

And then do a goodness of fit test on that.

And this is going to tell us that we have a non significant p value,

meaning we don't differ significantly from the pluson distribution.

We can do the same for the scrolling data and see we don't differ significantly.

And then the same for the voice data, and we also don't differ significantly.

In fact if we go back and look at the voice data distribution this is pretty

classic plus on so it's no surprise that we're more similar there.

Okay so now we'll finally move to the actual Poisson progression GLM analysis.

We're going to use the capital A anova call again so

we'll set the contrast as we've described and

then we use the actual direct GLM call the generalized linear model call.

And we indicate we want the Poisson family of analysis.

That's the distribution and

it also within it encodes the default link function which is the log link.

And so we'll set that model and report data nova.

And we'll see that we have significant

result from that chi-squared test telling us

there is a difference and if we look at again at the box

plots we would expect there would be obviously a significant difference there.

So we have an overall significant test.

Incidentally we can plot the residuals as we've done in the past for

the linear model.

And we can see that, in fact, they're patterned and

do not fall normally about the normal line there, the normal distribution line,

but here's what's fun about generalized linear models using plus on distribution.

There is no normality assumption in such a model, so

it's okay that we have data that looks like this, in fact we wouldn't expect it

to fall normally like we would for a regular linear model using a nova.

So that's an example of how we can get away from some of those assumptions.

So because we had a significant overall test let me go back here and

see our box plots.

We can conduct pair-wise comparisons among the different levels

of the technique factor.

And we do that, as we've seen in the past, with the mult comp library.

And what we can see here is we have three statistically significant p values,

all of which relate to the pair wise differences here.

And that doesn't surprise us.

None of these look like they should be similar to the other, so

we'd expect three significant pair wise differences.

And this is reported with a Z value.

So, now we've seen three different generalized linear models,

analyzing categorical or nominal responses, ordinal responses,

and Poisson count responses.

Let's go to our analyses table and see where this has brought us.

Here we are at our table of analyses.

And I've highlighted the generalized linear model, or

GLM, that we have just seen and used to revisit some of our data.

We looked at nominal logistic regression,

which we used to analyze categorical responses.

We've looked at ordinal logistic regression that we used to

analyze Likert scale or ordinal responses.

And we used Poisson regression to look at count data, analyzing error counts.

Technically, generalized linear models are not non-parametric, and

least not in the sense that they use ranks like the other non-parametric tests we've

seen above.

But, I've put generalized linear models in this column because

we would go to a GLM when we have a between subject study and we have data

that wouldn't normally conform to our usual parametric linear model ANOVAs.

Just like the GLM generalized the linear model, we'll now turn to the last

row where we can handle within subjects effects and look at linear

mixed models and it's generalization, generalized linear mixed models.

We'll go to that next.

Explore nosso catálogo

Registre-se gratuitamente e obtenha recomendações, atualizações e ofertas personalizadas.

O Coursera proporciona acesso universal à melhor educação do mundo fazendo parcerias com as melhores universidades e organizações para oferecer cursos on-line.

© 2019 Coursera Inc. Todos os direitos reservados.

Baixar na App Store

Baixar no Google Play