[BLANK_AUDIO]. So now let's look at factual questions from a different way. In this segment we'll talk about memory and recall and this is all dancing around questions about specific events in the past and conduct of activities, so behavioral frequencies and things of that nature. So with specific events we might want to know whether they occurred or not. So, for example, "Since January 2013 have you looked for a job?" And here you're asked about a specific event, which requires the respondent to comprehend what you mean with looking and what you mean with looking for a job, in particular. So definition of look and job is a comprehension task that the respondent has to master. Recall that from Unit 2 when we talked about the cognitive response process model. But what is different here about past behavior, when we ask of factual questions, is that there's an additional recall task. So you have to remember everything you did since January 2013. Now if you have been in a stable job, no problem there, nothing has happened. But if the time frame would be different or if you have done a lot of different things this might be a more challenging question. So then we can look about when did something occur. For example, "What age did you first try a cigarette?" Here we ask the respondent about a specific event, just like in the example before, we ask about past behavior, but we also ask him to recall something else, which is the specific date something occurred. These questions you find a lot in the public health service, for example. They try to measure overall exposure or links of behavior of some form. Another type is counts of activities and, you know, this is also part of this set of other biographical memory questions, very hard to do. So, here we would, for example, ask, "Since January 2013, how many times have you stayed in the hospital overnight?" Again a comprehension, a recall task, and, in addition, an estimation for an exact count, because you don't want to know the total number aggregated across all events that happen at that point in time. And then, finally, an example for frequency. "When you were growing up, how often did you attend religious services?" This, implicitly in the way it's worded, is encouraging the respondent to talk about frequency as a rate. So, "how often" is usually the signal for this. I'll talk about the different challenges with these type of questions as we go forward now in the segments. Now why is this difficult? There are a couple of things that are related to error in recall of these past behavioral questions. First one we'll talk about is encoding, storage, and retrieval. Encoding of information is key to recalling information. If it never entered your memory, can't come out again, right? So, research has shown that information has to be considered salient or distinctive to enter long term memory. And salience is individually defined, what is salient for me might be not at all salient for you, and vice versa. So, if you design your questionnaire, think about your respondents set, your population that you tried to ask question about and ask yourself, how important is this to the respondent, and does it stand out from other activities, and is this even reportable. Because if not, then you might have trouble recalling such events. The deeper encoding effects accessibility, so, you know, that if its not stored very strongly in your memory, it might be hard to retrieve it. Also, if something is not encoded at all, as I said earlier, if it's not in memory there's nothing to recall, which leads to underreporting, not motivated underreporting, but just underreporting errors. This has big implications for questionnaire design. If you do try to measure such past behaviour where you think it was not salient and the respondent might not have encoded this enough to retrieve it, you might need to rely on alternative data sources, for example, you know, bills and receipts. If you are in charge of the Bureau of Labor Statistics Consumer Expenditure Survey and you want to ask how many T-shirts someone bought in the year 2013, so over a year from today, you would probably have trouble getting that number right. But if receipts are collected, then you have the chance to look at this more closely. Here's an example from a study that Roger Tourangeau did. It was published in edited volume under the title "Remembering What Happened: Memory Errors and Survey Reports." A good read and this table here is sort of core summary table of the results. This was a study where the researchers asked parents that had just brought their kids into the doctor office. When they returned out of it, they did ask about the type of immunization that these kids got. And even so that just happened, you can see that parents tend to over- and underreport immunizations and that there is a big net error, a lot of overall underreporting for hepatitis B and DTP, whereas measles, mumps, and rubella is reported much more often. So lower rates of underreporting and also higher rates of overreporting, a total net error of plus 17.1%. This is interesting because, you know, you would think people, you know, only forget over time, but you see here that these parents probably never really knew what happened. But as I said before, you always get an answer in the questionnaire and therefore should be cautioned relying on those reports of past behavior. Now the different methods that help with the recall. We differentiate between free recall, cued recall, and recognition. The free recall is the most difficult of all. This would be in the example before, "What type of immunizations did your child get?" Whereas you could give a cued recall, you give some examples of what this could be. Or you even have recognition where you tell the respondent all the sets of possible immunization and then you just check it off. Likewise if you were to ask for, you know, social media use and you ask, "Which social media sites have you visited in the past year?" it might be much harder to do that in a free recall task because there are so many, but if you see the list of them you can just pick what fits there. Or at least give some examples so people do know what you mean and that helps them to remember. Now, the retrieval is also affected on what type of forgetting happens. So people might, you know, have stored this correctly and might be able to retrieve it with cued to recall or recognition, But if they have forgotten in the mean time, then it might be much harder to do so. So, the elements that affect forgetting are time, the older the event is, the more likely they will be forgotten. The less salient or more mundane the event is, the more likely it is to be forgotten. The more repetitive and routine, the more likely it is to forget an individual event. I mean, you do know that you drink coffee or that you eat bagels, but you might not recall every single time you did that. And here too we have clear implications for questionnaire design. You need to, first of all, give time for a task so that people can take the time to remember. It is helpful to give some example cues. It's helpful to shorter reference periods, we'll talk about that in a second in more detail. And, alternatively, you might want to use an event history calendar, so a visualization of a calendar where people can place events and time that can help mapping what happened in the past. Now, to get a sense of how difficult or easy it is to get this questions answered, time yourself when you answer the following questions. "Have you ever dropped a class as an undergraduate?" "...taken any antibiotics?" "...nNot worn a seatbelt while driving?" "Have you been to the movie theater in the last month?" "How old were you when you got your first kiss?" And "How many times have you seen a doctor last year?" Now, usually, people struggle, you know, with some of these questions. For example the second to last question. While this clearly is a comprehension problem going in the way of answering this question because some people might interpret this as a first romantic kiss where others might remember this as, you know, first kiss of someone's parent, relative, what not and then, you know, "Well, I probably got that before times that I can even remember." Likewise in the last question there's an issue here for remembering all the times we've seen a doctor but also of course what does last year mean? Is that from today, until 12 months from today? Or is it in 2013. So depending on when in the year the survey happens, you might have different answers to this particular question. So, in the next segment we will talk about exactly those things, reference period, and trying to reconstruct what happened in the past.
