[MUSIC] Welcome back to Introduction to Clinical Trials. This session is going to be about comparing the relative merits of randomized clinical trials to observational studies, and to try to address the question whether clinical trials are still the gold standard for health research evidence. So we're going to talk about three general areas in this lecture. First, briefly I'm going to talk about some frameworks for evaluating evidence from clinical studies. Second, I'm going to look at some studies that compare results of randomized clinical trials to non-randomized studies of the same type of interventions. And finally, I'm going to review some high profile cases where the results were in conflict between clinical trials and observational studies. And at the bottom of the slide I've got a little glossary box for you so you can go back and refer to that if some of the abbreviations don't make sense later on. So to start out with a framework for evaluating evidence for a particular healthcare intervention. And we have to remember that what we're trying to do is to evaluate a body of evidence and put together evidence from clinical trials and observational studies. And the focus really isn't on comparing whether one particular trial is better than a particular observational study. It's really synthesizing the evidence. But we need some guidelines for how to weight evidence. And here's one proposed framework for how we should weight evidence. And the idea of this pyramid is that at the bottom of the pyramid is unsystematic clinical observations, sort of case series, and those would have the least weight in developing guidelines. That we would be most suspect of those types of studies that might have bias, that they don't have an appropriate control group. And then we might go up to physiological studies that use surrogate outcomes, which may provide some evidence, but don't provide us evidence of the clinical effect of treatments. And then you can see in this particular framework that we have observational studies and single studies versus a systematic review of observational studies where you put a lot of observational studies together in what is commonly referred to as meta analysis. And then, at the top, we have single randomized trials or systematic review at the very top of more than one clinical trial. So that framework has been elaborated on in this GRADE program, which is a system for grading recommendations, assessment, and development of clinical evidence that is a framework for combining evidence from randomized trials and observational studies, and coming to an overall decision about a particular medical intervention. So we start here with study designs. And you note that randomized clinical trials start at being considered high quality evidence and observational studies not as high quality of evidence. But there are factors that we might look at in a particular trial or a particular group of trials that might lower the quality of that evidence. And they have defined five areas to look at. So whether there was a risk of bias in a particular trial, did they have good allocation concealment, if they had a subjective outcome was it masked assessment? And those kinds of things to see if there might have been bias in the clinical trial. And they would lower the weight they put on that evidence if they found that bias. And the same might be true for an observational study if they felt like the control group and the experimental group were very different and there was a lot of confounding. They might lower that evidence from the observational study. And other areas are inconsistency that there is lot of heterogeneity. In the results from different studies, if the evidence isn't really directly relevant to the question your answering, perhaps the question your looking at is whether a drug works in a pediatric population, and all your studies are adult population. That would be one form of indirectness, that it's not directly applicable to your population and the clinical question. Imprecision would mean that there were wide confidence intervals. And publication bias would be looking at, if the complete body of information is out there to be reviewed. And then there are some features of a particular set of evidence that may strengthen it. That if they saw a large treatment effect, if there was a dose response. And if they really thought that all of the confounding was either taken care of or would actually operate against the results, so that it was making the result more conservative. So that's kind of more elaborate framework to grade individual studies and also to know how to weight them in final decision about what kind of recommendation. I want to remind you of the key strengths of a randomized clinical trials, and why it's considered a gold standard. And the key strength is randomization, that we can break the link between prognosis and prescription. That there isn't confounding by indication that patients that come in with a particular set of risk factors aren't given a particular drug. So that we can ensure that we have comparable groups. And that is really the biggest strength of randomized clinical trials. And it allows us to apply the statistical methodology based on random sampling. We also have, in randomized clinical trials, generally more standardization of the treatments, so that we can be more precise about exactly what we're comparing, the experimental treatment and the control treatment. That can vary across different types of trials. But it is generally more constrained than in an observational study. And we also usually have a standardization of the outcomes assessment. So we select primary outcomes, we have rigorous protocols about how they're measured. And try to ensure that it's unbiased, whether that's by masked assessment or masking of the treatment groups. And overall, then we assume everything else between the groups is equal. Those are some of the key factors that allow clinical trials to provide us with unbiased estimates of treatment effectiveness. So that ends our brief discussion of the framework of how we evaluate evidence.
