Welcome back. In the last lecture, we implemented our solution, and we said we'll do a pilot. In this lecture, we'll analyze the results of our pilot. We'll learn from some of the mistakes, and then we'll iterate our solution to make it better. This will be the study and act part of the plan, do, study, act, or the PDSA cycle. Now, the Institute of Healthcare Improvement defines the study part as having these components. Once you've done your pilot and the study part you analyze your data, study the results. Compare the data to your predictions. What went wrong or what was working okay? And reflect on what was learned from the pilot. Then, the act part is you act on what you learn by doing some modifications in your workflow. Then, prepare to start the PDSA cycle again, and plan for the next test. In our case, the study and act part of the PDSA cycle were happening in very close proximity both in time and space. Let me explain. Every seven days, our team of clinicians and people who could make changes in the information systems used to meet and they used to see what was working okay and what was not working, and they used to make some changes. They used to come back again in seven days and see again what worked or what did not work and make another change. So, this was a very rapid cycle of improvement and iteration. Some of you may recognize, this is the typical Agile Methodology in which business people and developers must work together daily throughout the project. An regular interval, they meet, they reflect, they change their behavior of the workflow to see how it can be done better. Adapted to healthcare. I would say, the Agile Methodology is where workflow engineers and stakeholders including clinicians, must work together frequently if not daily throughout their project. Again, they meet at regular intervals, see what is working, what is not working, modify their workflow and their behavior, and improve, and act again. Let's put on our informatics hat again and see what are the characteristics of the information system that allows an Agile workflow reengineering in healthcare. When we started our pilot, our data source was an extract of fall risk score collected in excel sheet. Our method of analysis was via some MATLAB codes that we had written on one desktop, it was not web-based. Our delivery of information to the therapist was actually a print out of the list of patients that should be seen that day, and somebody used to email them or hand-deliver those lists to the therapist at the patient's bedside. Now, this might seem very primitive, but as you can imagine, this is very flexible. If I had to go back and change the algorithm of the data analysis, it would probably be very easy to do and could be done in a very short amount of time. That helps us to learn from our mistakes. I will show you two examples about where we had some unintended consequences and we had to modify the pilot. Let's start with the first one. As I said, we were using the Fall Risk Score as a surrogate measure of function. So, there was a 24-year-old gentlemen who was admitted one day prior. Our algorithm showed that this patient had a high Fall Risk Score and therefore, he should be seen today by physical and occupational therapists because he was on track to being missed on that particular day. What happens? The therapist walk up to the patient to actually see the patient or treat the patient, and we find that the patient is walking in the hallway, ready to be discharged. Obviously, this patient does not need any physical or occupational therapy. Why did we fail? Well, we failed because if we look back, the reason for admission for this gentleman was because he had alcohol intoxication. In about 24 hours, he was done with the effects of alcohol intoxication. He was like any other 24-year-old gentlemen walking down the road. So, when we look back, what are the modifications that we're needed in our pilot, so that we did not repeat this kind of mistake again? The Fall Risk Score, we were using it as a surrogate measure of function. We should not have been using that because this is the typical type of patients that will show a high Fall Risk Score, but they would probably not need therapy. That's what we learned in our pilots. So, in our production version, in our final version, we did not used the Fall Risk Score, we used the AM-PAC scores, that is the activity and mobility scores for post-acute care setting, which is a true measure of function. Because we had very flexible data source and method of collection of data in our pilot, we could make these changes without investing a lot of money or time. Let's go to the next example. As I said in our pilot, we used to print out list of patients and hand-deliver to a therapist or actually email them. We thought rightfully that this is a very inefficient method of doing things, and we would build them a web-based dashboard that they could login and see which patients needed to be seen. That was our assumption that therapists would look at a web-based dashboard to see which patients are a priority for treatment. Very poor adoption when we rule it out. Why was that? We went back, talked with the therapist. They said they did not have time to find a computer, log in, look at a dashboard between treating patients. So, we had to walk back. We said, that we agree that paper-based system is not a very efficient system, we will keep the web-based dashboard. But therapy coordinators and team leaders would look at the dashboard, and then page the treating therapists who are actually working with the patients saying that the web-based dashboard is saying that you should be looking at this patient. So, that's how we had to take a step back and redesign our workflow. To put it into perspective, this is what happened. When we started our pilot, we were printing out a list of patients. The therapy coordinators were looking at this list of patients and paging the therapist, or they were actually taking this list of patients and taking it to the therapists. This was an inefficient workflow. So, we said that each therapist will look at a web-based dashboard. Obviously, this does not work as we discussed before. We had to modify it saying that, okay, therapy coordinators will look at a web-based dashboard. Now, they could look at this dashboard from wherever they were in the hospital. But [inaudible] the therapy, so that the person who was treating the patient at the bedside does not have to log in and look at the dashboard. Now, this is something that we could do because we learned very early in our pilot. At that time, our information systems were very flexible and so that we could make these changes. So, if you look at this diagram that shows that between the pilot to our production, the data source and the method of storing the data had changed. We started with the Fall Risk Score. In the production version, we were using the AM-PAC. We started with MATLAB scripts running on a desktop for data analysis. Then, we had Ruby code running on a server. We started with a printed list of patients with somebody would hand-deliver or paging the therapist with. To a web dashboard, that the therapists were not looking, but a different set of clinicians were looking at, and then paging the bedside therapist. So, there was a huge difference between the pilot and the production. But one thing did not change, and that was the purpose. Our purpose was to redesign the workflow, so that we don't miss any patients who need therapy treatments. That is called redesign to the purpose while not being merit to a particular process. That completes the study and act part of our PDSA cycle. So, let's see what did we learn during this analyze and iterate phase of our pilot. One, the adaptation of Agile in healthcare workflow redesign; to work together, meet frequently, rapidly iterate to accommodate clinical needs and new evidence. Second, which I cannot emphasize enough, don't be afraid to walk back. The whole idea of the pilot is to learn from your mistakes. Reengineer to the purpose. What is the clinical purpose? What are you trying to achieve? Not to any particular process. I hope you enjoyed this lecture. I will see you in the next one.
