Hi. Let's talk a bit about verification and validation. This final phase of the UX process. We'll talk about why it's important. We'll also start to dig into things like how many users to test with, how we make tests stronger and better, and how we leverage thinking aloud when we're doing user based testing. Verify and validate. A lot of people use these interchangeably and they're pretty close. Generally, verification is establishing how accurate or true something is. Validation is whether or not something is acceptable. That's really the difference. The way we use it in an engineering terminology, is we generally verify requirements and we validate user acceptance. That's the way it works. I don't want to miss this quote from Glaser. "Yes, no, and wow." Wow seems like a good thing to shoot for everyday. We'll talk a little bit about verification versus validation. Again, we'll get into why we're doing this. We're going to see the V diagram again. Then we'll talk about users strengthening and thinking aloud. Verification versus validation. Again, in most engineering environments, it can get a little fuzzy, what you're talking about there. Verification usually goes back towards the engineering requirements. Validation, usually, is more around customer acceptance and maybe around testing for standards like UL acceptance testing. Verification and validation, together certainly represent the whole of the testing that we're doing, and there's a lot of crossover between the two. Why do we do it? Well, we've gone through all this effort to analyze, plan, research, and design, and we have been doing some testing along the way. We've been trying to get the users to take a look at what we're doing. Here in this phase, we really just try to solidify that final step towards our products. Final design criteria. It's certainly true that we also have to be careful when we're testing it, this final stage, as to how much of the testing we can really respond to. Depending on what we're making, we may not have a lot of latitude to make changes as far into the process. The changes we do respond to will be the most severe issues that we find. Something to think about as you're prioritizing the things that you find and how you're going to respond to them. Again, in our phased approach, we're at the tail end where we're really focusing on test cycles. Did we get through our requirements? Did we make our users happy? If we have to make a change, we're going to have to think about that pretty hard. Now, we're on the other side of the V diagram, so we're looking at the tests that we're going to use, to make sure that we met the requirements and the user acceptance level that we're hoping to get to. Again, verification is usually against things like the design and the architecture, and the system requirements. Where acceptance testing is probably more around user requirements and maybe requirements for some certification. Testing with users is really important and we want to do it all the way through the process, but there is a misconception that bringing in users must take time and effort. It's true that if you're going to do some formal usability testing, this can be some work, but really, with our discount methods, we can bring people into the process pretty quickly and we can learn about the entire user experience. It's really not just usability testing. It's really about what the users see in what they're using. Here's a list of guidelines for running a UX testing, and it brings up a number of good things to think about. One is to make sure you've got your tasks lined up, that you know what you're going to expect the user to do. Don't blame the user for things that go wrong during the test, blame the prototype. Sometimes, the user might find some interesting errors or some different ways of thinking about things. It's important that you're always positive towards their input, and that you give them lots of chances to comment, using open-ended questions like why to draw them out. Make sure that you're neutral about what's going on. Don't guide the users during the testing, and again, getting them to think aloud during the tests is a really important skill and we'll talk more about that. Let's touch on the number of users for a test and how that might influence what you're doing. It's surprisingly true that if you test multiple times with a low number of users, and you make changes in between those tests, that you'll actually uncover more issues than if you did one single test with a large number of users. This is Steve Krug's diagram of this, and really, it highlights the fact that you're going to find different issues if you have the opportunity to make some changes between the tests. Of course, you're only going to work on the worst problems that you find, and depending on what stage you're in, you may not want to make those changes. So you have to think about when it's appropriate to have what number of users tested to get what results. Nielsen generally recommends around five people for a usability test, but as you'll see and we'll talk about in this statistical section, that's not going to give you statistical levels of strength in what you find. So if you're looking for something qualitative, 2, 3, 5, 10, 11 users is going to give you a lot of feedback. You probably don't need to get into the 20s, 30s, 40s until you're trying to make some statistical proof around something in your interface. This is a great checklist for trying to improve your usability tests. It can be applied to any of the test methods that we're going to talk about in the next lecture. The most important things that are here are things like defining goals, defining tasks, knowing the number of users you're going to use, where you're going to test, doing a pilot test to make sure that the test works the way you expect, and thinking hard about what kind of metrics you're going to collect, and using a test plan to guide you. These are really great guidelines, so do read through them and keep this handy when you're planning your own testing. Another important part of usability testing is getting your users to think aloud. This isn't really natural. If they have their heads down and they're working on an interface, or working on a task, they probably will more naturally be quiet. Your goal is almost as a therapist, to get them to talk while they're doing that thing, and explain why they're doing what they're doing, or what they think about what they're doing. There's a list of questions here. This is drawn from another Steve Krug list. There's a lot more available at the full list, that talk about things that you could say to your users depending on what you see happening. So if the participant, for instance, seems surprised by something they see, you could ask them if that was what they expected to happen and draw them into a conversation about what they're seeing. A really important thing to consider, there's whole books on thinking aloud and developing your ability to facilitate usability tests. In summary, verification and validation probably started way before this phase. It's certainly, we did some in the design phase. We may have even done some in the research phase. It's important to remember that if we do that testing earlier, we're going to find those issues and fix them when it's cheaper to do so. No matter when you test, it's always valuable. You just need to decide whether it's appropriate to your design and your project. Next up, we'll go through that list of testing methods, to give you some ideas about how to get the best out of your users, working with your system. Thanks.
