Now, I'd like to talk to you about Requirements Coverage. It's not really enough that we have one test for every requirement. We want to test requirements thoroughly. And what we want to do here is we want to try and account for likely programmer mistakes. So, what I'm going to give you is an incomplete list of likely programmer mistakes for different kinds of requirements. So, let's look at some general principles. First, for each requirement, we'd like to look at both success and failure cases. So, we'd like to feed in values that are going to give you successful outcomes and also that are going to give you erroneous outcomes that are likely to occur in practice. So, like a user wishes to withdraw more money than they have in their account. We should definitely have a test case for that to make sure that we don't accidentally give them that money. And the other thing we're going to want to do is we're going to look at the structure of the types that we're using in our requirement. So, it turns out that there are things that programmers just have trouble with. For numbers, when we're dealing with integers, we want to check integer values near the limits. So, if you give this method that you're testing a MAX_INT or a MIN_INT value, is it going to cause trouble. And also we want to check floating point behavior when unusual floating point numbers are used. So, you can give it really big or really small floating point numbers. And there are also conditions in floating point numbers that involve infinite numbers. You can represent an infinite number in floats. You can also represent things that are not a number, that's kind of an odd concept. And so, depending on how robust we want our code to be, we should test it with these values. And then, again, if we have in the requirement relational boundaries, we should try and check them at, immediately above, and immediately below the boundaries that are in the requirement. So, for strings, we usually want to test things like the empty string. Is the function robust if we don't pass in any value? And also a very long string. So, oftentimes, the way our software goes wrong when we have security problems is because we have a buffer overflow, because we didn't test what the system would do when we fed it a string that's longer than we intended. And the other thing is that we often want to test strings with unexpected characters. So, one character that in C is used to terminate a string is called the null character, which is just the bit pattern 0 within one byte. And in Java, you can actually embed that in a string. And so, depending on how robust you want your code to be, you may want to test these unusual characters in your test regiment. And finally, for objects, there are a whole variety of things you might want to do but one of the things you usually want to do is check for null cases. So, what happens if I pass in a null object into this test? So, let's look at a concrete example of a bank account. Now, we have some text that describes a user story. The user shall be able to withdraw money from a savings account. The account balance will be decreased by the amount withdrawn. The account does not allow overdrafts, so an error should be returned if the amount withdrawn exceeds the account balance. So, we have this little savings account example and one of the methods on there is withdraw, and it can throw an overdraft exception if you exceed the balance. So, what would be some good tests? I want you to think about this for a while and see if you can write down a handful of tests that you think are going to adequately test this requirement that we have here, or actually, it's a user story, it's more than one requirement. And suppose we start with a balance of $100. So let me walk you through what I think would be some good tests for this. So the first is the common case. So let's say we have $100 in there. If we withdraw $50, we should have $50 leftover in the account and we should also have $50 in hand. That's harder than a normal case. But another case is a limit case. So suppose we withdraw exactly $100 and check the balance is equals to $0. So this is where that boundary value case comes in. Now, let's try and withdraw just $100.01 and check that the request fails with OverdraftException. So this is checking to see whether we've got that boundary right at the top. And then we'll withdraw $99.99 and check that the request succeeds. Then we have $0.01 left in our account. So we've now covered that boundary. And then, we can start thinking of interesting numeric cases. First, we could try and withdraw $0. And the interesting thing about this is that, off the top of my head, I don't know whether this is an error or not. So, it may be that withdrawing $0 just leaves everything unchanged and leaves you with no money. Or it could be that we consider this an illegal argument, because it's not really changing the structure of the account and it's likely to indicate an error. But this is the kind of thinking like a tester that you have to do. Another thing that should certainly fail is, I shouldn't be able to withdraw negative amounts of money. That might be a way that a user could maliciously add money to their own account. So we don't want to be able to do that. Also, we want to be able to test those numeric boundaries. So, let's pretend that, in Java we can figure out what the MAX_INT is, and we try and withdraw that much money. Sometimes, these kinds of values cause the structure of an integer to overflow its value. So, the way numbers are defined in computers, if you take the MAX_INT and you add one to it, because there's a maximum value with this which is essentially all ones, and then you add one to it, what happens is it rolls over and you get all zeros. And if you're looking at signed integers which can have positive and negative values, what happens is it goes from 0, 1, 1, 1, 1, 1, 1, 1, for lots of 1s and you add 1 to it, and then it becomes 1, 0, 0, 0, 0, 0, 0 which actually turns into the smallest representable integer. So, you go from 32,767 to negative 32,000 or something for a 16-bit integer. So, programs behave strangely when you get close to these integer boundaries. And it's a good idea to have tests that check those boundaries. And we could do the same thing with MIN_INT dollars. So, this is just a concrete example of testing a requirement thoroughly. It's beyond the scope of what we do in this course to talk about all the different ways that you can test the requirement thoroughly, but as thinking like a tester, what you want to be able to do is choose values that both indicate the common successful case and are likely to trigger a series of errors within the program you're testing.
