In this video, we're going to look at another way of storing tabular data in Python using nested dictionaries. Now, dictionaries have keys and values, so in this method, instead of just organizing the data spatially, we're going to have to know the names of the rows and the names of the columns. But if you know those, you can then structure your data as a dictionary. So, let's take a look at how we can do that. In order to talk about how to store our tabular data as nested dictionaries, we need some tabular data. So, in this video, I want to create a table of the software products with the most vulnerabilities. So my tabular data here is the Top 10 software products with the most vulnerabilities in 2017. Now, each row in my table is going to be a particular software product. And then I'm going to have four columns in my table. The first column will be the name of the product. The second column will be the vendor or whoever makes the product. The third column will be the type of the products such as an operating system or an application and so on. And then the final column will be the number of vulnerabilities that that piece of software had in 2017. All right. So you can see here my vulnerabilities2017 dictionary. You can see the open curly brace. It's big enough that you can't see the close curly brace here, but trust me it's there. And then I have a bunch of key value pairs. So let's just take a look at the first one here. The key is Android. That is the name of the product. It's Android. And then the value here is another dictionary. And so that dictionary is going to have keys and values as well, where the keys are the names of the columns and the values, well, are the value for that column. So the key in the outer dictionary is essentially the first column, it's the name of the row. And then in the inner dictionary, the keys are the names of the columns themselves, and the values are the value for that row in those columns. Now, I want you to realize that we have potential issue here. We don't have this column stored explicitly. The columns are actually in dictionaries that are values inside the outer dictionary. And so if we have those inner dictionaries have different keys than each other, it's going to be tough to figure out what exactly the columns are. So in order for this structure to work, you really need to know the names of the columns ahead of time, and you have to be disciplined about making sure that all the names of those columns are keys in those inner dictionaries. And that you don't mess up and some of them have different keys than others. You might want that in situations where you want to leave out data for a particular column for one row where it will have a subset of the columns, but you need to have a sort of understanding of what are the possible names of the columns in the entire table, and make sure those are what are used as the keys in inner dictionaries. Otherwise, it's going to get a little bit difficult to use. So, let's try to print this table out. So let's scroll down. Look. There's that closing brace. I told you it was there. So here's a piece of code that will allow me to print this dictionary out. First, I'm going to print a header or I'm going to print the names of the columns. And I can do this because I already know what they are. If I'm constructing this dictionary, I have got to know what they are. I'm going to print the nice line there so that it looks all nice. And then I'm going to iterate over the dictionary. So I'm going to iterate over the items of the dictionary because I need both the keys and the values. So, I'm going to call those products and values. Then for each product, I need to construct the row that I want to print out. So again, I'm going to use a format string and I'm going to format the product and then the three columns that are stored within the inner dictionary, the vendor, the type and the number. And then I can print that row out, and I should get a nice table. So let's do this. And here we go. Here, you can see that it does look like a table, everything looks great. I was able to print the headers, and then I was able to print each row. And you'll notice that the rows print nicely in decreasing order of the number of vulnerabilities. However, that's just luck. As we've talked about before, dictionaries when you iterate over them, you are not guaranteed to get the elements in any particular order. I've could have gotten these back in any order, and then when I printed them out, I might have had a problem if I really wanted it to be ordered in decreasing number of vulnerabilities. If you wanted to do that, you'd have to actually write some code here to figure out how to sort the rows and print rows out in the way that you wanted, which could get complicated if you have a lot of rows. Now, often, it's not that you want to be able to print out your tabular data nicely, you just want to work with it. And so let's see how we might do that if we want to access elements of our table with this structure. Well, I'd be surprised to realize, it's actually pretty easy. If I know the names of the rows and the names of the columns, I can just use them to index directly into my dictionary of dictionaries. So you can see here, I want to figure out how many vulnerabilities that Windows 7 have. Okay. Well, I index into my vulnerabilities2017 dictionary first with the product name, Windows 7. That gets me the correct row. And then I index with the column name number which will give me the column. And so I should be able to immediately print out the number of vulnerabilities that Windows 7 had. And I can, it's a 174. So contrast that to other things that we've looked at. If I were storing this as the list of lists, that would have been a little bit more complicated, unless I knew the indices of the row and the column I was looking for. Otherwise, I had to write some code to figure that out. And let's take a look at how I might actually do some other things here. Let's say I wanted to figure out which product has the most vulnerabilities. Well, let's loop over the elements of the number of column and figure out which one has the highest number, and then I can print that product out. So I'm going to keep track of it. I'm going to call it maxproduct and maxnumber. As I start out maxproduct as none. I don't know which product has the most vulnerabilities. Maxnumber number as minus one. And now, I can loop over all the items in the dictionary, the products and the values again. And I just check. If values bracket number is greater than maxnumber, I found a product that had more vulnerabilities than I found before. I saved that away by saving the name of the product in maxproduct and saving the number of vulnerabilities that product had in maxnumber, and once I complete the iteration, I'm guaranteed to now have figured out which product had the most vulnerabilities. And I can print that out. Let's take a look to see where are in the most vulnerabilities. It happened to be Android with 564 security vulnerabilities. So hopefully this gives you a feel for how you could use nested dictionaries to store your tabular data, and what you might need to do if you want to operate on that data. So now you've seen how we can use nested dictionaries to store tabular data. There are a couple of caveats here right. The first one, I need to know the names of the rows and the names of the columns. I have to use those as keys in dictionaries. I need to use the row names as the keys in the outer dictionary and the column names as the keys in the inner dictionary. And you can get yourself into trouble here because there's a bunch of inner dictionaries. You have to make sure that you're using the same column names in each inner dictionary. If you don't, well, you can probably imagine what's going to happen. Think about the code that we wrote to find who had the most vulnerabilities. Well, what if not everybody called that number? Or what if some of those inner dictionaries were missing the key number? Then you'd have a little bit more tricky time actually being able to manipulate the data. The other thing that we need to worry about is the fact that dictionaries do not have an order. So this is really only useful when you're not really concerned about the order of the rows, and maybe even the order of the columns, because it's not going to preserve it for you if you want to like actually printed out when you iterate over those dictionaries, you're going to get things back in any order that Python decides to give them to you. And so it may not even print out the same way across multiple runs of your program. But as long as you're willing to deal with that, dictionaries have this nice property where it's very easy to index into them and find the value of a piece of data somewhere inside of your table. All you need to know is the row name and the column name, and there you go. So now you've seen there are trade-offs here. Seen to multiple different ways of storing tabular data in Python. Hopefully, that gives you a feel for the kinds of things you need to think about.
