Let's do a fast review of the steps involved when doing machine learning on GCP. These are the steps involved in any machine learning project, but we'll focus on doing them with Google Cloud Platform Tools. Like most software libraries, TensorFlow contains multiple abstraction levels, tf layers, tf losses et cetera. These are high level representations of useful neural network components. These modules provide components that are useful when building custom neural network models. You often don't need a custom ML model, the estimator API is a high level API. It knows how to do distributed training, it knows how to evaluate, how to create a checkpoint, how to save a model, how to set it up for TensorFlow Serving. It comes with everything done in a sensible way that fits most ML models in production. In this course, we will work with TensorFlow at the tf estimator level of abstraction. Cloud ML Engine is orthogonal to this hierarchy. Regardless of which abstraction level you're writing your code at, CMLE gives you a managed service for training and deploying TensorFlow models. If you have data that fits in memory, pretty much any machine learning framework will work. Once your data sets get larger, your data will not fit into memory and you need a more sophisticated performant ML framework. This is where TensorFlow comes in. You can use TensorFlow estimators not just for deep learning but also for things like Boosted Regression Trees. But as we discussed in the first specialization, there are ways to architect deep neural networks so that they get the benefits of bagging and boosting, and so we will simply focus on one technology, Neural Networks. In real-world problems, there's often very little benefit to a different machine learning algorithm. You should spend any time and resources you have on getting better data. But as we said, many machine learning frameworks can handle toy problems. But what are some of the important things to think about when it comes to building effective machine learning models? The first and most important thing is that you need to figure out a way to train the model on as much data as you can. Don't sample the data, don't aggregate the data, use as much data as you can. As your data size increases, batching and distribution become extremely important. You will need to take your data and split it into batches, and then you need to train, but then you need to also distribute your training over many machines. This is not as simple as MapReduce, where things are embarrassingly parallel. Things like gradient descent optimizations are not embarrassingly parallel. You will need parameter servers that form a shared memory that's updated during each epoch. Sometimes people think they can take a shortcut in order to keep the training simple by getting a bigger and bigger machine with lots of GPUs. They often regret that decision because at some point, you will hit the limit of whatever single machine you're using. Scaling out is the answer, not scaling up. Another common shortcut that people take is to sample their data, so that it's small enough to do machine learning on the hardware they happened to have. They're leaving substantial performance gains on the table if they do that. Using all the available data and devising a plan to collect even more data is often the difference between ML that doesn't work and machine learning that appears magical. It's rare that you can build an effective machine learning model from just the raw data. Instead, you have to do feature engineering to get great machine learning models. So, the second thing you need to build effective machine learning is that you need feature engineering. Many of the major improvements to machine learning happen when human insights come into the problem. In machine learning, you bring human insights, what your experts know about the problem in the form of new features. You will need to pre-process the data, you will need to scale the data, encode it et cetera, and you need to create new features, and you need to do these two things on the large dataset, and it needs to be distributed, and it needs to be done on the cloud. The third thing that you need for effective machine learning is to use an appropriate model architecture. Different types of problems are better addressed with different types of models. For example, if you have a text classification problem, you want to be able to use CNNs and RNNs, things that we will look at in this specialization. This is where TensorFlow comes in. TensorFlow is the number one machine learning software repository. We, Google that is, we open-sourced TensorFlow because it can enable so many other companies to build great machine learning models. TensorFlow is highly performant. You can train models on CPUs, GPUs, TPUs, et cetera. Another advantage, you're also not locked in when you work with Cloud ML on GCP because the code that you write, TensorFlow, is based on open-source. So, why use TensorFlow? Because it can work with big data, it can capture many types of feature transformations, and it has implementations of many kinds of model architectures.
