Here are a few quotes on why machine learning is so important today.
Bill Gates said "If you invent a breakthrough in artificial intelligence,
so machines can learn,
that is worth ten Microsofts."
Sundar Pichai, the CEO of Google,
said "Machine learning is a core,
transformative way by which we're re-thinking how we're doing everything."
Finally, Tom Dietterich said
"Computers are already more intelligent than humans on many tasks,
including remembering things, doing arithmetic,
doing calculus, trading stocks and landing aircrafts."
Despite all the promises of AI,
there are some people however,
that fear that AI will bring about the rise of robots that will take over the world.
This hype comes from Hollywood movies,
from reporters hyping a story to get viewers and from
scientists wanting attention to their work.
The reality is paraphrasing and ranging that worrying
about evil robots is like worrying about the overpopulation of Mars.
Instead, as a society we should worry about the need to re-educate
people whose jobs will disappear as AI advances.
For example, autonomous vehicles will likely make transportation safer,
more comfortable and efficient.
But will also reduce the demand for jobs in the transportation sector.
Other sectors will similarly be affected.
And as a society,
we will be better off by preparing for these economic restructuring.
Okay, enough with the public policy discussion.
Now, let's look at the types of machine learning algorithms and
the differences and similarities between classical machine learning and deep learning.
Machine learning, which includes deep learning,
can be broadly split up into four categories;
supervised, unsupervised, semi-supervised, and reinforcement learning.
While there is active research in all areas,
supervised learning is the most common type used in industry.
Supervised learning maps input data to labels and therefore records label data to train.
Unsupervised learning attempts to discover patterns in unlabeled data.
One of the reasons for the lack of wide adoption of unsupervised learning,
is that it is often easier to spend a week or two labeling
data than months of research to get your unsupervised algorithm to deliver.
Semi-supervised learning uses a combination of
both labeled and unlabeled data to train to model.
Reinforcement learning, is used to teach an agent
to perform certain actions based on rewards.
However, it usually takes time to get the reward for a given action.
In this course, I will refer to machine learning algorithms
that are not deep learning as classical machine learning algorithms.
Let's explore the differences between classical machine learning and deep learning.
We will use the task of image classification.
In classical machine learning,
you have an image with N by N pixels.
Can you engineer a set of K features,
where K is typically much less than the dimensions of the image N squared.
These features are selected by a domain expert.
For example, to design a car classifier,
an expert may decide to select the ratio of
the height versus length or the number of circular regions in the image.
I am making up these features.
In practice, a computer which an expert would use more advanced features.
Each image is map to a K-dimensional feature representation.
Then you apply your varied algorithm such as SVM or logistic regression to
learn to associate these patterns of features with an identity vehicle.
I'm showing various machine learning algorithms.
And it's okay if you don't know what they are.
The K features that are selected should be unique for their particular object.
In these two example,
we have selected two features that is K equals
two for each image from five different classes;
vehicles, animals, faces, fruits, and chairs.
Each class is clustered together and separated from other classes.
There is these two features that the expert selected are very good features.
This is a made up example, in practice,
two features are typically not sufficient.
Supervised learning using the machine algorithms listed here,
allows for the determination of decision boundaries.
However, in practice choosing good features is very challenging.
It may be better to learn these features.
This is one of the benefits of deep learning.
Deep learning is End-to-End,
meaning that we pass the end by end pixels directly to
an deep network and we provide the desired output labels.
The data is passed through a series of
linear and non-linear transformations that
map the data to a distribution over all the labels.
The weights of these transformations are learned during the training phase.
These weights can also be called feature extractors.
That is, we can view a deep network as a series of
feature extractors that extract features of higher complexity,
the deeper in the network you are.
The network learns to extract good features without the need of a domain expert.
This marks a conceptual shift in thinking from how do we engineer the best features
to how do we engineer a model that will find the best features automatically?
Now, that we have compared deep learning to classical machine learning,
let's discuss some common deep learning frameworks that
are used to construct deep learning models.
There are a number of common frameworks,
you can think of these frameworks as libraries for deep learning,
that facilitate designing deep learning models.
They hide the low-level implementations from the deep learning practitioner.
Frameworks such as Neon, Caffe2,
and Caffe are well suited for industry due to their stability, scalability, and speed.
Frameworks such as the Theano,
PyTorch and Torch7 are well suited for
research-based applications due to their flexibility and ability to be debugged.
Frameworks such as Tensorflow,
and MXNet attempt to call both industry and research domains.
There are other popular frameworks such as CNTK, PaddlePaddle, Chainer,
Keras, Lasagne, BigDL, and DL4J.
While in this course,
we will focus on Neon,
the concepts are general and apply to various frameworks.
Neon is an open source framework available on github.
It is designed for speed in both design and performance.
Benchmarks such as the common end benchmark shows that
Neon is the fastest framework for a variety of workload.
Intel is committed to
Neon's performance leadership across all the main hardware platform.
Neon is built on top of Python and optimized at the low level with C++,
Guta C++ and SAS.
Python is fast to prototype and allows Neon users to
take advantage of the rich ecosystem of existing packages.
Training a model only requires the six steps shown here.
Once you import the right libraries,
the lines of code shown are sufficient to train a simple model.
First, specify the backend.
For example, CPU.
Second, load the data.
For example, load the MNIST data set.
MNIST is a popular data set with images for the digits zero through nine.
Third, specify the model architecture.
In this case, a two layer network with each layer having ten units.
Fourth, define the training parameters.
Fifth, train your model.
Then six, evaluate the model.
In the next lecture, we'll study the various units,
layers, and hyperparameters that needs to be defined.
A common question is how do you choose a model?
Deep learning practitioners often rely on models from Babylonian researchers as a basis.
That is, you find an existing model and a hyperparameters that
work on a similar problem and adapted to work for your task.
Here are the various functions supported by Neon.
The list continues to grows.
Neon supports various backends,
can easily done or various datasets,
offers various techniques to initialize
the model weight and various optimizers to train the weights.
A model in Neon can use various activations and layers
as well as various cost functions and metrics to evaluate the results.
I encourage you to download and look through the provided examples.
In the first exercise,
you will become familiar with the MNIST dataset, the Gaussian initializer,
the Gradient Descent with Momentum Optimizer,
and you will use Rectified linear unit activations,
linear layers, multiclass cross entropy cost, and accuracy metric.
Today, we started by looking at some applications
of deep learning including image and speech recognition.
And then, discuss the history of
deep learning and how it compares to classical machine learning.
Finally, we looked for
some common deep learning frameworks and gave a brief introduction to Neon.
In the next lecture,
we will look at deep learning models with a more detailed lengths by
discussing the architecture and hyperparameters of a deep learning model.
