0:00

[SOUND] Hi, in this video,

we will overview the basic principles of TensorFlow.

What is TensorFlow?

It's a deep learning framework that we will use in Jupyter Notebooks with Python

3 kernel.

We will overview Python API for TensorFlow because it is at present,

the most complete and the easiest to use.

I should mention that APIs in other languages also exist,

like Java or C++, but they are not as complete.

0:30

So what is TensorFlow?

TensorFlow is, essentially, two things.

First it is a tool to describe computational graphs.

The foundation of computation in TensorFlow is the graph object.

It holds a network of interconnected nodes, each representing one operation.

And it gets inputs from other operations and outputs something as well.

And TensorFlow is also a run time for execution of these graphs.

You can execute them on CPU, GPU, or even custom hardware,

like Tensor Processing Unit.

You can also run it on one node or in distributed mode.

And all these is a second nature of TensorFlow, which is a run time for

execution.

So why this name?

There is one possible explanation.

You can actually see that inputs to any operation will be a collection of tensors,

which are multi-dimensional arrays.

Output will be a collection of tensors as well.

So we will have a graph of operations,

each of which transforms tensors into another tensors.

So it's a kind of a flow of tensors, right?

But who cares?

Let's get into the details, and

the first thing we need to know is how our input might look like.

There are actually three different kind of inputs.

The first one is a placeholder.

A placeholder is just a place for

a tensor, which will be fed during graph execution.

For example, it can be input features.

You already know the size of your features and the type of your features,

but you might not have them right now when you define your graph.

2:11

So you can define a placeholder for that easily.

Another kind of input is a variable.

This is a tensor with some value that is stored and

updated during our graph execution.

For example, it could be a weights matrix in our MLP.

2:35

And we also have a third type of input which is a constant.

This is a tensor with constant value that cannot be changed.

And you can define it easily as well, using Numpy or any other inputs.

Now we know how our input might look like.

We can have placeholders, variables, and constants.

Let's see how our operation might look like.

Here, we define a placeholder x of size something by ten.

We define a variable w of size 10 by 20,

which will be initialized randomly uniformly.

And we also have a third line,

which is our first operation, which has metrics multiplication in Python 3.

3:16

So that is a metrics product.

You can replace it with tf.matmul in Python 2, and you'll be good.

Now let's try to print the value of that.

What you will have in the output is just a notice from Tensor Fow that it

understands that your zed is a tensor, and it has shaped something by 20.

So the metrics multiplication is done right.

3:45

And TensorFlow actually creates a default graph just after importing.

All the operations will go there by default.

You can get it with tf.get_default_graph,

which will return the instance of tf.Graph.

You can also create your own graphs and define operations there,

just like in the example, but we will not do that, we just don't need that.

4:08

You can clear the default graph like tf.reset_default_graph, and

you need to do that because we will use Jupyter Notebooks.

And in Jupyter Notebooks, you can actually run them several times and

nobody already understands in what sequence they were called.

So let's say for example that you have this line that defines a placeholder.

4:29

If you accidentally run it three times, then you can actually see that

in your graph, you will have three operations that output these placeholders.

So your graph is effectively clustered, and

you need to reset it somehow, to clean it somehow.

For that, you need that operation tf.reset_default_graph.

Okay, now let's look at how operations and tensors are interconnected.

Every node in our graph is an operation, which means that it takes a collection

of tensors as input, and it outputs a collection of tensors as well.

It can list all nodes with get_operations, and

you will actually see that you have an operation of type placeholder.

To get output tensors of that operation, you will have to call .outputs, and

you will see a list of tensors that our operation outputs.

And here is just a tensor of size something by 10 of type float32.

5:26

But to run a graph, we'll actually need a different object.

We will need a session object that encapsulates the environment in which

operation objects are executed and tensor objects are evaluated.

To create a session, you just call tf.InteractiveSession() and

it gets you the s object that is our session.

Now let's try to define a simple graph, which has two constants, a and

b, and which does their multiplication.

If we try to print the value of c, we will just see that that is some tensor

of empty shape, which means it is a scalar and of type float32.

To actually do some execution,

you will have to run in your session that object c, that operation c.

And this is when you get the actual result of computation.

You have the value of 30 which is correct.

6:21

So why do we need a different object?

Why do we need a session object?

Because you might notice that you define your graph in Python, but for sure,

those operations will not be executed in Python because it is pretty slow.

Operations are written in C++.

And can be executed on either a CPU or a GPU, or even tensor processing unit.

A session object owns necessary resources to execute your graph,

such as variable objects that can occupy RAM on CPU or GPU.

It is important to release these resources when they're no longer required with

tf.Session.close().

But there is one problem with variables, though.

Variables has an initial value, and you can say in Python how you want to

initialize it, like random uniformly or random normally or any other way.

The problem is that you execute that variable in a different environment,

like on GPU, for example.

That means that you need to initialize its value just on that GPU.

And to do that, you need to run some code that will get that initial

value in graph execution environment.

This is done with a call in your session to global_variables_initializer.

Without it, you will get attempting to use uninitialized value errors.

Let's see at the example.

7:52

And we compute our product c.

We already know that to round this graph, we need an interactive session.

Which will own the resources and which can run it on GPU, CPU, or any other device.

Then in that session, we try to run our operation c,

and we will get the error, attempting to use uninitialized value.

This is happening because you used variable b, and

in that environment, our variable b is non-existent.

You need to initialize it, and to do that, you need to run in your session

the operation global_variables_initializer and just write it after it.

You will be able to run your operation c, and you will get the desired result.

8:35

Another example is placeholder.

Let's try and replace our a with a placeholder.

That means that we say to TensorFlow that we are expecting a tensor of size 2 by 2.

It will have a floating point type, but we don't have it right now,

just be prepared for it.

8:55

Let's try to run it.

So we create an interactive session.

We will run global variables initializer because we have a variable b in our graph

and we need to initialize it.

And we already know about that.

Let's try and run our operation c.

This time, we have another error.

You must feed a value for a placeholder tensor, it says.

So to run it properly, you actually need to feed the values that will be

put in places of those placeholders.

9:25

And you can do that with a simple Python dictionary,

which has a key as a placeholder and a value, for example, a NumPy array.

And you will get the result that you are expecting.

9:38

So to summarize this video, you should understand several things.

First is that a TensorFlow is a tool for defining and

running computational graphs, but they are run in different environments.

Nodes of a graph are operations that convert a collection of

tensors into another collection of tensors.

In Python API, you just define the graph.

You don't execute it along the way.

Although I should notice that in recent versions of TensorFlow,

there is an eager execution mode, which can do precisely the later thing.

It can execute the graph along the way, but

will not overview it because it is still in preview.

10:31

And this is why you need to know that you need to

free those resources if they're no longer required.

So that's it for the basics of TensorFlow.

In the next video, we will train our first simple model.