Explore
For Enterprise
Join for Free
Log In

Artes e Humanas
Negócios
Ciência da Computação
Ciência de Dados
Tecnologia da informação
da saúdesaúde
Matemática e Lógica
Desenvolvimento Pessoal
Ciência e Engenharia Física
Ciências Sociais
Línguas
Notas
Certificados

Explore todo o Coursera

Navegar
Pesquisar
For Enterprise
Entrar
Inscreva-se gratuitamente

A Generic Model

Loading...

Internet das Coisas: Tecnologias Multimídia

Universidade da Califórnia, San Diego

4.4 (92 classificações) | 13K alunos inscritos

Curso 5 de 6 no Internet das Coisas — IoTInternet das coisas - IoT Programa de cursos integrados

Content is an eminent example of the features that contributed to the success of wireless Internet. Mobile platforms such as the Snapdragon™ processor have special hardware and software capabilities to make acquisition, processing and rendering of multimedia content efficient and cost-effective. In this course, you will learn the principles of video and audio codecs used for media content in iTunes, Google Play, YouTube, Netflix, etc. You will learn the file formats and codec settings for optimizing quality and media bandwidth and apply them in developing a basic media player application. Learning Goals: After completing this course, you will be able to: 1. Explain the tradeoffs between media quality and bandwidth for content delivery. 2. Extract and display metadata from media files. 3. Implement and demonstrate a simple media player application using DragonBoard™ 410c.

Visualizar o programa do curso

Avaliações

4.4 (92 classificações)

5 stars
62 ratings
4 stars
17 ratings
3 stars
4 ratings
2 stars
6 ratings
1 star
3 ratings

JC

Apr 19, 2018

excellent, I like the digital image processing, and the use of different free software tools for it.

MI

Mar 03, 2017

So much I have learned in this course and so thankful to UCSD and Coursera.

Na lição

Codecs

In this module our esteemed Professor Harinath Garudadri will talk about coders and decoders (Codecs). This will allow us to make better use of our multimedia choices when working with the DragonBoardTM 410c. We want to look at the motivation behind using Codecs, the different ways to take advantage of redundancies when using codecs and finally the ability to take advantage of different receiver / transmitter combinations. If we are able to understand the way that information is sent and received over the data plane we can create and use the right codecs.

Introduction to Lesson 10:57

A Generic Model12:29

Audio Processing7:28

Image Processing6:59

Video Processing6:41

Quality Evaluation6:52

Ministrado por

Harinath Garudadri
Associate Research Scientist
Ganz Chockalingam
Principal Engineer

Transcrição

So I will start with a generic model for multimedia processing.

[COUGH] Before we get to that,

you would have already realized that communication of data is very expensive.

Either we use a dedicated circuit switch network or on the internet.

We saw in the previous course that wideband speech can take up to 224 kbps.

In the CD quality audio, the sampling rate and the bits per sample for

the increased dynamic range we have in CD quality audio,

we need about 1.4 megabits per second.

Usually the payload rates are higher when you increase the,

include the overhead, like [COUGH] forward data correction, markers for

random access in the media, so on and so forth.

So let's just look at the content itself in the payload just so

that we can keep things apples to apples.

For still images, there are three components.

Red, blue, and green streams that come from typical camera sensors.

Each of them requires eight bits per pixel.

So, if you have a 1024 by 1024 image, we need

3.14 megabytes for each image.

In the case of video, we have similar images coming from the sensor

attached at 24 frames per second, 30 frames per second, or

even 60 frames per second in some of the new handsets.

So you can see that the data rates will very quickly reach gigabits per second.

So these rates are not really compatible with the Internet services.

And we need to compress the data either for transmissions, storage and retrieval.

So a generate block diagram that does this, [COUGH] is this.

It leverages some of the tools we already discussed in the previous course.

So for now, ignore the yellow blocks and

follow the signal from source to destination.

The time frequency decomposition block push the signals in a space

where we can take advantage of known properties of the signal.

The inverse time frequency decomposition typically results, if you do things

correctly, in perfect reconstruction of the original signal, if we do

not have any of the intermediate processes that are shown in the block diagram.

In the case of voice, this was LPC analysis where we took advantage

of the known properties of the source that is the human speech production system.

This is not so in the case of image and

video processing systems as we will see shortly.

So in the case of audio, we actually take advantage of

known properties of the sync, that is the human auditory system.

What exactly do we listen to when we are listening to a CD?

So the time frequency block itself does not give us any data rate reduction.

It is the quantization block that gives us more steady reduction in the data read.

The dotted feedback path in the receiver represents

techniques such as backward adaptation we saw in the waveform coding,

such as g.726 in course four.

We also saw in the previous course that not all patterns that are current

there in the text are the process data that comes with the Quantizer.

They all have uniform distributions just like the English text.

So Entropy Coding provides additional reductions in the data rate

by taking advantage of this non-uniform distributions.

The RTP block provides for real time transmission or

the internet for storage and retrieval.

There are similar protocols such as TCP/IP, HTML streaming, etc.

The feedback path provided by RTCP around the communication block,

it's very, very useful as we saw in the voice case.

This becomes even more important for the video streaming case,

because every short channeler has significant impact on the video quality.

Now, let's take a look at the yellow blocks on the sender side.

When there is temporal or spacial or

spectral redundancy in the signals, we can actually

predict what the next frame or the next sample is going to be.

And then we can subtract this from the incoming data

to reduce the bit rate further.

This is what we did in the CELP case, that is the Code-excited linear prediction.

And there are similar things that we can do for the other types of signals.

So just to recap, here is the spirit of processing we do at the end quarter.

The time-frequency decomposition puts the signal in a different space,

so that we can take advantage of the redundancy in the signals and

ignore the irrelevant parts.

These are the two operative words.

Redundancy and irrelevancy using our prior knowledge of the signals.

For redundancy, think of a pure tone.

It has all the samples in the time domain but you need only

two samples to represent it very compactly in the frequency domain.

Similarly if you have short transients in the time domain

you get to have very wide spectral distribution.

[COUGH] So you can have very compact representation in the time domain or

transient and very compact representation in the frequency domain for

tonal sounds [COUGH].

For the irrelevancy part, the more you know about the signals,

the source and the sync, the more irrelevant parts that we can

identify and take advantage of them as we do this process.

And as we discussed before, the quantization step implements

a lossy bitrate reduction scheme.

We simply assign fewer bits to the parts that are less

relevant based on what we already know.

Then comes the Entropy Coding.

There are many types of entropy coding systems that we can leverage

in the pipeline.

But, for now, I want you to realize, recognize, that the entropy coding

is a loss-less compression and quantization is a lossy compression.

[COUGH] So let's look at some of the options we have for

the time frequency decomposition.

We already saw LPC in the previous course.

Other than that, the short-time fourier transform is the basic work hours

because it doesn't make any assumption about the source or the sync.

The picture at the bottom left shows the tiling of components in the STFT.

The resolution in the time domain along the x axis [COUGH] and

the frequency domain on the y axis.

The magnitude in each of the ties in this representation

is the intensity which is same as this pictogram pictures we saw before.

The other common technique that is used a lot in the visual processing

is the Gabor Transform proposed by Dennis Gabor in the late 40s.

This is very much similar to STFT except it uses Gaussian

window in the time domain before taking FFT on each of the sharp segments.

We call that, if you use short windows in the STFT, we have very good

temporal resolution, but this results in a very poor spectral resolution.

On the other hand, if we want very fine spectral resolution,

we need to have very long time domain segments.

And this will end up at very poor temporal resolution.

This is actually related to Heisenberg's uncertainty

principle as applied to signal processing.

In the Gabor transform, because we use a Gaussian window in the time domain,

it is also a Gaussian window in the frequency domain.

So, we have the most compact representation

in this class of time-frequency representations.

The next one that you'll come across a lot in JPEG, JPEG 2000,

and other video processing is the Wavelet Transform.

It uses a different basis functions, as opposed to Short-time Fourier, or Gabor.

Instead of the exponential functions with constant window length anywhere in

the tiling, we use much shorter windows

in the time domain for higher frequencies.

This results in very fine spectral resolution in the lower frequencies and

very fine temporal resolution in the higher frequencies.

In terms of that disease very similar to the human perception

of audio and video signals.

I just realized that the figure on the bottom right

does not quite show the varying frequency resolution in the Y axis.

I should remember to re-draw this in future.

I do encourage you to take a look at the links here, at least even very briefly,

to see if these are the topics you want to explore in future,

because each one of them is very big.

It can be a complete course or a lesson by itself.

So, I put, for

the entropy coding, two links up here for you to do some further digging.

[COUGH] The Huffman coding,

we talked about it in course four with respect to the English text.

It is essentially a variable length code and

it is computationally extremely simple.

So it has been used in many, many practical implementations when

the processing in the mobile platforms was still at very premium.

So, simple example.

Let's say we have four symbols, a1 through a4, with probabilities of occurrence and

given some corpus, sorted in the descending order.

So, you need a large representative database to generate these

probabilities for all the symbols and real systems but let's just look at this form.

In the process of building the Huffman tables,

what happens is a1 gets assigned the shortest code word, it is 0 in this case.

[COUGH] The Huffman is also a prefix code.

This means that a2, a3 and all rest of the code works.

They shall not start with the code word for a1.

They all have unique prefix.

So as you go to lower probability symbols, we have to find longer code words

that do not have the earlier code words as the prefix.

Explore nosso catálogo

Registre-se gratuitamente e obtenha recomendações, atualizações e ofertas personalizadas.

O Coursera proporciona acesso universal à melhor educação do mundo fazendo parcerias com as melhores universidades e organizações para oferecer cursos on-line.

© 2019 Coursera Inc. Todos os direitos reservados.

Baixar na App Store

Baixar no Google Play

Coursera
Sobre
Liderança
Carreiras
Lista de cursos
Certificados
Certificados MasterTrack™
Notas
For Enterprise
For Government

Comunidade
Learners
Partners
Developers
Beta Testers
Translators

Conectar
Blog
Facebook
LinkedIn
Twitter
Instagram
YouTube
Tech Blog

Mais
Termos
Privacidade
Ajuda
Acessibilidade
Imprensa
Contato
Diretório
Afiliados