0:01

Hello.

Â Welcome back to the course on Audio Signal Processing for Music Applications.

Â We are in the last week of the course and we just touching some small topics

Â to basically wrap up the course, and give you some relevant

Â complimentary topics that I believe you should be interested in.

Â So for example, in this lecture I want to just review a few things of every week,

Â and highlight some of the core topics that we have been going over.

Â So that you can see kind of

Â 0:54

what we consider as the basic ideas that we cover.

Â And then we'll go through the different topics like the first one on

Â the sound spectra and basically on the DFT and STFT.

Â Then we will talk about sinusoids and harmonics.

Â Then we will go over the residual and stochastic components, and

Â the idea of modeling sound with sinusoid or component.

Â And then we'll talk about the two applications that we use

Â 1:47

this course and seeing what other topics could be continuations of these.

Â So let's go through everything.

Â And first, the idea is the spectrogram, so this is a view of

Â a spectrogram of this piano sound that we have seen in the class.

Â And this captures a little bit of the essence of what we are doing.

Â We are basically starting from this spectral representation of a sound.

Â The idea is a sound,

Â 2:17

the basic representation is the time domain waveform.

Â But for us that's really not that useful, so

Â what we are doing first is to go to the frequency domain to the spectrum.

Â And this has some perceptual motivation

Â 2:45

basically we are doing some of these analyses ourselves

Â in order then to understand better what a sound or a piece of music is.

Â So there is some perceptual motivation behind the idea that the frequency domain.

Â The spectrum of a sound is a very basic

Â view of a sound, after which we can do a lot of things from.

Â And the kind of things we have been doing can be captured with this diagram.

Â So we have been starting from the good signal, x of n,

Â 3:27

and all the analysis we have done have quite a bit of these.

Â Basically the idea is to start with the Fast transform,

Â the FFT and then obtain the peaks from the spectrum.

Â Out of that we can obtain the partials, the harmonics of a sound.

Â And these can be subtracted from the,

Â basically the regional signal, and obtain this residual.

Â Okay, so this is the basic analysis that

Â a lot of the things we have been doing go thru.

Â And out of that we can obtain interesting features.

Â So we can do feature analysis to obtain the fundamental frequency we can obtain

Â some ways of describing some sounds, but we can also transform these features.

Â We can transform the sinusoid.

Â We can transform this residual, and obtain a new sound, a synthesized sound

Â 4:22

that can be a modified version of the input sound.

Â Or if we don't make any transformations,

Â ideally, it should be very similar to the input sound.

Â So this is the kind of that basic framework within which we have elaborated

Â all our analysis description and synthesis techniques.

Â So let's now go through some of these individual aspects of all this framework.

Â The first one was the idea of a spectrum of a sound and the idea that we're

Â going to start from a sound and obtain a spectrum, and we had two variants.

Â We had the single frame version basically.

Â And that's what we call the Discrete Fourier Transform in which we just analyze

Â a fragment of a sound or a sound that is very short and obtain a single spectrum.

Â And then we went over the time variant version of that,

Â which is the short-time Fourier transform.

Â So that instead of having a single spectrum, we have a sequence of spectra,

Â and that's the X sub l of k, which is

Â this idea of time varying frequency representation of a sound.

Â And you can see that is the first model that was useful for us.

Â The first analysis synthesis model that could capture any sound.

Â In fact, this was an identity system, therefore we could analyze and

Â synthesize any sound.

Â 6:14

Strictly speaking the DFT is also a sinusoidal model, but

Â this sinusoidal model is a little bit different.

Â These are the idea of stable sinusoids,

Â of sinusoids within a sound that have some stability,

Â some coherence and that are really representing something

Â meaningful from the acoustics of the sound.

Â And that was the sinusoidal model.

Â And the harmonic model was a step beyond that in the sense of there is

Â quite a large family of sounds that these sinusoids have a harmonic relationship.

Â So the harmonic sounds have a series of harmonics, but

Â there are multiples of fundamental frequency.

Â So we can use that restriction so

Â if we have sounds that have that type of behavior,

Â then the analysis can be done in a more restricted way and

Â we can obtain a much more powerful representation.

Â The idea of this harmonic model allows us to

Â represent a sound in a very compact way and

Â at the same time have a lot of potential for describing and transforming the sound.

Â 7:33

Then we saw that these sinusoidal,

Â these harmonic components of sounds do not capture everything in the sound.

Â There is a part of the sound that is left out, and

Â this is the residual or it's the overcasting component.

Â So when we have analyzed the sinusoids of the harmonics of the sound

Â We can actually subtract them from the original sound and obtain residual,

Â that is what is left and is sometimes quite relevant.

Â Sometimes that's a very small part of the sound that can be discarded,

Â so it's not perceptually relevant.

Â But in many cases,

Â this is an important part of the sound that needs to be preserved.

Â It needs to be captured.

Â And we can just capture ICTs and that will need to residual component or

Â we can modulate with the stochastic model, with the idea of filtered white noise.

Â So in the bottom, a representation the residual is approximated with

Â this idea of a time bearing filter, Through which we put white noise.

Â So we have this complete model of Sinusoidal plus stochastic components,

Â and that captures many sounds.

Â Not all the sounds are properly modeled this way.

Â But, quite a large family of sounds either sinusoidal plus stochastic or

Â harmonic plus stochastic can be used to model many sounds.

Â And that then yields many potentials for

Â capturing the essence of the sound or being able to modify the sound.

Â And that what brought us to the idea of transforming sounds.

Â When we have these type of representations We have these harmonics or

Â these sinusoids.

Â And we have the frequencies, the amplitudes, and the faces, and

Â they can be processed.

Â They can be manipulated quite a lot.

Â We can change quite a bit their values, and the stochastic component too.

Â And things like time is stretching or a shift in the frequencies or

Â doing arbitrary changes.

Â In fact, in the class we went over some common transformations, but

Â there are many more that we could do that go beyond what we cover in class.

Â 9:56

So apart from the transformation of sounds,

Â we also talked about describing sounds.

Â This is a huge field.

Â In fact, we only talk about a small part of it.

Â The concept of describing sounds cover a wide variety of abstractions.

Â And we mentioned we use this diagram to show the different levels of

Â obstruction or description of sounds.

Â We stayed very much in the low levels of descriptions and the physical sensorial,

Â and some perceptual type descriptors that

Â are useful to describe sounds and music signals.

Â But this is a very interesting area of application of this spectral analyses

Â towards this imagined broader feel of describing

Â sounds and music that are not just a collection of frames.

Â But there is a complete pieces of music and complete music collections and

Â the type of problems are quite different from what we taught before.

Â But anyway, so that was a good introduction to describing sounds and

Â music using the techniques we have been talking the previous weeks.

Â And then finally, in the last lecture we just kind of open up

Â a door of saying okay, and what's beyond that, and beyond that there is a lot.

Â There is a lot within the audio signal processing field.

Â So audio signal processing is much more than what we have been looking at, and

Â you can explore many other methodologies to analyze, describe and transform sounds.

Â And even more than that, sounds and music, which is our target,

Â a kind of information that we are trying to understand.

Â It's much more than audio.

Â So we just hinted at the idea that we can use many other sources of information

Â to analyze describe sounds in music.

Â And that's a very new area that is being explored in the last few years,

Â and that opens up very interesting new application and development areas.

Â 12:17

And that's all, so all the slides are in the SMS Tools.

Â And hopefully that was just a very brief summary

Â of some highlights of the lectures that we went through the course.

Â And maybe that help you understand some of this overall view of the course.

Â And how we see the coherent of the topics that we covered.

Â So, thank you very much and I will see you in the next lecture.

Â Bye bye!

Â