In this class you will learn the basic principles and tools used to process images and videos, and how to apply them in solving practical problems of commercial and scientific interests. Digital images and videos are everywhere these days – in thousands of scientific (e.g., astronomical, bio-medical), consumer, industrial, and artistic applications. Moreover they come in a wide range of the electromagnetic spectrum - from visible light and infrared to gamma rays and beyond. The ability to process image and video signals is therefore an incredibly important skill to master for engineering/science students, software developers, and practicing scientists. Digital image and video processing continues to enable the multimedia technology revolution we are experiencing today. Some important examples of image and video processing include the removal of degradations images suffer during acquisition (e.g., removing blur from a picture of a fast moving car), and the compression and transmission of images and videos (if you watch videos online, or share photos via a social media website, you use this everyday!), for economical storage and efficient transmission. This course will cover the fundamentals of image and video processing. We will provide a mathematical framework to describe and analyze images and videos as two- and three-dimensional signals in the spatial, spatio-temporal, and frequency domains. In this class not only will you learn the theory behind fundamental processing tasks including image/video enhancement, recovery, and compression - but you will also learn how to perform these key processing tasks in practice using state-of-the-art techniques and tools. We will introduce and use a wide variety of such tools – from optimization toolboxes to statistical techniques. Emphasis on the special role sparsity plays in modern image and video processing will also be given. In all cases, example images and videos pertaining to specific application domains will be utilized.
Wiener v.s. Constrained Least-Squares Restoration Filter
Loading...
Do curso por Northwestern University
Fundamentals of Digital Image and Video Processing
899 classificações
Na lição
Image Recovery : Part 2
In this module we look at the problem of image and video recovery from a stochastic perspective. Topics include: Wiener restoration filter, Wiener noise smoothing filter, maximum likelihood and maximum a posteriori estimation, and Bayesian restoration algorithms.
Conheça os instrutores
Aggelos K. KatsaggelosJoseph Cummings Professor
Department of Electrical Engineering and Computer Science
Hello, and welcome back.
In this segment we compare the Wiener filter we obtained in the previous segment
with the Constrained Least Squares filter we derived last week.
We'll see that the mathematical expressions differ only in one term,
the so called stabilizing term.
So, if instead of measuring the image and
noise power spectra from the available noise image, we model them instead.
So we use a specific model for the power spectrum of the image as a function of
the stabilizing term that was elected by the CLS filter and
also we use a specific model for the power spectrum of the noise.
We assume it's wide and Gaussian.
Then the results of the two filters, of the Wiener and the CLS filter,
become identical.
We utilize the same simulated degraded image we used extensively last week, to
point out the similarities and differences between the Wiener and the CLS filter.
So let us proceed with the coverage of this material.
Here is the expression again of the frequency response of the noncausal Wiener
restoration filter we just derived.
I have divided numerator and denominator by P of ff and
this is the form we end up with.
A common assumption is that the noise is white,
Which means its power spectrum is flat, and more specifically,
it's equal to the variance of the noise.
So in this case, this is the form of the Wiener Restoration Filter.
Let us compare it now with another filter we
derived a bit earlier in the course which is the Constrained Least Squares filter.
So this is now the frequency response of the CLS filter.
By comparing the two, it's crystal clear that they're very similar.
And the differences, let's look at this for between this term and this term.
It can be shown that the power spectrum is real and more specifically,
it's not negative.
So P of ff omega 1, omega 2 is greater than or
equal than 0 for all omega 1, omega 2.
So therefore, see that the alpha
is equal to the variance of the white noise.
And then the magnitude of this
filter here is there is one over the spectral
density of the image, then the two filters become identical.
Clearly, when alpha is 0 and the noise variance is 0,
both filters reduce to the square's filter,
or generalized inverse filter that we talked about.
So, one way to think about is that for the CLS, I choose C, and alpha is shown here,
and then the CLS gives me exactly the same answer as the Wiener filter.
Or going the other way around, you might model the auto-correlation, and
therefore the power spectrum density of the image in such a way that at the end,
this equation down here holds, and you get then a CLS filter.
Conceptually, however, both filters strive to achieve the same result.
By that, I mean if you recall C omega 1, omega 2 is a frequency response,
is the magnitude here, of course, of a high pass filter.
And the same is depicted by 1/P of ff, omega 1, omega 2.
The other correlation for most images has this decaying
exponential shape that I discussed earlier.
And then the fourier transform of that signal is also a low pass type
of signal in the frequency domain then 1 over this is a high pass type of filter.
So again, what we derived here is at the high level,
some equivalence you might say between the Wiener and the Constrained Least Squares.
And then more specifically, if the parameters involved is chosen in a way
shown here and explain, then they give us exactly the same result.
But in general, they try to achieve the same objective which is to
regularize and make the high frequencies of the the inverse
restoration filter be tapered off, and not become very high.
Because this will introduce noise amplification.
Of course we saw that the mechanics, the paths we followed to obtain the Wiener and
the Constrained Least Squares filters are distinctly different.
Let us look at some experimental results now.
We show here the power spectrum of the camera and image.
The 0,0 value is quite high, and it's outside the scale of the figure.
And it has this decaying nature that we mentioned earlier,
and it's also no negative.
Here we show the spectral density of the noise is white noise,
therefore, its spectrum is flat.
This spectrum is not computed from the image.
But since it's a synthetic experiment,
we do know exactly what the variance of the noise is.
And that's what we show here.
We show here what is often called the stabilizing term.
So this is the term of the denominator of the filter, the Wiener or the CLS.
That is added to the square of the frequencies responsible
the degradation system.
So for the Wiener filter is the tab of the power spectrum of the noise
divided by the power spectrum of the image.
The noise in this example is white, so
this is equal to the variance of the noise, which of course is the numerator.
It's rather maybe hard to see exactly the shape of this filter here,
but it's high pass in nature.
It's a constant divided by the spectrum of the image, we saw that it has
this decaying shape so one of these low pass will give me high pass.
So indeed they have high values at high frequencies here, here and here.
And it has small values at low frequencies,
as 0,0 is in the center here as I indicated of this plot.
But it's clearly not smooth.
And that's the case, indeed, every time we work with real data.
They don't behave as smoothly and as nicely as synthetic data.
To compare this, we show here the stabilizing term for the CLS filter.
Is this term here and this is here for 2D laplasia.
So it's a high pass filter, of course, and it's rather smooth and nice you might say.
But both of them in shape are high pass filters and
they're going to be added to the low pass degradation filter,
the denominator of the Wiener filter over there, left, and the CLS on the right.
We're using this example, the same degradation system we've been using
throughout this material on image restoration so the degradation is due to
horizontal motion between the camera and the scene over 8 pixels.
So what we show here is a magnitude of the frequency response of the degradation
system, its rectified sync, and it's constant along
one of the frequency axis as we showed this multiple times, earlier.
We see here the frequency response of the Wiener restoration filter for
the parameters of the problem we've been describing.
It's a bit hard to see the exact shape again due to the fact
that the power spectrum of the image is based on actual data.
It is however the inverted sync function but
in the denominator we've added that stabilizing term.
It's because small values here at 0,0 frequencies then it goes
up a little bit in the mid-frequencies and then tapers off at high frequencies.
We compare this with the CLS filter for
the this value of the regularization parameter.
So the structure is definitely
more clear since the frequency response of the CLS filter is considerably smoother.
To have a clearer view of the two dimensional Wiener restoration filter,
we show here one slice of its frequency response.
The slice at omega 2 = 0.
Again for the parameters, we computed the power spectrum of the image directly from
the original image, which of course in a realistic application is not available.
And also since we added the noise, we have exact knowledge of its variance and
we utilize the exact noise variance.
So, we see here the inverted sync that is tapered off at high
frequencies due to the stabilizing term.
So the value is 1 here at 0, 0, and then you can see the shape
of the inverted sync that are exact zeros at these locations.
And then it tapers off nicely at high frequencies.
We compare these with the same slice of the frequency response of
the CLS filter which was implemented in an iterative fashion and
here it's shown after a 330 durations.
And this is the value of the regularization parameter that was used.
The scale is the same, so we have the same high value here.
So the filter's identical here.
And, they're very, very similar at low frequencies.
However, at higher frequencies, the CLS Filter has higher values.
Like here, I guess, it's around 5, while the same frequency's around 2.
We also show the slice of the frequency response of
the CLS when it's implemented directly.
The same again via regularization parameter, in both cases,
2D Laplacian was used.
So now that scale here, the high value's 30.
So I have these higher peaks here and also the values similarly
to the previous comment are higher at high frequencies.
So again, this is fine while it's 2 with the Wiener filter.
So by and large, based on this qualitative comparison,
we expect both these, and this filter to produce
noisier restorations than the Wiener filter.
But of course with the Constraint Least Squares filter, we have control over
the regularization parameter, and increasing it or decreasing.
Increasing it, we can change the values of the filter at
high frequencies, change the shape.
So we show here on the left the noisy-blurred images is the same
example we've been using throughout this segment of the course.
1D motion blur over 8 pixels and the blur signal to noise ration is 20 dB.
On the left, you see the restored image utilizing this non-causal
Wiener restoration filter that we have analyzed and
developed in this segment of the course.
And again these are the spectra that
were utilized either using the original image or from exact knowledge of noise.
So this is, in some sense, the upper bound of the performance of the Weiner filter
because we have exact knowledge in the synthetic experiment of these parameters.
In practice, we have to find means to provide good estimates,
obtain good estimates of Pff and Pww.
The improvement in signal-to-noise ratio for this particular case is 3.93 dB.
By and large, is a rather good restoration.
The noise is not terribly amplified and
some of the sharpness of the image has been restored.
We show here again the result of the Wiener restoration filter.
We're gonna compare it with a restored image obtained by the iterative
CLS filter.
This is the parameter of alpha, and this is shown after 330 durations.
We did show that frequency points of this filter in area slides.
As was argued there, it's expected this restored image by this filter was
going to be noisier than the result of the Wiener restoration and
this is indeed the case by comparing the two images.
Of course, noise amplification is traded for sharpness in the image.
So this image here is sharper than the Wiener restoration.
We also compare it with the result of the direct CLS restoration.
This now is even noisier than this and noisier than this.
And in both cases the ISNR is negative.
Of course with the CLS filter,
I have different means to control its performance.
One is through the value of the regularization parameter.
Two is through the selection of C.
We'll be using a Laplacian here.
Three's through the iterative implementation.
And in that case, we can use the number of iterations as a means of regularization.
And four, of course, is the introduction of adaptivity that we talked earlier.
The Wiener filter is a very celebrated filter.
It has found many applications in addition to image restoration.
And with respect to image restoration for
a long time was somehow the golden standard that
everybody was comparing their results against the Wiener restoration filter.
Explore nosso catálogo
Registre-se gratuitamente e obtenha recomendações, atualizações e ofertas personalizadas.
O Coursera proporciona acesso universal à melhor educação do mundo fazendo parcerias com as melhores universidades e organizações para oferecer cursos on-line.
© 2019 Coursera Inc. Todos os direitos reservados.
