As we just saw, local averaging has some very interesting mathematical properties.
It's a very simple operation as well and is very useful as we saw a few examples
before. But we have also observed that it can
blur the information in the image, it can blur the edges.
So, how do we solve that? We have a number of different directions
to go. One is to basically replace this local
averaging by a different local operation and we're going to see that very soon.
The other is, we may ask ourself, are we only limited to average in the local
neighborhood? Maybe we can do a smarter average.
Remember that averaging is like a heat flow.
So, shall we basically average between a very hot pixel and a very cold pixel or
shall we try only to average between pixels that are similar and try to stop
averaging pixels that are very different. And that's, a very interesting concept
that can actually be implementing a very, very variety of ways in a very different
number of directions. But let's just explain one which is
extremely elegant. In part it's elegant because it's very
simple and it's actually called non-local means.
Instead of looking at averages in the local neighborhood, we're going to look
at averages in the whole image. Let's see why we can do that and how we
do that. The idea is very simple.
Let's just observe this image. We actually see that images have
repetition. Sometimes these are called
self-similarities. So for example, if we look at this local
neighborhood, it's very similar to this local neighborhood, and very similar to
this local neighborhood. We also observed that here this local
neighborhood is very similar to this local neighborhood,
even similar to this. They're far away, but they're very
similar, the same here.
So, before, we were thinking about averaging only pixels inside here, maybe
we should actually average neighborhoods that are similar.
Maybe we should average these three, so we can replace this pixel by certain
relationship with others, but we should not mix this with these because the local
neighborhoods are very different. So, the basic idea of non local means or
non local averaging is to try to average, try to enjoy the properties of averaging
but not restrict that to a local neighbourhood.
So why are we doing that, and how are we doing that?
So, the basic idea is that if we assume that these are identical things,
identical structures, but maybe with different noise.
Let's see the effects of that. Let's assume that we have a certain
pixel, lets call it P, that's what we observed.
It's the clean pixel plus some noise. But the noise, basically is a random
variable, so it's always slightly different.
So, we observed the pixel a number of times.
We observed one with a certain noise. We observe it again, same pixel,
but with a different noise. And we can observe it multiple times,
let's say that we observe this n times, always the same pixel but with different
realizations of the noise. If we average all these,
we can easily show these are basically properties of signal analysis,
that the noise goes down proportional the square of the number of times that we
average. So we are averaging always the same
pixels, the noise is the random variable,
and basically, the noise goes down proportional to n^.
The more we average, the more we reduce noise.
Of course, it's very important that we always average exactly the same pixel.
The noise is the only thing that changes. So this is one of the motivations of non
local means to say, okay maybe I can find repeated structures that are identical or
at least very similar, the noise is different all around and I can reduce
that noise by averaging just because of this.
And the basic idea then, is to look for similar neighborhoods first and average
them. So if I want to replace this pixel in the
center, here, I look at the neighborhood. I look for similar neighborhoods and I
replace the pixel by the center, by the average of the center of the similar
neighborhoods. And how do I find similar neighborhoods?
So there's a whole literature on non local means,
that basically includes two parts. How do I find similar neighborhoods?
That's the first part. Once I found them, how do I average them?
Do I weighted average them? Do I compute a different relationship between them?
But once again, the simplest for both options is look for neighborhoods by
comparing, let's say, the mean square error between them and make sure that
it's very small. That's a very, very simple operation.
You go around the image you look at neighborhoods, lets say 3x3 or 5x5 or
Guillermo SapiroProfessor
