A gentle introduction to bilateral filtering and its applications

A Gentle Introduction to Bilateral Filtering and its Applications How does bilateral filter relates with other methods? Fredo Durand (MIT CSAIL) Slides by Pierre Kornprobst (INRIA) 0:35 Many people worked on… edge-preserving restoration Bilateral filter Partial differential equations Anisotropic diffusion Robust statistics Local mode filtering Goal: Understand how does bilateral filter relates with other methods Bilateral filter Partial differential equations Local mode filtering Robust statistics Local mode filtering principle Spatial window Smoothed local histogram You are going to see that BF has the same effect as local mode filtering Let’s prove it! • • • • • Define global histogram Define a smoothed histogram Define a local smoothed histogram What does it mean to look for local modes? What is the link with bilateral filter? Definition of a global histogram • Formal definition of histogram H at intensity i: Where is the Dirac symbol (zero everywhere except at 0) • A sum of Dirac, « a sum of ones » # pixels 1 1 1 1 1 1 1 1 1 1 intensity # pixels Smoothing the histogram 1 1 1 1 1 1 1 1 1 1 intensity Smoothing the histogram # pixels 1 1 1 1 1 1 1 1 1 1 intensity # pixels 1 1 1 1 1 1 intensity # pixels 1 1 1 1 1 1 intensity # pixels 1 1 1 1 intensity # pixels intensity # pixels This is it! intensity Definition of a local smoothed histogram • We introduce a « smooth window » Smoothing of intensities where Spatial window And that’s the formula to have in mind! Definition of local modes A local mode i verifies Local modes? • Given • We look for • Result: Summary • A local mode i verifies: • Hey! That looks like bilateral filter!!! One iteration of the bilateral filter amounts to converge to the local mode DIscussion • The bilateral filter goes to a LOCAL mode, not necessarily the global mode • Often desirable: mode closest to input pixel • Sometimes not: impulse noise case – Recall the use of the median as pre-filter – amounts to going to the global mode Take home message #1 Bilateral filter is equivalent to mode filtering in local histograms [Van de Weijer, Van den Boomgaard, 2001, etc] Goal: Understand how does bilateral filter relates with other methods Bilateral filter Partial differential equations Local mode filtering Robust statistics Robust statistics? • Goals: Reduce the influence of outliers • Minimizing a cost Error norm • In standard robust statistics Iq are measured data, Ip is a robust average of the data [Huber 81, Hampel 86] Robust statistics? • In our case: the output at a pixel should be a robust smoothing of its neighbors Error norm • Minimizing a cost • Extended local formulation [Huber 81, Hampel 86] How to choose the error norm? • Least square pays a big penalty for big errors – problem in the presence of outliers How to choose the error norm? • Strong differences must not be too penalizing, otherwise, everything will be smoothed! How to minimize the cost function? • Gradient descent and iterative scheme Doesn’t cost too much No influence In gradient Getting closer to bilateral filter • Rewrite introducing a new function Getting closer to bilateral filter • Rewrite introducing a new function • g has the same qualitative behavior than a Gaussian • Now this operator reminds us about bilateral filter! No influence Really the same? • Iteratively reweighted least square M-estimator etal,average 1986]: M-estimators Weighted of the data and •[Hampel W_estimators are essentially equivalent and solve the same minimization problem W-estimator Take home message #2 Bilateral filter can be interpreted in term of robust statistics since it is related to a cost function! [Durand, Dorsey, 2002, Black, Marimont, 1998, etc] Goal: Understand how does bilateral filter relates with other methods Bilateral filter Partial differential equations Local mode filtering Robust statistics Disclaimer • We will shrink the neighborhood • This will lose some properties of the bilateral filter • But although partial, this parallel is insightful What do I mean by PDEs? • Images live in a continuous domain • Two kinds of formulations – Variational approach – Evolving a partial differential equation Recall robust statistics PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs PDEs Images are continuous Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Robust statistics Robust statistics Robust St Images are discrete Some technical results to establish • Considering the Yaroslavsky Filter • When (operation similar to M-estimators) At a very local scale, the asymptotic behavior of the integral operator corresponds to a diffusion operator [Buades, Coll, Morel, 2005] The PDE world at a glance Tschumperle, Deriche Perez, Gangnet, Blake Tschumperle, Deriche Sussman Desbrun etal Breen, Whitaker Discussion • We shrunk the kernel • Take home message #3 Bilateral filter is a discretization of a particular kind of a PDEbased anisotropic diffusion. [Barash 2001, Elad 2002, Durand 2002, Buades, Coll, Morel, 2005] Welcome to the PDE-world! [Kornprobst 2006] Summary Bilateral filter is one technique for anisotropic diffusion and it makes the bridge between several frameworks. From there, you can explore news worlds! Local mode filtering Bilateral filter Anisotropic diffusion PDEs Robust statistics Questions?