Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Irregularity Index for Vector-Valued Morphological Operators

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Mathematical morphology is a valuable theory of nonlinear operators widely used for image processing and analysis. Although initially conceived for binary images, mathematical morphology has been successfully extended to vector-valued images using several approaches. Vector-valued morphological operators based on total orders are particularly promising because they circumvent the problem of false colors. On the downside, they often introduce irregularities in the output image. This paper proposes measuring the irregularity of a vector-valued morphological operator by the relative gap between the generalized sum of pixel-wise distances and the Wasserstein metric. Apart from introducing a measure of the irregularity, referred to as the irregularity index, this paper also addresses its computational implementation. Precisely, we distinguish between the ideal global and the practical local irregularity indexes. The local irregularity index, which can be computed more quickly by aggregating values of local windows, yields a lower bound for the global irregularity index. Computational experiments with natural images illustrate the effectiveness of the proposed irregularity indexes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. The source-codes for the Loewner morphological approach can be found at https://www.math.tu-cottbus.de/INSTITUT/lsnmwr/kleefeld/SourcePRL/.

  2. The Julia’s source-code for the global irregularity index is available at https://github.com/mevalle/Irregularity-Index.

  3. The Julia’s source-code for the local irregularity index is available at https://github.com/mevalle/Irregularity-Index.

  4. https://optimaltransport.github.io/.

  5. https://github.com/JuliaOptimalTransport/OptimalTransport.jl.

References

  1. Angulo, J.: Morphological colour operators in totally ordered lattices based on distances: application to image filtering, enhancement and analysis. Comput. Vis. Image Underst. 107(1–2), 56–73 (2007)

    Article  Google Scholar 

  2. Aptoula, E., Lefèvre, S.: A comparative study on multivariate mathematical morphology. Pattern Recognit. 40(11), 2914–2929 (2007)

    Article  Google Scholar 

  3. Birkhoff, G.: Lattice Theory, 3rd edn. American Mathematical Society, Providence (1993)

    MATH  Google Scholar 

  4. Burgeth, B., Didas, S., Kleefeld, A.: A unified approach to the processing of hyperspectral images. In: Burgeth, B., Kleefeld, A., Naegel, B., Passat, N., Perret, B. (eds.) Mathematical Morphology and Its Applications to Signal and Image Processing, pp. 202–214. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-20867-7_16

    Chapter  MATH  Google Scholar 

  5. Burgeth, B., Kleefeld, A.: An approach to color-morphology based on Einstein addition and Loewner order. Pattern Recognit. Lett. 47, 29–39 (2014)

    Article  Google Scholar 

  6. Chevallier, E., Angulo, J.: The irregularity issue of total orders on metric spaces and its consequences for mathematical morphology. J. Math. Imaging Vis. 54(3), 344–357 (2016). https://doi.org/10.1007/s10851-015-0607-7

    Article  MathSciNet  MATH  Google Scholar 

  7. Dougherty, E.R., Lotufo, R.A.: Hands-On Morphological Image Processing. SPIE Press (2003)

  8. Fatras, K., Zine, Y., Flamary, R., Gribonval, R., Courty, N.: Learning with minibatch Wasserstein : asymptotic and gradient properties. In: Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, pp. 2131–2141. PMLR (2020). http://proceedings.mlr.press/v108/fatras20a.html

  9. Foley, J.D., Dam, A.V., Huges, J.F., Feiner, S.K.: Computer Graphics: Principles and Practice, 2nd edn. Addison-Wesley (1990)

  10. Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 2nd edn. Prentice-Hall, Upper Saddle River (2002)

    Google Scholar 

  11. Goutsias, J., Heijmans, H.J.A.M., Sivakumar, K.: Morphological operators for image sequences. Comput. Vis. Image Underst. 62, 326–346 (1995)

    Article  Google Scholar 

  12. Haykin, S.: Neural Networks and Learning Machines, 3rd edn. Prentice-Hall, Upper Saddle River (2009)

    Google Scholar 

  13. Heijmans, H.J.A.M.: Mathematical morphology: a modern approach in image processing based on algebra and geometry. SIAM Rev. 37(1), 1–36 (1995)

    Article  MathSciNet  Google Scholar 

  14. Levkowitz, H.: Color Theory and Modeling for Computer Graphics, Visualization, and Multimedia Applications. Kluwer Academic Publishers, Norwell (1997)

    Book  Google Scholar 

  15. Levkowitz, H., Herman, G.T.: GLHS: a generalized lightness, hue, and saturation color model. CVGIP Graph. Models Image Process. 55(4), 271–285 (1993). https://doi.org/10.1006/cgip.1993.1019

    Article  Google Scholar 

  16. Lézoray, O.: Complete lattice learning for multivariate mathematical morphology. J. Vis. Commun. Image Represent. 35, 220–235 (2016). https://doi.org/10.1016/j.jvcir.2015.12.017

    Article  Google Scholar 

  17. Najman, L., Talbot, H. (eds.): Mathematical Morphology: From Theory to Applications. Wiley, Hoboken (2013). https://doi.org/10.1002/9781118600788

    Book  Google Scholar 

  18. Peyré, G., Cuturi, M.: Computational optimal transport. Found. Trends Mach. Learn. 11(5–6), 1–257 (2019). https://doi.org/10.1561/2200000073

    Article  MATH  Google Scholar 

  19. Pitié, F.: Advances in colour transfer. IET Comput. Vis. 14(6), 304–322 (2020). https://doi.org/10.1049/iet-cvi.2019.0920

    Article  Google Scholar 

  20. Pitié, F., Kokaram, A.C., Dahyot, R.: N-dimensional probability density function transfer and its application to colour transfer. In: Proceedings of the IEEE International Conference on Computer Vision, vol. II, pp. 1434–1439 (2005). https://doi.org/10.1109/ICCV.2005.166

  21. Ronse, C.: Why mathematical morphology needs complete lattices. Signal Process. 21(2), 129–154 (1990)

    Article  MathSciNet  Google Scholar 

  22. Rubner, Y., Tomasi, C., Guibas, L.J.: Earth mover’s distance as a metric for image retrieval. Int. J. Comput. Vis. 40(2), 99–121 (2000). https://doi.org/10.1023/A:1026543900054

    Article  MATH  Google Scholar 

  23. Sangalli, M., Valle, M.E.: Color mathematical morphology using a fuzzy color-based supervised ordering. In: Barreto, G.A., Coelho, R. (eds.) Fuzzy Inf. Process., pp. 278–289. Springer, Berlin (2018)

    Google Scholar 

  24. Sangalli, M., Valle, M.E.: Approaches to multivalued mathematical morphology based on uncertain reduced orderings. In: Burgeth, B., Kleefeld, A., Naegel, B., Passat, N., Perret, B. (eds.) Mathematical Morphology and Its Applications to Signal and Image Processing, pp. 228–240. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20867-7_18

  25. Schmitzer, B.: Stabilized sparse scaling algorithms for entropy regularized transport problems. SIAM J. Sci. Comput. 41(3), A1443–A1481 (2019). https://doi.org/10.1137/16M1106018

    Article  MathSciNet  MATH  Google Scholar 

  26. Schölkopf, B., Smola, A.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2002)

    Google Scholar 

  27. Serra, J.: The “false colour’’ problem. In: Wilkinson, M.H.F., Roerdink, J.B.T.M. (eds.) Mathematical Morphology and Its Application to Signal and Image Processing. Lecture Notes in Computer Science, vol. 5720, pp. 13–23. Springer, Berlin (2009)

    Google Scholar 

  28. Soille, P.: Morphological Image Analysis. Springer Verlag, Berlin (1999)

    Book  Google Scholar 

  29. Valle, M.E., Francisco, S., Granero, M.A., Velasco-Forero, S.: Measuring the irregularity of vector-valued morphological operators using Wasserstein metric. In: Lindblad, J., Malmberg, F., Sladoje, N. (eds.) Discrete Geometry and Mathematical Morphology. DGMM 2021. Lecture Notes in Computer Science, vol. 12708, pp. 512–524. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-76657-3_37

  30. Valle, M.E., Valente, R.A.: Mathematical morphology on the spherical CIELab Quantale with an application in color image boundary detection. J. Math. Imaging Vis. 57(2), 183–201 (2017). https://doi.org/10.1007/s10851-016-0674-4

    Article  MathSciNet  MATH  Google Scholar 

  31. van de Gronde, J., Roerdink, J.: Group-invariant colour morphology based on frames. IEEE Trans. Image Process. 23(3), 1276–1288 (2014)

    Article  MathSciNet  Google Scholar 

  32. Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)

    MATH  Google Scholar 

  33. Velasco-Forero, S., Angulo, J.: Mathematical morphology for vector images using statistical depth. In: Soille, P., Pesaresi, M., Ouzounis, G.K. (eds.) Mathematical Morphology and Its Applications to Image and Signal Processing, pp. 355–366. Springer, Berlin (2011)

    Chapter  Google Scholar 

  34. Velasco-Forero, S., Angulo, J.: Supervised ordering in Rp: application to morphological processing of hyperspectral images. IEEE Trans. Image Process. 20(11), 3301–3308 (2011). https://doi.org/10.1109/TIP.2011.2144611

    Article  MathSciNet  MATH  Google Scholar 

  35. Velasco-Forero, S., Angulo, J.: Random projection depth for multivariate mathematical morphology. IEEE J. Sel. Top. Signal Process. 6(7), 753–763 (2012). https://doi.org/10.1109/JSTSP.2012.2211336

    Article  Google Scholar 

  36. Velasco-Forero, S., Angulo, J.: Vector ordering and multispectral morphological image processing. In: Celebi, M.E., Smolka, B. (eds.) Advances in Low-Level Color Image Processing, pp. 223–239. Springer, Dordrecht (2014). https://doi.org/10.1007/978-94-007-7584-8_7

    Chapter  Google Scholar 

  37. Villani, C.: Optimal Transport, Grundlehren der mathematischen Wissenschaften, vol. 338. Springer, Berlin (2009). https://doi.org/10.1007/978-3-540-71050-9

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universidade Estadual de Campinas, Campinas, SP, Brazil

    Marcos Eduardo Valle

  2. Instituto Federal de Educação, Ciência e Tecnologia de São Paulo, São Paulo, SP, Brazil

    Samuel Francisco & Marco Aurélio Granero

  3. Center of Mathematical Morphology, Mines ParisTech, PSL Research University, Paris, France

    Santiago Velasco-Forero

Authors
  1. Marcos Eduardo Valle
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samuel Francisco
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marco Aurélio Granero
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Santiago Velasco-Forero
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcos Eduardo Valle.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the São Paulo Research Foundation (FAPESP) under Grant No. 2019/02278-2.

Appendix A Supervised and Unsupervised Morphological Approaches

Appendix A Supervised and Unsupervised Morphological Approaches

Let us briefly review the supervised and unsupervised vector-valued morphological approaches, whose details can be found in [34, 35].

In a supervised ordering, the surjective mapping \(\rho :{\mathbb {V}} \rightarrow {\mathbb {L}}\) is defined using a set \(F \subset {\mathbb {V}}\) of foreground values and a set \(B \subset {\mathbb {V}}\) of background values such that \(F \cap B = \emptyset \). Given the sets F and B, the mapping \(\rho \) is expected to satisfy the inequality \(\rho ({\varvec{f}}) > \rho ({\varvec{b}})\) for \({\varvec{f}}\in F\) and \({\varvec{b}}\in B\). Considering \({\mathbb {V}} \subset {\mathbb {R}}^d\) and \({\mathbb {L}} \subset {\mathbb {R}}\), the decision function of a SVM can be used to accomplish this goal [26, 32, 34]. Precisely, consider sets \(F = \{{\varvec{f}}_1,\ldots ,{\varvec{f}}_K\} \subset {\mathbb {R}}^d\) and \(B = \{{\varvec{b}}_1,\ldots ,{\varvec{b}}_M\} \subset {\mathbb {R}}^d\) of foreground and background values, respectively. An SVM-based morphological approach is obtained by considering the mapping \(\rho _S:{\mathbb {R}}^d \rightarrow {\mathbb {R}}\) given by

$$\begin{aligned} \rho _{S}({\varvec{x}})= \sum _{{\varvec{f}}\in F} \alpha _i \kappa ({\varvec{x}},{\varvec{f}}) - \sum _{{\varvec{b}}\in B} \beta _j \kappa ({\varvec{x}},{\varvec{b}}), \quad \forall {\varvec{x}}\in {\mathbb {R}}^d, \end{aligned}$$
(33)

where \(\kappa :{\mathbb {R}}^d \times {\mathbb {R}}^d \rightarrow {\mathbb {R}}\) is a Mercer kernel [26]. For example, Gaussian radial basis function kernel is given by

$$\begin{aligned} \kappa ({\varvec{x}},{\varvec{y}}) = e^{-\frac{1}{2\sigma }\Vert {\varvec{x}}-{\varvec{y}}\Vert _2^2}, \quad \forall {\varvec{x}}, {\varvec{y}}\in {\mathbb {R}}^d, \end{aligned}$$
(34)

where \(\sigma >0\) is a parameter. Moreover, \(\alpha _1,\ldots ,\alpha _K\) and \(\beta _1,\ldots ,\beta _M\) solve the quadratic optimization problem

$$\begin{aligned} {\left\{ \begin{array}{ll} \text{ maximize } &{} \displaystyle \sum _{i=1}^{K}\alpha _{i}+\sum _{j=1}^{M}\beta _{j} -\dfrac{1}{2}\sum _{i, l =1}^{K} \alpha _{i}\alpha _{l} \kappa ({\varvec{f}}_i, {\varvec{f}}_l) \\ &{}\quad - \, \dfrac{1}{2}\sum _{j, l =1}^{M} \beta _{j}\beta _{l} \kappa ({\varvec{b}}_j, {\varvec{b}}_l) \\ &{} \quad \displaystyle {+ \, \dfrac{1}{2}\sum _{i =1}^{K} \sum _{j=1}^{M}\alpha _{i}\beta _{j} \kappa ({\varvec{f}}_i, {\varvec{b}}_j)}\\ \text{ subject } \text{ to } &{} \displaystyle {\sum _{i=1}^{K}\alpha _i - \sum _{j=1}^{M}\beta _j = 0}, \\ &{} 0 \le \alpha _i, \beta _j \le C, \end{array}\right. } \end{aligned}$$
(35)

where the parameter \(C>0\) controls the trade-off between the classification error and the margin of separation between background and foreground values [12, 26].

In an unsupervised morphological approach, the mapping \(\rho :{\mathbb {V}} \rightarrow {\mathbb {L}}\) is determined using a set of unlabeled values. The statistical depth projection-based approach, for example, determines the mapping \(\rho \) based on “anomalies” with respect to a background composed of the majority of pixel values of an image [35]. Formally, suppose \({\mathbb {V}} \subset {\mathbb {R}}^d\) and \({\mathbb {L}} \subset {\mathbb {R}}\). Given a training sample represented by a matrix \(\mathbf{X } = [{\varvec{x}}_1, \ldots , {\varvec{x}}_n] \in {\mathbb {R}}^{d \times n}\), the projection depth function \(\rho _P^*:{\mathbb {R}}^d \rightarrow {\mathbb {R}}\) is defined by

$$\begin{aligned} \rho _P^*({\varvec{x}}) = \sup _{{\varvec{u}}\in {\mathbb {S}}^{d-1}} \dfrac{|{\varvec{u}}^{T}{\varvec{x}}- \text {MED}({\varvec{u}}^{T}\mathbf{X })|}{\text {MAD}({\varvec{u}}^T\mathbf{X })}, \quad \forall {\varvec{x}}\in {\mathbb {R}}^d, \end{aligned}$$
(36)

where \({\mathbb {S}}^{d-1} = \{{\varvec{x}}\in {\mathbb {R}}^d:||{\varvec{x}}||_{2} = 1\}\), \(\text {MED}:{\mathbb {R}}^n \rightarrow {\mathbb {R}}\) is the median operator, and \(\text {MAD}:{\mathbb {R}}^n \rightarrow {\mathbb {R}}\) is the median absolute deviation from the median operator. Recall that the median absolute deviation from the median is given by

$$\begin{aligned} \text {MAD}({\varvec{t}}) = \text {MED}(|{\varvec{t}}- {\varvec{1}}_n\text {MED}({\varvec{t}})|), \end{aligned}$$
(37)

where \({\varvec{1}}_n \in {\mathbb {R}}^n\) denotes the vector of ones and the absolute value \(|\cdot |\) is computed in a component-wise manner. In practice, we compute the depth projection function by replacing the supremum with the maximum on a finite set of elements in the hypersphere \({\mathbb {S}}^{d-1}\). Formally, the function \(\rho _P:{\mathbb {R}}^d \rightarrow {\mathbb {R}}\) given by

$$\begin{aligned} \rho _P({\varvec{x}}) = \max _{{\varvec{u}}\in {\mathbb {U}}} \dfrac{|{\varvec{u}}^{T}{\varvec{x}}- \text {MED}({\varvec{u}}^{T}\mathbf{X })|}{\text {MAD}({\varvec{u}}^T\mathbf{X })}, \quad \forall {\varvec{x}}\in {\mathbb {R}}^d, \end{aligned}$$
(38)

where \({\mathbb {U}} = \{{\varvec{u}}_1, {\varvec{u}}_2, \ldots , {\varvec{u}}_k \} \subset {\mathbb {S}}^{d-1}\), is taken as an approximation of the theoretical depth projection function \(\rho _P^*\). The projection depth morphological approach is defined by ranking the vector-values according to the mapping \(\rho _P:{\mathbb {R}}^d \rightarrow {\mathbb {R}}\) given by (38) together with a look-up table.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Valle, M.E., Francisco, S., Granero, M.A. et al. Irregularity Index for Vector-Valued Morphological Operators. J Math Imaging Vis 64, 754–770 (2022). https://doi.org/10.1007/s10851-022-01092-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-022-01092-0

Keywords

Search

Navigation