Cited By
View all- Reif UWeinmann A(2021)Clothoid fitting and geometric Hermite subdivisionAdvances in Computational Mathematics10.1007/s10444-021-09876-547:4Online publication date: 26-Jun-2021
Inpainting for Fringe Projection Profilometry Based on Geometrically Guided Iterative Regularization
Pages 5531 - 5542
Abstract
Conventional fringe projection profilometry methods often have difficulty in reconstructing the 3D model of objects when the fringe images have the so-called highlight regions due to strong illumination from nearby light sources. Within a highlight region, the fringe pattern is often overwhelmed by the strong reflected light. Thus, the 3D information of the object, which is originally embedded in the fringe pattern, can no longer be retrieved. In this paper, a novel inpainting algorithm is proposed to restore the fringe images in the presence of highlights. The proposed method first detects the highlight regions based on a Gaussian mixture model. Then, a geometric sketch of the missing fringes is made and used as the initial guess of an iterative regularization procedure for regenerating the missing fringes. The simulation and experimental results show that the proposed algorithm can accurately reconstruct the 3D model of objects even when their fringe images have large highlight regions. It significantly outperforms the traditional approaches in both quantitative and qualitative evaluations.
References
[1]
P. S. Huang and S. Zhang, “Fast three-step phase-shifting algorithm,” Appl. Opt., vol. 45, no. 21, pp. 5086–5091, Jul. 2006.
[2]
Y. Wang, K. Liu, Q. Hao, D. L. Lau, and L. G. Hassebrook, “Period coded phase shifting strategy for real–time 3D structured light illumination,” IEEE Trans. Image Process., vol. 20, no. 11, pp. 3001–3013, Nov. 2011.
[3]
Y. Wang, K. Liu, Q. Hao, X. Wang, D. L. Lau, and L. G. Hassebrook, “Robust active stereo vision using Kullback–Leibler divergence,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 3, pp. 548–563, Mar. 2012.
[4]
Z. Song, R. Chung, and X.-T. Zhang, “An accurate and robust strip-edge-based structured light means for shiny surface micromeasurement in 3D,” IEEE Trans. Ind. Electron., vol. 60, no. 3, pp. 1023–1032, Mar. 2013.
[5]
M. Holroyd, J. Lawrence, and T. Zickler, “A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance,” ACM Trans. Graph., vol. 29, no. 4, pp. 99:1–99:12, Jul. 2010.
[6]
J. Park and A. C. Kak, “3D modeling of optically challenging objects,” IEEE Trans. Vis. Comput. Graphics, vol. 14, no. 2, pp. 246–262, Mar./Apr. 2008.
[7]
C. Hermans, Y. Francken, T. Cuypers, and P. Bekaert, “Depth from sliding projections,” in Proc. IEEE Conf. CVPR, Jun. 2009, pp. 1865–1872.
[8]
S. K. Nayar, X.-S. Fang, and T. Boult, “Separation of reflection components using color and polarization,” Int. J. Comput. Vis., vol. 21, no. 3, pp. 163–186, Feb. 1997.
[9]
T. Chen, H. Lensch, C. Fuchs, and H.-P. Seidel, “Polarization and phase-shifting for 3D scanning of translucent objects,” in Proc. CVPR, Jun. 2007, pp. 1–8.
[10]
S. K. Nayar and M. Gupta, “Micro phase shifting,” in Proc. IEEE Conf. CVPR, Jun. 2012, pp. 813–820.
[11]
T. Chen, H.-P. Seidel, and H. Lensch, “Modulated phase-shifting for 3D scanning,” in Proc. IEEE Conf. CVPR, Jun. 2008, pp. 1–8.
[12]
M. Gupta, A. Agrawal, A. Veeraraghavan, and S. G. Narasimhan, “A practical approach to 3D scanning in the presence of interreflections, subsurface scattering and defocus,” Int. J. Comput. Vis., vol. 102, nos. 1–3, pp. 33–55, 2013.
[13]
V. Couture, N. Martin, and S. Roy, “Unstructured light scanning to overcome interreflections,” in Proc. IEEE Int. Conf. ICCV, Nov. 2011, pp. 1895–1902.
[14]
S. K. Nayar, G. Krishnan, M. D. Grossberg, and R. Raskar, “Fast separation of direct and global components of a scene using high frequency illumination,” ACM Trans. Graph., vol. 25, no. 3, pp. 935–944, Jul. 2006.
[15]
P. Tan, S. Lin, L. Quan, and H.-Y. Shum, “Highlight removal by illumination-constrained inpainting,” in Proc. Int. Conf. Comput. Vis., vol. 1. Oct. 2003, pp. 164–169.
[16]
R. T. Tan and K. Ikeuchi, “Separating reflection components of textured surfaces using a single image,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 2, pp. 178–193, Feb. 2005.
[17]
H.-L. Shen and Z.-H. Zheng, “Real-time highlight removal using intensity ratio,” Appl. Opt., vol. 52, no. 19, pp. 4483–4493, Jul. 2013.
[18]
H.-L. Shen, H.-G. Zhang, S.-J. Shao, and J. H. Xin, “Chromaticity-based separation of reflection components in a single image,” Pattern Recognit., vol. 41, no. 8, pp. 2461–2469, Aug. 2008.
[19]
H. Kim, H. Jin, S. Hadap, and I. Kweon, “Specular reflection separation using dark channel prior,” in Proc. IEEE Conf. CVPR, Jun. 2013, pp. 1460–1467.
[20]
P. Koirala, P. Pant, M. Hauta-Kasari, and J. Parkkinen, “Highlight detection and removal from spectral image,” J. Opt. Soc. Amer. A, vol. 28, no. 11, pp. 2284–2291, 2011.
[21]
S. A. Shafer, “Using color to separate reflection components,” Color Res. Appl., vol. 10, no. 4, pp. 210–218, 1985.
[22]
J. W. Park and K. H. Lee, “Inpainting highlights using color line projection,” IEICE Trans. Inf. Syst., vol. E90-D, no. 1, pp. 250–257, Jan. 2007.
[23]
J. H. Massig and J. Heppner, “Fringe-pattern analysis with high accuracy by use of the Fourier-transform method: Theory and experimental tests,” Appl. Opt., vol. 40, no. 13, pp. 2081–2088, May 2001.
[24]
C. Roddier and F. Roddier, “Interferogram analysis using Fourier transform techniques,” Appl. Opt., vol. 26, no. 9, pp. 1668–1673, May 1987.
[25]
M. Dai and Y. Wang, “Fringe extrapolation technique based on Fourier transform for interferogram analysis,” Opt. Lett., vol. 34, no. 7, pp. 956–958, Apr. 2009.
[26]
T.-C. Hsung, D. P.-K. Lun, and W. W. L. Ng, “Efficient fringe image enhancement based on dual-tree complex wavelet transform,” Appl. Opt., vol. 50, no. 21, pp. 3973–3986, 2011.
[27]
W. W.-L. Ng and D. P.-K. Lun, “Effective bias removal for fringe projection profilometry using the dual-tree complex wavelet transform,” Appl. Opt., vol. 51, no. 24, pp. 5909–5916, Aug. 2012.
[28]
L. Meng, S. Fang, P. Yang, L. Wang, M. Komori, and A. Kubo, “Image-inpainting and quality-guided phase unwrapping algorithm,” Appl. Opt., vol. 51, no. 13, pp. 2457–2462, May 2012.
[29]
H. Wu, L. Wang, X. Yu, H. Bai, and D. Wu, “Exploration on exemplar-based encoding image inpainting of coded structured light system,” in Proc. 6th Int. Forum Strategic Technol. (IFOST), vol. 2. Aug. 2011, pp. 1092–1095.
[30]
A. Criminisi, P. Pérez, and K. Toyama, “Region filling and object removal by exemplar-based image inpainting,” IEEE Trans. Image Process., vol. 13, no. 9, pp. 1200–1212, Sep. 2004.
[31]
O. G. Guleryuz, “Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising—Part II: Adaptive algorithms,” IEEE Trans. Image Process., vol. 15, no. 3, pp. 555–571, Mar. 2006.
[32]
O. G. Guleryuz, “Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising—Part I: Theory,” IEEE Trans. Image Process., vol. 15, no. 3, pp. 539–554, Mar. 2006.
[33]
F. Li and T. Zeng, “A universal variational framework for sparsity-based image inpainting,” IEEE Trans. Image Process., vol. 23, no. 10, pp. 4242–4254, Oct. 2014.
[34]
Y.-R. Li, L. Shen, and B. W. Suter, “Adaptive inpainting algorithm based on DCT induced wavelet regularization,” IEEE Trans. Image Process., vol. 22, no. 2, pp. 752–763, Feb. 2013.
[35]
J.-F. Cai, R. H. Chan, and Z. Shen, “A framelet-based image inpainting algorithm,” Appl. Comput. Harmon. Anal., vol. 24, no. 2, pp. 131–149, Mar. 2008.
[36]
M. Takeda and K. Mutoh, “Fourier transform profilometry for the automatic measurement of 3D object shapes,” Appl. Opt., vol. 22, no. 24, pp. 3977–3982, 1983.
[37]
X. Su and W. Chen, “Fourier transform profilometry: A review,” Opt. Lasers Eng., vol. 35, no. 5, pp. 263–284, May 2001.
[38]
T.-C. Hsung, D. P. Lun, and W. W. L. Ng, “Zero spectrum removal using joint bilateral filter for Fourier transform profilometry,” in Proc. IEEE Int. Conf. VCIP, Nov. 2011, pp. 1–4.
[39]
T.-C. Hsung and D. P.-K. Lun, “On optical phase shift profilometry based on dual tree complex wavelet transform,” in Proc. IEEE Int. Conf. Image Process., Sep. 2010, pp. 337–340.
[40]
I. W. Selesnick, R. G. Baraniuk, and N. C. Kingsbury, “The dual-tree complex wavelet transform,” IEEE Signal Process. Mag., vol. 22, no. 6, pp. 123–151, Nov. 2005.
[41]
N. Kingsbury, “A dual-tree complex wavelet transform with improved orthogonality and symmetry properties,” in Proc. Int. Conf. Image Process., vol. 2. Sep. 2000, pp. 375–378.
[42]
G. Yu and G. Sapiro, “DCT image denoising: A simple and effective image denoising algorithm,” Image Process. Line, vol. 1, 2011. [Online]. Available: https://doi.org/10.5201/ipol.2011.ys-dct
[43]
A. N. Akansu and P. R. Haddad, Multiresolution Signal Decomposition: Transforms, Subbands, and Wavelets. San Francisco, CA, USA: Academic, 2000, p. 499.
[44]
J.-L. Starck, E. J. Candes, and D. L. Donoho, “The curvelet transform for image denoising,” IEEE Trans. Electron. Packag. Manuf., vol. 11, no. 6, pp. 670–684, Jun. 2002.
[45]
D. L. Donoho, “De-noising by soft-thresholding,” IEEE Trans. Inf. Theory, vol. 41, no. 3, pp. 613–627, May 1995.
[46]
N. Otsu, “A threshold selection method from gray-level histograms,” IEEE Trans. Syst., Man, Cybern., vol. 9, no. 1, pp. 62–66, Jan. 1979.
[47]
J. Kittler and J. Illingworth, “Minimum error thresholding,” Pattern Recognit., vol. 19, no. 1, pp. 41–47, 1986.
[48]
C. Bishop, Pattern Recognition and Machine Learning (Information Science and Statistics). Secaucus, NJ, USA: Springer-Verlag, 2006.
[49]
B. Budianto and D. P. K. Lun, “Inpainting for fringe projection profilometry based on iterative regularization,” in Proc. Int. Conf. Digit. Signal Process., Aug. 2014, pp. 668–672.
[50]
T. Hastie, R. Tibshirani, and J. Friedman, “The EM algorithm,” in The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. New York, NY, USA: Springer-Verlag, 2009, pp. 272–279.
[51]
S. Masnou, “Disocclusion: A variational approach using level lines,” IEEE Trans. Image Process., vol. 11, no. 2, pp. 68–76, Feb. 2002.
[52]
F. Cao, Y. Gousseau, S. Masnou, and P. Pérez, “Geometrically guided exemplar-based inpainting,” SIAM J. Imag. Sci., vol. 4, no. 4, pp. 1143–1179, Jan. 2011.
[53]
J. F. Hughes, A. van Dam, J. D. Foley, M. McGuire, D. F. Sklar, and S. K. Feiner, Computer Graphics: Principles and Practice. Reading, MA, USA: Addison-Wesley, 2013.
[54]
E. Bertolazzi and M. Frego, “G1 fitting with clothoids,” Math. Methods Appl. Sci., vol. 38, no. 5, pp. 881–897, Mar. 2015.
[55]
A. Criminisi, P. Perez, and K. Toyama, “Object removal by exemplar-based inpainting,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2. Jun. 2003, pp. II-721–II-728.
[56]
D. Tschumperlé and R. Deriche, “Vector-valued image regularization with PDEs: A common framework for different applications,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 4, pp. 506–517, Apr. 2005.
[57]
W. Dong, G. Shi, and X. Li, “Nonlocal image restoration with bilateral variance estimation: A low-rank approach,” IEEE Trans. Image Process., vol. 22, no. 2, pp. 700–711, Feb. 2013.
Index Terms
- Inpainting for Fringe Projection Profilometry Based on Geometrically Guided Iterative Regularization
Index terms have been assigned to the content through auto-classification.
Recommendations
Transformer-Based Fringe Restoration for Shadow Mitigation in Fringe Projection Profilometry
Pattern RecognitionAbstractFringe Projection Profilometry (FPP) is a widely recognized technique for deriving 3D profiles from images. Despite numerous methodologies developed to determine depth in FPP, the inherent triangulation setup of camera, projector, and object often ...
The Height Variance Range for One Frequency Fringe Pattern Profilometry
CGIV '10: Proceedings of the 2010 Seventh International Conference on Computer Graphics, Imaging and VisualizationThe upper limit on the deepest step of the surface shape that can be accurately determined is an important performance measure associated with the fringe projection profilometry. This metric is evaluated as the variance of height between two adjacent ...
Robust Single-Shot Fringe Projection Profilometry Based on Morphological Component Analysis
In a fringe projection profilometry (FPP) process, the captured fringe images can be modeled as the superimposition of the projected fringe patterns on the texture of the objects. Extracting the fringe patterns from the captured fringe images is an ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2015.
Publisher
IEEE Press
Publication History
Published: 01 December 2015
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025
Other Metrics
Citations
Cited By
View all- Reif UWeinmann A(2021)Clothoid fitting and geometric Hermite subdivisionAdvances in Computational Mathematics10.1007/s10444-021-09876-547:4Online publication date: 26-Jun-2021