research-article

A Compressive Sensing Approach to Describe Indoor Scenes for Blind People

Authors:

Mohamed Lamine Mekhalfi,

Naif AlajlanAuthors Info & Claims

IEEE Transactions on Circuits and Systems for Video Technology, Volume 25, Issue 7

Pages 1246 - 1257

https://doi.org/10.1109/TCSVT.2014.2372371

Published: 01 July 2015 Publication History

Abstract

This paper introduces a new portable camera-based method for helping blind people to recognize indoor objects. Unlike state-of-the-art techniques, which typically perform the recognition task by limiting it to a single predefined class of objects, we propose here a completely different alternative scheme, defined as coarse description. It aims at expanding the recognition task to multiple objects and, at the same time, keeping the processing time under control by sacrificing some information details. The benefit is to increment the awareness and the perception of a blind person to his direct contextual environment. The coarse description issue is addressed via two image multilabeling strategies which differ in the way image similarity is computed. The first one makes use of the Euclidean distance measure, while the second one relies on a semantic similarity measure modeled by means of Gaussian process estimation. To achieve fast computation capability, both strategies rely on a compact image representation based on compressive sensing. The proposed methodology was assessed on two indoor datasets representing different indoor environments. Encouraging results were achieved in terms of both accuracy and processing time.

References

[1]

L. Dandona and R. Dandona, “Revision of visual impairment definitions in the international statistical classification of diseases,” BMC Med., vol. 4, pp. 1–7, Mar. 2006.

[2]

M. da Silva Cascalheira, P. Pinho, D. Teixeira, and N. B. de Carvalho, “Indoor guidance system for the blind and the visually impaired,” IET Microw., Antennas, Propag., vol. 6, no. 10, pp. 1149–1157, Jul. 2012.

[3]

I. Ulrich and J. Borenstein, “The GuideCane—Applying mobile robot technologies to assist the visually impaired,” IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 31, no. 2, pp. 131–136, Mar. 2001.

Digital Library

[4]

S. Shoval, J. Borenstein, and Y. Koren, “Auditory guidance with the Navbelt—A computerized travel aid for the blind,” IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 28, no. 3, pp. 459–467, Aug. 1998.

Digital Library

[5]

M. R. Strakowski, B. B. Kosmowski, R. Kowalik, and P. Wierzba, “An ultrasonic obstacle detector based on phase beamforming principles,” IEEE Sensors J., vol. 6, no. 1, pp. 179–186, Feb. 2006.

[6]

S. Pundlik, M. Tomasi, and G. Luo, “Collision detection for visually impaired from a body-mounted camera,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Jun. 2013, pp. 41–47.

[7]

V. Pradeep, G. Medioni, and J. Weiland, “Robot vision for the visually impaired,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Jun. 2010, pp. 15–22.

[8]

M. Radvanyi, B. Varga, and K. Karacs, “Advanced crosswalk detection for the bionic eyeglass,” in Proc. 12th Int. Workshop Cellular Nanoscale Netw. Appl. (CNNA), Feb. 2010, pp. 1–5.

[9]

G. Balakrishnan, G. Sainarayanan, R. Nagarajan, and S. Yaacob, “Stereopsis method for visually impaired to identify obstacles based on distance,” in Proc. IEEE 1st Symp. Multi-Agent Secur. Survivability, Dec. 2004, pp. 580–583.

[10]

F. M. Hasanuzzaman, X. Yang, and Y. Tian, “Robust and effective component-based banknote recognition for the blind,” IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 42, no. 6, pp. 1021–1030, Nov. 2012.

Digital Library

[11]

D. López-de-Ipiña, T. Lorido, and U. López, “BlindShopping: Enabling accessible shopping for visually impaired people through mobile technologies,” in Proc. 9th Int. Conf. Toward Useful Services Elderly People Disabilities, 2011, pp. 266–270.

[12]

E. Tekin and J. M. Coughlan, “An algorithm enabling blind users to find and read barcodes,” in Proc. Workshop Appl. Comput. Vis. (WACV), Dec. 2009, pp. 1–8.

[13]

H. Pan, C. Yi, and Y. Tian, “A primary travelling assistant system of bus detection and recognition for visually impaired people,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops (ICMEW), Jul. 2013, pp. 1–6.

[14]

T. J. J. Tang, W. L. D. Lui, and W. H. Li, “Plane-based detection of staircases using inverse depth,” in Proc. ACRA, 2012, pp. 1–10.

[15]

X. Chen and A. L. Yuille, “Detecting and reading text in natural scenes,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR), vol. 2. Jun./Jul. 2004, pp. II-366–II-373.

[16]

X. Yang and Y. Tian, “Robust door detection in unfamiliar environments by combining edge and corner features,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Jun. 2010, pp. 57–64.

[17]

S. Wang and Y. Tian, “Camera-based signage detection and recognition for blind persons,” in Proc. 13th Int. Conf. Comput. Helping People Special Needs, 2012, pp. 17–24.

[18]

J. Shotton, J. Winn, C. Rother, and A. Criminisi, “TextonBoost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context,” Int. J. Comput. Vis., vol. 81, no. 1, pp. 2–23, 2009.

Digital Library

[19]

N. Razavi, J. Gall, and L. Van Gool, “Scalable multi-class object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2011, pp. 1505–1512.

[20]

T. Deselaers, D. Keysers, R. Paredes, E. Vidal, and H. Ney, “Local representations for multi-object recognition,” in Pattern Recognition. Berlin, Germany: Springer-Verlag, 2003, pp. 305–312.

[21]

K. Mikolajczyk, B. Leibe, and B. Schiele, “Multiple object class detection with a generative model,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2006, pp. 26–36.

[22]

C. Pantofaru, C. Schmid, and M. Hebert, “Object recognition by integrating multiple image segmentations,” in Proc. 10th Eur. Conf. Comput. Vis. (ECCV), 2008, pp. 481–494.

[23]

D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289–1306, Apr. 2006.

Digital Library

[24]

E. J. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489–509, Feb. 2006.

Digital Library

[25]

M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311–4322, Nov. 2006.

Digital Library

[26]

J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, “Robust face recognition via sparse representation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 2, pp. 210–227, Feb. 2009.

Digital Library

[27]

V. M. Patel and R. Chellappa, “Sparse representations, compressive sensing and dictionaries for pattern recognition,” in Proc. 1st Asian Conf. Pattern Recognit. (ACPR), Nov. 2011, pp. 325–329.

[28]

A. Morelli Andrés, S. Padovani, M. Tepper, and J. Jacobo-Berlles, “Face recognition on partially occluded images using compressed sensing,” Pattern Recognit. Lett., vol. 36, pp. 235–242, Jan. 2014.

Digital Library

[29]

J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process., vol. 19, no. 11, pp. 2861–2873, Nov. 2010.

Digital Library

[30]

S. Rao, R. Tron, R. Vidal, and Y. Ma, “Motion segmentation in the presence of outlying, incomplete, or corrupted trajectories,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 10, pp. 1832–1845, Oct. 2010.

Digital Library

[31]

J. Mairal, M. Elad, and G. Sapiro, “Sparse representation for color image restoration,” IEEE Trans. Image Process., vol. 17, no. 1, pp. 53–69, Jan. 2008.

Digital Library

[32]

B. Shen, W. Hu, Y. Zhang, and Y.-J. Zhang, “Image inpainting via sparse representation,” in Proc. IEEE ICASSP, Apr. 2009, pp. 697–700.

[33]

L. Lorenzi, F. Melgani, and G. Mercier, “Missing-area reconstruction in multispectral images under a compressive sensing perspective,” IEEE Trans. Geosci. Remote Sens., vol. 51, no. 7, pp. 3998–4008, Jul. 2013.

[34]

A. Quattoni, M. Collins, and T. Darrell, “Transfer learning for image classification with sparse prototype representations,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2008, pp. 1–8.

[35]

J. Wright, Y. Ma, J. Mairal, G. Sapiro, T. S. Huang, and S. Yan, “Sparse representation for computer vision and pattern recognition,” Proc. IEEE, vol. 98, no. 6, pp. 1031–1044, Jun. 2010.

[36]

E. J. Candès and T. Tao, “Decoding by linear programming,” IEEE Trans. Inf. Theory, vol. 51, no. 12, pp. 4203–4215, Dec. 2005.

Digital Library

[37]

D. L. Donoho, Y. Tsaig, I. Drori, and J.-L. Starck, “Sparse solution of underdetermined systems of linear equations by stagewise orthogonal matching pursuit,” IEEE Trans. Inf. Theory, vol. 58, no. 2, pp. 1094–1121, Feb. 2012.

Digital Library

[38]

C. K. I. Williams and D. Barber, “Bayesian classification with Gaussian processes,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 12, pp. 1342–1351, Dec. 1998.

Digital Library

[39]

C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning. Cambridge, MA, USA: MIT Press, 2006.

Digital Library

[40]

Y. Bazi and F. Melgani, “Gaussian process approach to remote sensing image classification,” IEEE Trans. Geosci. Remote Sens., vol. 48, no. 1, pp. 186–197, Jan. 2010.

[41]

H. Bay, T. Tuytelaars, and L. Van Gool, “SURF: Speeded up robust features,” in Proc. 9th Eur. Conf. Comput. Vis. (ECCV), 2006, pp. 404–417.

[42]

SURF-Based Object Detection. [Online]. Available: http://www.mathworks.it/it/help/vision/examples/object-detection-in-a-cluttered-scene-using-point-feature-matching.html, accessed Jun. 25, 2014.

[43]

R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge, U.K.: Cambridge Univ. Press, 2003.

Digital Library

[44]

P. H. S. Torr and A. Zisserman, “MLESAC: A new robust estimator with application to estimating image geometry,” Comput. Vis. Image Understand., vol. 78, no. 1, pp. 138–156, 2000.

Digital Library

[45]

SparseLab. [Online]. Available: http://sparselab.stanford.edu, accessed Feb. 1, 2014.

[46]

C. E. Rasmussen and K. I. Williams. Gaussian Process Soft-Ware. [Online]. Available: http://www.Gaussianprocess.org/gpml/code/matlab/ doc/, accessed Jun. 25, 2014.

Cited By

Prasanna DTripathi S(2024)A Review on Tongue Based Assistive Technology, Devices and FPGA Processors Using Machine Learning ModuleWireless Personal Communications: An International Journal10.1007/s11277-024-10897-8134:1(151-170)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1007/s11277-024-10897-8
Müller KEngel CLoitsch CStiefelhagen RWeber G(2022)Traveling More Independently: A Study on the Diverse Needs and Challenges of People with Visual or Mobility Impairments in Unfamiliar Indoor EnvironmentsACM Transactions on Accessible Computing10.1145/351425515:2(1-44)Online publication date: 19-May-2022
https://dl.acm.org/doi/10.1145/3514255
Engel CMüller KConstantinescu ALoitsch CPetrausch VWeber GStiefelhagen R(2020)Travelling more independently: A Requirements Analysis for Accessible Journeys to Unknown Buildings for People with Visual ImpairmentsProceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3373625.3417022(1-11)Online publication date: 26-Oct-2020
https://dl.acm.org/doi/10.1145/3373625.3417022
Show More Cited By

Index Terms

A Compressive Sensing Approach to Describe Indoor Scenes for Blind People
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
  2. Machine learning

Index terms have been assigned to the content through auto-classification.

Recommendations

A Fast and Comprehensive Indoor Scene Understanding Approach: For Blind People
CICSYN '14: Proceedings of the 2014 Sixth International Conference on Computational Intelligence, Communication Systems and Networks

Scene perception and understanding is regarded as a major prospect for blind and visually impaired people. To date, there is no system than can completely recover the sight to them. However, various designs intended to improve their quality of life have ...
Exploring Art with a Voice Controlled Multimodal Guide for Blind People
TEI '19: Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction

There is an increasing concern to improve the accessibility of artworks for blind people. Much of the effort has been focused on helping the visually impaired people to access the exhibition facilities, but the works of art hosted there are still ...
An insight into smartphone-based assistive solutions for visually impaired and blind people: issues, challenges and opportunities
Abstract
Blind people are confronting a number of challenges in performing activities of daily life such as reading labels on a product, identification of currency notes, exploring unknown spaces, identifying the appearance of an object of interest, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Circuits and Systems for Video Technology

IEEE Transactions on Circuits and Systems for Video Technology Volume 25, Issue 7

July 2015

177 pages

ISSN:1051-8215

Issue’s Table of Contents

Copyright © 2014.

Publisher

IEEE Press

Publication History

Published: 01 July 2015

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Prasanna DTripathi S(2024)A Review on Tongue Based Assistive Technology, Devices and FPGA Processors Using Machine Learning ModuleWireless Personal Communications: An International Journal10.1007/s11277-024-10897-8134:1(151-170)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1007/s11277-024-10897-8
Müller KEngel CLoitsch CStiefelhagen RWeber G(2022)Traveling More Independently: A Study on the Diverse Needs and Challenges of People with Visual or Mobility Impairments in Unfamiliar Indoor EnvironmentsACM Transactions on Accessible Computing10.1145/351425515:2(1-44)Online publication date: 19-May-2022
https://dl.acm.org/doi/10.1145/3514255
Engel CMüller KConstantinescu ALoitsch CPetrausch VWeber GStiefelhagen R(2020)Travelling more independently: A Requirements Analysis for Accessible Journeys to Unknown Buildings for People with Visual ImpairmentsProceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3373625.3417022(1-11)Online publication date: 26-Oct-2020
https://dl.acm.org/doi/10.1145/3373625.3417022
Guo RMa SHan Y(2019)Image captioning: from structural tetrad to translated sentencesMultimedia Tools and Applications10.1007/s11042-018-7118-778:17(24321-24346)Online publication date: 1-Sep-2019
https://dl.acm.org/doi/10.1007/s11042-018-7118-7

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents