research-article

LiveMaps: Learning Geo-Intent from Images of Maps on a Large Scale

Authors:

Michael R. Evans,

Ahmad Mahmoody,

Dragomir Yankov,

Florin Teodorescu,

Pavel BerkhinAuthors Info & Claims

SIGSPATIAL '17: Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

Article No.: 31, Pages 1 - 9

https://doi.org/10.1145/3139958.3139965

Published: 07 November 2017 Publication History

Abstract

Image search is a popular application on web search engines. Issuing a location-related query on an image search engine often returns multiple images of maps among the top ranked results. Traditionally, clicking on such images either opens the image in a new browser tab or takes users to a web page containing the image. However, finding the area of intent on an interactive web map (e.g., Bing Maps) is a manual process. In this paper, we describe a novel system, LiveMaps, for analyzing and retrieving an appropriate map viewport for a given image of a map. This provides annotation of images of maps returned by image search engines, allowing users to directly open a link to an interactive map centered on the location of interest.

LiveMaps works in several stages. It first checks whether the input image represents a map. If yes, then the system attempts to identify what geographical area this map image represents. In the process, we use textual as well as visual information extracted from the image. Finally, we construct an interactive map object capturing the geographical area inferred for the image. Evaluation results on a dataset of labeled map images indicate our system constructs precise map representations while also achieving good levels of coverage.

References

[1]

2017. Bing Images. (2017). http://www.bing.com/images

[2]

2017. Bing Maps. (2017). http://www.bing.com/maps

[3]

Sean Bell and Kavita Bala. 2015. Learning visual similarity for product design with convolutional neural networks. ACM Transactions on Graphics (TOG) 34, 4 (2015), 98.

Digital Library

[4]

Pavel Berkhin, Michael R. Evans, Florin Teodorescu, Wei Wu, and Dragomir Yankov. 2015. A New Approach to Geocoding: BingGC (SIGSPATIAL '15). ACM, New York, NY, USA.

Digital Library

[5]

Gal Chechik, Varun Sharma, Uri Shalit, and Samy Bengio. 2010. Large scale online learning of image similarity through ranking. Journal of Machine Learning Research 11, Mar (2010), 1109--1135.

Digital Library

[6]

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09.

[7]

Zhengming Ding, Nasser M Nasrabadi, and Yun Fu. 2016. Task-driven deep transfer learning for image classification. In Acoustics, Speech and Signal Processing (ICASSP), 2016 IEEE International Conference on. IEEE, 2414--2418.

[8]

Michael R. Evans, Dragomir Yankov, Pavel Berkhin, Pavel Yudin, Florin Teodorescu, and Wei Wu. 2017. LiveMaps - Converting Map Images into Interactive Maps. In SIGIR '17. Shinjuku, Japan.

Digital Library

[9]

Jerome H Friedman. 2001. Greedy function approximation: a gradient boosting machine. Annals of statistics (2001).

[10]

Andrew Gallagher, Dhiraj Joshi, Jie Yu, and Jiebo Luo. 2009. Geo-location inference from image content and user tags. In CVPR Workshops 2009. IEEE, 55--62.

[11]

Moritz Hardt and Tengyu Ma. 2016. Identity Matters in Deep Learning. (2016). http://arxiv.org/abs/1611.04231

[12]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR16.

[13]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.

Digital Library

[14]

Sihan Li, Jiantao Jiao, Yanjun Han, and Tsachy Weissman. 2016. Demystifying ResNet. (2016). http://arxiv.org/abs/1611.01186

[15]

Yunpeng Li, Noah Snavely, Daniel P Huttenlocher, and Pascal Fua. 2016. Worldwide pose estimation using 3d point clouds. In Large-Scale Visual Geo-Localization. Springer, 147--163.

[16]

Heng Liu, Tao Mei, Jiebo Luo, Houqiang Li, and Shipeng Li. 2012. Finding perfect rendezvous on the go: accurate mobile visual localization and its applications to routing. In ACM Multimedia '12. ACM, 9--18.

Digital Library

[17]

Jiebo Luo, Dhiraj Joshi, Jie Yu, and Andrew Gallagher. 2011. Geotagging in multimedia and computer vision - survey. Multimedia Tools and Applications 51, 1 (2011), 187--211.

Digital Library

[18]

Sinno Jialin Pan and Qiang Yang. 2010. A survey on transfer learning. IEEE Transactions on knowledge and data engineering 22, 10 (2010), 1345--1359.

Digital Library

[19]

Arun Sacheti and Eason Want. 2015. The Image Graph-Powering the Next Generation of Bing Image Search. https://blogs.bing.com/search-quality-insights/2015/04/22/the-image-graph-powering-the-next-generation-of-bing-image-search. (2015).

[20]

Hanan Samet and Aya Soffer. 1996. Marco: Map retrieval by content. IEEE Transactions on Pattern Analysis and Machine Intelligence 18, 8 (1996), 783--798.

Digital Library

[21]

Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. (2014). http://arxiv.org/abs/1409.1556

[22]

Krysta M Svore and CJ Burges. 2011. Large-scale learning to rank using boosted decision trees. Scaling Up Machine Learning: Parallel and Distributed Approaches 2 (2011).

[23]

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In CVPR. IEEE Computer Society, 1--9.

[24]

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2818--2826.

[25]

Benjamin E Teitler, Michael D Lieberman, Daniele Panozzo, Jagan Sankaranarayanan, Hanan Samet, and Jon Sperling. 2008. NewsStand: A new view on news. In Proceedings of the 16th ACM SIGSPATIAL international conference on Advances in geographic information systems. ACM, 18.

Digital Library

[26]

Jiang Wang, Yang Song, Thomas Leung, Chuck Rosenberg, Jingbin Wang, James Philbin, Bo Chen, and Ying Wu. 2014. Learning fine-grained image similarity with deep ranking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1386--1393.

Digital Library

[27]

Zhou Wang and Eero P Simoncelli. 2005. Translation insensitive image similarity in complex wavelet domain. In Acoustics, Speech, and Signal Processing, 2005. Proceedings.(ICASSP'05). IEEE International Conference on, Vol. 2. IEEE, ii--573.

[28]

Benjamin P. Wing and Jason Baldridge. 2011. Simple Supervised Document Geolocation with Geodesic Grids. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies -Volume 1 (HLT '11). Association for Computational Linguistics, Stroudsburg, PA, USA, 955--964. http://dl.acm.org/citation.cfm?id=2002472.2002593

Digital Library

[29]

Ruimao Zhang, Liang Lin, Rui Zhang, Wangmeng Zuo, and Lei Zhang. 2015. Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Transactions on Image Processing 24, 12 (2015), 4766--4779.

Digital Library

[30]

Yan-Tao Zheng, Ming Zhao, Yang Song, Hartwig Adam, Ulrich Buddemeier, Alessandro Bissacco, Fernando Brucher, Tat-Seng Chua, and Hartmut Neven. 2009. Tour the world: building a web-scale landmark recognition engine. In Computer vision and pattern recognition, 2009. CVPR 2009. IEEE conference on. IEEE, 1085--1092.

[31]

Fuzhen Zhuang, Xiaohu Cheng, Ping Luo, Sinno Jialin Pan, and Qing He. 2015. Supervised Representation Learning: Transfer Learning with Deep Autoencoders. In IJCAI. 4119--4125.

Digital Library

Cited By

Kang YGao SRoth R(2024)Artificial intelligence studies in cartography: a review and synthesis of methods, applications, and ethicsCartography and Geographic Information Science10.1080/15230406.2023.2295943(1-32)Online publication date: 16-Jan-2024
https://doi.org/10.1080/15230406.2023.2295943
Hu YGui ZWang JLi M(2021)Enriching the metadata of map images: a deep learning approach with GIS-based data augmentationInternational Journal of Geographical Information Science10.1080/13658816.2021.196840736:4(799-821)Online publication date: 23-Aug-2021
https://doi.org/10.1080/13658816.2021.1968407

Index Terms

LiveMaps: Learning Geo-Intent from Images of Maps on a Large Scale
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Information extraction
    2. Specialized information retrieval
      1. Multimedia and multimodal retrieval
        Image search

Recommendations

LiveMaps: Converting Map Images into Interactive Maps
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

Image search is a popular application on web search engines. Issuing a location-related query in image search engines often returns multiple images of maps among the top ranked results. Traditionally, clicking on such images either opens the image in a ...
Why People Search for Images using Web Search Engines
WSDM '18: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining

What are the intents or goals behind human interactions with image search engines? Knowing why people search for images is of major concern to Web image search engines because user satisfaction may vary as intent varies. Previous analyses of image ...
Learning to re-rank: query-dependent image re-ranking using click data
WWW '11: Proceedings of the 20th international conference on World wide web

Our objective is to improve the performance of keyword based image search engines by re-ranking their original results. To this end, we address three limitations of existing search engines in this paper. First, there is no straight-forward, fully ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGSPATIAL '17: Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 2017

677 pages

ISBN:9781450354905

DOI:10.1145/3139958

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSPATIAL: ACM Special Interest Group on Spatial Information

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SIGSPATIAL'17

Sponsor:

SIGSPATIAL

SIGSPATIAL'17: 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 7 - 10, 2017

CA, Redondo Beach, USA

Acceptance Rates

SIGSPATIAL '17 Paper Acceptance Rate 39 of 193 submissions, 20%;

Overall Acceptance Rate 220 of 1,116 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
161
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kang YGao SRoth R(2024)Artificial intelligence studies in cartography: a review and synthesis of methods, applications, and ethicsCartography and Geographic Information Science10.1080/15230406.2023.2295943(1-32)Online publication date: 16-Jan-2024
https://doi.org/10.1080/15230406.2023.2295943
Hu YGui ZWang JLi M(2021)Enriching the metadata of map images: a deep learning approach with GIS-based data augmentationInternational Journal of Geographical Information Science10.1080/13658816.2021.196840736:4(799-821)Online publication date: 23-Aug-2021
https://doi.org/10.1080/13658816.2021.1968407

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents