research-article

Finding Weather Photos: Community-Supervised Methods for Editorial Curation of Online Sources

Authors:

David A. Shamma,

Lyndon Kennedy,

Jeff YuanAuthors Info & Claims

CSCW '16: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing

Pages 86 - 96

https://doi.org/10.1145/2818048.2819989

Published: 27 February 2016 Publication History

Abstract

There are many cues that can be used to curate media from social networking websites. Beyond metadata, group behavior provide a strong community-based signal for surfacing images, which we show in a user-defined curatorial task. In a departure from mirco-task crowdwork, we observe that the curation inherent in online photo communities guides the discoverability and consumption of the media, which in turn provides a strong signal that can be used in new editorial tasks in a community-supervised manner. We use this approach in tandem with other more conventional multimedia methods (i.e.\ computer vision and contextual metadata) to form a broad multimodal approach to retrieval and recommendation. We present a large-scale system implementation on a real-world curative task for weather images on a web-scale dataset. Finally, we conduct an evaluation of this system using professional editors and find substantial improvements in editorial efficiency.

Supplementary Material

Shamma-suppl.txt (86_shamma_supmat.txt)

Shamma Supplemental Materials

Download
4.23 KB

References

[1]

Ricardo Baeza-Yates, Berthier Ribeiro-Neto, and Yoelle Maarek. 2011. Modern Information Retrieval: The Concepts and Technology behind Search. Addison-Wesley, Chapter Web Retrieval.

[2]

Subhabrata Bhattacharya, Rahul Sukthankar, and Mubarak Shah. 2011. A Holistic Approach to Aesthetic Enhancement of Photographs. ACM Transactions on Multimedia Computing, Communications, and Applications 7S, 1, Article 21 (Nov. 2011), 21 pages.

Digital Library

[3]

Donald E. Broadbent and Margaret H.P. Broadbent. 1987. From detection to identfication: Response to multiple targets in rapid serial visual presentation. Perception & Psychophysics 42, 2 (1987), 105–113.

[4]

Dick C. A. Bulterman, Pablo Cesar, and Rodrigo Laiola Guimarães. 2013. Socially-aware Multimedia Authoring: Past, Present, and Future. ACM Transactions on Multimedia Computing, Communications, and Applications 9, 1s, Article 35 (Oct. 2013), 23 pages.

Digital Library

[5]

Shuo Chang, Vikas Kumar, Eric Gilbert, and Loren G. Terveen. 2014. Specialization, Homophily, and Gender in a Social Curation Site: Findings from Pinterest. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '14). ACM, New York, NY, USA, 674–686.

Digital Library

[6]

William J Clancey. 1997. Situated cognition: On human knowledge and computer representations. Cambridge University Press.

Digital Library

[7]

Ritendra Datta, Dhiraj Joshi, Jia Li, and James Z Wang. 2006. Studying Aesthetics in Photographic Images Using a Computational Approach. In Proceedings of the European Conference on Computer Vision, Vol. 3. 288–301.

Digital Library

[8]

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.

[9]

Nicholas A. Diakopoulos. 2015. The Editor's Eye: Curation and Comment Relevance on the New York Times. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15). ACM, New York, NY, USA, 1153–1157.

Digital Library

[10]

Motahhare Eslami, Aimee Rickman, Kristen Vaccaro, Amirhossein Aleyasen, Andy Vuong, Karrie Karahalios, Kevin Hamilton, and Christian Sandvig. 2015. 'I Always Assumed That I Wasn'T Really That Close to {Her}': Reasoning About Invisible Algorithms in News Feeds. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 153–162.

Digital Library

[11]

Kenneth M. Ford, Clark Glymour, and Patrick J. Hayes. 1997. On the Other Hand - Cognitive Prostheses. AI Magazine 18, 3 (1997), 104.

[12]

Eric Gilbert, Saeideh Bakhshi, Shuo Chang, and Loren Terveen. 2013. 'I Need to Try This'?: A Statistical Overview of Pinterest. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 2427–2436.

Digital Library

[13]

Rebecca Gulotta, Haakon Faste, and Jennifer Mankoff. 2012. Curation, Provocation, and Digital Identity: Risks and Motivations for Sharing Provocative Images Online. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 387–390.

Digital Library

[14]

Douglas Harper. 2002. Talking about pictures: A case for photo elicitation. Visual studies 17, 1 (2002), 13–26.

[15]

R.R. Hoffman, P.J. Hayes, and K.M. Ford. 2001. Human-centered computing: thinking in and out of the box. IEEE Intelligent Systems 16, 5 (Sep 2001), 76–78.

Digital Library

[16]

Liang-Chi Hsieh, W. H. Hsu, and Hao-Chuan Wang. 2014. Investigating and predicting social and visual image interestingness on social media by crowdsourcing. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 4309–4313.

[17]

Katsuhiko Ishiguro, Akisato Kimura, and Koh Takeuchi. 2012. Towards Automatic Image Understanding and Mining via Social Curation. In Proceedings of the IEEE International Conference on Data Mining. 906–911.

Digital Library

[18]

Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional Architecture for Fast Feature Embedding. arXiv preprint arXiv:1408.5093 (2014).

[19]

Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David A. Shamma, Michael Bernstein, and Li Fei-Fei. 2015. Image Retrieval using Scene Graphs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3668–3678.

[20]

Joseph B. Kadane and Lara J. Wolfson. 1998. Experiences in Elicitation. Journal of the Royal Statistical Society. Series D (The Statistician) 47, 1 (1998), pp. 3–19. http://www.jstor.org/stable/2988424

[21]

Yan Ke, Xiaoou Tang, and Feng Jing. 2006. The Design of High-Level Features for Photo Quality Assessment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 419–426.

Digital Library

[22]

Aisling Kelliher. 2014. Critical Multimedia. IEEE MultiMedia 21, 4 (Oct 2014), 4–7.

[23]

Andruid Kerne, Andrew M. Webb, Steven M. Smith, Rhema Linder, Nic Lupfer, Yin Qu, Jon Moeller, and Sashikanth Damaraju. 2014. Using Metrics of Curation to Evaluate Information-Based Ideation. ACM Trans. Comput.-Hum. Interact. 21, 3, Article 14 (June 2014), 48 pages.

Digital Library

[24]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems.

Digital Library

[25]

Yiwen Luo and Xiaoou Tang. 2008. Photo and Video Quality Evaluation: Focusing on the Subject. In Proceedings of the European Conference on Computer Vision, Vol. 3. 386–399.

Digital Library

[26]

Catherine C Marshall and Frank M Shipman III. 1995. Spatial hypertext: designing for change. Commun. ACM 38, 8 (1995), 88–97.

Digital Library

[27]

Mor Naaman. 2012. Social Multimedia: Highlighting Opportunities for Search and Mining of Multimedia Data in Social Media Applications. Multimedia Tools and Applications 56, 1 (Jan. 2012), 9–34.

Digital Library

[28]

Adrian Popescu, Symeon Papadopoulos, and Ioannis Kompatsiaris. 2014. USEMP at MediaEval Placing Task. Working notes of MediaEval. (2014).

[29]

Mary C. Potter. 1976. Short-term conceptual memory for pictures. Journal of Experimental Psychology: Human Learning and Memory 2, 5 (September 1976), 509–522.

[30]

Emilee Rader and Rebecca Gray. 2015. Understanding User Beliefs About Algorithmic Curation in the Facebook News Feed. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 173–182.

Digital Library

[31]

Judith Alice Redi, Tobias HoãYfeld, Pavel Korshunov, Filippo Mazza, Isabel Povoa, and Christian Keimel. 2013. Crowdsourcing-based Multimedia Subjective Evaluations: A Case Study on Image Recognizability and Aesthetic Appeal. In Proceedings of the ACM International Workshop on Crowdsourcing for Multimedia (CrowdMM '13). ACM, New York, NY, USA, 29–34.

Digital Library

[32]

F. Ribeiro, D. Florencio, and V. Nascimento. 2011. Crowdsourcing subjective image quality evaluation. In Proceedings of the IEEE International Conference on Image Processing. 3097–3100.

[33]

S. Rudinac, M. Larson, and A. Hanjalic. 2013. Learning Crowdsourced User Preferences for Visual Summarization of Image Collections. IEEE Transactions on Multimedia 15, 6 (Oct 2013), 1231–1243.

Digital Library

[34]

David A. Shamma, Ryan Shaw, Peter L. Shafton, and Yiming Liu. 2007. Watch What I Watch: Using Community Activity to Understand Content. In Proceedings of the International Workshop on Workshop on Multimedia Information Retrieval (MIR '07). ACM, New York, NY, USA, 275–284.

Digital Library

[35]

Aaron Shaw, Haoqi Zhang, Andrés Monroy-Hernández, Sean Munson, Benjamin Mako Hill, Elizabeth Gerber, Peter Kinnaird, and Patrick Minder. 2014. Computer Supported Collective Action. ACM interactions 21, 2 (March 2014), 74–77.

Digital Library

[36]

H. R. Sheikh, A. C. Bovik, and G. de Veciana. 2005. An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Transactions on Image Processing 14, 12 (2005), 2117–2128.

Digital Library

[37]

Zoe Strickler. 1999. Elicitation Methods in Experimental Design Research. Design Issues 15, 2 (1999), pp. 27–39. http://www.jstor.org/stable/1511840

[38]

Lucy A. Suchman. 1987. Plans and Situated Actions: The Problem of Human-machine Communication. Cambridge University Press, New York, NY, USA.

Digital Library

[39]

Vladimir Vapnik. 1998. Statistical Learning Theory. Wiley, New York.

[40]

Annika Wolff and Paul Mulholland. 2013. Curation, Curation, Curation. In Proceedings of the 3rd Narrative and Hypertext Workshop (NHT '13). ACM, New York, NY, USA, Article 1, 5 pages.

Digital Library

[41]

Lexing Xie, David A. Shamma, and Cees Snoek. 2014. Content is Dead...Long Live Content: The New Age of Multimedia-Hard Problems. IEEE MultiMedia 21, 1 (Jan 2014), 4–8.

[42]

Qianqian Xu, Qingming Huang, and Yuan Yao. 2012. Online crowdsourcing subjective image quality assessment. In Proceedings of the ACM International Conference on Multimedia. 359–368.

Digital Library

[43]

Yan Xu, Joshua Ratcliff, James Scovell, Gheric Speiginer, and Ronald Azuma. 2015. Real-time Guidance Camera Interface to Enhance Photo Aesthetic Quality. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1183–1186.

Digital Library

[44]

Keiji Yanai, Keita Yaegashi, and Bingyu Qiu. 2009. Detecting Cultural Differences Using Consumer-generated Geotagged Photos. In Proceedings of the International Workshop on Location and the Web (LOCWEB '09). ACM, New York, NY, USA, Article 12, 4 pages.

Digital Library

[45]

Wayne Zachary, Robert Hoffman, Beth Crandall, Tom Miller, and Christopher Nemeth. 2012. 'Rapidized' Cognitive Task Analysis. IEEE Intelligent Systems 27, 2 (March 2012), 61–66.

Digital Library

[46]

Fang-Lue Zhang, Miao Wang, and Shi-Min Hu. 2013. Aesthetic Image Enhancement by Dependence-Aware Object Recomposition. IEEE Transactions on Multimedia 15, 7 (2013), 1480–1490.

Digital Library

[47]

Xuan Zhao and Siân E. Lindley. 2014. Curation Through Use: Understanding the Personal Value of Social Media. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 2431–2440.

Digital Library

Cited By

Chen YMonroy-Hernández AWehrman IOney SLasecki WVaish R(2020)Sifter: A Hybrid Workflow for Theme-based Video Curation at ScaleProceedings of the 2020 ACM International Conference on Interactive Media Experiences10.1145/3391614.3393657(65-73)Online publication date: 17-Jun-2020
https://dl.acm.org/doi/10.1145/3391614.3393657
Porter EKrafft PKeegan B(2020)Visual Narratives and Collective Memory across Peer-Produced Accounts of Contested Sociopolitical EventsACM Transactions on Social Computing10.1145/33731473:1(1-20)Online publication date: 26-Feb-2020
https://dl.acm.org/doi/10.1145/3373147
Fernando PWeiler JKuznetsov SBrewster SFitzpatrick GCox AKostakos V(2019)A Rough Sketch of the Freehand Drawing ProcessProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300312(1-13)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300312
Show More Cited By

Recommendations

Tagged photos: concerns, perceptions, and protections
CHI EA '09: CHI '09 Extended Abstracts on Human Factors in Computing Systems

Photo sharing has become a popular feature of many online social networking sites. Many of the photo sharing applications on these sites, allow users to annotate photos with those who are in them. A number of researchers have examined the social uses ...
Producing timely recommendations from social networks through targeted search
AAMAS '09: Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 2

There has been a significant increase in interest and participation in social networking websites recently. For many users, social networks are indispensable tools for sharing personal information and keeping abreast with updates by their acquaintances. ...
Demographics, weather and online reviews: a study of restaurant recommendations
WWW '14: Proceedings of the 23rd international conference on World wide web

Online recommendation sites are valuable information sources that people contribute to, and often use to choose restaurants. However, little is known about the dynamics behind participation in these online communities and how the recommendations in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CSCW '16: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing

February 2016

1866 pages

ISBN:9781450335928

DOI:10.1145/2818048

General Chairs:
Darren Gergle
Northwestern University
,
Meredith Ringel Morris
Microsoft Research
,
Program Chairs:
Pernille Bjørn
University of Copenhagen
,
Joseph Konstan
University of Minnesota

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 February 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CSCW '16

Sponsor:

SIGCHI

CSCW '16: Computer Supported Cooperative Work and Social Computing

February 27 - March 2, 2016

California, San Francisco, USA

Acceptance Rates

CSCW '16 Paper Acceptance Rate 142 of 571 submissions, 25%;

Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

Upcoming Conference

CSCW '25

Sponsor:
sigchi

Computer-Supported Cooperative Work and Social Computing

October 18 - 22, 2025

Bergen , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
110
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)2

Reflects downloads up to 16 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen YMonroy-Hernández AWehrman IOney SLasecki WVaish R(2020)Sifter: A Hybrid Workflow for Theme-based Video Curation at ScaleProceedings of the 2020 ACM International Conference on Interactive Media Experiences10.1145/3391614.3393657(65-73)Online publication date: 17-Jun-2020
https://dl.acm.org/doi/10.1145/3391614.3393657
Porter EKrafft PKeegan B(2020)Visual Narratives and Collective Memory across Peer-Produced Accounts of Contested Sociopolitical EventsACM Transactions on Social Computing10.1145/33731473:1(1-20)Online publication date: 26-Feb-2020
https://dl.acm.org/doi/10.1145/3373147
Fernando PWeiler JKuznetsov SBrewster SFitzpatrick GCox AKostakos V(2019)A Rough Sketch of the Freehand Drawing ProcessProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300312(1-13)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300312
Chancellor SKalantidis YPater JDe Choudhury MShamma DMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)Multimodal Classification of Moderated Online Pro-Eating Disorder ContentProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025985(3213-3226)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3025985
Bica MPalen LBopp CLee CPoltrock SBarkhuus LBorges MKellogg W(2017)Visual Representations of DisasterProceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing10.1145/2998181.2998212(1262-1276)Online publication date: 25-Feb-2017
https://dl.acm.org/doi/10.1145/2998181.2998212

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents