research-article

Real-World Product Deployment of Adaptive Push Notification Scheduling on Smartphones

Authors:

Tadashi Okoshi,

Kota Tsubouchi,

Hideyuki TokudaAuthors Info & Claims

KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Pages 2792 - 2800

https://doi.org/10.1145/3292500.3330732

Published: 25 July 2019 Publication History

Abstract

The limited attentional resource of users is a bottleneck to delivery of push notifications in today's mobile and ubiquitous computing environments. Adaptive mobile notification scheduling, which detects opportune timings based on mobile sensing and machine learning, has been proposed as a way of alleviating this problem. However, it is still not clear if such adaptive notifications are effective in a large-scale product deployment with real-world situations and configurations, such as users' context changes, personalized content in notifications, and sudden external factors that users commonly experience (such as breaking news). In this paper, we construct a new interruptibility estimation and adaptive notification scheduling with redesigned technical components. From the deploy study of the system to the real product stack of Yahoo! JAPAN Android application and evaluation with 382,518 users for 28 days, we confirmed several significant results, including the maximum 60.7% increase in the users' click rate, 10 times more gain compared to the previous system, significantly better gain in the personalized notification content, and unexpectedly better performance in a situation with exceptional breaking news notifications. With these results, the proposed system has officially been deployed and enabled to all the users of Yahoo! JAPAN product environment where more than 10 million Android app users are enjoying its benefit.

References

[1]

Utku Acer, Afra Mashhadi, Claudio Forlivesi, and Fahim Kawsar. 2015. Energy Efficient Scheduling for Mobile Push Notifications. In Proceedings of the 12th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services on 12th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (MOBIQUITOUS'15). 100--109.

Digital Library

[2]

Piotr D. Adamczyk and Brian P. Bailey. 2004. If not now, when?: the effects of interruption at different moments within task execution. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). 271--278.

Digital Library

[3]

Brian P. Bailey and Joseph A. Konstan. 2006. On the need for attention-aware systems: Measuring effects of interruption on task performance, error rate, and affective state. Computers in Human Behavior, Vol. 22, 4 (2006), 685 -- 708.

[4]

James "Bo" Begole, Nicholas E. Matsakis, and John C. Tang. 2004. Lilsys: Sensing Unavailability. In Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work (CSCW '04). 511--514.

Digital Library

[5]

Mary Czerwinski, Edward Cutrell, and Eric Horvitz. 2000. Instant messaging: Effects of relevance and timing. In People and computers XIV: Proceedings of HCI, Vol. 2. British Computer Society, 71--76.

[6]

Joel E. Fischer, Chris Greenhalgh, and Steve Benford. 2011. Investigating Episodes of Mobile Phone Activity As Indicators of Opportune Moments to Deliver Notifications. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). 181--190.

Digital Library

[7]

D. Garlan, D.P. Siewiorek, A. Smailagic, and P. Steenkiste. 2002. Project Aura: toward distraction-free pervasive computing. Pervasive Computing, IEEE, Vol. 1, 2 (april-june 2002), 22 --31.

Digital Library

[8]

Google Inc. {n. d.}. Making Your App Location-Aware - Android Developers . https://developer.android.com/intl/ja/training/location/index.html .

[9]

Sandy Gould, Duncan Brumby, Anna Cox, Victor González, Dario Salvucci, and Niels Taatgen. 2012. Multitasking and interruptions: a SIG on bridging the gap between research on the micro and macro worlds. In CHI'12 Extended Abstracts on Human Factors in Computing Systems. 1189--1192.

Digital Library

[10]

Joyce Ho and Stephen S. Intille. 2005. Using Context-aware Computing to Reduce the Perceived Burden of Interruptions from Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '05). 909--918.

Digital Library

[11]

Eric Horvitz and Johnson Apacible. 2003. Learning and Reasoning About Interruption. In Proceedings of the 5th International Conference on Multimodal Interfaces (ICMI '03). 20--27.

Digital Library

[12]

Eric Horvitz, Paul Koch, and Johnson Apacible. 2004. BusyBody: Creating and Fielding Personalized Models of the Cost of Interruption. In Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work (CSCW '04). 507--510.

Digital Library

[13]

Scott Hudson, James Fogarty, Christopher Atkeson, Daniel Avrahami, Jodi Forlizzi, Sara Kiesler, Johnny Lee, and Jie Yang. 2003. Predicting Human Interruptibility with Sensors: A Wizard of Oz Feasibility Study. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). 257--264.

Digital Library

[14]

Shamsi T. Iqbal and Brian P. Bailey. 2006. Leveraging Characteristics of Task Structure to Predict the Cost of Interruption. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06). 741--750.

Digital Library

[15]

Shamsi T. Iqbal and Brian P. Bailey. 2005. Investigating the Effectiveness of Mental Workload As a Predictor of Opportune Moments for Interruption. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). 1489--1492.

Digital Library

[16]

Shamsi T. Iqbal and Brian P. Bailey. 2010. Oasis: A Framework for Linking Notification Delivery to the Perceptual Structure of Goal-directed Tasks. ACM Transactions on Computer-Human Interaction, Vol. 17, 4, Article 15 (Dec. 2010), bibinfonumpages28 pages.

Digital Library

[17]

Shamsi T. Iqbal and Eric Horvitz. 2010. Notifications and Awareness: A Field Study of Alert Usage and Preferences. In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (CSCW '10). 27--30.

Digital Library

[18]

Daniel Kahneman. 1973. Attention and effort .Prentice-Hall, Inc.

[19]

J. G. Kreifeldt and M. E. McCarthy. 1981. Interruption as a test of the user-computer interface. In JPL Proceeding of the 17 th Annual Conference on Manual Control. 655--667.

[20]

Abhinav Mehrotra, Mirco Musolesi, Robert Hendley, and Veljko Pejovic. 2015. Designing Content-driven Intelligent Notification Mechanisms for Mobile Applications. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '15). 813--824.

Digital Library

[21]

James T Milord and Raymond P Perry. 1977. A Methodological Study of Overloadx. The Journal of General Psychology, Vol. 97, 1 (1977), 131--137.

[22]

Darren Newtson and Gretchen Engquist. 1976. The perceptual organization of ongoing behavior. Journal of Experimental Social Psychology, Vol. 12, 5 (1976), 436--450.

[23]

Tadashi Okoshi, Julian Ramos, Hiroki Nozaki, Jin Nakazawa, Anind K. Dey, and Hideyuki Tokuda. 2015a. Attelia: Reducing User's Cognitive Load due to Interruptive Notifications on Smart Phones. In Proceedings of IEEE International Conference on Pervasive Computing and Communications 2015 (PerCom '15).

[24]

Tadashi Okoshi, Julian Ramos, Hiroki Nozaki, Jin Nakazawa, Anind K. Dey, and Hideyuki Tokuda. 2015b. Reducing Users' Perceived Mental Effort Due to Interruptive Notifications in Multi-device Mobile Environments. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '15). 475--486.

Digital Library

[25]

Tadashi Okoshi, Kota Tsubouchi, Masaya Taji, Takanori Ichikawa, and Hideyuki Tokuda. 2017. Attention and Engagement-Awareness in the Wild : A Large-Scale Study with Adaptive Notifications. In Proceedings of IEEE International Conference on Pervasive Computing and Communications 2017 (PerCom '17) . 100--110.

[26]

Veljko Pejovic and Mirco Musolesi. 2014. InterruptMe : Designing Intelligent Prompting Mechanisms for Pervasive Applications. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '14). 395--906.

Digital Library

[27]

Martin Pielot, Tilman Dingler, Jose San Pedro, and Nuria Oliver. 2015. When Attention is Not Scarce - Detecting Boredom from Mobile Phone Usage. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '15). 825--836.

Digital Library

[28]

Benjamin Poppinga, Martin Pielot, Niels Henze, Nuria Oliver, Karen Church, and Alireza Sahami Shirazi. 2015. Smarttention, Please! Intelligent Attention Management on Mobile Devices. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI '15). 1066--1069.

Digital Library

[29]

Richard M Shiffrin and Walter Schneider. 1977. Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. Psychological review, Vol. 84, 2 (1977), 127.

[30]

Herbert A Simon. 1971. Designing organizations for an information-rich world. Computers, communication, and the public interest, Vol. 37 (1971), 40--41.

[31]

Cheri Speier, Joseph S Valacich, and Iris Vessey. 1999. The influence of task interruption on individual decision making: An information overload perspective. Decision Sciences, Vol. 30, 2 (1999), 337--360.

[32]

Robert J. Sternberg and Karin Sternberg. 2012. Cognitive Psychology 6 ed.). Cengage Learning.

[33]

G. H. (Henri) ter Hofte. 2007. Xensible Interruptions from Your Mobile Phone. In Proceedings of the 9th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '07). 178--181.

Digital Library

[34]

Alvin Toffler. 1990. Future shock .Bantam.

[35]

Kota Tsubouchi and Tadashi Okoshi. 2017. People's Interruptibility In-the-wild: Analysis of Breakpoint Detection Model in a Large-scale Study. In Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers (UbiComp '17). ACM, New York, NY, USA, 922--927.

Digital Library

[36]

Alexandra Voit, Benjamin Poppinga, Dominik Weber, Matthias Bo?hmer, Niels Henze, Sven Gehring, Tadashi Okoshi, and Veljko Pejovic. 2016. UbiTtention: Smart & Ambient Notification and Attention Management. In Adjunct Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '16). 1520--1523.

Digital Library

[37]

Dominik Weber, Alireza Sahami Shirazi, Sven Gehring, Niels Henze, Benjamin Poppinga, Martin Pielot, and Tadashi Okoshi. 2016. Smarttention, Please!: 2Nd Workshop on Intelligent Attention Management on Mobile Devices. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI '16). 914--917.

Digital Library

[38]

Fred RH Zijlstra, Robert A Roe, Anna B Leonora, and Irene Krediet. 1999. Temporal factors in mental work: Effects of interrupted activities. Journal of Occupational and Organizational Psychology, Vol. 72, 2 (1999), 163--185.

Cited By

Lee YWu MChang CChang XChang Y(2024)Investigating User-perceived Impacts of Contextual Factors on Opportune MomentsProceedings of the ACM on Human-Computer Interaction10.1145/36765148:MHCI(1-28)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676514
Ito MTsubouchi KNishio NShimosaka MTaya ASezaki KNishiyama YKostakos VKay JHoang T(2024)Investigating Acceptable Voice-based Notification Timings through Earable Devices: A Preliminary Field StudyCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3677579(30-34)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3677579
Mathis LBubeck CPeissner M(2024)When to Approach the User: Investigating Suitable Context Factors for Proactive Voice Assistance in Automated CarsProceedings of Mensch und Computer 202410.1145/3670653.3670672(299-309)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1145/3670653.3670672
Show More Cited By

Index Terms

Real-World Product Deployment of Adaptive Push Notification Scheduling on Smartphones
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools
    2. Ubiquitous and mobile devices
      1. Smartphones

Recommendations

Understanding notification stress of smartphone messenger app
CHI EA '14: CHI '14 Extended Abstracts on Human Factors in Computing Systems

Today, many smartphone users experience stress from receiving notifications all the time. In this work-in-progress paper, we explore the relationships between users' stress levels from receiving smartphone notifications and notification setting ...
Towards end-user development of REST client applications on smartphones

HTML5 can be used to develop client applications by composing REST web services within the context of Web 2.0. However, the possibility of implementing cross-platform smartphone applications with REST services needs to be studied. Accordingly, we ...
DaPanda: detecting aggressive push notifications in Android apps
ASE '19: Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering

Mobile push notifications have been widely used in mobile platforms to deliver all sorts of information to app users. Although it offers great convenience for both app developers and mobile users, this feature was frequently reported to serve malicious ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

July 2019

3305 pages

ISBN:9781450362016

DOI:10.1145/3292500

General Chairs:
Ankur Teredesai
KenSci
,
Vipin Kumar
University of Minnesota
,
Program Chairs:
Ying Li
EV Analysis Corporation
,
Rómer Rosales
LinkedIn
,
Evimaria Terzi
Boston University
,
George Karypis
University of Minnesota

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '19

Sponsor:

KDD '19: The 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 4 - 8, 2019

AK, Anchorage, USA

Acceptance Rates

KDD '19 Paper Acceptance Rate 110 of 1,200 submissions, 9%;

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
1,196
Total Downloads

Downloads (Last 12 months)104
Downloads (Last 6 weeks)5

Reflects downloads up to 02 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lee YWu MChang CChang XChang Y(2024)Investigating User-perceived Impacts of Contextual Factors on Opportune MomentsProceedings of the ACM on Human-Computer Interaction10.1145/36765148:MHCI(1-28)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676514
Ito MTsubouchi KNishio NShimosaka MTaya ASezaki KNishiyama YKostakos VKay JHoang T(2024)Investigating Acceptable Voice-based Notification Timings through Earable Devices: A Preliminary Field StudyCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3677579(30-34)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3677579
Mathis LBubeck CPeissner M(2024)When to Approach the User: Investigating Suitable Context Factors for Proactive Voice Assistance in Automated CarsProceedings of Mensch und Computer 202410.1145/3670653.3670672(299-309)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1145/3670653.3670672
Chen YWoodward JBista DZhang XSingh IObajemu OShankar MRoss KRuiz JAnthony L(2024)Investigating Contextual Notifications to Drive Self-Monitoring in mHealth Apps for Weight MaintenanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641993(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641993
Quin FWeyns DGalster MSilva C(2024)A/B testingJournal of Systems and Software10.1016/j.jss.2024.112011211:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.jss.2024.112011
Pinto VOliveira LDe Sousa YDanyi SLima D(2023)Smart Notifications – An ML-based Framework to Boost User EngagementProceedings of the 29th Brazilian Symposium on Multimedia and the Web10.1145/3617023.3617053(25-31)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3617023.3617053
Lin CWu CHuang PLai YHo YChung CChang Y(2023)Scanning or Simply Unengaged in Reading? Opportune Moments for Pushed News Notifications and Their Relationship with Smartphone Users' Choice of News-reading ModesProceedings of the ACM on Human-Computer Interaction10.1145/36042687:MHCI(1-26)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604268
Chang XHsu FLiang EChiou ZChuang HTseng FLin YChang Y(2023)Not Merely Deemed as Distraction: Investigating Smartphone Users’ Motivations for Notification-InteractionProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581146(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581146
Zhang YZhao MLi CTou WChen HNiu DYin CHe YGuo F(2023)Online Volume Optimization for Notifications via Long Short-Term Value ModelingAdvances in Knowledge Discovery and Data Mining10.1007/978-3-031-33380-4_2(16-28)Online publication date: 27-May-2023
https://doi.org/10.1007/978-3-031-33380-4_2
Alghamdi ARiasat HIqbal MAshraf MAlshahrani AAlshamrani A(2022)Intelligence and Usability Empowerment of Smartphone Adaptive FeaturesApplied Sciences10.3390/app12231224512:23(12245)Online publication date: 30-Nov-2022
https://doi.org/10.3390/app122312245
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents