Article

Evaluating the accuracy of data collection on mobile phones: a study of forms, sms, and voice

Authors:

Somani Patnaik,

Emma Brunskill,

William ThiesAuthors Info & Claims

ICTD'09: Proceedings of the 3rd international conference on Information and communication technologies and development

Pages 74 - 84

Published: 17 April 2009 Publication History

Abstract

While mobile phones have found broad application in reporting health, financial, and environmental data, there has been little study of the possible errors incurred during mobile data collection. This paper provides the first (to our knowledge) quantitative evaluation of data entry accuracy on mobile phones in a resource-poor setting. Via a study of 13 users in Gujarat, India, we evaluated three user interfaces: 1) electronic forms, containing numeric fields and multiple-choice menus, 2) SMS, where users enter delimited text messages according to printed cue cards, and 3) voice, where users call an operator and dictate the data in real-time.

Our results indicate error rates (per datum entered) of 4.2% for electronic forms, 4.8% for SMS, and 0.45% for voice. These results caused us to migrate our own initiative (a tuberculosis treatment program in rural India) from electronic forms to voice, in order to avoid errors on critical health data. While our study has some limitations, including varied backgrounds and training of participants, it suggests that some care is needed in deploying electronic interfaces in resource-poor settings. Further, it raises the possibility of using voice as a low-tech, high-accuracy, and cost-effective interface for mobile data collection.

References

[1]

Achieving the Health Millennium Development Goals in Asia and the Pacific: Policies and Actions Within Health Systems and Beyond. United Nations Publications, Dec. 2007.

[2]

V. Anantraman, T. Mikkelsen, R. Khilnani, V. S. Kumar, N. R. Machiraju, A. Pentland, and L. Ohno-Machado. Handheld computers for rural healthcare: Experiences from research concept to global operations. In Workshop on Development by Design, 2002.

[3]

A. Bernabe-Ortiz, W. H. Curioso, M. A. Gonzales, W. Evangelista, J. M. Castagnetto, C. P. Carcamo, J. P. Hughes, P. J. Garcia, G. P. Garnett, and K. K. Holmes. Handheld computers for self-administered sensitive data collection: A comparative study in Peru. BMC Medical Informatics and Decision Making, 8:11, 2008.

[4]

I. Blaya and H. S. F. Fraser. Development, implementation and preliminary study of a PDA-based bacteriology collection system. Proceedings of the AMIA Annual Symposium, 2006:4145, 2006.

[5]

J. A. Blaya, W. Gomez, P. Rodriguez, and H. Fraser. Cost and implementation analysis of a personal digital assistant system for laboratory data collection. The International Journal of Tuberculosis and Lung Disease, 12(8):921-7, Aug. 2008.

[6]

Cost effectiveness study report for the PDA data capture and transmission. SATELLIFE, Sept. 2004. http://www.healthnet.org/coststudy.php.

[7]

W. H. Curioso, B. T. Karras, P. E. Campos, C. Buenda, K. K. Holmes, and A. M. Kimball. Design and implementation of Cell-PREVEN: A real-time surveillance system for adverse events using cell phones in Peru. Proceedings ofthe AMIA Annual Symposium, 2005:176180, 2005.

[8]

B. DeRenzi, N. Lesh, T. Parikh, C. Sims, W. Maokla, M. Chemba, Y. Hamisi, D. S. Hellenberg, M. Mitchell, and G. Borriello. E-IMCI: Improving pediatric health care in low-income countries. In CHI, 2008.

Digital Library

[9]

L. Diero, I. Rotich, I. Bii, B. Mamlin, R. Einterz, I. Kalamai, and W. Tierney. A computer-based medical record system and personal digital assistants to assess and follow patients with respiratory tract infections visiting a rural Kenyan health centre. BMC Medical Informatics and Decision Making, 6(1):21, 2006.

[10]

T. Engebretsen. Acceptance of information technology by health research projects in low-income countries: Intention to use and acceptance of using EpiHandy (IUAUE). Master's thesis, Agder University College, Dec. 2005.

[11]

Evaluation of the SATELLIFE PDA project, Feb. 2003. Bridges.org.

[12]

D. Forster, R. H. Behrens, H. Campbell, and P. Byass. Evaluation of a computerized field data collection system for health surveys. Bulletin of the World Health Organization, 69(1):107-11, 1991.

[13]

R. Fynn. Remote HIV/AIDS patient monitoring tool using 3G/GPRS packet-switched mobile technology. In Appropriate Healthcare Technologies for Developing Countries, pages 129-138, 2006.

[14]

S. Grisedale, M. Graves, and A. Griinsteidl. Designing a graphical user interface for healthcare workers in rural India. In CHI, 1997.

Digital Library

[15]

T. Groves. SatelLife: Getting relevant information to the developing world. British Medical Journal, 313(7072): 1606-9, 1996.

[16]

P. C. Gupta. Survey of sociodemographic characteristics of tobacco use among 99, 598 individuals in Bombay, India using handheld computers. Tobacco Control, 5(2): 114-120, June 1996.

[17]

R. N. Jamison, R. H. Gracely, S. A. Raymond, J. G. Levine, B. Marino, T. J. Herrmann, M. Daly, D. Fram, and N. P. Katz. Comparative study of electronic vs. paper VAS ratings: A randomized, crossover trial using healthy volunteers. Pain, 99(1-2):341-7, Sept. 2002.

[18]

D. R. Kaufman and I. B. Starren. A methodological framework for evaluating mobile health devices. Proceedings of the AMIA Annual Symposium, 2006:978, 2006.

[19]

H. Ladd and N. Sewankambo. Uganda health information network: Technical report (submitted to the International Development Research Centre). SATELLIFE & Uganda Chartered HealthNet, Dec. 2004. http://www.healthnet.org/idrcreport.html.

[20]

S. O. Lal, F. W. Smith, J. P. Davis, H. Y. Castro, D. W. Smith, D. L. Chinkes, and R. E. Barrow. Palm computer demonstrates a fast and accurate means of bum data collection. The Journal of Burn Care & Rehabilitation, 21(6):559-61; discussion 558, 2000.

[21]

S. I. Lane, N. M. Heddle, E. Arnold, and I. Walker. A review of randomized controlled trials comparing the effectiveness of hand held computers with paper methods for data collection. BMC Medical Informatics and Decision Making, 6:23, 2006.

[22]

Mali: Mobile service helps bring down infant mortality, 2007. Balancing Act News, Issue No. 364, http://www.balancingactafrica.com/news/back/balancing-act_364.html.

[23]

I. S. McBride, R. T. Anderson, and J. L. Bahnson. Using a hand-held computer to collect data in an orthopedic outpatient clinic: A randomized trial of two survey methods. Medical Care, 37(7):647-51, July 1999.

[24]

M. A. Missinou, C. H. O. Olola, S. Issifou, P.-B. Matsiegui, A. A. Adegnika, S. Borrmann, D. Wypij, T. E. Taylor, and P. G. Kremsner. Short report: Piloting paperless data entry for clinical research in Africa. Amer. Journ. of Trop. Medicine and Hygiene, 72(3):301-303, Mar. 2005.

[25]

Mobilizing against malaria. Expanding Horizons (by Nokia), pages 12- 13, Feb. 2007.

[26]

Nokia CEO message, 2008. http://www.nokia.com/A4942317.

[27]

T. S. Parikh. Using mobile phones for secure, distributed document processing in the developing world. Pervasive Computing, 4(2), 2005.

Digital Library

[28]

T. S. Parikh, P. Javid, S. K, K. Ghosh, and K. Toyama. Mobile phones and paper documents: Evaluating a new approach for capturing microfinance data in rural India. In CHI, 2006.

Digital Library

[29]

P. Quinn, I. Goka, and H. Richardson. Assessment of an electronic daily diary in patients with overactive bladder. BJU International, 91(7):647- 52, May 2003.

[30]

J. Selanikio. Advantages of using handheld computers for data collection. DataDyne. http://www.datadyne.org/files/Brief-HandheldAdvantages.pdf

[31]

J. Selanikio and R. Donna. Datadyne brief DataDyne. http://www.datadyne.org/files/DataDyne_briefpdf

[32]

K. Shirima, O. Mukasa, J. Schellenberg, F. Manzi, D. John, A. Mushi, M. Mrisho, M. Tanner, H. Mshinda, and D. Schellenberg. The use of personal digital assistants for data entry at the point of collection in a large household survey in southern Tanzania. Emerging Themes in Epidemiology, 4(1):5, 2007.

[33]

D. Skinner, U. Rivette, and C. Bloomberg. Evaluation of use of cellphones to aid compliance with drug therapy for HIV patients. AIDS Care, 19(5):605-607, 2007.

[34]

SMS data management. Dimagi, Inc., 2008. http://www.dimagi.com/ content/sms-data-management.html.

[35]

B. Tiplady, G. Crompton, M. Dewar, F. Boellert, S. Matusiewicz, L. Campbell, and D. Brackenridge. The use of electronic diaries in respiratory studies. Drug Information Journal, 31(3):759-764, 1997.

[36]

I. Walker, C. Sigouin, I. Sek, T. Almonte, I. Carruthers, A. Chan, M. Pai, and N. Heddle. Comparing hand-held computers and paper diaries for haemophilia home therapy: A randomized trial. Haemophilia, 10(6):698-704, Nov. 2004.

Cited By

Cross AGupta NLiu BNair VKumar AKuttan RIvatury PChen ALakshman KRodrigues RD'Souza GChittamuru DRao RRade KVadera BShah DChoudhary VChadha VShah AKumta SDewan PThomas BThies WChandwani RSingh PKumar NVeeraraghavan R(2019)99DOTSProceedings of the Tenth International Conference on Information and Communication Technologies and Development10.1145/3287098.3287102(1-12)Online publication date: 4-Jan-2019
https://dl.acm.org/doi/10.1145/3287098.3287102
Pal JDasika AHasan AWolf JReid NKameswaran VYardi PMackay AWagner AMukherjee BJoshi SSantra SPandey PSaif U(2017)Changing data practices for community health workersProceedings of the Ninth International Conference on Information and Communication Technologies and Development10.1145/3136560.3136582(1-12)Online publication date: 16-Nov-2017
https://dl.acm.org/doi/10.1145/3136560.3136582
Kinai AMutahi JBore NKarumba SSridharan SDiriye AWeldemariam KFerrario MPernici B(2017)Deploying large scale school census hubProceedings of the 39th International Conference on Software Engineering: Software Engineering in Society Track10.1109/ICSE-SEIS.2017.7(47-56)Online publication date: 20-May-2017
https://dl.acm.org/doi/10.1109/ICSE-SEIS.2017.7
Show More Cited By

Evaluating the accuracy of data collection on mobile phones: a study of forms, sms, and voice
1. Human-centered computing

Recommendations

Mobile Phones in Data Collection: A Systematic Review

Mobile phones are increasingly popular tools not only for daily use but also for research purposes. The authors systematically searched related literature using mobile phones as a tool for data collection and found 171 publications consisting of review, ...
Mobile Data Collection Frameworks: A Survey
Mobidata '15: Proceedings of the 2015 Workshop on Mobile Big Data

Mobile phones equipped with powerful sensors have become ubiquitous in recent years. Mobile sensing applications present an unprecedented opportunity to collect and analyze information from mobile devices. Much of the work in mobile sensing has been ...
Secure data storage for mobile data collection systems
MEDES '12: Proceedings of the International Conference on Management of Emergent Digital EcoSystems

Wireless network infrastructures, notably cellular networks, are becoming a vital element for exchanging electronic data in low income countries. Several key sectors are already leveraging on cellular networks: mobile financial transactions have already ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICTD'09: Proceedings of the 3rd international conference on Information and communication technologies and development

April 2009

496 pages

ISBN:9781424446629

Sponsors

IEEE
ExxonMobil
Microsoft: Microsoft
IBM: IBM

In-Cooperation

SIGCAS: ACM Special Interest Group on Computers and Society

Publisher

IEEE Press

Publication History

Published: 17 April 2009

Check for updates

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 22 of 116 submissions, 19%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cross AGupta NLiu BNair VKumar AKuttan RIvatury PChen ALakshman KRodrigues RD'Souza GChittamuru DRao RRade KVadera BShah DChoudhary VChadha VShah AKumta SDewan PThomas BThies WChandwani RSingh PKumar NVeeraraghavan R(2019)99DOTSProceedings of the Tenth International Conference on Information and Communication Technologies and Development10.1145/3287098.3287102(1-12)Online publication date: 4-Jan-2019
https://dl.acm.org/doi/10.1145/3287098.3287102
Pal JDasika AHasan AWolf JReid NKameswaran VYardi PMackay AWagner AMukherjee BJoshi SSantra SPandey PSaif U(2017)Changing data practices for community health workersProceedings of the Ninth International Conference on Information and Communication Technologies and Development10.1145/3136560.3136582(1-12)Online publication date: 16-Nov-2017
https://dl.acm.org/doi/10.1145/3136560.3136582
Kinai AMutahi JBore NKarumba SSridharan SDiriye AWeldemariam KFerrario MPernici B(2017)Deploying large scale school census hubProceedings of the 39th International Conference on Software Engineering: Software Engineering in Society Track10.1109/ICSE-SEIS.2017.7(47-56)Online publication date: 20-May-2017
https://dl.acm.org/doi/10.1109/ICSE-SEIS.2017.7
Raza ARazaq SRaja ANaru RGibran ASabri ANiaz HSaleem MSaif U(2016)Real-time Automated Surveys among Low-literate Masses using Voice-based Telephone ServicesProceedings of the 7th Annual Symposium on Computing for Development10.1145/3001913.3006626(1-4)Online publication date: 18-Nov-2016
https://dl.acm.org/doi/10.1145/3001913.3006626
Holeman IYembric ABrown DKane DKatanu JAbbyad MSharma R(2016)Design and Implementation of an Open Source 'Thin SIM' System for Collecting Data & Supporting Global Health CareProceedings of the 7th Annual Symposium on Computing for Development10.1145/3001913.3001923(1-10)Online publication date: 18-Nov-2016
https://dl.acm.org/doi/10.1145/3001913.3001923
DeRenzi BWacksman JDell NLee SLesh NBorriello GEllner AToyama K(2016)Closing the Feedback LoopProceedings of the Eighth International Conference on Information and Communication Technologies and Development10.1145/2909609.2909652(1-10)Online publication date: 3-Jun-2016
https://dl.acm.org/doi/10.1145/2909609.2909652
Dell NKumar NKaye JDruin ALampe CMorris DHourcade J(2016)The Ins and Outs of HCI for DevelopmentProceedings of the 2016 CHI Conference on Human Factors in Computing Systems10.1145/2858036.2858081(2220-2232)Online publication date: 7-May-2016
https://dl.acm.org/doi/10.1145/2858036.2858081
Perrier TDeRenzi BAnderson RSathiaseelan AZennaro MBelding EChen J(2015)USSDProceedings of the 2015 Annual Symposium on Computing for Development10.1145/2830629.2830645(13-21)Online publication date: 1-Dec-2015
https://dl.acm.org/doi/10.1145/2830629.2830645
Pervaiz FPerrier TPhongphila SAnderson RChib AKam MBurrell J(2015)User errors in SMS based reporting systemsProceedings of the Seventh International Conference on Information and Communication Technologies and Development10.1145/2737856.2737877(1-4)Online publication date: 15-May-2015
https://dl.acm.org/doi/10.1145/2737856.2737877
White JDuggirala MChib AKam MBurrell J(2015)Speech-interface prompt designProceedings of the Seventh International Conference on Information and Communication Technologies and Development10.1145/2737856.2737861(1-4)Online publication date: 15-May-2015
https://dl.acm.org/doi/10.1145/2737856.2737861
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

Media

Figures

Other

Tables

View Table of Contents