Abstract
Background
Patient-administered computerized questionnaires represent a novel tool to assist primary care physicians in the delivery of preventive health care.
Objective
The aim of this study was to assess patient-reported ease of use with a self-administered tablet computer-based questionnaire in routine clinical care.
Design
All patients seen in a university-based primary care practice were asked to provide routine screening information using a touch-screen tablet computer-based questionnaire. Patients reported difficulty using the tablet computer after completion of their first questionnaire.
Patients
Ten thousand nine hundred ninety-nine patients completed the questionnaire between January 2004 and January 2006.
Measurements
We calculated rates of reporting difficulty (no difficulty, some difficulty, or a lot of difficulty) using the tablet computers based on patient age, sex, race, educational attainment, marital status, and number of comorbid medical conditions. We constructed multivariable ordered logistic models to identify predictors of increased self-reported difficulty using the computer.
Results
The majority of patients (84%) reported no difficulty using the tablet computers to complete the questionnaire, with only 3% reporting a lot of difficulty. Significant predictors of reporting more difficulty included increasing age [odds ratio (OR) 1.05, 95% confidence interval (CI) 1.05–1.05)]; Asian race (OR 2.3, 95% CI 1.8–2.9); African American race (OR 1.4, 95% CI 1.2–1.6); less than a high school education (OR 3.0, 95% CI 2.6–3.4); and the presence of comorbid medical conditions (1–2: OR 1.3, 95% CI 1.2–1.5; ≥3: OR 1.7 95% CI 1.5–2.1).
Conclusions
The majority of primary care patients reported no difficulty using a self-administered tablet computer-based questionnaire. While computerized questionnaires present opportunities to collect routine screening information from patients, attention must be paid to vulnerable groups.
Key words: computerized questionnaires, patient screening, health services, primary care
BACKGROUND
Patient self-administered computerized questionnaires represent a novel tool to assist physicians in the delivery of screening and preventive health care.1 Using a self-completed computerized questionnaire, patients can report completion of preventive health services as well as pain and health status.2,3 Computer algorithms can score and present information from validated questionnaires such as the RAND-36 or Center for Epidemiologic Studies Depression Scale in real time, enabling use during routine patient encounters. Physicians can then respond to issues identified while saving time on data collection.3,4 Computerized questionnaires can also employ skip patterns, obviating the need for patients to answer redundant questions while also ensuring that all relevant questions are completed. Additionally, prior research has found that information collected via computer is valid compared to paper and pencil surveys.2 Participants may be more candid with computers,5 increasing the desirability of computer administration for sensitive topics such as alcohol use. Given this promise, it is critical to determine if patients are comfortable using computers during routine primary care encounters.
Numerous evidence-based recommendations exist regarding preventive health services and chronic disease management in primary care settings,6–14 all of which are important for patient care and well-being. However, time constraints,15 lack of centralized information,16 and the need to prioritize acute medical care and chronic disease management17 have limited effective implementation of screening within the primary care setting. Additionally, primary care recognition of mental health problems is often lacking.18,19
While the use of computers to collect behavioral and mental health screening information holds great potential to be successful2,3, such systems must be accessible to a primary care population. This population includes older adults, those with lower education attainment, as well as those with high disease burden,20 which creates competing priorities for office visit time. If such a system is not usable for these subgroups of the overall primary care population, then its utility will be limited. Unfortunately, information regarding acceptability of computerized surveys in routine clinical practice is lacking.
OBJECTIVE
Our practice uses a patient-administered, touch-screen tablet computer-based questionnaire as part of routine patient care. This study includes data on patient reported ease of use with the computer tablets and examines variability in ease of use based on age, sex, race, educational attainment, and number of comorbid medical conditions. Our study involves a larger population than has been previously reported in an office-based setting as standard of care, as opposed to a research setting. Based on our preliminary experience,21 we hypothesized that older patients and those with lower educational attainment would have more difficulty using the system, but that their overall ease of use would remain high.
METHODS
We have developed the Functional Assessment Screening Tablet (FAST),21 a touch-screen computerized patient-completed screening tool used in our primary care practice. Over the last 2 years, we have used the FAST within a busy, urban, university-based primary care practice with an annual patient encounter volume of approximately 44,000. Every primary care patient in our outpatient setting uses a touch-screen computer to complete routine questionnaires. At check-in, patients are given a tablet computer and instructed on use of the attached “pen” to answer questions. They are asked to return the computer to a medical assistant when called to see their physician and all questions are complete. Staff trained to assist the patients with the tablet computers circulate in the waiting room and check with patients to ensure they do not have questions or difficulty. The FAST computers are wirelessly networked, allowing patients to proceed through the office while completing their questionnaires.
All questions are answered using radial buttons or check boxes and do not require free-text response. Items are assessed at varying intervals, including only once (sex, race/ethnicity), yearly (marital status, educational attainment, self-assessment of current weight as healthy, physical activity, social support, living will), biennially (RAND-36), with every visit (comorbid medical conditions, pain rating scale, change in appetite, change in weight, prescription refills needed), or at varying intervals based on response (tobacco use, alcohol use). Additionally, for comprehensive physical visits, patients complete a review of systems.
A summary of the current responses along with prior responses is provided to the physician at the time of the visit for use during the patient encounter. Because questions vary based on time intervals, the time for the FAST completion also varies. A comprehensive visit takes approximately 25 minutes to complete all questions, while questions for a routine follow-up visit can be completed in less than 5 minutes. There have been no modifications to our office schedule to accommodate the time required by patients to complete the FAST.
Our primary study outcome of difficulty using the computer is assessed the first time the patient completes the FAST. At this time, patients are asked: “Did you have trouble using the computer to answer these questions?”, with potential response options including no difficulty; some difficulty; or a lot of difficulty. This question is not repeated during subsequent visits. As described above, demographic variables also assessed include age, sex, race (Caucasian, African American, Native American, Asian, Hawaiian/ Pacific Islander, or Other), ethnicity (Hispanic versus non-Hispanic), educational attainment (elementary school, junior high, high school, some college, completed college, or graduate degree), and marital status (single, widowed, divorced, married, committed relationship). We dichotomized marital status as non-married (single, widowed, divorced) versus married (married or committed relationship) and educational attainment as ≤ high school (elementary school, junior high, or high school) versus ≥ some college (some college, completed college, or graduate degree). Patients report the presence of 10 possible comorbid medical conditions (heart disease, heart failure, depression, stroke, emphysema or lung disease, arthritis, non-skin cancer, diabetes, and high blood pressure). We further categorized these as 0, 1–2, or ≥3 comorbid medical conditions. The University of Pittsburgh Institutional Review Board approved this project.
ANALYSIS
Participant characteristics were summarized using frequencies and measures of central tendencies. We used χ2 techniques to compare categorical variables and univariable ordered logistic regression for continuous variables. Predictors with univariable significance of p ≥ 0.1 were included in multivariable ordered logistic models. Sex and age were included in all models. A separate analysis, paralleling the whole group analysis, was conducted for respondents ≥65. We conducted separate analyses in older adults because they may feel less comfortable using and have different requirements (e.g., vision) to use computer systems than younger adults.22–25 Age 65 was chosen a priori because it is the age of Medicare eligibility in the USA and it has been used by others.25
RESULTS
A total of 11,044 unique individuals completed the FAST. Forty-five individuals were missing race/ethnicity(n = 45) and/ or educational attainment information (n = 37), leaving 10,999 patients who completed the FAST (Table 1). The mean age of patients was 47 years, and 60% of patient population was female. The study population was well educated, with almost 60% overall, and nearly 40% of those ≥65 years old having at least a college education. However, 22% of all patients and 44% of those ≥65 had a high school education or less. Approximately one half of participants were married. More patients ≥65 years old reported ≥3 comorbid medical conditions compared to the overall population (30% versus 12%).
Table 1.
Characteristic | Total population (n = 10,999) | Age ≥65 (n = 1,707) |
---|---|---|
(n, %) | (n, %) | |
Age (mean ± SD) | 47 ± 17 | 74 ± 7 |
Female | 6,818 (62) | 1,050 (62) |
Race | ||
Native American | 43 (0.4) | 7 (0.4) |
Asian | 799 (7) | 70 (4) |
African American | 1,991 (18) | 326 (19) |
Caucasian | 7,922 (72) | 1,289 (76) |
Other | 244 (2) | 15 (1) |
Ethnicity (Hispanic) | 297 (3) | 37 (2) |
Education | ||
≤High school | 2,412 (22) | 748 (44) |
≥Some college | 8,587 (78) | 959 (56) |
Marital status (married) | 5,685 (52) | 873 (51) |
Comorbid medical conditions | ||
0 | 4,920 (45) | 243 (14) |
1–2 | 4,792 (44) | 960 (56) |
≥3 | 1,287 (12) | 504 (30) |
Computer difficulty | ||
None | 9,231 (84) | 1,016 (60) |
Some | 1,387 (13) | 471 (28) |
A lot | 381 (3) | 220 (13) |
SD Standard deviation
Overall Population
The majority of patients (84%) reported having no difficulty using the FAST, whereas only 3% of patients reported having a lot of difficulty (Table 1). In univariable analyses, patient age, race, educational attainment, marital status, and number of comorbid medical conditions were all associated with level of difficulty using the computer (Table 2). In multivariable analyses, significant predictors of reporting more difficulty using the FAST tool included increasing age [odds ratio (OR) 1.05, 95% confidence interval (CI) 1.05–1.05)]; Asian race (OR 2.3, 95% CI 1.8–2.9); African American race (OR 1.4, 95% CI 1.2–1.6); Other race (OR 2.1, 95% CI 1.4–3.0), less than a high school education (OR 3.0, 95% CI 2.6–3.4); and the presence of comorbid medical conditions (1–2: OR 1.3, 95% CI 1.2–1.5; ≥3: OR 1.7 95% CI 1.5–2.1; Table 3).
Table 2.
Characteristic Age (mean ± SD) | Total population (n = 10,999) | Age ≥65 (n = 1,707) | ||||||
---|---|---|---|---|---|---|---|---|
None† 45 ± 16 | Some† 57 ± 16 | A lot† 66 ± 16 | p value < 0.001 | None† 72 ± 6 | Some† 74 ± 7 | A lot† 77 ± 7 | p value < 0.001 | |
Gender | 0.9 | 0.01 | ||||||
Female | 5,715 (84) | 865 (13) | 238 (3) | 596 (57) | 307 (29) | 147 (14) | ||
Male | 3,516 (84) | 522 (12) | 143 (3) | 420 (64) | 164 (25) | 73 (11) | ||
Race | <0.001 | <0.001 | ||||||
Native American | 31 (72) | 8 (19) | 4 (9) | 4 (57) | 2 (29) | 1 (14) | ||
Asian | 685 (86) | 94 (12) | 20 (3) | 41 (59) | 16 (23) | 13 (19) | ||
African American | 1,471 (74) | 393 (20) | 127 (6) | 124 (38) | 130 (40) | 72 (22) | ||
Caucasian | 6,840 (86) | 864 (11) | 218 (3) | 841 (65) | 316 (25) | 132 (10) | ||
Other | 204 (84) | 28 (11) | 12 (5) | 6 (40) | 7 (47) | 2 (13) | ||
Ethnicity | 0.4 | 0.7 | ||||||
Hispanic | 241 (81) | 43 (14) | 13 (4) | 23 (62) | 11 (30) | 3 (8) | ||
Non-Hispanic | 8,990 (84) | 1,344 (13) | 368 (3) | 993 (59) | 460 (28) | 217 (13) | ||
Education | <0.001 | <0.001 | ||||||
≤High School | 1526 (63) | 643 (27) | 243 (10) | 316 (42) | 279 (37) | 153 (20) | ||
≥Some College | 7,705 (90) | 744 (9) | 138 (2) | 700 (73) | 192 (20) | 67 (7) | ||
Marital status | <0.001 | <0.001 | ||||||
Single/widowed/divorced | 4,271 (80) | 800 (15) | 243 (5) | 419 (50) | 270 (32) | 145 (17) | ||
Married | 4,960 (87) | 587 (10) | 138 (2) | 597 (68) | 201 (23) | 75 (9) | ||
Comorbid medical conditions | <0.001 | <0.001 | ||||||
0 | 4518 (92) | 335 (7) | 67 (1) | 177 (73) | 41 (17) | 25 (10) | ||
1–2 | 3,879 (81) | 735 (15) | 178 (4) | 588 (61) | 374 (29) | 98 (10) | ||
≥3 | 834 (65) | 317 (25) | 136 (11) | 251 (50) | 156 (31) | 97 (19) |
*Due to rounding, percents may not sum to 100%
†Amount of reported difficulty with the FAST computers
Table 3.
Characteristic | Total population (n = 10,999) | Age ≥65 (n = 1,707) |
---|---|---|
Age | 1.05 (1.05–1.05) | 1.05 (1.03–1.07) |
Gender (female ref) | 1.1 (.9–1.4) | |
Race | ||
Native American | 1.7 (.8–3.4) | 0.7 (.2–3.3) |
Asian | 2.3 (1.8–2.9) | 2.0 (1.3–3.4) |
African American | 1.4 (1.2–1.6) | 1.8 (1.4–2.3) |
Caucasian (ref) | 1.0 | 1.0 |
Other | 2.1 (1.4–3.0) | 3.1 (1.2–7.8) |
Education | ||
≤High school/GED | 3.0 (2.6–3.4) | 2.7 (2.2–3.4) |
≥Some college (ref) | 1.0 | 1.0 |
Marital Status (single† ref) | 0.7 (0.6–0.7) | 0.8 (0.6–0.9) |
Comorbid medical conditions | ||
0 (ref) | 1.0 | 1.0 |
1–2 | 1.3 (1.2–1.5) | 1.4 (.99–1.9) |
≤3 | 1.7 (1.5–2.1) | 1.8 (1.3–2.6) |
*Difficulty measured as “Some” or “A lot”. Odds ratio (95% confidence interval)
†Single: single, widowed, or divorced
Patients ≥65 Years
While 60% of patients ≥65 years reported having no difficulty, 13% of those ≥65 reported having a lot of difficulty. The remaining one third (28%) reported some difficulty using the FAST, representing a higher rate of difficulty than the overall population. Univariable analyses were similar to the overall population, with patient age, educational attainment, marital status, and number of comorbid conditions all associated with level of difficulty using the computer (Table 2). Unlike the overall population, women ≥65 years old were more likely to report some or a lot of difficulty compared to men (p = 0.01). Multivariable analyses yielded similar results to that seen in the overall population (Table 3).
DISCUSSION
Patients in our practice reported very little difficulty using a tablet computer-based assessment tool. While this varied by age, race, and educational attainment, less than 15% of the elderly population (≥65 years old) and less than 5% of the overall population reported a lot of difficulty. It is a concern, however, that up to one third of people in any potentially vulnerable group reported some difficulty using the FAST. It is unclear, based on our assessment, if these are people with minor difficulties that are easily overcome with repetition or guidance or if they represent individuals with more substantial problems that will impede efforts to increase future use. Understanding this problem is critical for future success of computer-based tools such as the one we evaluated.
Our results are in concordance with the majority of literature in this area. The majority of people find computers relatively easy to use within patient care settings.2,20 Our evaluation is the largest of its kind, as the FAST was implemented as standard of care within our practice setting as opposed to a research protocol, therefore, representing a more diverse population than has previously been reported.2,20,26 Our findings bode well for the potential of using computerized questionnaires to enhance prevention strategies and the use of validated functional status, quality of life, and mental health instruments within the primary care setting. These tools may serve to make the primary care provider more effective at delivering recommended preventive services while also enhancing the identification and treatment of subtle physical disabilities and mental health problems such as depression and anxiety. The efficiency of having the patient provide this information before the visit allows time to address ongoing medical needs as well as preventive health services.
We are cognizant of the small percentage of patients who reported difficulty using computers. We believe that this issue can be overcome with adequate staffing and staff training to offer and provide assistance to patients with computer difficulties, as well as using technology to overcome visual, reading, or language limitations. Computers allow the use of increased font sizes for those with visual difficulty. They also provide an opportunity to use audio and voice recognition as well as translation software to overcome visual, reading, and language limitation. While we have not yet implemented these solutions and do not have quantitative longitudinal data, our experience suggests that the majority of computer-related difficulties might be overcome with these technologies and repeated use. Further analyses should focus on the longitudinal patterns of FAST use based on initial reported difficulty using the system as well as potential changes in self-reported difficulty with repeated use.
It should also be noted that our population has much higher educational attainment than the Pittsburgh area as a whole, likely skewed by our affiliation with a university community including students and faculty. Based on 2000 United States census data, only 24% of the Pittsburgh area has completed college or obtained a graduate degree, compared to more than 50% in our study. This may have implications for the generalizability of our findings to other settings.
Our study may also be limited by patient self-selection, as some patients decline to use the FAST and we do not have information regarding the numbers or demographic characteristics of these patients. However, more than 11,000 unique patients completed the questionnaire, indicating that we are capturing a reasonable portion of our population. Our registration staff has been trained to encourage patients to use the FAST, and we provide assistance for patients with difficulty. While we do not assess vision, language proficiency, or literacy, the alternative, a paper and pencil form, does nothing to address these limitations. We are beginning a program to assess health literacy as standard of care in our population.
Finally, using standardized instruments is time-consuming for our patients, and we struggle with the challenge of educating them that the patient encounter is more than the “face time” with the physician. We continue to work with our patients to arrive with sufficient time to complete preventive health screening prior to the visit.
CONCLUSIONS
We found that a computerized screening instrument is of limited difficulty for the majority of patients to use in a routine practice setting. This tool allows us to collect routine screening and prevention information using standardized instruments and provide physicians with results for immediate use during the patient encounter. Systems such as the FAST may help facilitate more effective delivery of necessary preventive services by providing physicians with additional time and information during the patient encounter. Future work should focus on physician experiences with this system, as well as the reasons patients have difficulty using touch-screen tablets and whether difficulty diminishes over time. By identifying and rectifying these barriers, we can use technology to enhance the effective delivery of primary care.
Acknowledgments
The FAST is supported by the University of Pittsburgh’s Division of General Internal Medicine. Dr. Hess has grant support from the National Institutes of Health, Department of Health and Human Services, through the National Institute on Aging AG024254. Dr. McTigue has grant support from the National Institutes of Health, Department of Health and Human Services, through the National Institute of Diabetes and Digestive and Kidney Diseases DK067192.
Conflict of Interest Dr. Hess has received research support from BIONOVO for A phase II, double-blind, placebo-controlled, randomized clinical trial assessing safety and efficacy of MF101 for hot flashes and menopausal symptoms in postmenopausal women. No other authors have conflicts to disclose.
References
- 1.Bachman JW. The patient-computer interview: a neglected tool that can aid the clinician. Mayo Clin Proc. 2003;78(1):67–78. [DOI] [PubMed]
- 2.Bliven BD, Kaufman SE, Spertus JA. Electronic collection of health-related quality of life data: validity, time benefits, and patient preference. Qual Life Res. 2001;10(1):15–22. [DOI] [PubMed]
- 3.McGuire M, Bakst K, Fairbanks L, et al. Cognitive, mood, and functional evaluations using touchscreen technologies. J Nerv Ment Dis. 2000;188(12):813–7. [DOI] [PubMed]
- 4.Jette AM, Davies A, Cleary PD, et al. The functional status questionnaire: reliability and validity when used in primary care. J Gen Intern Med. 1986;1(3):143–9. [DOI] [PubMed]
- 5.Feigelson ME, Dwight SA. Can asking questions by computer improve the candidness of responding? A meta-analytic perspective. Consult Psychol J: Prac Res. 2000;52(4):248–55. [DOI]
- 6.US Preventive Services Task Force. Genetic risk assessment and BRCA mutation testing for breast and ovarian cancer susceptibility: recommendation statement. Ann Intern Med. 2005;143(5):355–61. [DOI] [PubMed]
- 7.US Preventive Services Task Force. Screening for suicide risk: recommendation and rationale. Ann Intern Med. 2004;140(10):820–1. [DOI] [PubMed]
- 8.US Preventive Services Task Force. Screening and behavioral counseling interventions in primary care to reduce alcohol misuse: recommendation statement. Ann Intern Med. 2004;140(7):554–6. [DOI] [PubMed]
- 9.US Preventive Services Task Force. Screening for family and intimate partner violence: recommendation statement. Ann Intern Med. 2004;140(5):382–6. [DOI] [PubMed]
- 10.US Preventive Services Task Force. Screening for thyroid disease: recommendation statement. Ann Intern Med. 2004;140(2):125–7. [DOI] [PubMed]
- 11.US Preventive Services Task Force. Screening for obesity in adults: recommendations and rationale. Ann Intern Med. 2003;139(11):930–2. [DOI] [PubMed]
- 12.US Preventive Services Task Force. Screening for type 2 diabetes mellitus in adults: recommendations and rationale. Ann Intern Med. 2003;138(3):212–4. [DOI] [PubMed]
- 13.US Preventive Services Task Force. Screening for colorectal cancer: recommendation and rationale. Ann Intern Med. 2002;137(2):129–31. [DOI] [PubMed]
- 14.US Preventive Services Task Force. Screening for depression: recommendations and rationale. Ann Intern Med. 2002;136(10):760–4. [DOI] [PubMed]
- 15.Yarnall KS, Pollak KI, Ostbye T, Krause KM, Michener JL. Primary care: is there enough time for prevention? Am J Public Health. 2003;93(4):635–41. [DOI] [PMC free article] [PubMed]
- 16.Tsui JI, Dodson K, Jacobson TA. Cardiovascular disease prevention counseling in residency: resident and attending physician attitudes and practices. J Natl Med Assoc. 2004;96(8):1080–3. 1088–91. [PMC free article] [PubMed]
- 17.Melin I, Karlstrom B, Berglund L, Zamfir M, Rossner S. Education and supervision of health care professionals to initiate, implement and improve management of obesity. Patient Educ Couns. 2005;58(2):127–36. [DOI] [PubMed]
- 18.Rollman BL, Hanusa BH, Gilbert T, Lowe HJ, Kapoor WN, Schulberg HC. The electronic medical record. A randomized trial of its impact on primary care physicians’ initial management of major depression [corrected]. Arch Intern Med. 2001;161(2):189–97. [DOI] [PubMed]
- 19.Rollman BL, Hanusa BH, Lowe HJ, Gilbert T, Kapoor WN, Schulberg HC. A randomized trial using computerized decision support to improve treatment of major depression in primary care. J Gen Intern Med. 2002;17(7):493–503. [DOI] [PMC free article] [PubMed]
- 20.Bock B, Niaura , Fontes A, Bock F. Acceptability of computer assessments among ethnically diverse, low-income smokers. Am J Health Promot. 1999;13(5):299–304. [DOI] [PubMed]
- 21.Hess , Matthews K, McNeil M, Chang CCH, Kapoor W, Bryce C. Health services research in the privacy age. J Gen Intern Med. 2005;20(11):1045–9. [DOI] [PMC free article] [PubMed]
- 22.Dickinson A, Gregor P. Designing computer systems for and with older users. Behav Inf Technol. 2007;26(4):273–4. [DOI]
- 23.Hawthorn D. Interface design and engagement with older people. Behav Inf Technol. 2007;26(4):333–41. [DOI]
- 24.Dickinson A, Arnott J, Prior S. Methods for human–computer interaction research with older people. Behav Inf Technol. 2007;26(4):343–52. [DOI]
- 25.Larkin-Lieffers PA. The older adult and public library computer technology: a pilot study in a Canadian setting. Libri. 2000;50:225–34. [DOI]
- 26.Main D, Quintela J, Araya-Guerra , Holcomb S, Pace W. Exploring patient reactions to pen-tablet computers: a report from CaReNet. Ann Fam Med. 2004;2(5):421–4. [DOI] [PMC free article] [PubMed]