Cultural adaptation and validation of the Health Literacy Questionnaire (HLQ): robust nine-dimension Danish language confirmatory factor model

The complexity of modern healthcare and the many health messages being promoted have led to health literacy being a key consideration for health promotion and improving the quality of health services (Nutbeam 2000; Protheroe et al. 2009; Sørensen et al. 2012; Kickbusch et al. 2013; Norgaard et al. 2014). Furthermore, the need for people to manage their health themselves, including using various health technologies, requires individuals to have a wide range of health literacy competencies.

The World Health Organization defined health literacy as “the cognitive and social skills which determine the motivation and ability of individuals to gain access to, understand and use information in ways which promote and maintain good health” (World Health Organization 1998). From the consumer’s perspective the required competencies include not only being able to read and understand health information, but also being able to navigate the health system, communicate and engage with healthcare providers, engage in critical appraisal of health information, and advocate for one’s right to health services (HLS-EU Consortium 2012; Kickbusch et al. 2013; Osborne et al. 2013). There is, therefore, great potential to improve public health and clinical medicine by addressing health literacy in organisations on many levels through clinician training, organisational policy and planning, health protection, and public health policy.

In Denmark, and many other countries, there is a need for data about population health literacy in order to cultivate an inclusive health system that meets the needs of individuals and diverse population groups (Sorensen et al. 2014).

The focus of previous health literacy measurement methods was on functional health literacy: word recognition tests, reading ability and numeracy (Mårtensson and Hensing 2012; Haun et al. 2014). Recent instruments, such as the Health Literacy Questionnaire (HLQ) (Osborne et al. 2013) and the European Health Literacy Survey (HLS-EU) (HLS-EU Consortium 2012), assess a wider range of health literacy components. Data derived from the HLQ are designed to assist with the diagnosis of the diverse health literacy needs of individuals and communities in both health promotion and clinical settings. It has nine scales that generate comprehensive profiles of the health literacy of individuals and groups. The profiles then assist practitioners, planners and policy makers to understand the health literacy needs of communities and this in turn assists in planning, designing and evaluating interventions.

The HLQ has been widely translated and applied in research, evaluation and monitoring (Deakin University 2016). However, in order for data from a new measure to be regarded as sufficiently robust to make decisions about individuals, communities or organisations, or to compare across settings, psychometric evidence is required that demonstrates that it is culturally and linguistically appropriate and has strong measurement properties.

In the development of the English language HLQ the items were chosen to best represent the constructs that had been generated through a ‘grounded validity-driven’ approach which included both qualitative and quantitative techniques among both consumers and professionals (Buchbinder et al. 2011). The meaning of the constructs was established by this grounded process and validated by the psychometric analyses reported in the original paper (Osborne et al. 2013). A strict translation protocol was then instituted to ensure that this meaning was carried through accurately to all non-English-language versions. The psychometric analyses in the present paper have sought to replicate the results in the original paper and thus validate the meaning of the Danish scales as equivalent to that established by the original procedures.

The aims of this study were to translate the HLQ to Danish and test the psychometric properties of the translated version in a Danish validation sample. Adaptation was done through a rigorous cultural and linguistic translation procedure, followed by psychometric analyses. The outcome will inform stakeholders and the research community if the Danish HLQ data enable valid and reliable decisions to be made about health literacy in Danish settings.

Setting and data collection

The study was undertaken in primary care settings (community health centres and general practices), public places and workplaces with wide geographic and sociodemographic variation. These settings were consistent with populations and areas in which future application of the HLQ is planned. Data were gathered by health professionals and students by face-to-face interview in both the cognitive testing and validation samples using a standardised protocol.

Health Literacy Questionnaire (HLQ)

The HLQ was designed using a grounded, validity-driven approach and initially tested in diverse samples of individuals in Australian communities and shown to have strong construct validity, reliability and acceptability to clients and clinicians (Osborne et al. 2013; Batterham et al. 2014). It was designed for administration by pen and paper self-administration or by interview to ensure inclusion of people who cannot read or have other difficulties with self-administration.

The HLQ contains 44 questions that cover nine conceptually distinct areas of health literacy:

1. 1.

   Feeling understood and supported by healthcare providers (four items).

2. 2.

   Having sufficient information to manage my health (four items).

3. 3.

   Actively managing my health (five items).

4. 4.

   Social support for health (five items).

5. 5.

   Appraisal of health information (five items).

6. 6.

   Ability to actively engage with healthcare providers (five items).

7. 7.

   Navigating the healthcare system (six items).

8. 8.

   Ability to find good health information (five items).

9. 9.

   Understand health information well enough to know what to do (five items).

Response options for each scale were determined by the content and nature of the items. For scales 1–5 four-point ordinal response options are used (strongly disagree, disagree, agree and strongly agree), while for scales 6–9 five-point ordinal response options are used (cannot do, very difficult, quite difficult, quite easy and very easy).

Translation, cultural adaptation, and item strength equivalence

A standardised procedure (Hawkins and Osborne 2010) was used including recommendations from a range of organisations (Guillemin et al. 1993; Wild et al. 2005; Koller et al. 2007). The translation process included an initial discussion of cultural differences between the Australian and Danish health systems and the presence of local dialects in Denmark. It involved two forward translators, two backward translators, the authors, and was chaired by one of the HLQ developers (RHO). The process involved three main steps:

1. 1.

   Professional translators, with Danish as their native language, were briefed and provided with the English HLQ and detailed information about the intent of each of the nine scale constructs and each of the items within those scales. The item intent document included narratives about precisely what each element of the HLQ items mean and, in some cases, what these do not mean.

2. 2.

   Back translation was undertaken by two native English speakers with excellent Danish language skills.

3. 3.

   Finally, a translation consensus meeting was conducted by an expert panel comprising the four translators and four researchers in the fields of population health, health promotion and patient care (HTM), healthcare systems and medicine (ON, LK), and questionnaire development and translation methodology (RHO). Each translated item was examined in turn. Panel members confirmed that: (a) the principal scale construct embodied in each Danish item matched the original English item; (b) the translated items were suitable for males and females, for a wide age range, for people with low linguistic skills, limited experience of health and the healthcare system, and who either did or did not have a health problem; and (c) the relative ‘difficulty’ (in choosing the positive response options agree/strongly agree and quite easy/very easy) of each item was equivalent between the languages. When a word from a Danish dialect was used in the translation, panel members were required to consider how ubiquitous the word is across the country, or if a more commonly-used word with the same meaning would be more appropriate.

The level of comprehensibility and cognitive equivalence of the preliminary translated Danish HLQ (Willis 1999; Wild et al. 2005) was field tested through face-to-face cognitive interviews with 15 persons across gender and education categories and from different parts of Denmark. The cognitive testing involved initial administration of items using paper and pen format with careful observation of each respondent. The interviewer then reviewed items with the respondent and asked specific questions about items they had hesitated on or appeared to have found difficult in answering. Respondents were asked ‘What were you thinking about when you were answering that question?’ This process elicited the cognitive process behind the answers. A prompt was used if needed: ‘Why did you select that response option?’

Ethics

The study was approved by the Danish Data Protection Agency (j.no: 2013-41-2270). According to Danish law, when survey-based studies are undertaken in accordance with the Helsinki Declaration, specific approval by an ethics committee and written informed consent is not required. Potential respondents were provided with information about the survey and its purpose, including that participation was voluntary. The completion of the survey by participants was then considered to be implied consent.

Statistical analysis

Analyses were conducted with STATA version 13.0 and 14.0 and Mplus version 7.11. Descriptive statistics were generated for each item to determine the extent of missing values and to demonstrate the range of responses by providing difficulty estimates within and across the nine scales. For scales with disagree/agree response options, the relative strength of an item (how difficult it is to score highly) was calculated as the proportion responding disagree and strongly disagree (low scores) as against agree or strongly agree (high scores). For scales with response options cannot do to very easy, the difficulty level was calculated as the proportion responding cannot do, very difficult, or quite difficult as against quite easy and very easy (Raykov and Marcoulides 2011).

As Cronbach’s α is frequently a biased estimate of population reliability, unbiased estimates of composite reliability were generated (Raykov 2007). However, α was also calculated for comparison with other studies. Given that the HLQ scales were specified a priori, confirmatory factor analysis (CFA) was undertaken. Using one-factor CFA, a model was fitted to the data for each previously confirmed scale (Osborne et al. 2013). The response options were scored as ordinal variables (1–4 for the strongly disagree–strongly agree scales; 1–5 for the cannot do–very easy scales) and the models were fitted using the weighted least squares mean and variance adjusted (WLSMV) estimator available in Mplus. Unstandardised and standardised factor loadings, an estimate of the variance in the measured variable explained by the latent variable (R²), and associated standard errors are provided in Mplus together with fit statistics (χ ², comparative fit index—CFI, Tucker–Lewis index—TLI, and root mean square error of approximation—RMSEA). Indicative threshold values for the tests of ‘close fit’ used in this analysis were CFI > 0.95; TLI > 0.95; RMSEA < 0.06 while a value of <0.08 for the RMSEA was taken to indicate a “reasonable” fit (Browne and Cudeck 1993; Yu 2002; West et al. 2012).

Mplus also provides statistics that can be used to facilitate model improvement by suggesting fixed parameters (e.g., in the case of single-factor models, correlations among residual variances) that might be freely estimated. In Mplus, these statistics include standardised residuals, modification indices (MIs) and the associated change in a parameter if the modification is included in the model (standardised expected parameter change—SEPC).

A full nine-factor CFA model with no correlated residuals or cross-loadings was fitted to the data to investigate discriminant validity. As the restrictions that are typically placed on multi-factor CFA models frequently result in a strong upwards bias in the estimation of inter-factor correlations, this analysis was followed-up by fitting the nine-factor model using Bayesian structural equation modelling (BSEM) (Marsh et al. 2010). By using small-variance priors, BSEM allows models to be fitted that have the flexibility to estimate small variations from the strictly zero constraints on the residual correlations and cross-loadings in a typical multi-factor CFA model (Muthén and Asparouhov 2012). For this analysis, the variance of the Bayesian priors for the cross-loadings was set at 0.02 such that there was a 95 % probability that the cross-loadings would be within the range ±0.28. Similarly, the variance for the residual correlations was set to give a 95 % probability that the correlations were within the range of ±0.2 (Muthén and Asparouhov 2012).

HLQ scale scores were calculated as unit-weighted sums of the constituent items averaged by the number of items in the scale such that the nominal range of the scale scores was 1–4 for scales 1–5 and 1–5 for scales 6–9. One-way ANOVA was used to investigate mean differences on the HLQ scale scores across a range of sociodemographic variables. Effect sizes (ES) and their 95 % confidence intervals (CI) for standardised differences in means between sociodemographic groups were calculated using Cohen’s ‘d’ with interpretation of ES as follows: “small” ES > 0.20–0.50, “medium” ES approximately 0.50–0.80, and “large” ES > 0.80.

Cognitive testing

Cognitive testing revealed that almost all items were well understood. Only minor re-wording was required.

Validation sample characteristics

Table 1 shows the sociodemographic characteristics and health conditions of the 481 respondents in the validation sample. The median age was 53 years, with the youngest being 17, and the oldest 92. There were more women than men (a percentage-point difference of 19.2), while 18.9 % of respondents lived alone. Over 50 % had completed a high school or higher and 60 % reported having a longstanding illness or disability. The most frequent chronic conditions reported were: musculoskeletal disorders, cardiovascular disease, diabetes, cancer, chronic obstructive pulmonary disease and mental disorders. Just over a quarter of respondents lived in the capital area of Denmark and 63 % lived in other Danish cities across different geographical regions.

Psychometric properties of the Danish HLQ

Response to the HLQ items was high (missing answers: 0.2–1.7 %). Tables 2 and 3 present the psychometric properties of the items and scales. At the item level, there were few missing data, with the highest proportion being for item 7.6 ‘Work out what is best care for you’ (1.7 %). The relative strength of the items within scales and between scales varied modestly. The scales that were easiest to score highly were 4. ‘Social support for health’ (average item difficulty 0.14) and 2. ‘Having sufficient information to manage my health’ (0.16). The scales that were hardest to score highly were 7. ‘Navigating the healthcare system’ (0.36) and 5. ‘Appraisal of health information’ (0.36). The easiest item was found in scale 4. ‘Social support for health’ [4.4 ‘I have at least one person who can come to medical appointments with me’ (0.07)], and the hardest item was in scale 7. ‘Navigating the healthcare system’ [7.5 ‘Find out what healthcare services you are entitled to’ (0.51)]. The scale with the smallest range of difficulty was 8. ‘Ability to find good health information’ (hardest 0.21, easiest 0.15, range 0.06), whereas 1. ‘Feeling understood and supported by healthcare providers’ had the largest range of difficulties (hardest 0.36, easiest 0.11, range 0.25).

The model fit for all scales (with some inclusion of correlated errors related mostly to linguistic overlap) was generally very good, demonstrating that the scales are homogeneous, although, after model modification, the RMSEA remained unacceptably high for scale 1 (Table 3) suggesting some further association between the item residuals in this scale. Initially four scales (scales 2, 3, 7 and 8) returned a satisfactory close fit for the one-factor models. For five scales, the close fit statistics were initially not satisfactory due to the presence of correlated residuals which, when included in the model (maximum 2), ranged from 0.23 (scale 4) to 0.49 (scale 1).

For each scale there were high loadings on almost all items [all above 0.6 except for one each in scale 5 (0.49) and scale 7 (0.50)]. The median loading was 0.79. A composite reliability and Cronbach’s α of ≥0.8 was observed for all scales except scale 5. ‘Appraisal of health information’ (composite reliability = 0.77, α = 0.76). The median composite reliability was 0.84 (α = 0.83), with the highest for scale 8. ‘Ability to find good health information’ (composite reliability = 0.87, α = 0.87). In summary, the reliability for all scales was good (0.8–0.9) except for scale 4. These findings are in the same range as what was observed in the original psychometric studies of the English HLQ.

A nine-factor CFA model was fitted to the 44 items with no cross-loadings or correlated residuals allowed. Given the very restricted nature of the model, the fit was quite satisfactory: χ ²_WLSMV (866 df) = 2459.31, p < 0.0001, CFI = 0.934, TLI = 0.930 and RMSEA = 0.062. While the CFI and TLI are lower than the pre-specified cut-off, this is not surprising given the large number of parameters in the model set precisely to 0.0. Also, the CFI and TLI tend to underestimate the goodness-of-fit of models with large numbers of measured variables compared with the RMSEA as in this analysis (Kenny and McCoach 2003). The ranges of the factor loadings in this model were: scale 1. ‘Feeling understood and supported by healthcare providers’: 0.78–0.95; scale 2. ‘Having sufficient information to manage my health’: 0.77–0.86; scale 3. ‘Actively managing my health’: 0.69–0.89; scale 4. ‘Social support for health’: 0.62–0.84; scale 5. ‘Appraisal of health information’: 0.58–0.83; scale 6. ‘Ability to actively engage with healthcare providers’: 0.69–0.90; scale 7. ‘Navigating the healthcare system’: 0.57–0.81; scale 8. ‘Ability to find good health information’: 0.79–0.89; and scale 9. ‘Understanding health information well enough to know what to do’: 0.69–0.86.

Inter-factor correlations in the nine-factor model ranged from 0.41 (scales 4 and 9) to 0.93 (scales 8 and 9). Inter-factor correlations were >0.80 for scales 6/7 = 0.82, 7/8 = 0.87, 6/9 = 0.84, 7/9 = 0.92 and 8/9 = 0.93. It is frequently argued that an inter-factor correlation of >0.80 to >0.85 indicates a potential lack of discriminant validity (Brown 2006). This suggests that there may be a lack of discriminant validity in second part of the HLQ, particularly for scales 7, 8 and 9.

By allowing some flexibility in the estimation of residual correlations and cross-loadings the nine-factor BSEM model was an excellent fit to the data (the posterior predictive probability value was 0.49 compared with the target value of 0.50). There were nine statistically significant cross-loadings, the largest being 0.28. Correlations between scales 6–9 were 6/7 = 0.72, 6/8 = 0.63, 6/9 = 0.67, 7/8 = 0.83, 7/9 = 0.85, 8/9 = 0.84.

Table 4 shows pattern of HLQ scores according to sociodemographic variables. Females were somewhat higher than males only on 8. ‘Ability to find good health information’. Younger people had substantially higher scores on 4. ‘Social support for health’ than older people. There were differences on six HLQ variables for education, all indicating that people with higher education have higher health literacy than those with less education. The strongest effects were seen for 8. ‘Ability to find good health information’. There are no significant differences between those who did or didn’t have Danish as their mother tongue. Five scales showed that those with a long term illness or disability had higher health literacy, the largest effect was seen for 4. ‘Social support for health’ and 2. ‘Having sufficient information to manage my health’. The strongest pattern of all exogenous variables was with self-rated health. All scales demonstrated a difference between those rating themselves as having 'Excellent' or 'Very good' health compared with 'Good' or worse health . The effect was strongest for 4. ‘Social support for health’, 2. ‘Having sufficient information to manage my health’, and 3. ‘Actively managing my health’.

In this study, we applied rigorous linguistic and cultural adaption methods to the HLQ to produce a high quality Danish version. We used a process we call measurement adaptation whereby the meanings of the original concepts were carefully reproduced. Consequently, data from a diverse validation sample of the Danish population demonstrated that the translated HLQ has a strong psychometric structure, and the reliability of the original HLQ was maintained. Within each scale, the Danish items have a range of difficulty that should enable the Danish HLQ to be sensitive to differences between groups and change over time. Importantly, respondents clearly understood all the translated items. This series of rigorous psychometric and practical tests indicate that the Danish HLQ is robust and suitable for application in Danish settings, and that the reproduction of the nine scales in a different language and setting attests to the fundamental separate elements embodied in each of the HLQ constructs (Osborne et al. 2013).

The cognitive interviews generated valuable information. It was demonstrated that the translated items were understood according to the item intents by Danish people from a wide range of backgrounds.

Within scales there was some item residual correlation. Such correlations can mean construct complexity (i.e., sub-constructs) or item redundancy (Boyle 1991). In some cases, this seemed to be due to limited choices of words in the Danish language. For example, two separate words exist in English for ‘asking’ and ‘discussing’ (items 6.4 and 6.5) but not in Danish in this context. For items 5.1 and 5.3, the correlation is thought to relate to the positioning of these items on Bloom’s taxonomy (Amer 2006). That is, the items measure the easy and difficult ends of the same dimension: item 5.1 (0.27) is a relatively easy item on which to score highly, and item 5.3 (0.41) is a relatively hard item. While there is some conceptual overlap across these items, this scale, like most HLQ scales, has a good range of item strength and therefore has wide coverage of the target construct.

The majority of the Danish HLQ items load highly on their respective factors and the scales have good reliability. The fit of both single-factor and nine-factor models to the data was generally good, indicating scale homogeneity. Further, scales 1–6 showed clear discriminant validity, while discriminant validity was not clearly established for scales 7–9. By allowing for small correlated item residuals and cross-loadings the BSEM analysis showed inter-factor correlations among scales 6–9 that were somewhat smaller, however three remained >0.80 but ≤0.85. The overlap between scales 7–9 suggests that there may be a higher-order factor or causal linkages determining the stronger association between the concepts measured by these scales. Strong associations between these scales have been noted in previous analyses of the English HLQ (Osborne et al. 2013). Scales 8 and 9 focus on the ability to locate (scale 8) and appraise (scale 9) health information, while scale 7 (health system navigation) may be seen as a closely linked outcome of these abilities.

Every item clearly loaded on its own factor, with only three of the 44 items loading <0.6. Given that every scale comprised only four, five or six items, this demonstrates parsimony: that is, a minimum number of items are administered to respondents to reliably capture the full breadth of the intended constructs (Boyle 1991). Many questionnaires achieve high reliability through the inclusion of large numbers of items, or of items that have only minor linguistic differences and conceptual redundancy (Boyle 1991). In the Danish HLQ, only one scale had reliability below our nominal 0.8 cut-off: 5. ‘Appraisal of health information’ (0.77). This scale included the most difficult items and asked respondents about a range of challenging tasks such as comparing, checking up and finding out about information. These concepts are high in Bloom’s taxonomy (Krathwohl 2002; Amer 2006) and are likely to be more difficult for respondents to attend to and process successfully. Thus respondents may have found it challenging to judge their own critical appraisal ability, and this possibly contributed to the lower, but acceptable, reliability of the scale.

The wide ranging difficulty of items within a scale was an important and deliberate step in the development of the HLQ as it is intended to make the questionnaire sensitive to small differences at both the low score (e.g., strongly disagree or cannot do) and high score (e.g., strongly agree or very easy) ends of the scales. The ability to measure differences between subgroups was demonstrated in a recent study that used two of the scales from the Danish HLQ (scales 6 and 9—the understanding and engagement dimensions of health literacy) (Bo et al. 2014). The response options were slightly different because the extreme cannot do category in this general survey was omitted. However, the estimates clearly pointed out that 8–20 % of the Danish population have difficulties with these two dimensions of health literacy. Further, the study showed that the elderly, immigrants, those with limited formal education, and low-income groups were more likely to have limited health literacy skills. The findings are in line with Dutch results using the European Health Literacy Survey, where having a low level of education, or a low perceived social status, or being male were found to be modestly related to low health literacy scores, mainly for accessing and understanding health information (van der Heide et al. 2013).

The HLQ scales were related to the sociodemographic subgroups in our sample in reasonably expected ways. While the items were written specifically to avoid potential bias in gender, age and education, the data indicated that women were slightly better than men in their ability to find good information, however differences were not observed in other domains. Younger people (<65 years), compared with older people (≥65 years), reported more ‘Social support for health’ (scale 4), probably related to isolation in older people. As expected, education was clearly related to health literacy, but primarily in those HLQ domains with content covering finding and processing information (scales 2, 5, 7, 8 and 9) and not with domains related to feeling understood or engaging with health care professionals (scales 1, 3 and 6). In previous research we have seen language variables, such as the language spoken at home, to be strongly related to health literacy (Beauchamp et al. 2015). In the current study, with 93 % having Danish as their mother tongue and where no questionnaires were offered in other languages, almost no association between language and health literacy was observed. Only a small effect was observed for the scale with content requiring the highest language skills, 5. ‘Appraisal of health information’, where those without Danish reported slightly higher scores. It is possible that this small group included nationalised professionals, however future studies need to explore this in more detail.

Having a long term illness or disability was moderately and positively related to several health literacy domains. This is consistent with the notion that health literacy develops in individuals as they gain experience with managing illnesses, working with practitioners over many years, and through overcoming information barriers (Paasche-Orlow and Wolf 2007). Finally, we expect that good health literacy is a determinant, antecedent or consequence of good health (Paasche-Orlow and Wolf 2007; Berkman et al. 2011; Batterham et al. 2016). Consequently, we observe this pattern through all nine HLQ domains positively related to self-rated health.

Strengths and limitations

It is intended that the HLQ can be used in diverse settings, and with people with broad-reaching sociodemographic profiles and various health conditions. A strength of this study is that the validation sample was drawn from communities across a wide range of locations, with an array of health conditions, and who were young and old. The face-to-face administration of the questionnaire ensured a high participation rate, particularly from people with limited reading and writing abilities who usually are excluded from taking part in questionnaire studies.

There are several indications that the Danish HLQ has good psychometric properties. The use of modern psychometric procedures permitted the application of highly demanding tests, and the analysis has clearly shown that the HLQ multi-dimensional construct of health literacy comprises nine separate and cogent scales. A nine-factor CFA model demonstrated good construct validity. A model with no residual correlations or cross-loadings was an acceptable fit to the data while a very good fit was achieved with a BSEM analysis that allowed modest correlated residuals and cross-loadings.

The stringent linguistic, cultural and measurement adaptation procedures are likely to have contributed to the strong psychometric performance of the Danish HLQ. This procedure includes an extensive item intent document that explains what the elements of individual items mean and do not mean. This document, along with the intensive translation consensus meeting, ensures the translators are supported to capture the precise meaning, context, difficulty, and measurement juxtaposition of one item with other items within each scale.

A strength of the HLQ is that it has sensitivity to group differences related to illness or disability, and self-rated health. This has also been demonstrated in other settings (Beauchamp et al. 2015; Zhang et al. 2016; Vamos et al. 2016). While it is important that differences between groups are demonstrated, research needs to be done to demonstrate sensitivity to change. This issue is being addressed in a range of ongoing intervention studies where the HLQ is part of the assessments of a range of outcomes (Batterham et al. 2014; Livingston et al. 2014; Redfern et al. 2014; Banbury et al. 2014; Barker et al. 2015; Griva et al. 2015).

Another possible limitation that needs further investigation is the somewhat high inter-correlations between Danish HLQ scales 7, 8 and 9. These data suggest that there may be a lack of discriminant validity between these scales; however, this may also be explained by higher-order models and causal relationships between these constructs. This provides guidance for further work, particularly in exploring how the scales respond, with careful exploration of rates of change (or no change) over time.

Finally, this study only explored administration of the HLQ in face-to-face interviews. This is a strength of the study because it overcomes illiteracy of respondents, but future work should continue to explore other modes of administration including self-administered written formats, and confirm whether the HLQ produces valid information that results in improved services across the Danish society.

In conclusion, this study demonstrates that the Danish HLQ has strong construct and content validity, and high composite reliability. As such, the HLQ is now available for application in Denmark with nine scales providing a robust multi-dimensional approach to understanding health literacy. It is incumbent on the developers of questionnaires to demonstrate the measurement properties of new questionnaires, and that the data they return to consumers, practitioners, policymakers and researchers provide valid inferences and are reliable. The findings of this study may be an important contribution to health literacy research, and the data support the web of evidence of measurement validity of the interpretation of HLQ scores for use in national and international population health and healthcare.