Development and Validation of a New Short-Form Health Literacy Instrument (HLS-SF12) for the General Public in Six Asian Countries
No comprehensive short-form health literacy (HL) survey tool has been available for general use across Asia.
This study aimed to develop and validate a short-form HL instrument derived from the 47-item European Health Literacy Questionnaire (HLS-EU-Q47).
A population survey (N = 10,024) was conducted from 2013 to 2015 using the HLS-EU-Q47 in 1,029 participants from Indonesia, 1,845 from Kazakhstan, 462 from Malaysia, 1,600 from Myanmar, 3,015 from Taiwan, and 2,073 from Vietnam. Validation of the short form was evaluated by principle component analysis, internal consistency, Pearson correlation, and regression analysis.
Based on responses from six countries, a 12-item short-form HL questionnaire (HLS-SF12) was developed, retaining the conceptual framework of the HLS-EU-Q47 and accounting for the high variance of the full-form (i.e., 90% in Indonesia, 91% in Myanmar, 93% in Malaysia, 94% in Taiwan, and 95% in both Kazakhstan and Vietnam). The HLS-SF12 was demonstrated to have adequate psychometric properties, including high reliability (Cronbach's alpha = .85), good criterion-related validity, a moderate and high level of item-scale convergent validity, no floor or ceiling effect, and good model-data-fit throughout the populations in these countries.
The HLS-SF12 was shown to be a valid and reliable tool for HL surveys in the general public in six Asian countries. [HLRP: Health Literacy Research and Practice. 2019;3(2):e90–e102.]
Plain Language Summary:
A health literacy survey was conducted from 2013 to 2015 in six Asian countries using the European Health Literacy Questionnaire (HLS-EU-Q47). The collected data were used to develop and validate a comprehensive short-form questionnaire. A health literacy questionnaire with 12 items (HLS-SF12) that retains the original conceptual framework of the HLS-EU-Q47 was demonstrated to be reliable and valid.
The purpose of health literacy (HL) is to help people make health-related decisions and take appropriate actions to manage their health (Sørensen et al., 2012). The ability to find, understand, appraise, and apply information related to health is essential in medical services, and it is also important in disease prevention and health promotion (Sørensen et al., 2012). Strengthening HL is seen as a solution to increase the compliance and adherence of people with both medication and nonmedication regimens in chronic and acute diseases (Miller, 2016), to empower disease self-management skills (Mackey, Doody, Werner, & Fullen, 2016), and to guarantee the behavior changes (Duong, Sørensen, et al., 2018; Guntzviller, King, Jensen, & Davis, 2016; Yokokawa et al., 2016). Therefore, it is important for health care providers to quickly evaluate and understand people's HL comprehensively to facilitate effective interventions and improve health outcomes.
The 47-item European Health Literacy Questionnaire (HLS-EU-Q47) was developed (Sørensen et al., 2013), based on the comprehensive definition and conceptual model of HL (Sørensen et al., 2012). The HLS-EU-Q47 had been demonstrated as a valid tool to assess HL of the general public in Taiwan (Duong et al., 2015), Japan (Nakayama et al., 2015), eight European countries (HLS-EU Consortium, 2012; Pelikan & Ganahl, 2017; Sørensen et al., 2015), and six Asian countries (Duong, Aringazina, et al., 2017).
The purpose of the HLS-EU-Q47 was to study the various dimensions of HL in depth, so it has many questions. It was not developed as a quick screening tool. Several short-form questionnaires were developed to provide quicker assessment of HL of study participants, including the Short Test of Functional Health Literacy in Adults (S-TOHFLA) (Baker, Williams, Parker, Gazmararian, & Nurss, 1999), the Rapid Estimate of Adult Literacy in Medicine-Revised (REALM-R) (Bass, Wilson, & Griffith, 2003), as well as the short-form Mandarin Health Literacy Scale (SHEAL), which is being used in clinical and research settings (Lee, Tsai, Tsai, & Kuo, 2012). These short-form questionnaires have certain advantages and disadvantages, such as not including the comprehensive aspects of HL. A comprehensive short-form was validated for the general public in Taiwan (Duong, Chang, et al., 2017), but was not available for the general public or general publication in other Asian countries. Therefore, it is important to develop and validate a comprehensive short-form questionnaire based on the HLQ-EU-Q47 data that could be conducted in the general public in six Asian countries with the hope that it could be used to screen for HL in Asia.
The aim of this study was to develop and validate a short-form HL questionnaire from the HLS-EU-Q47 in six Asian countries.
Sources of Data
This study used data from a cross-sectional survey in six Asian countries using the HLS-EU-Q47. It was conducted among people age 15 years and older, between the years of 2013 and 2015, and by different multistage sampling methods with different population structures (Duong, Aringazina, et al., 2017). The overall samples of 10,024 participants were included in the analysis, including 1,029 from Indonesia, 1,845 from Kazakhstan, 462 from Malaysia, 1,600 from Myanmar, 3,015 from Taiwan, and 2,073 from Vietnam (Duong, Aringazina, et al., 2017).
The surveys were approved by the Institutional Review Boards (IRB) in all partner countries: Dian Nuswantoro University, Indonesia (Number 33/EC/FKM/2014); Kazakhstan School of Public Health (Number A043); University Malaya Medical Centre, Malaysia (MEC Reference Number 896.34); University of Medicine 1, Yangon, Myanmar; the Joint IRB of Taipei Medical University, Taiwan (Number 201305007); and Hanoi School of Public Health, Vietnam (Number 014-254/DD-YTCC).
HL was measured by the HLS-EU-Q47 (Sørensen et al., 2013). The perceived difficulty of each health-related task was rated on 4-point Likert scales (1 = very difficult, 2 = difficult, 3 = easy, and 4 = very easy), which were translated and validated in Asian countries (Duong, Aringazina, et al., 2017).
The HL matrix was constructed from 4 steps of information processing (finding health information, understanding health information, judging health information, applying health information) in three health domains (health care, disease prevention, health promotion), which created a total of 12 dimensions or components of HL (Sørensen et al., 2012). The combination of 47 items that comprised the indices, including a general HL index consisting of all the items, three subindices covering three domains of health, four subindices covering 4 stages of information processing, and 12 specific sub-subindices corresponding to 12 single cells in the matrix (HLS-EU Consortium, 2012). The general HL index and three subindices according to health care, disease prevention, and health promotion were commonly used in European countries (HLS-EU Consortium, 2012), and Asian countries (Duong, Aringazina, et al., 2017; Nakayama et al., 2015).
The HL indices were standardized to unified metrics from 0 to 50 using the formula [index = (mean – 1) × (50/3)], in which the Index was the specific index calculated, Mean was the mean of all participating items for each person, 1 was the minimal possible value of the mean (leading to a minimum value of the index of 0), 3 was the range of the mean, and 50 was the chosen maximum value of the new metric. An index value was thus obtained in which 0 represented the lowest HL and 50 the highest HL (HLS-EU Consortium, 2012).
The short-form HL questionnaire was developed using data collected in Taiwan and further validated using datasets from six Asian countries (Figure 1). Because validation of a short-form test should be conducted independently using independent subject samples (Coste, Guillemin, Pouchot, & Fermanian, 1997; Goetz et al., 2013), the samples from Taiwan were randomly split into two groups: sample N1 = 1,514; and sample N2 = 1,501.
Principal component analysis. The Kaiser-Meyer Olkin Measure (KMO) of sampling adequacy was employed to determine the suitability of the data for factor analyses and was set to be >.60, whereas Bartlett's test of sphericity was set to <.05. These measures demonstrated a sufficient sample size and indicated adequate correlations between the variables, so the variables could be reduced to a smaller number of components (Kaiser, 1974). To determine the numbers of components to be retained, an updated guideline for shortening scales by Goetz et al. (2013) was employed to take the conceptual model into account. The 12 components, representing 12 dimensions of the HL conceptual model, were retained purposely in the principal component analysis (PCA), whereas the oblique rotation (i.e., promax) was recruited (Field, 2013).
Linear regression analyses. The use of regression analysis for developing the short-form questionnaire was applied in previous studies (Campbell et al., 2014; Jones, Brennan, Parker, & Jamieson, 2014)
It had been suggested that an equal number of HL items in each component/subdomain could contribute to a more stable measure and enhance internal consistency (Goetz et al., 2013; Kimberlin & Winterstein, 2008). In addition, the number of items in the short form was expected to be less than or equal to one-half of the items in its full-form (Goetz et al., 2013). Therefore, the selected 24 items with the highest factor loadings from the 12 components were analyzed by linear regression model to estimate the standardized-coefficient values (24 items were treated as independent variables, and HL index of 47 items was treated as the dependent variable). This set of 24 items was named as subset A. Two subsets with 12 items were developed; one was selected by factor loading only (subset B), and the other was selected first by factor loading and further by the standardized-coefficient values (subset C). Two linear regressions were conducted to examine how much variance of the full-form scale could be accounted for separately by the two selected item subsets (two sets of 12 items [subset B and subset C] were treated as independent variables and HL index of full-form was treated as dependent variable) to get the adjusted R2 values that explained the variance of the full form.
Psychometric properties of the short-form health literacy questionnaire
Item-scale convergent validity. This was examined by the correlation between the items and their own theoretical scale (Hays & Hayashi, 1990), and was determined by the Pearson correlation coefficient. In this study, the correlation between HL items and their sub-HL indices was examined.
Criterion-related validity analysis. The correlation between short form and full form was examined by Pearson correlation, which provided evidence of criterion-related validity (Kimberlin & Winterstein, 2008).
Construct validity. Confirmatory factor analyses (CFA) were used to examine the model-fit-indices of the three-factor model. The items representing 12 dimensions/components of HL loaded into three health domains as three factors in the CFA were health care, disease prevention, and health promotion.
Reliability analysis. Internal consistency was tested with Cronbach's alpha value designated ≥.70 for satisfactory reliability (Cronbach & Shavelson, 2004).
Floor and ceiling effects analyses. Due to the scale of the survey, floor and ceiling effect analyses were limited in their responsiveness. The floor or ceiling effects referred to a high percentage of participants scoring the possible lowest score or achieving the possible highest score, respectively. Therefore, minimal floor and ceiling effects were recommended for the HLS-EU-Qs scales, and ≥15% at floor or ceiling levels was considered a significant effect (Terwee et al., 2007).
All statistical analyses were performed using the SPSS Version 20, and Amos version 22. The significance level was set at p < .05. The statistical strategy is illustrated in Figure 1.
Development of the New Short-Form Health Literacy Questionnaire
In the current study, a PCA was conducted on the first dataset in Taiwan (N1 = 1,514) with oblique rotation (promax) method (Field, 2013). The KMO value was .96 for the whole scale at a satisfactory level (Hutcheson & Sofroniou, 1999), whereas the KMO values for individual items were >.90, which is also well above the acceptable limit of .50. The results indicated the adequacy of sample sizes for the analysis (Field, 2013).
The PCA was performed to obtain an eigenvalue of each component in the survey data based on the conceptual framework of HL (Goetz et al., 2013; Sørensen et al., 2012) and to maintain the 12 components, which explained 67.3% of the variance. The eigenvalues of the 12 components before oblique rotation ranged from 0.78 to 16.56. After oblique rotation (promax), the eigenvalues ranged from 5.48 to 9.99, with the factor loadings of these selected items shown in Table 1. The average communality value of .67 was greater than the required value of .60, demonstrating satisfactory approach (Field, 2013). In addition, positive correlations were observed between 12 components (from .24 to .55), which indicated that the promax rotation method was used properly (Field, 2013).
|Conceptual Dimension Item No.a||Components||Standardized Coefficient (beta)|
|1||2||3||4||5||6||7||8||9||10||11||12||Subset A||Subset Bb||Subset Cc|
The subset A contained 24 items with the highest factor loadings (Table 1) and included HLS-EU-Q47 items 1, 2, 5, 6, 10, 12, 14, 15, 17, 18, 21, 23, 25, 26, 30, 31, 33, 34, 38, 39, 42, 43, 45, and 46. The subset B contained 12 items of the short form selected by the highest factor loadings in each component and included HLS-EU-Q47 items 2, 5, 10, 15, 18, 21, 26, 31, 33, 38, 42, and 45.
Linear regression analysis was conducted on 24 items with those of high factor loading values in subset A. The standardized-coefficient values were obtained to make a more stringent selection, resulting in subset C, which included HLS-EU-Q47 items 2, 6, 10, 15, 18, 23, 26, 30, 33, 39, 43, and 45 (Table 1).
The results showed that the 24-item short-form (i.e., subset A) explained 97% of the variance of the full-form (R2 = .97), whereas the 12-item short form selected by factor loading (subset B) explained 92% (R2 =.92), and the 12-item short form selected by factor loading and standardized-coefficient values (subset C) explained 94% (R2 = .94) (Table 1). The subset C with these specifically selected 12 items was then named the Health Literacy Short-Form 12 (HLS-SF12), which retained 12 dimensions of the conceptual framework and explained more variance of the full form (94%) than subset B (92%). The HLS-SF12 was shown to explain 94% of the variance of the full form, which is only 3% less than the 24-item subset A (which had 97% variance). Because of the significant reduction in number of question items, and its representativeness of the full form, the HLS-SF12 was chosen for further empirical psychometric testing.
Validation of the HLS-SF12
Psychometric analyses were performed on the second Taiwan dataset (N2 =1,501) and datasets from the other five Asian countries (Indonesia [N = 1,029], Kazakhstan [N = 1,845], Malaysia [N = 462], Myanmar [N = 1,600], and Vietnam [N = 2,073]). All of these items were shown with the satisfactory criterion of item-scale convergent validity (average item-scale correlation ranged from .59 to .80) (Table 2). Internal consistency of HLS-SF12 was satisfied with Cronbach's alpha values from .79 in Indonesia to .90 in Kazakhstan, and there was no floor or ceiling effect, as the percentages of people with the lowest scores or the highest scores of HL at floor or ceiling were less than 15%. The percentage of the score at floor ranged from 0.1% to 0.3%, and ceiling ranged from 1.9% to 9.5% for overall HLS-SF12 scale and its subscales (Table 2).
|Indonesia (N = 1,029)||Kazakhstan (N = 1,845)||Malaysia (N = 462)||Myanmar (N = 1,600)||Taiwan (N2= 1,501)||Vietnam (N = 2,073)|
|Item-scale convergent validity, mean of correlations (range)|
|Gen-HL||.55 (.49–.60)||.69 (.65–.75)||.62 (.55–.68)||.59 (.38–.68)||.59 (.50–.66)||.64 (.58–.69)|
|HC-HL||.63 (.59–.67)||.74 (.69–.77)||.70 (.66–.73)||.69 (.65–.73)||.66 (.60–.72)||.73 (.70–.74)|
|DP-HL||.69 (.63–.74)||.77 (.75–.79)||.73 (.71–.76)||.73 (.70–.75)||.69 (.65–.73)||.74 (.72–.77)|
|HP-HL||.69 (.68–.72)||.80 (.77–.82)||.71 (.67–.76)||.67 (.61–.71)||.73 (.70–.75)||.73 (.70–.77)|
|Reliability (Cronbach's alpha)|
|Floor effects (%)|
|Ceiling effect (%)|
The HLS-SF12 index score was calculated by the same formula as the full form (HLS-EU Consortium, 2012), which provided HLS-SF12 index scores of 30.5 ± 6.4 in Indonesia, 31.6 ± 9.5 in Kazakhstan, 32.7 ± 7.9 in Malaysia, 29.2 ± 9.3 in Myanmar, 34.3 ± 6.9 in Taiwan, and 29.5 ± 9.5 in Vietnam. The correlation between HL index scores of HLS-SF12 and HLS-EU-Q47 by the Pearson correlation coefficient was satisfactory with rho-value of .97 in Taiwan, .94 in Indonesia, .97 in Kazakhstan, .96 in Malaysia, .95 in Myanmar, and .97 in Vietnam, which indicated satisfactory criterion-related validity. The variance of the full form was explained by the HLS-SF12, with 90% in Indonesia, 91% in Myanmar, 93% in Malaysia, 94% in Taiwan, 95% in Kazakhstan, and 95% in Vietnam.
Finally, the construct validity was analyzed by CFA with maximum likelihood algorithm estimation (Kline, 2013). The root mean square error of approximation (RMSEA) value of goodness-of-fit indices (GFI) for HLS-SF12 was ≤.06, and GFI, adjusted GFI, confirmatory fix index, incremental fit index, and normed fit index ranged from .92 to .98 for six Asian countries (Table 3), which indicated good model-data-fit (Floyd & Widaman, 1995). The correlations among the three HL domains health care, disease prevention, and health promotion were significantly strong with values ranging from .71 to .96 (Figure 2).
|Absolute Model Fit||Incremental Fit||Parsimonious Fit|
Development of the New Short-Form Health Literacy Questionnaire
PCA is one of the methods of choice to reduce the number of items of an instrument by eliminating specific items with low factor loading (Goetz et al., 2013). In our study, PCA was used to develop a scale to measure what needed to be measured, instead of measuring what respondents know about in the item response. In addition, HL was measured as the ability needed to help people improve their health (Sørensen et al., 2012). Therefore, the PCA was used adequately in this study. Based on the results of PCA using data collected in Taiwan, a short-form HLS-EU with 12 items, named HLS-SF12, was developed by selection through factor loading and standardized-coefficient values while maintaining the 12 dimensions/components of the comprehensive HL model (Sørensen et al., 2012). The strength of HLS-SF12 as compared with HLS-EU-Q16 was that HLS-SF12 covered 12 dimensions of health literacy with 12 questions, and the 4-point Likert scale responses could be transformed into HL indices from 0 to 50. The HLS-EU-Q16, which was also derived from the HLS-EU-Q47, covered 11 of the 12 dimensions but with more questions, and the responses were transformed into “yes” or “no” and scored from 1 to 16, which might increase the measurement bias (Pelikan, Rothlin, Ganahl, & Peer, 2014). Therefore, we feel that the HLS-SF12 could be a useful tool in an Asian context.
Validation of the HLS-SF12
The HLS-SF12 was as valid as the HLS-EU-Q47 in detecting differences among subgroups of the population with levels of HL. In addition, HLS-SF12 was strongly correlated with the HLS-EU-Q47, with adequate evidence of criterion-related validity (Kimberlin & Winterstein, 2008). The HLS-SF12 was shown with the satisfactory criterion of item-scale convergent validity, with all items correlated with their own scales at a value of > .40 (Hays & Hayashi, 1990). It illustrated good model-data-fit (Floyd & Widaman, 1995), which supported a hypothetical construct of HLS-SF12 in different countries.
The HLS-SF12 also showed high internal consistency, with the Cronbach's alpha value similar with those in the original HLS-EU survey (Cronbach's alpha = .87∼.97) (HLS-EU Consortium, 2012), and in Taiwan (Cronbach's alpha = .96) (Duong et al., 2015). This suggested that these items were homogenous in reflecting the HL of these participants. In addition, there was minimal floor/ceiling effect to show the robust reliability of those subscales (Terwee et al., 2007).
In comparison with other short-form tools for HL, the HLS-SF12 was shorter than 26-item S-TOFHLA (Baker et al., 1999), and HLS-EU-Q16 (Pelikan, Rothlin, Ganahl, & Peer, 2014), but a bit longer than HLS-EU-Q6 (Pelikan et al., 2014), the 8-item REALM-R (Bass et al., 2003), and the 11-item SHEAL in Taiwan (Lee et al., 2012). Most importantly, HLS-SF12 was able to maintain the theoretical framework and comprehensive construct of the HLS-EU-Q47 (Sørensen et al., 2012). Overall, the HLS-SF12 was able to explain 94% of the variance of the HLS-EU-Q47 full-form in Asian countries, which is slightly higher than 92% of the HLS-EU-Q16 in European countries. This may be caused by an equal number of items of the HLS-SF12 in each component that could provide stable measurement and enhance internal consistency (Goetz et al., 2013; Kimberlin & Winterstein, 2008). On the other hand, HLS-SF12 could better represent the theoretical construct of HL in all of the 12 dimensions. Moreover, HLS-SF12 could possibly provide a reliable indicator for its general HL, and the three indices for three highly correlated dimensions of HL (health care, disease prevention, and health promotion), as well as the indices for four competencies of information processing. This could serve different interests of health care providers when assessing HL in the public.
Finally, in this analysis, HLS-SF12 was comparable with the HLS-EU-Q47 in illustrating the slight differences in HL in several Asian countries. The HL levels measured by the HLS-SF12 and HLS-EU-Q47, respectively, ranged from scores of 29.5 ± 9.5 vs. 29.6 ± 9.1 in Vietnam, 29.2 ± 9.3 vs. 31.3 ± 8.7 in Myanmar, 30.5 ± 6.4 vs. 31.4 ± 5.8 in Indonesia, 31.6 ± 9.5 vs. 31.6 ± 9.3 in Kazakhstan, 32.7 ± 7.9 vs. 32.9 ± 7.2 in Malaysia, and 34.3 ± 6.9 vs. 34.4 ± 6.6 in Taiwan (Duong, Aringazina, et al., 2017).
The HLS-SF12 was developed and validated across several countries in Asia using three-factor models based on the conceptual framework of HL, and specifically on three dimensions of health. The measurement invariance in different groups was also examined previously (Sudbury-Riley, FitzPatrick, & Schulz, 2017). The metric invariance was not investigated using the single-factor model in the current study. However, the scale homogeneity was supported by the item-scale correlation, floor and ceiling effects (Table 2), and inter-factor correlations in the CFA shown in Figure 2. This was mentioned in a previous study on measurement invariance of the HL questionnaire (Elsworth, Beauchamp, & Osborne, 2016).
The development and validation of the short form based on a single cross-sectional survey conducted on a population in Taiwan and other Asian countries may pose restrictions on the external validity and reproducibility. However, it is common to develop and test single samples in instrument development and testing (Norris & Lecavalier, 2010), including several other cross-sectional studies on the development of short-forms (Coste et al., 1997; Li & Lopez, 2007; Norris & Lecavalier, 2010). On the other hand, it would be necessary to further validate the HLS-SF12 by other advanced study designs.
The HLS-SF12 was developed and based on the HLSEU-Q47. The HLS-SF12 maintained all the 12 components and presented the original conceptual dimensions of the HLS-EU-Q47 with good validity. We feel it would be useful for easy and accurate assessment of HL in larger populations or clinical settings in Asia.
- Baker D. W., Williams M. V., Parker R. M., Gazmararian J. A., Nurss J. (1999). Development of a brief test to measure functional health literacy. Patient Education and Counseling, 38(1), 33–42.
- Bass P. F., Wilson J. F., Griffith C. H. (2003). A shortened instrument for literacy screening. Journal of General Internal Medicine, 18(12), 1036–1038.
- Campbell H. S., Hall A., Sanson-Fisher R., Barker D., Turner D., Taylor-Brown J. (2014). Development and validation of the Short-Form Survivor Unmet Needs Survey (SF-SUNS). Supportive Care in Cancer, 22(4), 1071–1079.
- Coste J., Guillemin F., Pouchot J., Fermanian J. (1997). Methodological approaches to shortening composite measurement scales. Journal of Clinical Epidemiology, 50(3), 247–252.
- Cronbach L. J., Shavelson R. J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64(3), 391–418.
- Duong T. V., Aringazina A., Baisunova G., Nurjanah , Pham T. V., Pham K. M., Chang P. W. (2017). Measuring health literacy in Asia: Validation of the HLS-EU-Q47 survey tool in six Asian countries. Journal of Epidemiology, 27(2), 80–86.
- Duong T. V., Chang P. W., Yang S.-H., Chen M.-C., Chao W.-T., Chen T., Huang H.-L. (2017). A new comprehensive short-form health literacy survey tool for patients in general. Asian Nursing Research, 11(1), 30–35.
- Duong T. V., Lin I.-F., Sørensen K., Pelikan J. M., Van den Broucke S., Lin Y.-C., Chang P. W. (2015). Health literacy in Taiwan: A population-based study. Asia-Pacific Journal of Public Health, 27(8), 871–880.
- Duong T. V., Sørensen K., Pelikan J., Van den Broucke S., Lin I. F., Lin Y.-C., Chang P. (2018). Health-related behaviors moderate the association between age and self-reported health literacy among Taiwanese women. Women & Health, 56(6), 632–646.
- Elsworth G. R., Beauchamp A., Osborne R. H. (2016). Measuring health literacy in community agencies: A Bayesian study of the factor structure and measurement invariance of the health literacy questionnaire (HLQ). BMC Health Services Research, 16, 508.
- Field A. (2013). Discovering statistics using IBM SPSS statistics (4th ed.). London, UK: Sage. Google Scholar
- Floyd F. J., Widaman K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7(3), 286–299.
- Goetz C., Coste J., Lemetayer F., Rat A.-C., Montel S., Recchia S., Guillemin F. (2013). Item reduction based on rigorous methodological guidelines is necessary to maintain validity when shortening composite measurement scales. Journal of Clinical Epidemiology, 66(7), 710–718.
- Guntzviller L. M., King A. J., Jensen J. D., Davis L. A. (2016). Self-efficacy, health literacy, and nutrition and exercise behaviors in a low-income, Hispanic population. Journal of Immigrant and Minority Health, 19(2), 489–493.
- Hays R. D., Hayashi T. (1990). Beyond internal consistency reliability: Rationale and user's guide for multitrait analysis program on the microcomputer. Behavior Research Methods, Instruments, & Computers, 22(2), 167–175.
- HLS-EU Consortium. (2012). Comparative report of health literacy in eight EU member states. The European Health Literacy Project 2009–2012. Retrieved from: https://www.healthliteracyeurope.net/hls-eu Google Scholar
- Hutcheson G., Sofroniou N. (1999). The multivariate social scientist. London, UK: Sage.
- Jones K., Brennan D., Parker E., Jamieson L. (2014). Development of a short-form health literacy dental scale (HeLD-14). Community Dentistry and Oral Epidemiology, 43(2), 143–151.
- Kaiser H. F. (1974). An index of factorial simplicity. Psychometrika, 39(1), 31–36.
- Kimberlin C. L., Winterstein A. G. (2008). Validity and reliability of measurement instruments used in research. American Journal of Health-System Pharmacy, 65(23), 2276–2284.
- Kline R. B. (2013). Exploratory and confirmatory factor analysis. In (Eds.),Applied quantitative analysis in the social sciences (pp. 171–207). New York, NY: Routledge. Google Scholar
- Lee S.-Y. D., Tsai T.-I., Tsai Y.-W., Kuo K. N. (2012). Development and validation of the short-form Mandarin Health Literacy Scale. Taiwan Journal of Public Health, 31(2), 184–194. Google Scholar
- Li H. C. W., Lopez V. (2007). Development and validation of a short form of the Chinese version of the State Anxiety Scale for Children. International Journal of Nursing Studies, 44(4), 566–573.
- Mackey L. M., Doody C., Werner E. L., Fullen B. (2016). Self-management skills in chronic disease management: What role does health literacy have?Medical Decision Making, 36(6), 741–759.
- Miller T. A. (2016). Health literacy and adherence to medical treatment in chronic and acute illness: A meta-analysis. Patient Education and Counseling, 99(7), 1079–1086.
- Nakayama K., Osaka W., Togari T., Ishikawa H., Yonekura Y., Sekido A., Matsumoto M. (2015). Comprehensive health literacy in Japan is lower than in Europe: A validated Japanese-language assessment of health literacy. BMC Public Health, 15(1), 505.
- Norris M., Lecavalier L. (2010). Evaluating the use of exploratory factor analysis in developmental disability psychological research. Journal of Autism and Developmental Disorders, 40(1), 8–20.
- Pelikan J. M., Ganahl K. (2017). Measuring health literacy in general populations: Primary findings from the HLS-EU Consortium's health literacy assessment effort. Studies in Health Technology and Informatics, 240, 34–59. Google Scholar
- Pelikan J. M., Rothlin F., Ganahl K., Peer S. (
2014, October). Measuring comprehensive health literacy in general populations – the HLS-EU instruments. Paper presented at the Second International Conference of Health Literacy and Health Promotion, Taipei, Taiwan. Google Scholar
- Sørensen K., Pelikan J. M., Rothlin F., Ganahl K., Slonska Z., Doyle G., Brand H. (2015). Health literacy in Europe: Comparative results of the European health literacy survey (HLS-EU). European Journal of Public Health, 25(6), 1053–1058.
- Sørensen K., Van den Broucke S., Brand H., Fullam J., Doyle G., Pelikan J., Slonszka Z. (2012). Health literacy and public health: A systematic review and integration of definitions and models. BMC Public Health, 12(1), 80.
- Sørensen K., Van den Broucke S., Pelikan J., Fullam J., Doyle G., Slonska Z., Brand H. (2013). Measuring health literacy in populations: Illuminating the design and development process of the European Health Literacy Survey Questionnaire (HLS-EU-Q). BMC Public Health, 13(1), 948.
- Sudbury-Riley L., FitzPatrick M., Schulz P. J. (2017). Exploring the measurement properties of the eHealth Literacy Scale (eHEALS) among Baby Boomers: A multinational test of measurement invariance. Journal of Medical Internet Research, 19(2), e53.
- Terwee C. B., Bot S. D. M., de Boer M. R., van der Windt D. A. W. M., Knol D. L., Dekker J., de Vet H. C. W. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60(1), 34–42.
- Yokokawa H., Fukuda H., Yuasa M., Sanada H., Hisaoka T., Naito T. (2016). Association between health literacy and metabolic syndrome or healthy lifestyle characteristics among community-dwelling Japanese people. Diabetology & Metabolic Syndrome, 8, 30.