International Journal of

Environmental Research
and Public Health

Article
Changes in Attitudes toward Mental Illness in
Healthcare Professionals and Students
Yin-Yi Lien † , Hui-Shin Lin † , Chi-Hsuan Tsai, Yin-Ju Lien * and Ting-Ting Wu
Department of Health Promotion and Health Education, National Taiwan Normal University, 162, Heping East
Road Section 1, Taipei 106, Taiwan
* Correspondence: yjlien@ntnu.edu.tw
† These authors contributed equally to this work.




Received: 23 September 2019; Accepted: 19 November 2019; Published: 22 November 2019


Abstract: Mental-illness-related stigma not only exists in the public but also in healthcare systems.
Healthcare providers (HCPs) who have stigmatizing attitudes or behaviors might be thought of as a
key barrier to mental health service use, and influence the quality of healthcare. Although cumulative
projects have been conducted to reduce stigma related to mental illness among HCPs around the
world, little is known about whether the attitudes of HCPs toward mental illness have changed
over time. Research on this topic is mixed with respect to whether attitudes of HCPs toward mental
illness have become more or less positive. The aim of the current study was to help clarify this issue
using a cross-temporal meta-analysis of scores on the Social Distance Scale (SDS), Opinions about
Mental Illness (OMI), and Community Attitudes towards Mental Illness (CAMI) measures among
health care professionals and students (N = 15,653) from 1966 to 2016. Our results indicated that
both social distance (β = −0.32, p < 0.001) and attitudes (β = 0.43, p = 0.007) of HCPs toward mental
illness have become increasingly positive over time. These findings provide empirical evidence to
support that the anti-stigma programs and courses have positive effects on HCPs and can inform
future anti-stigma programs focusing on improving the attitudes of HCPs toward mental illness,
thereby improving the quality of healthcare provided.

Keywords: mental illness; stigma; attitude; healthcare providers

1. Introduction
Mental-illness-related stigma is a focus of global public health problems. To challenge stigma
associated with mental illness, the World Psychiatric Association (WPA) constructed a global program
known as “Open the Doors” to fight the stigma and discrimination of mental illness in 1996 [1].
Many countries have also conducted mental health campaigns. For example, the Australian campaign
“Beyond Blue” was established to address depression-related issues and promote awareness among
the community [2]. A national campaign called “Time to Change”, which aimed to reduce stigma
and discrimination against people with mental health disorders, was launched in 2009 in England [3].
The German campaign “Nuremberg Alliance against Depression” was an intervention program to
increase awareness among the public [4]. Time trend studies have evaluated the effects on attitudes
toward people with mental illness among the public, and the inconsistent results have been found,
with evidence of positive change [5–8], negative change [8–10], or no change [4,8–11]. The evolution
of public attitudes towards people with mental illness has mainly been studied in Western countries
(e.g., Germany [4,8,9], Australia [5], England [5,7], and Sweden [6]). Little is known about the change
of public attitudes towards mental illness in non-Western countries.

Int. J. Environ. Res. Public Health 2019, 16, 4655; doi:10.3390/ijerph16234655 www.mdpi.com/journal/ijerph
Int. J. Environ. Res. Public Health 2019, 16, 4655 2 of 14

Stigmatizing attitudes are not only confined to the public, but are also prevalent among healthcare
providers (HCPs) [12,13]. Accumulating evidence reveals that many people with mental illness report
that HCPs, working on both mental and physical health services, are an important source of stigma
and discrimination in many countries worldwide [14,15]. Mental-illness-related stigma within the
healthcare system and among HCPs has been identified as a major barrier to treatment and recovery as
well as a significant source of poorer quality of physical care for persons with mental illness [12,16].
In other words, stigmatizing attitudes or behaviors by HCPs have the potential to lead to a lack of
attention to patients’ medical needs, mismanagement of patients with mental illness, and even social
marginalization [17].
Furthermore, another major issue regarding stigma toward mental illness among HCPs is that
it might cause staff shortages in psychiatry. A systematic review indicated that underlying stigma
among medical students towards mental illness has been suggested as an influential factor in shaping
the negative views toward a career in psychiatry [18]. Psychiatry has been facing a shortage of
specialists [19,20], and the shortage of psychiatrists might cause a growing mental health care system
crisis. Research has shown that the shrinking psychiatrist workforce was likely to affect access to
care for people with mental illnesses [21]. Under the circumstances, mental-illness-related stigma is
increasingly seen as a fundamental cause of population health inequalities and a major challenge for
public health.
The problem of mental illness-related stigma within healthcare is an area receiving increased
attention and concern [16,17,22]. A great amount of effort has been made around the world to
reduce mental-illness-related stigma among HCPs. Education Not Discrimination (END) is one of
the components of the Time to Change program, which aims to reduce mental health stigma among
healthcare professionals and professional students [23]. Furthermore, an anti-stigma initiative of the
Mental Health Commission of Canada (MHCC) known as Opening Minds (OM) has conducted a large
series of evaluations of anti-stigma programs targeting various HCPs since 2009 [24]. The OM program
has had a dual focus addressing stigma within healthcare as a workplace, as well as addressing stigma
within consumer–provider interactions and quality of care. There is substantial research evaluating the
attitude toward mental illness among HCPs over the decades in Western [25–27] and non-Western
countries [28,29]. However, questions remain as to whether these changes in the attitudes of HCPs
toward mental illness are moving in a positive direction and whether the changes are influenced by
regions. Under the circumstances, there is a need to determine the evolution of the attitudes towards
mental illness among HCPs.

2. Method

2.1. Search Methods

We conducted a literature search in the electronic databases PubMed, MEDLINE, PsychINFO,
PsycARTICLES, and ERIC, using the terms (“stigma” OR “knowledge” OR “stereotype” OR “attitude”
OR “prejudge” OR “behavior” OR “discrimination” OR “social distance”) AND (“mental illness”
OR “mental disease” OR “mental health” OR “mental health literacy” OR “psychiatry illness” OR
“psychiatry disorder” OR “schizophrenia” OR “depress *”) AND (“student” OR “professional” OR
“clinicians” OR “physicians” OR “health staff” OR “medical personnel” OR “healthcare provider”)
AND (“survey” OR “scale” OR “measurement”). We searched for peer-reviewed journal articles
regarding attitudes toward mental illness in a diverse group of healthcare professionals and students
that appeared until 28 February 2019. In addition, we also performed a manual search of references
cited by the published original studies, relevant reviews, and meta-analysis articles. Furthermore,
we contacted the experts in the field of attitude research and asked them about any relevant studies to
expand the initial search. Before data collection, ethical approval was obtained from the Institutional
Review Board of National Taiwan Normal University (ID: 201808HS12).
Int. J. Environ. Res. Public Health 2019, 16, 4655 3 of 14

2.2. Study Selection

This review followed PRISMA guidelines [30], and the protocol is registered with the PROSPERO
database of systematic reviews (PROSPERO: CRD42018112875) [31]. Figure. 1 describes the flow of
candidate and eligible articles. We retained all reports on studies that met the following criteria. First,
the focus of the study was on the healthcare professionals and students. Studies investigating the
beliefs or attitudes of professionals or students in healthcare fields (e.g., medical, nursing, social work,
psychology, pharmacy, occupational therapy, physical therapy) were included. Second, there are
many measures to assess the attitudes and beliefs among healthcare professionals and students,
such as the Social Distance Scale (SDS) [32], Reported and Intended Behaviour Scale (RIBS) [33],
Opinions about Mental Illness (OMI) [34], Community Attitudes towards Mental Illness (CAMI) [35],
Mental Illness Clinicians Attitude (MICA) [36], and Opening Minds Stigma Scales for Health Care
Providers (OMS-HC) [37]. Typically, the Social Distance Scale is regarded as a proxy measure of
mental-health-related stigma [32,38–41]. OMI, CAMI, MICA and OMS-HC were designed to assess
attitudes towards people with mental illness. Especially, MICA and OMS-HC were recently developed
for healthcare professionals. As we were interested in sustained time trends of the attitudes and beliefs
of healthcare professionals and students, MICA and OMS-HC were not included in this study because
the two measures have not been in use for long enough. Accordingly, we included studies in which the
outcome was measured using the Social Distance Scale (SDS), Opinions about Mental Illness (OMI),
and Community Attitudes towards Mental Illness (CAMI). Third, we included only survey studies for
the following reasons: (a) For our study period we found only two intervention studies used SDS,
OMI or CAMI as outcome measures. The publications are scarce in this period. (b) In survey studies
the samples are usually comprised of a wide range of study participants, but in intervention studies
small numbers of people are tested. (c) Additionally, survey studies have been conducted all over the
world, whereas intervention studies were conducted in only a few countries. Fourth, we only included
studies published in English. We also attempted to contact authors for missing data and the experts in
the field of psychiatry about any relevant studies to expand the initial search. The contents of abstracts
or full-text manuscripts identified through the literature search were reviewed independently by two
authors in duplicate to determine whether they met the eligibility criteria for inclusion. Disagreements
between two authors were resolved by consensus with discussion.

2.3. Relevant Measures

There are a number of measures assessing attitudes and social distance toward mental illness [42,43].
With regard to investigating attitudes toward mental illness, one popular measure is the Opinions
about Mental Illness (OMI) [34]. The 51-item OMI assesses five domains of attitudes toward people
with mental illness, including (1) authoritarianism; (2) social restrictiveness; (3) benevolence; (4) mental
hygiene ideology; and (5) interpersonal etiology. Participants are asked to rate each item on a 6-point
Likert scale ranging from strongly agree to strongly disagree. A modified version of the OMI scale
called the Community Attitudes towards Mental Illness (CAMI) scale [35] has three out of four factors
in common with the OMI scale: authoritarianism, benevolence, and social restrictiveness. CAMI can
detect the attitude of accepting psychiatric patients in a community. It requires the participants to
answer items on a 5-point Likert scale ranging from strongly agree to strongly disagree. A high score
on the subscales and the total scale indicates positive attitudes toward mental illness.
The Social Distance Scale (SDS) [32] is commonly used to measure social distance toward mental
illness, which is assessed as the level of desired future contact with people with mental health problems.
Many studies have established that the SDS has validity and reliability [32,44,45]. The original SDS
contains seven items, and each item asks the participant to use a 4-point Likert scale (1 = “Definitely
Unwilling” to 4 = “Definitely Willing”). As such, possible scores range from 7 to 28, with higher scores
indicating more social distance [32].
Int. J. Environ. Res. Public Health 2019, 16, 4655 4 of 14

2.4. Cross-Temporal Meta-Analysis

Cross-temporal meta-analysis [46–50] is similar to traditional meta-analysis in the procedures
for identifying and collecting data for studies. Instead of computing an effect size for each study
as in traditional meta-analysis, cross-temporal meta-analysis records means, standard deviations,
data collection year of study (as two years prior to publication, unless the year was otherwise
noted in the article), as used by previous studies [51,52], and other study characteristics are also
coded (e.g., sample number, region). In our cross-temporal meta-analysis we were interested in the
relationship between the mean scores on the outcome measures and the year that these data were
collected. In addition, we weighted the data in two ways. First, studies with larger sample sizes
providing better estimates of the population mean had a stronger influence on our findings [53].
Second, studies with smaller variances had a stronger influence on the final results. Cross-temporal
meta-analysis ultimately provides an index of the degree to which scores on a measure of outcome
have changed over time.
We analyzed how attitudes and social distance scores have changed over time, primarily by
examining correlations between mean scores and year of data collection. Subgroup analysis was used
to examine whether the observed effects was different by regions. Cross-temporal meta-analyses were
performed using Comprehensive Meta-Analysis Version 3 (Biostat, Englewood, NJ, USA) [54]. We fit
random-effects models, which took into account the between-study variations, to study the factors that
might affect social distance and attitude. Because of the different scales and measurement items in the
studies, we adjusted scores before conducting analyses [9]. This technique was used to compute the
correlation between attitudes and social distance mean scores and year.

3. Results

3.1. Data Identification and Extraction

Applying our study criteria, we identified 34 studies that used SDS, OMI, or CAMI and met the
inclusion criteria (see Figure 1): 18 studies using the SDS, 6 studies using OMI, and 10 studies using
CAMI. These studies were based on a combined sample size of 15,653 participants. Characteristics of
each study included in the review are documented in Tables 1 and 2.
Int. J. Environ. Res. Public Health 2019, 16, 4655 5 of 14
Int. J. Environ. Res. Public Health 2019, 16, x 5 of 13

Articles identified through database searching (in Additional articles identified

MEDLINE, ERIC, PsycARTICLES, PsycINFO, CINAHL) through reference search
(n=79,722) (n=0)

Articles excluded for reason:

not a survey study
not focused on healthcare professionals
not focused on general mental
illness/depression/schizophrenia
(n=79,502)
Studies assessed for inclusion criteria
(n=220)
Articles excluded for reason:
not use specific questionnaires
not English articles
Studies reviewed for further evaluation (n=191)
(n=29)

Articles included for reason:

contacted the authors for missing data
contacted the experts in the field of psychiatry
about any relevant study to expand the initial
search
(n=5)
Studies included in the meta analysis
(n=34)

Figure1.1.Flowchart
Figure Flowchartof
ofstudy
studyselection.
selection.

Table 1. Summary of characteristics of the 18 Social Distance Scale (SDS) studies.

Study Author, Year of Data Total Score,

Country Group N
Publication Date Collection μ (SD)

United
Crismon, 1990 [25] 1988 Pharmacists 165 15.87 (4.08)
States
Third-year pharmacy students 216 18.75 (5.04)
Bell et al., 2006 [55] Australia 2004
Pharmacy graduates 232 18.52 (5.00)
Volmer et al., 2008 [26] Estonia 2006 Pharmacy students 157 20.36 (3.88)
Australia,
Pharmacy students in India 106 18.75 (3.57)
Belgium,
Pharmacy students in Australia 241 19.65 (3.97)
India,
Bell et al., 2010 [56] 2006 Pharmacy students in Finland 130 18.05 (3.12)
Finland,
Pharmacy students in Estonia and Latvia 70 20.90 (4.04)
Estonia,
Pharmacy students in Belgium 102 19.61 (2.92)
Latvia
Hanzawa et al., 2012 [57] Japan 2009 Psychiatric nurses 215 19.76 (4.30)
Loch et al., 2013 [58] Brazil 2009 Psychiatrists 1414 14.00 (3.58)
United Mental health providers 205 14.87 (6.01)
Mittal et al., 2014 [59] 2011
States Primary care providers 146 16.23 (6.89)
General Practitioners 518 14.14 (5.18)
Reavley et al., 2014 [60] Australia 2012 Psychiatrists 506 14.14 (5.67)
Psychologists 498 12.25 (4.48)
Amarasuriya et al., 2015 [61] Sri Lanka 2013 Medical students 605 13.03 (4.02)
Dabby et al., 2015 [62] Canada 2012 Psychiatrists 68 10.47 (3.36)
Nursing professionals a 209 16.31 (5.06)
Mak et al., 2015 [28] Hong Kong 2011
Social work professionals a 150 13.23 (4.29)
Int. J. Environ. Res. Public Health 2019, 16, 4655 6 of 14

Table 1. Summary of characteristics of the 18 Social Distance Scale (SDS) studies.

Study Author, Year of Data Total Score,

Country Group N
Publication Date Collection µ (SD)
Crismon, 1990 [25] United States 1988 Pharmacists 165 15.87 (4.08)
Third-year pharmacy students 216 18.75 (5.04)
Bell et al., 2006 [55] Australia 2004
Pharmacy graduates 232 18.52 (5.00)
Volmer et al., 2008 [26] Estonia 2006 Pharmacy students 157 20.36 (3.88)
Pharmacy students in India 106 18.75 (3.57)
Australia, Belgium, Pharmacy students in Australia 241 19.65 (3.97)
Bell et al., 2010 [56] India, Finland, 2006 Pharmacy students in Finland 130 18.05 (3.12)
Estonia, Latvia Pharmacy students in Estonia and Latvia 70 20.90 (4.04)
Pharmacy students in Belgium 102 19.61 (2.92)
Hanzawa et al., 2012 [57] Japan 2009 Psychiatric nurses 215 19.76 (4.30)
Loch et al., 2013 [58] Brazil 2009 Psychiatrists 1414 14.00 (3.58)
Mental health providers 205 14.87 (6.01)
Mittal et al., 2014 [59] United States 2011
Primary care providers 146 16.23 (6.89)
General Practitioners 518 14.14 (5.18)
Reavley et al., 2014 [60] Australia 2012 Psychiatrists 506 14.14 (5.67)
Psychologists 498 12.25 (4.48)
Amarasuriya et al., 2015 [61] Sri Lanka 2013 Medical students 605 13.03 (4.02)
Dabby et al., 2015 [62] Canada 2012 Psychiatrists 68 10.47 (3.36)
Nursing professionals a 209 16.31 (5.06)
Social work professionals a 150 13.23 (4.29)
Medical professionals a 149 16.87 (5.13)
Nursing students a 203 12.81 (4.99)
Social work students a 207 13.86 (5.04)
Mak et al., 2015 [28] Hong Kong Medical students a 60 13.30 (4.88)
2011
Nursing professionals b 186 18.55 (4.77)
Social work professionals b 154 15.61 (4.34)
Medical professionals b 201 19.74 (4.96)
Nursing students b 203 16.17 (4.99)
Social work students b 185 17.99 (5.71)
Medical students b 52 16.73 (5.55)
O’Reilly et al., 2015 [63] Australia 2009 Pharmacists 186 17.81 (3.79)
Psychiatrists, nurses, clinical
Chiba et al., 2016 [29] Japan 2012 psychologists, pharmacists, occupational 307 15.22 (4.75)
therapists, social workers
Primary care nurses 91 15.83 (4.67)
Primary care physicians 55 16.88 (4.05)
Smith et al., 2017 [64] United States 2012 Mental health nurses 67 15.01 (4.81)
Psychiatrists 62 15.92 (5.07)
Psychologists 76 13.89 (3.91)
Students (social work) 296 18.14 (3.76)
Students (psychology) 419 17.18 (3.64)
Pranckeviciene et al., 2018 [65] Lithuanian 2015
Social workers 111 17.43 (4.00)
Psychologists 122 16.61 (3.37)
Tay et al., 2018 [27] United Kingdom 2015 Psychologists 678 12.18 (3.71)
Students (social work) 104 11.90 (3.77)
Students (counseling) 87 11.04 (3.20)
Students (psychology) 111 11.90 (3.94)
Tillman et al., 2018 [66] United States 2016
Social workers 23 10.01 (3.59)
Counselors 34 11.02 (3.24)
Psychologists 38 12.13 (3.16)
Perlman et al., 2019 [67] Australia 2016 Nurses 168 15.82 (3.76)
a b
The outcome measure is social distance of depression; The outcome measure is social distance of schizophrenia.
Int. J. Environ. Res. Public Health 2019, 16, 4655 7 of 14

Table 2. Summary of characteristics of the 16 OMI/CAMI Studies.

Study Author, Year of Data Total Score,

Scale Country Group N
Publication Date Collection µ (SD)
LeMay et al., Counselor candidates (male) 31 134.50 (14.66)
OMI United States 1966
1968 [68] Counselor candidates (female) 50 134.29 (13.13)
Physicians (British) 181 127.23 (19.52)
Great Britain, Physicians (Czechoslovakian) 103 110.75 (18.52)
Levine et al.,
OMI Czechoslovakia, 1968 Nurses (West German) 80 120.69 (22.52)
1972 [69]
Germany Nurses (British) 188 128.30 (20.18)
Nurses (Czechoslovakian) 116 105.35 (19.31)
Kirkby et al.,
OMI Australia 1977 Medical practitioners 37 129.18 (20.25)
1979 [70]
Murray et al., Supportive case managers 24 147.48 (16.56)
OMI United States 1997
1999 [71] Intense case managers 23 135.87 (17.30)
Smith et al., Health professionals and medical
CAMI United States 2006 168 113.87 (20.83)
2008 [72] students
Arvaniti et al., Health professionals and medical
OMI Greece 2006 580 147.38 (25.85)
2009 [73] students
Smith et al., Mental health students 58 143.10 (15.59)
CAMI United States 2008
2010 [74] Mental health professionals 58 141.40 (17.19)
Finland,
Chambers et al.,
CAMI Lithuania, Ireland, 2007 Nurses 810 134.00 (20.74)
2010 [75]
Italy, Portugal
Guise et al.,
CAMI United Kingdom 2009 Nurses 81 135.50 (17.07)
2010 [76]
O’ Connor et al., Medical students (third year) 140 159.20 (14.60)
CAMI Ireland 2010
2013 [77] Medical students (final year) 145 158.50 (16.50)
Kopera et al., Psychiatrists, psychotherapists
OMI Poland 2011 57 147.80 (13.96)
2015 [78] Medical students
Winkler et al.,
CAMI Czech Republic 2014 Medical doctors 1200 142.22 (16.30)
2016 [79]
Janouskova et al.,
CAMI Czech Republic 2016 Medical students 457 163.56 (18.68)
2017 [80]
Mosaku et al.,
CAMI Nigeria 2013 Health workers 112 115.60 (19.96)
2017 [81]
Siqueira et al.,
CAMI Brazil 2014 Health professionals 246 113.20 (14.80)
2017 [82]
Cremonini et al.,
CAMI Italy 2016 Health care professionals 120 160.77 (15.60)
2018 [83]
OMI: Opinions about Mental Illness scale; CAMI: Community Attitudes towards Mental Illness scale.

3.2. Correlation between Mean Scores of Social Distances, Attitude, and Years
The cross-temporal meta-analysis showed that the mean scores of social distance and attitudes
were positively associated with the year of data collection (β = −0.32, p < 0.001; β = 0.43, p = 0.007),
indicating that the desire of social distance and the attitudes toward people with mental illness in
healthcare professionals and students become positive over the period 1966–2016 (Figures 2 and 3).
To further examine the magnitude of change in SDS and attitude scores, we calculated the size of
increase in scores over time using the regression equation weighted by w. The regression equation
used the algebraic formula Yx1 = C1 + Bx1 , where Yx1 is the average SDS score for a particular year of
interest, x1 is the year of interest, B is the beta coefficient of −0.32 (p < 0.001), and C1 is the equation
constant of 650.59. We used the r2 estimator to conclude that only 17% of the variance in effects was
explained by the model. The regression equation of attitude was Yx2 = C2 + Bx2 , where Yx2 is the
average attitude score for a particular year of interest, the beta coefficient was 0.43 (p < 0.05), and C2 is
the equation constant (−714.94). r2 was 0.16, which means 16% of the variance in effects was explained
by this model.
Int. J. Environ. Res. Public Health 2019, 16, 4655 8 of 14
Int. J. Environ. Res. Public Health 2019, 16, x 8 of 13
Int. J. Environ. Res. Public Health 2019, 16, x 8 of 13

Metaregression
Figure2.2.Meta
Figure regressionofof mean
mean ofof Social
Social Distance
Distance Scale
Scale score
score andand
yearyear of data
of data collection
collection fromfrom
1988 1988
to 2016.
to 2016.
Figure 2. Meta regression of mean of Social Distance Scale score and year of data collection from 1988 to 2016.

Figure 3. Meta regression of mean of CAMI/OMI score and year of data collection from 1966 to 2016.
Figure
Figure
CAMI: 3. Meta
3. Meta regression
regression
Community of of
Attitudes mean
mean ofofCAMI/OMI
towards CAMI/OMI score
score
Mental Illness and
and
scale; year
year
OMI: collection
of data about
Opinions collection from
Mentalfrom 1966
1966
Illness toto2016.
scale. 2016.
CAMI:
CAMI: Community
Community Attitudestowards
Attitudes towardsMental
Mental Illness
Illnessscale;
scale;OMI:
OMI:Opinions about
Opinions Mental
about Illness
Mental scale.scale.
Illness
4. Discussion
4.Considering
Discussion regional difference, additional subgroup analysis was conducted only for social
Summarizing our findings, this study indicates that over a half century, HCPs’ attitudes
distance and not for OMI and CAMI because there was only one study using CAMI as an outcome
Summarizing
toward mental illnessourhave
findings, thisconsiderably.
increased study indicates that over
Moreover, therea has
halfbeen
century, HCPs’ reduction
a significant attitudes
measure
towardin the non-Western country (i.e., Nigeria). Moreover,
The subgroup has analysis aindicated that greater
of socialmental illness
distance have
from increased
people withconsiderably.
mental illness among there
HCPs over beenthe significant
past threereduction
decades.
meanof scores
social of social distance
distance from showed werewith
people also associated
mental withamong
the later yearsover
of data collection in both the
Meanwhile, our study that in both illness
Western HCPs
and non-Western the past three
countries, the decades.
attitudes
Western countries
Meanwhile,
among HCPsour = 14; βshowed
(n study
toward = −0.27, p =illness
that
mental 0.001)
in bothand non-Western
Western
have countries
and non-Western
improved in recent = 5;decades.
β = −0.73,
(n countries, p = 0.048).
the Reducing
attitudes
Thisamong
reveals that
HCPsthe desire
toward of social distance
mental fromhave
illness peopleimproved
with mentalinillness in healthcare
recent decades. professionals
Reducing
Int. J. Environ. Res. Public Health 2019, 16, 4655 9 of 14

and students became more favorable with the passage of time, regardless of region. Compared with
the findings from the original cross-temporal meta-regression model, similar findings were found after
including region as a covariate. Our findings suggest that the correlation between the year of data
collection and social distance was also independent of region.

4. Discussion
Summarizing our findings, this study indicates that over a half century, HCPs’ attitudes toward
mental illness have increased considerably. Moreover, there has been a significant reduction of social
distance from people with mental illness among HCPs over the past three decades. Meanwhile,
our study showed that in both Western and non-Western countries, the attitudes among HCPs toward
mental illness have improved in recent decades. Reducing mental-illness-related stigma among HCPs
has become a global campaign [13]. The deleterious impacts of stigma in healthcare have promoted
increased calls to action for health organizations to take leadership roles in tackling the problem [17,84],
such as the OM initiative and Time to Change programs in Western countries. Of note, there were also
some effective interventions which aimed to reduce the mental-illness-related stigma among HCPs in
non-Western countries such as Hong Kong [28], Japan [29,85], South Africa [86], and Turkey [87–89].
In addition, compared with traditional education, the involvement of consumers in the education
(i.e., contacting people with mental illness) of HCPs has been identified as a potentially effective strategy
in influencing more positive attitudes toward consumer involvement in mental health services [14,90].
Furthermore, an educational strategy called problem-based learning (PBL) is a common newer teaching
technique used in medical education in recent years. Research has suggested that the PBL method has
played an effective role in the development of positive attitudes toward psychiatric nursing and patients
as well as in the acquisition of the basic skills of psychiatric nursing [91]. Under the circumstances,
participating in anti-stigma programs and modernizing medical education might help HCPs generate
positive attitudes toward mental illness.
Although we found these positive improvements, the results of our study have some limitations.
First, mental illness is a general term for a group of illnesses that may influence a person’s thoughts,
perceptions, feelings, and behaviors. We only included studies evaluating stigma toward mental
illness, schizophrenia, and depression in our criteria since these are most common diagnoses used
in the mental health campaigns to reduce mental-illness-related stigma among HCPs. A range of
diagnosis-based specific mental disorder conditions (e.g., bipolar disorder and alcohol use disorder)
could be a target for future research. Second, potentially relevant studies were not included in this
study due to lack of access to an English version. However, most studies on evaluating HCPs’ stigma
toward mental illness took place in Western countries. The findings might not be easily translated
to Eastern cultures. Third, as in any meta-analysis, interpretations of the results of this study are
limited to the data reported by authors. Specifically, many authors do not report the specific year
of data collection, the ethnicity of their participants, or the means and standard deviations for all
variables. However, the goal of this cross-temporal meta-analysis was to examine the relationship
between time and attitude. This study also could not determine whether the change in attitude was
a purely generational effect or a time-period effect. As with any time-lag study including people of
only one age group, we cannot know if those in other age groups also changed. Finally, the use of an
attitude scale to assess outcome might be influenced by socially desirable responding.

5. Conclusions
This study provides further evidence in support of the importance of global and national programs
and new medical education methods in eliminating stigma toward mental illness among HCPs.
The findings also suggest that these efforts improve positive attitudes toward mental illness and
reduce the social distance from mentally ill people. As actions to fight the stigma toward mental
illness have continued, new trend analyses tracking present and future attitude changes are necessary.
Future research might focus on monitoring and evaluating the trends nationally as well as globally
Int. J. Environ. Res. Public Health 2019, 16, 4655 10 of 14

and determining if there are differences in cultural needs, reception, and reactions to different
campaign messages.

Author Contributions: Conceptualization, H.-S.L. and Y.-J.L.; Investigation, Y.-Y.L., H.-S.L., C.-H.T., and T.-T.W.;
Formal analysis, Y.-Y.L. and C.-H.T.; Writing the original draft, Y.-Y.L., H.-S.L., and C.-H.T.; Funding acquisition,
Y.-J.L.; Project administration, Y.-J.L.; Supervision and writing—review & editing, Y.-J.L.
Funding: This study was supported by Ministry of Science and Technology of Taiwan (MOST 107-2410-H-003-022-).
The Ministry of Science and Technology of Taiwan was not involved in the study design, data collection, analysis,
interpretation, or writing of the manuscript.
Conflicts of Interest: The authors declare no conflicts of interest.

References
1. Sartorius, N. Fighting schizophrenia and its stigma: A new World Psychiatric Association educational
programme. Br. J. Psychiatry 1997, 170, 297. [CrossRef] [PubMed]
2. Jorm, A.F.; Christensen, H.; Griffiths, K.M. The impact of beyondblue: The national depression initiative on
the Australian public’s recognition of depression and beliefs about treatments. Aust. N. Z. J. Psychiatry 2005,
39, 248–254. [CrossRef] [PubMed]
3. Time to Change. Available online: http://www.time-to-change.org.uk (accessed on 3 March 2019).
4. Makowski, A.C.; Mnich, E.E.; Ludwig, J.; Daubmann, A.; Bock, T.; Lambert, M.; Härter, M.;
Dirmaier, J.; Tlach, L.; Liebherz, S. Changes in beliefs and attitudes toward people with depression
and schizophrenia–results of a public campaign in Germany. Psychiatry Res. 2016, 237, 271–278. [CrossRef]
[PubMed]
5. Henderson, C.; Robinson, E.; Evans-Lacko, S.; Corker, E.; Rebollo-Mesa, I.; Rose, D.; Thornicroft, G. Public
knowledge, attitudes, social distance and reported contact regarding people with mental illness 2009–2015.
Acta Psychiatr. Scand. 2016, 134, 23–33. [CrossRef] [PubMed]
6. Hansson, L.; Stjernswärd, S.; Svensson, B. Changes in attitudes, intended behaviour, and mental health literacy
in the Swedish population 2009–2014: An evaluation of a national antistigma programme. Acta Psychiatr. Scand.
2016, 134, 71–79. [CrossRef] [PubMed]
7. Robinson, E.J.; Henderson, C. Public knowledge, attitudes, social distance and reporting contact with people
with mental illness 2009–2017. Psychol. Med. 2018. [CrossRef]
8. Angermeyer, M.C.; Matschinger, H.; Carta, M.G.; Schomerus, G. Changes in the perception of mental illness
stigma in Germany over the last two decades. Eur. Psychiatry 2014, 29, 390–395. [CrossRef]
9. Angermeyer, M.C.; Matschinger, H.; Schomerus, G. Attitudes towards psychiatric treatment and people with
mental illness: Changes over two decades. Br. J. Psychiatry 2013, 203, 146–151. [CrossRef]
10. Schomerus, G.; Schwahn, C.; Holzinger, A.; Corrigan, P.W.; Grabe, H.J.; Carta, M.G.; Angermeyer, M.C.
Evolution of public attitudes about mental illness: A systematic review and meta-analysis.
Acta Psychiatr. Scand. 2012, 125, 440–452. [CrossRef]
11. Mirnezami, H.F.; Jacobsson, L.; Edin-Liljegren, A. Changes in attitudes towards mental disorders and
psychiatric treatment 1976–2014 in a Swedish population. Nord. J. Psychiatry 2016, 70, 38–44. [CrossRef]
12. Henderson, C.; Noblett, J.; Parke, H.; Clement, S.; Caffrey, A.; Gale-Grant, O.; Schulze, B.; Druss, B.;
Thornicroft, G. Mental health-related stigma in health care and mental health-care settings. Lancet Psychiatry
2014, 1, 467–482. [CrossRef]
13. Thornicroft, G.; Mehta, N.; Clement, S.; Evans-Lacko, S.; Doherty, M.; Rose, D.; Koschorke, M.; Shidhaye, R.;
O’Reilly, C.; Henderson, C. Evidence for effective interventions to reduce mental-health-related stigma and
discrimination. Lancet 2016, 387, 1123–1132. [CrossRef]
14. O’Reilly, C.L.; Bell, J.S.; Chen, T.F. Consumer-led mental health education for pharmacy students. Am. J.
Pharm. Educ. 2010, 74, 167. [CrossRef] [PubMed]
15. Rong, Y.; Glozier, N.; Luscombe, G.M.; Davenport, T.A.; Huang, Y.; Hickie, I.B. Improving knowledge and
attitudes towards depression: A controlled trial among Chinese medical students. BMC Psychiatry 2011, 11,
36. [CrossRef]
16. Knaak, S.; Patten, S.; Ungar, T. Mental illness stigma as a quality-of-care problem. Lancet Psychiatry 2015, 2,
863–864. [CrossRef]
Int. J. Environ. Res. Public Health 2019, 16, 4655 11 of 14

17. Knaak, S.; Mantler, E.; Szeto, A. Mental illness-related stigma in healthcare: Barriers to access and care and
evidence-based solutions. Healthc. Manag. Forum 2017, 30, 111–116. [CrossRef]
18. Lyons, Z. Attitudes of medical students toward psychiatry and psychiatry as a career: A systematic review.
Acad. Psychiatry 2013, 37, 150–157. [CrossRef]
19. Barkil-Oteo, A. Psychiatry’s identity crisis. Lancet 2012, 379, 2428. [CrossRef]
20. Katschnig, H. Are psychiatrists an endangered species? Observations on internal and external challenges to
the profession. World Psychiatry 2010, 9, 21–28. [CrossRef]
21. Bishop, T.F.; Seirup, J.K.; Pincus, H.A.; Ross, J.S. Population of US practicing psychiatrists declined, 2003–2013,
which may help explain poor access to mental health care. Health Aff. 2016, 35, 1271–1277. [CrossRef]
22. Abbey, S.C.M.; Tranulis, C.; Moss, P.; Baici, W.; Dabby, L.; Gautam, M.; Paré, M. Stigma and discrimination.
Can. J. Psychiatry 2011, 56, 1–9. [PubMed]
23. Friedrich, B.; Evans-Lacko, S.; London, J.; Rhydderch, D.; Henderson, C.; Thornicroft, G. Anti-stigma
training for medical students: The Education Not Discrimination project. Br. J. Psychiatry 2013, 202, s89–s94.
[CrossRef] [PubMed]
24. Stuart, H.; Chen, S.P.; Christie, R.; Dobson, K.; Kirsh, B.; Knaak, S.; Koller, M.; Krupa, T.; Lauria-Horner, B.;
Luong, D. Opening minds in Canada: Background and rationale. Can. J. Psychiatry 2014, 59, 8–12. [CrossRef]
[PubMed]
25. Crismon, M.L.; Jermain, D.M.; Torian, S.J. Attitudes of pharmacy students toward mental illness. Am. J.
Hosp. Pharm. 1990, 47, 1369–1373. [CrossRef]
26. Volmer, D.; Mäesalu, M.; Bell, J.S. Pharmacy students’ attitudes toward and professional interactions with
people with mental disorders. Int. J. Soc. Psychiatry 2008, 54, 402–413. [CrossRef]
27. Tay, S.; Alcock, K.; Scior, K. Mental health problems among clinical psychologists: Stigma and its impact on
disclosure and help-seeking. J. Clin. Psychol. 2018, 74, 1545–1555. [CrossRef]
28. Mak, W.W.S.; Cheung, F.M.C.; Wong, S.Y.S.; Tang, W.K.; Lau, J.T.F.; Woo, J.; Lee, D.T.F. Stigma towards people
with psychiatric disorders. Hong Kong Med. J. 2015, 21 (Suppl. 2), 9–12.
29. Chiba, R.; Umeda, M.; Goto, K.; Miyamoto, Y.; Yamaguchi, S.; Kawakami, N. Psychometric properties
of the Japanese version of the Recovery Attitudes Questionnaire (RAQ) among mental health providers:
A questionnaire survey. BMC Psychiatry 2016, 16, 1–9. [CrossRef]
30. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G. Preferred reporting items for systematic reviews and
meta-analyses: The PRISMA statement. Ann. Intern. Med. 2009, 151, 264–269. [CrossRef]
31. PROSPERO. International Prospective Register of Systematic Reviews. Available online: http://www.crd.
york.ac.uk/prospero (accessed on September 20 2019).
32. Link, B.G.; Cullen, F.T.; Frank, J.; Wozniak, J.F. The social rejection of former mental patients: Understanding
why labels matter. Am. J. Sociol. 1987, 92, 1461–1500. [CrossRef]
33. Evans-Lacko, S.; Rose, D.; Little, K.; Flach, C.; Rhydderch, D.; Henderson, C.; Thornicroft, G. Development
and psychometric properties of the reported and intended behaviour scale (RIBS): A stigma-related behaviour
measure. Epidemiol. Psychiatr. Sci. 2011, 20, 263–271. [CrossRef] [PubMed]
34. Cohen, J.; Struening, E.L. Opinions about mental illness in the personnel of two large mental hospitals.
J. Abnorm. Soc. Psychol. 1962, 64, 349. [CrossRef] [PubMed]
35. Taylor, S.M.; Dear, M.J. Scaling community attitudes toward the mentally ill. Schizophr. Bull. 1981, 7, 225–240.
[CrossRef] [PubMed]
36. Gabbidon, J.; Clement, S.; van Nieuwenhuizen, A.; Kassam, A.; Brohan, E.; Norman, I.; Thornicroft, G.
Mental Illness: Clinicians’ Attitudes (MICA) Scale—Psychometric properties of a version for healthcare
students and professionals. Psychiatry Res. 2013, 206, 81–87. [CrossRef]
37. Modgill, G.; Patten, S.B.; Knaak, S.; Kassam, A.; Szeto, A.C.H. Opening minds stigma scale for health care
providers (OMS-HC): Examination of psychometric properties and responsiveness. BMC Psychiatry 2014,
14, 120. [CrossRef]
38. Angermeyer, M.C.; Daubmann, A.; Wegscheider, K.; Mnich, E.; Schomerus, G.; vd Knesebeck, O. The
relationship between biogenetic attributions and desire for social distance from persons with schizophrenia
and major depression revisited. Epidemiol. Psychiatr. Sci. 2015, 24, 335–341. [CrossRef]
39. Kermode, M.; Bowen, K.; Arole, S.; Pathare, S.; Jorm, A.F. Attitudes to people with mental disorders: A
mental health literacy survey in a rural area of Maharashtra, India. Soc. Psychiatry Psychiatr. Epidemiol. 2009,
44, 1087–1096. [CrossRef]
Int. J. Environ. Res. Public Health 2019, 16, 4655 12 of 14

40. Reavley, N.J.; Jorm, A.F. Recognition of mental disorders and beliefs about treatment and outcome: Findings
from an Australian national survey of mental health literacy and stigma. Aust. N. Z. J. Psychiatry 2011, 45,
947–956. [CrossRef]
41. Schomerus, G.; Matschinger, H.; Angermeyer, M.C. Causal beliefs of the public and social acceptance of
persons with mental illness: A comparative analysis of schizophrenia, depression and alcohol dependence.
Psychol. Med. 2014, 44, 303–314. [CrossRef]
42. Brohan, E.; Slade, M.; Clement, S.; Thornicroft, G. Experiences of mental illness stigma, prejudice and
discrimination: A review of measures. BMC Health Serv. Res. 2010, 10, 80. [CrossRef]
43. Charles, J.L.K.; Bentley, K.J. Measuring mental health provider-based stigma: Development and initial
psychometric testing of a self-assessment instrument. Community Ment. Health J. 2018, 54, 33–48. [CrossRef]
[PubMed]
44. Corrigan, P.W.; Watson, A.C. Understanding the impact of stigma on people with mental illness.
World Psychiatry 2002, 1, 16. [PubMed]
45. Moxham, L.; Taylor, E.; Patterson, C.; Perlman, D.; Brighton, R.; Sumskis, S.; Keough, E.; Heffernan, T. Can a
clinical placement influence stigma? An analysis of measures of social distance. Nurse Educ. Today 2016, 44,
170–174. [CrossRef] [PubMed]
46. Twenge, J.M. Changes in masculine and feminine traits over time: A meta-analysis. Sex Roles 1997, 36,
305–325. [CrossRef]
47. Twenge, J.M. Attitudes toward women, 1970–1995: A meta-analysis. Psychol. Women Q. 1997, 21, 35–51.
[CrossRef]
48. Twenge, J.M. The age of anxiety? The birth cohort change in anxiety and neuroticism, 1952–1993. J. Personal.
Soc. Psychol. 2000, 79, 1007. [CrossRef]
49. Twenge, J.M. Birth cohort changes in extraversion: A cross-temporal meta-analysis, 1966–1993.
Personal. Individ. Differ. 2001, 30, 735–748. [CrossRef]
50. Twenge, J.M. Changes in women’s assertiveness in response to status and roles: A cross-temporal
meta-analysis, 1931–1993. J. Personal. Soc. Psychol. 2001, 81, 133. [CrossRef]
51. Oliver, M.B.; Hyde, J.S. Gender differences in sexuality: A meta-analysis. Psychol. Bull. 1993, 114, 29.
[CrossRef]
52. Wells, B.E.; Twenge, J.M. Changes in young people’s sexual behavior and attitudes, 1943–1999: A
cross-temporal meta-analysis. Rev. Gen. Psychol. 2005, 9, 249–261. [CrossRef]
53. Hedges, L.V.; Becker, B.J. Statistical methods in the meta-analysis of research on gender differences. In The
Psychology of Gender: Progress through Meta-Analysis; The Johns Hopkins University Press: Baltimore, MD,
USA, 1986.
54. Borenstein, M.; Hedges, L.; Higgins, J.; Rothstein, H. Comprehensive Meta-Analysis Version 3.3. 070; Biostat:
Englewood, NJ, USA, 2014; Volume 104.
55. Bell, J.S.; Johns, R.; Chen, T.F. Pharmacy students’ and graduates’ attitudes towards people with schizophrenia
and severe depression. Am. J. Pharm. Educ. 2006, 70, 77. [CrossRef] [PubMed]
56. Bell, J.S.; Aaltonen, S.E.; Airaksinen, M.S.; Volmer, D.; Gharat, M.S.; Muceniece, R.; Vitola, A.; Foulon, V.;
Desplenter, F.A.; Chen, T.F. Determinants of mental health stigma among pharmacy students in Australia,
Belgium, Estonia, Finland, India and Latvia. Int. J. Soc. Psychiatry 2010, 56, 3–14. [CrossRef] [PubMed]
57. Hanzawa, S.; Nosaki, A.; Yatabe, K.; Nagai, Y.; Tanaka, G.; Nakane, H.; Nakane, Y. Study of understanding
the internalized stigma of schizophrenia in psychiatric nurses in Japan. Psychiatry Clin. Neurosci. 2012, 66,
113–120. [CrossRef] [PubMed]
58. Loch, A.A.; Hengartner, M.P.; Guarniero, F.B.; Lawson, F.L.; Wang, Y.P.; Gattaz, W.F.; Rössler, W. The more
information, the more negative stigma towards schizophrenia: Brazilian general population and psychiatrists
compared. Psychiatry Res. 2013, 205, 185–191. [CrossRef]
59. Mittal, D.; Corrigan, P.; Sherman, M.D.; Chekuri, L.; Han, X.; Reaves, C.; Mukherjee, S.; Morris, S.; Sullivan, G.
Healthcare providers’ attitudes toward persons with schizophrenia. Psychiatr. Rehabil. J. 2014, 37, 297.
[CrossRef]
60. Reavley, N.J.; Mackinnon, A.J.; Morgan, A.J.; Jorm, A.F. Stigmatising attitudes towards people with mental
disorders: A comparison of Australian health professionals with the general community. Aust. N. Z.
J. Psychiatry 2014, 48, 433–441. [CrossRef]
Int. J. Environ. Res. Public Health 2019, 16, 4655 13 of 14

61. Amarasuriya, S.D.; Jorm, A.F.; Reavley, N.J.; Mackinnon, A.J. Stigmatising attitudes of undergraduates
towards their peers with depression: A cross-sectional study in Sri Lanka. BMC Psychiatry 2015, 15, 129.
[CrossRef]
62. Dabby, L.; Tranulis, C.; Kirmayer, L.J. Explicit and implicit attitudes of Canadian psychiatrists toward people
with mental illness. Can. J. Psychiatry 2015, 60, 451–459. [CrossRef]
63. O’reilly, C.L.; Bell, J.S.; Kelly, P.J.; Chen, T.F. Exploring the relationship between mental health stigma,
knowledge and provision of pharmacy services for consumers with schizophrenia. Res. Soc. Adm. Pharm.
2015, 11, e101–e109. [CrossRef]
64. Smith, J.D.; Mittal, D.; Chekuri, L.; Han, X.; Sullivan, G. A comparison of provider attitudes toward serious
mental illness across different health care disciplines. Stigma Health 2017, 2, 327. [CrossRef]
65. Pranckeviciene, A.; Zardeckaite-Matulaitiene, K.; Marksaityte, R.; Endriulaitiene, A.; Tillman, D.R.; Hof, D.D.
Social distance in Lithuanian psychology and social work students and professionals. Soc. Psychiatry
Psychiatr. Epidemiol. 2018, 53, 849–857. [CrossRef]
66. Tillman, D.R.; Hof, D.D.; Pranckeviciene, A.; Endriulaitienė, A.; Markšaitytė, R.; Žardeckaitė-Matulaitienė, K.
Social distance from mental illness among counseling, social work, and psychology students and helping
professionals. J. Soc. Action Couns. Psychol. 2018, 10, 24–37.
67. Perlman, D.; Moxham, L.; Patterson, C.; Cregan, A. Stigmatization behavior of pre-registration nurses: Do the
self-determined psychological needs influence this? Issues Ment. Health Nurs. 2019, 40, 342–346. [CrossRef]
68. LeMay, M.L. Counselor candidates’ attitudes and opinions about mental illness. Couns. Educ. Superv. 1968,
8, 51–54. [CrossRef]
69. Levine, D. A cross-national study of attitudes toward mental illness. J. Abnorm. Psychol. 1972, 80, 111.
[CrossRef]
70. Kirkby, R.J.; James, A. Attitudes of medical practitioners to mental illness. Aust. N. Z. J. Psychiatry 1979, 13,
165–168. [CrossRef]
71. Murray, M.G.; Steffen, J.J. Attitudes of case managers toward people with serious mental illness.
Community Ment. Health J. 1999, 35, 505–514. [CrossRef]
72. Smith, A.L. An Investigation of Attitudes towards Adults with Mental Illness among Mental Health Professionals
In-Training, Non Mental Health Professionals In-Training, Mental Health Professionals, and Non Mental Health
Professionals; The University of North Carolina at Greensboro: Greensboro, NC, USA, 2008.
73. Arvaniti, A.; Samakouri, M.; Kalamara, E.; Bochtsou, V.; Bikos, C.; Livaditis, M. Health service staff’s attitudes
towards patients with mental illness. Soc. Psychiatry Psychiatr. Epidemiol. 2009, 44, 658–665. [CrossRef]
74. Smith, A.L.; Cashwell, C.S. Stigma and mental illness: Investigating attitudes of mental health and
non-mental-health professionals and trainees. J. Humanist. Couns. Educ. Dev. 2010, 49, 189–202. [CrossRef]
75. Chambers, M.; Guise, V.; Välimäki, M.; Botelho, M.A.R.; Scott, A.; Staniuliene, V.; Zanotti, R. Nurses’ attitudes
to mental illness: A comparison of a sample of nurses from five European countries. Int. J. Nurs. Stud. 2010,
47, 350–362. [CrossRef]
76. Guise, V.; Chambers, M.; Välimäki, M.; Makkonen, P. A mixed-mode approach to data collection: Combining
web and paper questionnaires to examine nurses’ attitudes to mental illness. J. Adv. Nurs. 2010, 66, 1623–1632.
[CrossRef] [PubMed]
77. O’Connor, K.; Brennan, D.; O’loughlin, K.; Wilson, L.; Pillay, D.; Clarke, M.; Casey, P.; Malone, K.; Lane, A.
Attitudes towards patients with mental illness in Irish medical students. Ir. J. Med Sci. 2013, 182, 679–685.
[CrossRef] [PubMed]
78. Kopera, M.; Suszek, H.; Bonar, E.; Myszka, M.; Gmaj, B.; Ilgen, M.; Wojnar, M. Evaluating explicit and implicit
stigma of mental illness in mental health professionals and medical students. Community Ment. Health J.
2015, 51, 628–634. [CrossRef] [PubMed]
79. Winkler, P.; Mladá, K.; Janoušková, M.; Weissová, A.; Tušková, E.; Csémy, L.; Evans-Lacko, S. Attitudes
towards the people with mental illness: Comparison between Czech medical doctors and general population.
Soc. Psychiatry Psychiatr. Epidemiol. 2016, 51, 1265–1273. [CrossRef]
80. Janoušková, M.; Weissová, A.; Formánek, T.; Pasz, J.; Bankovská Motlová, L. Mental illness stigma among
medical students and teachers. Int. J. Soc. Psychiatry 2017, 63, 744–751. [CrossRef]
81. Mosaku, K.S.; Wallymahmed, A.H. Attitudes of primary care health workers towards mental health patients:
A cross-sectional study in Osun state, Nigeria. Community Ment. Health J. 2017, 53, 176–182. [CrossRef]
Int. J. Environ. Res. Public Health 2019, 16, 4655 14 of 14

82. Siqueira, S.R.G.; Abelha, L.; Lovisi, G.M.; Sarução, K.R.; Yang, L. Attitudes towards the mentally ill: A study
with health workers at a university hospital in Rio de Janeiro. Psychiatr. Q. 2017, 88, 25–38. [CrossRef]
83. Cremonini, V.; Pagnucci, N.; Giacometti, F.; Rubbi, I. Health care professionals attitudes towards mental
illness: Observational study performed at a public health facility in northern Italy. Arch. Psychiatr. Nurs.
2018, 32, 24–30. [CrossRef]
84. WHO. Mental Health Action Plan 2013–2020; WHO: Geneva, Switzerland, 2014.
85. Mino, Y.; Yasuda, N.; Tsuda, T.; Shimodera, S. Effects of a one-hour educational program on medical students’
attitudes to mental illness. Psychiatry Clin. Neurosci. 2001, 55, 501–507. [CrossRef]
86. Kakuma, R.; Kleintjes, S.; Lund, C.; Drew, N.; Green, A.; Flisher, A.J. Mental Health Stigma: What is being
done to raise awareness and reduce stigma in South Africa? Afr. J. Psychiatry 2010, 13, 116–124. [CrossRef]
87. Bayar, M.R.; Poyraz, B.Ç.; Aksoy-Poyraz, C.; Arikan, M.K. Reducing mental illness stigma in mental health
professionals using a web-based approach. Isr. J. Psychiatry Relat. Sci. 2009, 46, 226–230. [PubMed]
88. ÜÇOk, A.; Soyguer, H.; Atakli, C.; Kuşcu, K.; Sartorius, N.; Duman, Z.C.; Polat, A.; Erkoç, Ş. The impact
of antistigma education on the attitudes of general practitioners regarding schizophrenia. Psychiatry Clin.
Neurosci. 2006, 60, 439–443. [CrossRef] [PubMed]
89. Altindag, A.; Yanik, M.; Ucok, A.; Alptekin, K.; Ozkan, M. Effects of an antistigma program on medical
students’ attitudes towards people with schizophrenia. Psychiatry Clin. Neurosci. 2006, 60, 283–288.
[CrossRef] [PubMed]
90. Happell, B.; Byrne, L.; McAllister, M.; Lampshire, D.; Roper, C.; Gaskin, C.J.; Martin, G.; Wynaden, D.;
McKenna, B.; Lakeman, R. Consumer involvement in the tertiary-level education of mental health
professionals: A systematic review. Int. J. Ment. Health Nurs. 2014, 23, 3–16. [CrossRef] [PubMed]
91. Cooper, C.; Carver, N. Problem based learning in mental health nursing: The students’ experience. Int. J.
Ment. Health Nurs. 2012, 21, 175–183. [CrossRef] [PubMed]

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).