JRC Statistical Audit of the

Sustainable Development Goals

Index and Dashboards

Papadimitriou, Eleni
Neves, Ana Rita
Becker, William

2019

EUR 29776 EN
This publication is a Technical report by the Joint Research Centre (JRC), the European Commission’s science and
knowledge service. It aims to provide evidence-based scientific support to the European policymaking process.
The scientific output expressed does not imply a policy position of the European Commission. Neither the
European Commission nor any person acting on behalf of the Commission is responsible for the use that might
be made of this publication.

Report completed in May 2019.

Contact information
European Commission
Joint Research Centre
Directorate for Competences
Monitoring, Indicators and Impact Evaluation Unit
Competence Centre on Composite Indicators and Scoreboards
E-mail: jrc-coin@ec.europa.eu
https://ec.europa.eu/jrc/en/coin
https://composite-indicators.jrc.ec.europa.eu/

EU Science Hub
https://ec.europa.eu/jrc

JRC116857

EUR 29776 EN

PDF ISBN 978-92-76-08995-7 ISSN 1831-9424 doi:10.2760/723763

Luxembourg: Publications Office of the European Union, 2019

© European Union, 2019

The reuse policy of the European Commission is implemented by Commission Decision 2011/833/EU of 12
December 2011 on the reuse of Commission documents (OJ L 330, 14.12.2011, p. 39). Reuse is authorised,
provided the source of the document is acknowledged and its original meaning or message is not distorted. The
European Commission shall not be liable for any consequence stemming from the reuse. For any use or
reproduction of photos or other material that is not owned by the EU, permission must be sought directly from
the copyright holders.

All content © European Union, 2019

How to cite this report: Papadimitriou, E.; Neves, A. R.; Becker, W., JRC Statistical Audit of the Sustainable
Development Goals Index and Dashboards, EUR 29776 EN, 2019, ISBN 978-92-76-08995-7,
doi:10.2760/723763, JRC116857.
Contents

Abstract ............................................................................................................... 2
1 Introduction ...................................................................................................... 3
2 Conceptual and statistical coherence .................................................................... 4
2.1 Relevance to the SDG Index framework .......................................................... 4
2.2 Data availability........................................................................................... 5
2.3 Identification and treatment of outliers ........................................................... 9
2.4 Normalisation .............................................................................................. 9
2.5 Weighting and aggregation ........................................................................... 9
2.6 Cross-correlation analysis ........................................................................... 10
2.7 Principal components analysis ..................................................................... 13
3 Impact of modelling assumptions on the SDG Index results .................................. 16
4 Communication on the SDG Index results ........................................................... 20
5 Conclusions .................................................................................................... 23
References ......................................................................................................... 24
Annex I – List of indicators included in the 2019 SDG Index ...................................... 25
Annex II - Median ranks of countries with 95% confidence intervals .......................... 28
Abstract

In 2015, the United Nations adopted the 2030 Agenda for Sustainable Development with
17 Sustainable Development Goals (SDGs) and 169 associated targets. All 193 United
Nations member states have committed to achieve sustainable development across its
three dimensions – economic, social, and environmental – in a balanced and integrated
manner. In order to assist countries in measuring their progress towards the achievement
of the SDGs, Bertelsmann Stiftung and the United Nations Sustainable Development
Solutions Network (SDSN) developed the Sustainable Development Goals Index and
Dashboards (SDG Index) in 2016. Since then, the SDG Index has been annually updated
and presently covers 162 countries. The European Commission’s Competence Centre on
Composite Indicators and Scoreboards (COIN) at the Joint Research Centre (JRC) was
invited by the SDSN to audit the 2019 edition of the SDG Index which will be launched on
the sidelines of the 2019 United Nations High-level Political Forum on Sustainable
Development. The audit presented herein aims to contribute to ensuring the transparency
of the SDG Index methodology and the reliability of the results. The report touches upon
data quality issues, the conceptual and statistical coherence of the framework and the
impact of modelling assumptions on the results. The fact that the SDGs are universal and
highly diverse in nature makes the work of aggregating into a single number quite
challenging from a statistical point of view. Nevertheless, the SDG Index is a remarkable
effort of synthetizing the 17 SDGs into a single measure. The index ranks are robust
enough, allowing meaningful conclusions to be drawn from the index.

2
1 Introduction

The 2030 Agenda for Sustainable Development and its 17 Sustainable Development Goals
(SDGs) was adopted by all 193 United Nations (UN) member states in 2015. The
implementation and success of this universal agenda will rely on all countries and will
require national sustainable development policies and multi-stakeholder partnerships.

Sound metrics are critical for turning the SDGs into practical tools for problem solving by
mobilising governments, academia, civil society and business; providing a report card to
track progress and ensure accountability; and serving as a management tool for the
transformations needed to achieve the SDGs by 2030. Countries are expected to voluntarily
establish national frameworks for monitoring progress made on the 17 SDGs. The UN High-
Level Political Forum plays a central role in following up and reviewing progress at the
global level.

In order to assist countries in the annual stocktaking of SDGs progress, Bertelsmann

Stiftung and the Sustainable Development Solutions Network (SDSN) launched in 2016 the
first edition of the Sustainable Development Goals Index and Dashboards (SDG Index).
The SDG Index is a composite measure of progress covering 85 indicators across all 17
goals. Now in its 2019 edition, the SDG Index includes 162 countries, while the dashboards
present data for all 193 UN member states. Additional metrics are also provided on the
dashboards and country profiles of members of the Organisation for Economic Co-operation
and Development (OECD).

The European Commission’s Competence Centre on Composite Indicators and Scoreboards

(COIN) at the Joint Research Centre (JRC) was invited by the SDSN to audit the 2019
edition of the SDG Index which will be launched on the sidelines of the 2019 United Nations
High-level Political Forum on Sustainable Development in July in New York.

The results of the audit presented herein aim at shedding light on the transparency and
reliability of the SDG Index. It is expected to contribute to enable policymakers and
advocates to derive more accurate and meaningful conclusions and to potentially guide
choices on priority setting and policy formulation.

The JRC statistical audit1 of the SDG Index focuses on two main issues: the statistical
coherence of the structure of indicators (Section 2) and the impact of key modelling
assumptions on the SDG Index ranking (Section 3). The audit follows three main steps:
the first focuses on the main descriptive statistics of the data and on a data analysis to
detect missing values and potential outliers; the second on the analysis of the statistical
coherence through a multilevel analysis of the correlations of the indicators and pillars,
and; the third, on the robustness analysis of the index and the testing of the impact of key
modelling assumptions. The results are supported by a spreadsheet in Excel format [1].
The JRC analysis also complements the reported country rankings for the SDG index with
confidence intervals in order to better appreciate the robustness of these ranks to the
computation methodology (in particular the exclusion of potentially problematic indicators,
weights and aggregation formula at the goals level).

An initial assessment on the 2018 edition of the SDG Index [2] [3] was undertaken by the
JRC in February 2019 [4]. The latest 2019 edition provided by the developers incorporated
many of the JRC suggestions and for some of the identified issues the developers provided
strong arguments for using a different approach.

1
The JRC statistical audit is based on the recommendations of the OECD & JRC (2008) Handbook on Composite Indicators
and on more recent research from the JRC. JRC audits of composite indicators are conducted upon request of their
developers. For more information see: https://composite-indicators.jrc.ec.europa.eu

3
2 Conceptual and statistical coherence

2.1 Relevance to the SDG Index framework

The conceptual framework of the SDG Index mirrors the 17 SDGs agreed by all UN member
states (Table 1). It includes 85 indicators (listed in Annex I) grouped into 17 goals, which
are subsequently aggregated into the SDG Index. The overall index is calculated as the
simple arithmetic average of the 17 goals.

While another structure could have been adopted, such as the triple bottom line framework
– Environmental, Social and Economic, or the 5Ps framework – People, Planet, Prosperity,
Peace and Partnership, the authors of the SDG Index decided to maintain the alignment
with the global SDGs framework and in this way assist countries to measure their baselines
and progress in each of the 17 SDGs. The choice of aggregating indicators in the 17 goals
to link to the 2030 global policy agenda [5] is conceptually well justified and responds to
a political need of tracking progress at goal level.

The indicators were selected based on five criteria: relevance to monitoring the
achievement of the SDGs; statistical adequacy; timeliness; data quality and coverage.
Expert consultation was used in the process of selecting the indicators.

The conceptual relevance of the indicators underpinning the SDG index framework is not
discussed in this report. One remark though, is that their number across SDGs is uneven,
ranging from SDG10 with only one indicator to SDG3 with 14 indicators. As acknowledged
by the authors, this means that those 14 indicators in SDG3 weight individually less than
the single indicator in SDG10.

Table 1. Conceptual framework of the SDG Index.

Number of
Sustainable Development Goal (SDG)
indicators
SDG1 No Poverty 2

SDG2 Zero Hunger 7

SDG3 Good Health and Well-being 14

SDG4 Quality Education 3

SDG5 Gender Equality 4

SDG6 Clean Water and Sanitation 5

SDG7 Affordable and Clean Energy 3

SDG8 Decent Work and Economic Growth 5

SDG9 Industry, Innovation and Infrastructure 6

SDG10 Reduced Inequality 1

SDG11 Sustainable Cities and Communities 3

SDG12 Responsible Consumption and Production 6

SDG13 Climate Action 4

SDG14 Life Below Water 4

SDG15 Life on Land 5

SDG16 Peace and Justice Strong Institutions 9

SDG17 Partnerships to Achieve the Goal 4

4
 2.2 Data availability

The 2019 SDG index was calculated for 162 countries. This coverage implies five additional
countries in comparison with the last edition (Maldives, Fiji, Sao Tome and Principe,
Vanuatu and Comoros). Additionally, the index is based on reliable and publicly available
data published by official international data providers (e.g. World Bank, WHO, ILO, among
others) and other international organisations including research centres and non-
governmental organisations. This is an important point given that the quality and adequacy
of the index lies not only on the index development, but also on obtaining reliable data.

Table 2 offers summary statistics for the indicators included in the SDG Index using the
raw data and highlights the cases in which specific issues were found in terms of data
coverage and presence of outliers. In the table some preliminary imputations made by the
developers’ team are included.

Moreover, for each indicator, sustainability “targets” were determined either based on
explicit/implicit SDGs targets, science-based targets or average performance of the best
performers [3]. At the same time, to remove the effect of extreme values, the developers
capped the data at the bottom 2.5th percentile as the minimum value for the normalisation.
These upper and lower bounds remain the same over the annual editions of the index and
are included in Table 2. The JRC recommended approach would be to only treat data in
specific cases where it is needed, however the developers argue that this approach is
adopted in order to facilitate comparability of the results.

5
 Table 2. Summary statistics of the indicators (raw data) included in the SDG Index.

Number of Missing Minimum Maximum Lower Upper

Goal Indicator Mean Skewness Kurtosis Direction
observations data (%) value value bound bound

1a 151 6.8 11.6 1.8 2.4 0.0 76.9 72.6 0.0 -1

SDG1
1b 151 6.8 21.9 1.1 0.0 0.0 93.1 51.5 0.0 -1
2a 154 4.9 11.0 1.8 2.9 1.2 61.8 42.3 0.0 -1
2b 159 1.9 17.8 0.6 -0.9 1.3 55.9 50.2 0.0 -1
2c 159 1.9 4.8 1.3 1.4 0.0 21.5 16.3 0.0 -1
SDG2 2d 161 0.6 18.3 -0.1 -1.1 2.1 37.9 35.1 2.8 -1
2e 159 1.9 3.5 2.7 15.0 0.2 21.5 0.2 8.6 1
2f 136 16.0 0.8 -0.5 0.1 0.3 1.3 1.2 0.0 -1
2g 152 6.2 2.3 0.0 -0.9 2.0 2.6 2.5 2.0 -1
3a 162 0.0 165.8 2.0 4.4 3.0 1360 814.0 3.4 -1
3b 162 0.0 12.8 0.8 -0.4 0.9 44.2 39.7 1.1 -1
3c 162 0.0 29.0 1.2 0.7 2.1 123.2 130.1 2.6 -1
3d 162 0.0 107.6 1.8 2.9 0.8 665 561.0 0.0 -1
3e 162 0.0 0.5 4.5 21.7 0.0 9.1 5.5 0.0 -1
3f 162 0.0 18.5 0.1 -0.8 7.8 30.6 31.0 9.3 -1
3g 162 0.0 90.5 0.9 0.1 7.0 324 368.8 0.0 -1
SDG3
3h 162 0.0 17.2 0.3 -0.7 2.9 45.4 33.7 3.2 -1
3i 162 0.0 72.1 -0.5 -0.6 52.9 84.2 54.0 83.0 1
3j 162 0.0 48.5 1.0 0.6 1.7 194 139.6 2.5 -1
3k 156 3.7 86.1 -1.4 0.9 20.2 100 23.1 100.0 1
3l 162 0.0 86.6 -1.7 2.4 37.0 99.0 41.0 100.0 1
3m 162 0.0 69.6 -0.1 -1.0 30.3 95.7 38.2 100.0 1
3n 156 3.7 5.5 -0.1 -0.7 2.7 7.9 3.3 7.6 1
4a 152 6.2 90.4 -2.0 5.5 36.8 100.0 53.8 100.0 1
SDG4 4b 136 16.0 88.6 -1.7 1.8 30.8 100.0 18.0 100.0 1
4c 151 6.8 74.6 -0.6 -0.7 10.0 116.1 45.2 100.0 1
5a 159 1.9 63.0 -0.6 -0.7 12.9 96.6 17.5 100.0 1
5b 157 3.1 88.7 -1.0 0.5 31.7 127.3 41.8 100.0 1
SDG5
5c 162 0.0 71.5 -1.0 0.7 8.4 110.3 21.5 100.0 1
5d 162 0.0 22.7 0.5 -0.1 0.0 61.3 1.2 50.0 1
6a 162 0.0 86.4 -1.3 0.6 36.6 100.0 40.0 100.0 1
6b 162 0.0 73.5 -0.8 -0.8 7.1 100.0 9.7 100.0 1
SDG6 6c 161 0.6 65.0 7.4 56.7 0.0 2603.5 100.0 12.5 -1
6d 156 3.7 9.9 4.9 28.2 0.1 148.2 42.6 0.1 -1
6e 156 3.7 26.6 1.0 -0.6 0.0 100.0 0.0 100.0 1
7a 162 0.0 82.1 -1.3 0.2 7.6 100.0 9.1 100.0 1
SDG7 7b 160 1.2 65.8 -0.6 -1.2 0.6 100.0 2.0 100.0 1
7c 133 17.9 1.7 6.4 45.2 0.1 22.6 5.9 0.0 -1
8a 158 2.5 -2.1 -0.9 2.9 -14.5 7.2 -14.7 5.0 1
8b 143 11.7 5.3 3.4 17.2 0.3 40.0 22.0 0.0 -1
SDG8 8c 151 6.8 59.2 0.0 -1.3 6.4 99.9 8.0 100.0 1
8d 162 0.0 7.3 1.7 2.9 0.1 28.5 25.9 0.5 -1
8e 161 0.6 0.8 4.7 26.6 0.0 12.4 6.0 0.0 -1

Notes: Indicators shaded in red have absolute skewness greater than 2.0 and kurtosis greater than 3.5 and/or
data coverage below 80%. The list of indicators is provided in Annex I.
* Only for the 51 High Income & OECD countries included in the country list. ** Excluding the High Income &
OECD countries.
Source: European Commission’s Joint Research Centre, 2019.

6
 Table 2. Summary statistics of the indicators (raw data) included in the SDG Index. (cont.)

Number of Missing Minimum Maximum Lower Upper

Goal Indicator Mean Skewness Kurtosis Direction
observations data (%) value value bound bound
9a 162 0.0 53.1 -0.1 -1.3 4.3 98.3 2.2 100.0 1
9b 162 0.0 64.1 0.9 1.8 0.0 243.4 1.4 100.0 1
9c 155 4.3 2.7 0.8 -0.3 1.6 4.4 1.8 4.2 1
SDG9
9d 162 0.0 20.0 1.0 0.2 0.0 94.3 0.0 91.0 1
9e 162 0.0 0.4 1.7 1.7 0.0 2.5 0.0 2.2 1
9f 132 18.5 0.8 1.7 2.5 0.0 4.3 0.0 3.7 1
SDG10 10a 148 8.6 42.1 0.5 -0.2 26.7 67.1 63.0 27.5 -1
11a 162 0.0 28.3 1.6 2.4 5.9 99.7 87.0 6.3 -1
SDG11 11b 152 6.2 84.2 -1.6 2.1 7.4 100.0 6.1 100.0 1
11c 156 3.7 57.6 -0.7 0.6 7.9 85.3 21.0 82.6 1
12a 146 9.9 1.3 1.9 5.2 0.1 5.7 3.7 0.1 -1
12b 154 4.9 8.1 0.8 -0.4 0.4 28.5 23.5 0.2 -1
12c 143 11.7 14.5 3.9 19.2 0.4 176.3 68.3 0.5 -1
SDG12
12d 161 0.6 2.0 0.6 6.3 -52.0 60.9 30.1 0.0 -1
12e 141 13.0 28.2 1.8 5.0 1.0 139.8 86.5 2.3 -1
12f 124 23.5 7.3 -0.6 11.1 -1223.4 965.4 432.4 0.0 -1
13a 162 0.0 4.5 3.3 16.1 0.0 47.5 23.7 0.0 -1
13b 160 1.2 0.0 -5.0 37.0 -19.5 4.3 3.2 0.0 -1
SDG13
13c 141 13.0 2421.3 3.4 12.2 0.0 31953 18000 0.0 -1
13d 148 8.6 4605.7 6.3 46.8 0.0 160773 44000 0.0 -1
14a 114 29.6 46.0 0.1 -1.2 0.0 99.6 0.0 100.0 1
14b 123 24.1 54.2 0.0 0.1 15.1 94.0 28.6 100.0 1
SDG14
14c 96 40.7 31.7 0.7 -0.1 0.1 100.0 90.7 0.0 -1
14d 111 31.5 32.2 0.8 -0.5 0.0 97.4 90.0 1.0 -1
15a 158 2.5 46.6 0.2 -1.1 0.0 99.4 4.6 100.0 1
15b 129 20.4 49.9 0.1 -1.2 0.0 100.0 0.0 100.0 1
SDG15 15c 162 0.0 0.9 -1.0 1.7 0.4 1.0 0.6 1.0 1
15d 138 14.8 0.2 3.5 14.4 0.0 2.9 1.5 0.0 -1
15e 160 1.2 6.5 6.5 54.5 0.0 140.2 26.4 0.1 -1
16a 162 0.0 7.1 3.6 16.1 0.3 82.8 38.0 0.3 -1
16b 148 8.6 0.3 0.6 -0.8 0.0 0.8 0.8 0.1 -1
16c 155 4.3 61.8 -0.1 -0.4 12.5 94.2 33.0 90.0 1
16d 144 11.1 4.3 0.4 -0.3 1.8 6.6 2.5 6.3 1
SDG16 16e 149 8.0 84.3 -1.7 1.8 2.7 100.0 11.3 100.0 1
16f 160 1.2 43.2 0.8 -0.3 13.0 88.0 13.0 88.6 1
16g 139 14.2 12.6 1.1 0.5 0.0 55.8 39.3 0.0 -1
16h 162 0.0 0.3 5.8 43.5 0.0 10.2 3.4 0.0 -1
16i 160 1.2 34.1 0.9 0.6 7.6 84.2 80.0 10.0 -1
17a 148 8.6 7.9 0.4 -0.1 1.0 17.9 0.0 15.0 1
17b1* 36* 29.4 0.4 1.2 0.1 0.1 1.0 0.1 1.0 1
SDG17
17b2** 95** 14.4 21.8 0.5 0.1 5.0 43.8 10.0 40.0 1
17c 162 0.0 0.2 3.9 14.2 0.0 5.0 5.0 0.0 -1

Notes: Indicators shaded in red have absolute skewness greater than 2.0 and kurtosis greater than 3.5 and/or
data coverage below 80%. The list of indicators is provided in Annex I.
* Only for the 51 High Income & OECD countries included in the country list. ** Excluding the High Income &
OECD countries.
Source: European Commission’s Joint Research Centre, 2019.

7
In general, the data coverage for the indicators included in the index is good, covering at
least 80% both at indicator and country level. Countries are included if data availability is
at least 80% at index level, however this is not the case at goal level where in some SDGs
there are countries which have no indicator data at all. In these cases, the developers
impute the missing value using the regional average score in the specific goal. For example,
Afghanistan misses both indicators in SDG1 (No poverty) so the SDG1 score that it gets is
the regional score for East Europe & Central Asia. This implies primarily to SDG10, but also
to SDG1, SDG4, SDG14, SDG15 and SDG17. SDG14 is a particular case since the countries
that miss data are the landlocked countries. The countries that miss more than 55% of
indicators on a specific goal (excluding SDG14) are listed in Table 3.

This is a fact that needs to be highlighted so that conclusions are carefully drawn for these
countries, since the results can be reflecting more a regional average than the particular
situation of the country. Therefore, the JRC recommends for the following editions of the
index to increase the number of indicators in these SDGs and/or focus specifically on
aforementioned countries trying to find alternative data sources.

Table 3. Countries missing more than 55% of indicators at goal level in the SDG Index.

SDG1 SDG4 SDG10 SDG15 SDG17

Afgha ni s tan Aus tra l i a Afgha ni s tan Jorda n Cuba
Ba hra i n Aus tri a Ba hra i n Kuwa i t
Cuba Bos ni a a nd Herzegovi na Bel i ze Montenegro
Kuwa i t Ca na da Cuba Tri ni da d a nd Toba go
Oma n Czech Republ i c Guya na
Qa tar Ga bon Kuwa i t
Sa udi Ara bi a Ha i ti New Zea l a nd
Syri a n Ara b Republ i c Netherl a nds Oma n
Uni ted Ara b Emi ra tes New Zea l a nd Qa tar
Yemen, Rep. Sl ova k Republ i c Sa udi Ara bi a
Zi mba bwe Turkmeni s tan Si nga pore
Uni ted Ki ngdom Suri na me
Uni ted States Tri ni da d a nd Toba go
Turkmeni s tan

Source: European Commission’s Joint Research Centre, 2019.

Besides the use of regional average values for imputing data for the cases above, there
are also around eight indicators with poor data coverage for which data is imputed on a
case-by-case basis [3]. The approaches used to impute the missing data are described on
the SDG Index detailed methodological paper, while the imputed data can be clearly
identified in the SDG Index dataset. These are important aspects contributing to increase
the transparency of the SDG Index.

8
 2.3 Identification and treatment of outliers

Potentially problematic indicators that could bias the overall index results were identified
on the basis of two measures related to the shape of the distributions: the skewness and
kurtosis. A practical rule used by the JRC [6] is that an indicator should be considered for
treatment if it has an absolute skewness greater than 2.0 and kurtosis greater than 3.5.

Based on this rule, Table 2 shows that initially there are 18 potentially problematic
indicators in the raw dataset which would require greater attention because of their skewed
distributions. After the lower and upper bound setting by the developers this number was
reduced. However, there are nine indicators which remain very skewed: HIV infections
(3e), Imported groundwater depletion (6d), CO2 emissions from fuel combustion (7c), Fatal
Accidents embodied in imports (8e), People affected by climate-related disasters (13c),
CO2 emissions embodied in fossil fuel exports (13d), Commodity-drive deforestation (15d),
Homicides (16a), Weapons exports (16h) and Tax Haven Score (17c). As suggested by the
JRC, the index developers applied different techniques to improve the distributions, such
as logarithmic transformations, but no major improvements were observed. Due to the
policy relevance of these indicators identified by the developers, they have decided to keep
them in the framework, however for completeness; the effect of removing these indicators
is investigated in the uncertainty analysis in Section 3.

2.4 Normalisation

As mentioned on section 2.2, the developers used boundaries on the lower and upper
bounds of the scale. The indicators’ values are normalised using the min-max normalisation
method on a scale of 0 to 100 using as minimum and maximum values the pre-set bounds.
The rescaling equation ensured that all rescaled variables were expressed as ascending
variables (i.e. higher values denoted better performance). In this way, the rescaled data
became easy to communicate to a wider public and to compare across all indicators.

2.5 Weighting and aggregation

The SDG Index is calculated using equal weighting for the underlying components. At goal
level, this is justified by the fact that all SDGs are considered as having equal importance
as part of the 2030 Agenda. At the indicator level, equal weighting was retained because
all alternatives were considered as being less satisfactory. However, assigning equal
weights to the indicators and goals do not necessarily guarantee an equal contribution of
the indicators or goals to the SDG Index [6] [7]. For example, considering that goals are
measured using an uneven number of indicators, the 14 global indicators under SDG3 are
effectively weighted less in the overall aggregation than the single indicator used to
measure SDG 10.

Regarding the aggregation formula, the arithmetic average is used at all levels to build the
SDG Index; at the first aggregation level (from indicators to goals) and at the second and
last aggregation level (from goals to the overall index). This means that the overall index
is calculated as the arithmetic average over the 17 SDGs. While arithmetic averages are
easy to interpret, they also allow perfect compensability between the variables, whereby a
high score on one variable can fully offset low scores in other variables. This may not
necessarily fit with the concept of sustainable development where having a high social
sustainability should not come at the cost of low environmental sustainability, although
this is often observed in practice - see the following section. The geometric average is an
alternative aggregation method which is non-compensatory and fits with the view that
scores in different dimensions of sustainability should not compensate one another. The

9
impacts of the aggregation formula as well as of the weighting scheme in the index results
will be discussed thoroughly in section 3.

2.6 Cross-correlation analysis

The statistical coherence of the SDG Index should be considered a necessary, though not
necessarily sufficient, condition for a sound index. Given that the present statistical
analysis is mostly based on correlations, the correspondence of the SDG Index to a real
world phenomenon needs to be critically addressed because “correlations do not
necessarily represent the real influence of the individual indicators on the phenomenon
being measured” [6]. This relies on the combination of statistical and conceptual
soundness. The cross-correlation analysis is used to address to what extent the data
support the conceptual framework. The 1% significance level is used to determine whether
the correlation between two variables is statistically significant.
In the ideal case, there should be positive significant correlations within every level of the
index, i.e. each indicator positively correlated with its goal and the index as well as each
goal correlated with the index. This effectively ensures that the overall index scores
adequately reflect the underlying indicator values. Redundancy should be avoided in the
framework because if two indicators are collinear, this amounts to double-counting (and
therefore over-weighting) the same phenomenon. It also increases the complexity, which
is contrary to good practices of data modelling, in which the simplest model that explains
the data (or phenomenon) is preferable (Occam’s Razor).
A detailed analysis of the correlation within and across goals confirms that most of the
indicators are more correlated to their own goal than to any other goal. A few exceptions
were found, but as the SDG Index conceptual framework is limited by the fixed structure
of the UN SDG official framework [8], those indicators cannot be simply transferred from
one goal to another, as acknowledged by the index developers. Overall, correlations within
each goal are significant and positive, but there are a few indicators which would require
greater attention due to their negative correlation with other indicators and with the goal.
Table 4 shows the correlation between indicators, their respective goal and the overall
index. Some indicators are negatively correlated with their respective goal and/or with the
index (highlighted in red), typically as a result of negative correlations with other
indicators. Other indicators are highly collinear (i.e. Pearson correlation coefficients greater
than 0.92) with their respective goal (highlighted in blue).

10
 Table 4. Correlations between the indicators, their respective goal and the overall index.

Respective Respective Respective

Indicator id Index Indicator id Index Indicator id Index
SDG SDG SDG
1a 0.95 0.79 5d 0.65 0.33 12f 0.77 -0.35
1b 0.98 0.89 6a 0.79 0.83 13a 0.70 -0.47
2a 0.66 0.78 6b 0.76 0.86 13b 0.56 -0.16
2b 0.71 0.85 6c 0.25 -0.06 13c 0.46 0.25
2c 0.69 0.64 6d 0.16 -0.06 13d 0.69 -0.08
2d -0.24 -0.54 6e 0.70 0.69 14a 0.55 0.34
2e 0.68 0.68 7a 0.95 0.83 14b 0.37 0.36
2f 0.55 0.41 7b 0.94 0.84 14c 0.50 -0.26
2g -0.32 -0.74 7c 0.51 0.35 14d 0.50 -0.48
3a 0.89 0.84 8a 0.71 0.65 15a 0.78 0.25
3b 0.93 0.88 8b 0.64 0.58 15b 0.81 0.22
3c 0.94 0.89 8c 0.74 0.77 15c 0.52 0.10
3d 0.70 0.59 8d 0.44 0.08 15d 0.36 0.35
3e 0.47 0.38 8e -0.14 -0.34 15e 0.22 -0.46
3f 0.64 0.57 9a 0.88 0.86 16a 0.49 0.31
3g 0.89 0.84 9b 0.84 0.79 16b 0.61 0.46
3h 0.79 0.77 9c 0.92 0.70 16c 0.72 0.50
3i 0.97 0.91 9d 0.89 0.63 16d 0.76 0.58
3j 0.83 0.76 9e 0.89 0.68 16e 0.68 0.73
3k 0.79 0.76 9f 0.88 0.66 16f 0.83 0.69
3l 0.62 0.59 10a 1.00 0.41 16g 0.75 0.80
3m 0.94 0.91 11a 0.73 0.51 16h -0.32 -0.43
3n 0.79 0.77 11b 0.78 0.68 16i 0.40 0.34
4a 0.84 0.70 11c 0.60 0.45 17a 0.58 0.62
4c 0.92 0.83 12a 0.73 -0.48 17b1 0.69 0.30
4b 0.92 0.80 12b 0.92 -0.79 17b2 0.91 0.48
5a 0.74 0.63 12c 0.51 -0.34 17c 0.34 -0.23
5b 0.71 0.71 12d 0.73 -0.52
5c 0.45 -0.01 12e 0.85 -0.53

Notes: Numbers represent the Pearson correlation coefficients between each indicator and the corresponding goal
as well as between each indicator and the overall index. Correlations that are not significant at the significance
level of α = 0.01 are highlighted in grey (critical value of 0.202). Very high correlations (i.e. Pearson correlation
coefficients greater than 0.92) are highlighted in blue and negative correlations in red.
Source: European Commission’s Joint Research Centre, 2019.

Table 5summarises the correlation coefficients between goals as well as between each goal
and the overall index. Values greater than 0.70 are desirable as they imply that the index
captures at least 50% (≈ 0.70 × 0.70) of the variation in the underlying goals and vice-
versa. This is the case for 11 out of 17 SDGs: from SDG1 to SDG9, SDG11 and SDG16.
SDG3 shows a very high correlation (Pearson correlation coefficient 0.93) which may
suggest redundancy. SDG10 and SDG17 have lower correlation coefficients but still
significant, suggesting that their importance is lower that this of the other goals. With
respect to the remaining goals, SDG14 and SDG15 show no significant correlation with the
overall index, while SDG12 and SDG13 present a negative relationship with the index. In
practice, this means that the highest scoring countries on the SDG Index are having some
of the lowest scores in SDG12 and SDG13, and vice versa, which can give the impression

11
that high-scoring countries score highly in all goals. This seems to be an unavoidable reality
in which environmental sustainability goes somewhat contrary to social sustainability, and
motivates the possibility of using a non-compensatory geometric mean, as discussed
earlier. In order to address this issue, a possible revision of the indicators which are not
significantly correlated or negatively correlated under each of these four goals could be
considered by the developers, particularly in the case where official SDGs indicators are
not adopted. In any case, the important is that this disparity between the SDG Index scores
and SDG12 and SDG13 should be made clear in the conclusions of the SDG Index, possibly
by presenting index scores additionally with these two goals. These issues are further
discussed in Section 4.

Table 5. Correlations between the goals and SDG Index.

SDG 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Index
1 1.00
2 0.49 1.00
3 0.84 0.64 1.00
4 0.77 0.61 0.84 1.00
5 0.35 0.54 0.59 0.61 1.00
6 0.69 0.66 0.81 0.73 0.68 1.00
7 0.88 0.50 0.85 0.81 0.46 0.71 1.00
8 0.50 0.60 0.68 0.62 0.59 0.63 0.51 1.00
9 0.66 0.66 0.82 0.67 0.59 0.75 0.68 0.63 1.00
10 0.36 0.29 0.36 0.17 0.01 0.21 0.19 0.24 0.38 1.00
11 0.54 0.46 0.68 0.66 0.64 0.69 0.59 0.53 0.55 0.10 1.00
12 -0.59 -0.53 -0.76 -0.59 -0.52 -0.67 -0.60 -0.46 -0.86 -0.32 -0.50 1.00
13 -0.28 -0.17 -0.33 -0.32 -0.18 -0.19 -0.27 -0.10 -0.32 -0.05 -0.13 0.50 1.00
14 -0.17 -0.01 -0.11 -0.10 0.09 -0.04 -0.13 0.06 -0.05 -0.19 -0.04 0.04 0.02 1.00
15 -0.09 0.15 -0.02 -0.03 0.17 0.12 -0.05 0.14 0.09 0.01 -0.01 -0.02 0.22 0.24 1.00
16 0.64 0.59 0.81 0.67 0.50 0.65 0.60 0.58 0.78 0.43 0.63 -0.72 -0.33 -0.11 0.04 1.00
17 0.19 0.07 0.21 0.22 0.14 0.12 0.29 -0.03 0.11 0.01 0.16 -0.12 -0.35 -0.02 -0.08 0.14 1.00
Index 0.84 0.71 0.93 0.86 0.67 0.86 0.86 0.73 0.83 0.40 0.73 -0.68 -0.20 -0.01 0.14 0.79 0.24 1.00

Notes: Numbers represent the Pearson correlation coefficients between the SDG Index goals and the overall
index. Correlations that are not significant at the significance level of α = 0.01 are highlighted in grey (critical
value of 0.202). Very high correlations (i.e. Pearson correlation coefficients greater than 0.92) are highlighted in
blue and negative correlations in red.
Source: European Commission’s Joint Research Centre, 2019.

12
 2.7 Principal components analysis
Principal components analysis (PCA) [9] [10] explores the correlation of all the indicators
simultaneously, highlighting, if present, some common trends that describe a common
concept among the indicators. It is here used to assess to what extent the conceptual
framework of the SDG Index is confirmed by statistical approaches.
The results of the PCA performed to the total group of 85 indicators show that there are
17 principal components with eigenvalues greater than 1 that explain almost 80% of the
total variance (Table 6). That suggests the presence of several drivers among the indicators
and is correctly accommodated by the use of the 17 goals as an intermediate step towards
the creation of the overall score.

Table 6 - Results of the Principal Components Analysis on the 85 indicators.

cumulative %
eigenvalue % of variance
of variance
PC1 33.38 39.27 39.27
PC2 6.39 7.51 46.78
PC3 4.57 5.38 52.16
PC4 3.37 3.96 56.12
PC5 2.54 2.99 59.11
PC6 2.18 2.56 61.68
PC7 1.99 2.34 64.02
PC8 1.82 2.14 66.16
PC9 1.70 2.00 68.16
PC10 1.51 1.78 69.94
PC11 1.44 1.69 71.63
PC12 1.26 1.48 73.11
PC13 1.16 1.37 74.48
PC14 1.13 1.33 75.82
PC15 1.05 1.23 77.05
PC16 1.04 1.22 78.27
PC17 0.96 1.13 79.39
PC18 0.87 1.02 80.41
PC19 0.86 1.01 81.42
PC20 0.81 0.95 82.37

Results shown for the first 20 out of 85 principal components (PC).

Source: European Commission’s Joint Research Centre, 2019.

At a second step, PCA is performed to the 17 goals that, after aggregation, form the overall
SDG Index score. Ideally, it is expected to have one principal component (PC) explaining
at least 70%-80% of the total variance in order to claim that there is a single latent
phenomenon behind the data. This is not the case in the SDG Index, as the results show
that there are four principal components that explain around 70% of the variance. From
the Table 7, the presence of a major driver is evident; the first component explains 50%
of the variance, although, still, there are three other components that are explaining
enough amount (eigenvalues >=1).

13
 Table 7 - Results of the Principal Components Analysis on the 17 goals.

% of cumulative %
eigenvalue
variance of variance
PC1 8.37 49.26 49.26
PC2 1.67 9.82 59.08
PC3 1.32 7.76 66.84
PC4 1.07 6.32 73.16
PC5 0.84 4.96 78.12
PC6 0.71 4.19 82.31
PC7 0.63 3.68 85.99
PC8 0.53 3.09 89.08
PC9 0.41 2.41 91.49
PC10 0.35 2.04 93.53
PC11 0.31 1.84 95.38
PC12 0.23 1.37 96.74
PC13 0.17 1.01 97.75
PC14 0.15 0.87 98.62
PC15 0.09 0.55 99.18
PC16 0.08 0.48 99.66
PC17 0.06 0.34 100.00

Source: European Commission’s Joint Research Centre, 2019.

Figure 1 shows in more detail that most goals form a group on the right quadrant, which
is explained by the first principal component. Then, it is possible to observe that goal 12
forms a second group opposite to the first (as suggested by the negative correlations). In
addition, a third group comprises goals 13, 14 and 15 orthogonal to the first two groups
and a fourth group includes goal 17, more close to the first one.

14
 Figure 1 – Factor map of the 17 goals of the SDG Index.

Source: European Commission’s Joint Research Centre, 2019.

15
3 Impact of modelling assumptions on the SDG Index results

The development of a composite indicator, like any model, involves assumptions and
subjective decisions. This section aims to test the impact of varying some of these
assumptions within a range of plausible alternatives in an uncertainty analysis. The
objective is therefore to try to quantify the uncertainty in the ranks of the SDG Index,
which can demonstrate the extent to which countries can be differentiated by their SDG
Index scores.

Although many assumptions made in the development of the SDG Index could be
examined, three particular assumptions were examined in this uncertainty analysis (see
Table 8). These were chosen as plausible alternative pathways in the construction of the
SDG Index, which can be relatively easily investigated.

Table 8. Conceptual framework of the SDG Index.

Assumption Alternatives

1. Indicator set Full set

Reduced set
2. Aggregation method (pillar level) Arithmetic average
Geometric average
3. Weights (pillar level) Randomly varied +/-25% from nominal values

The first is the inclusion of indicators: in the present audit, a number of statistically
“problematic” indicators were identified, which have issues in terms of skewness and
correlation (see section 2.3). For conceptual and communication reasons, these indicators
were retained in the final index, but the effect is tested here of removing all of these
indicators simultaneously, resulting in a “reduced set” of indicators which can be viewed
as an alternative approach to building the index. The second assumption which is varied is
the aggregation method. In the SDG Index, the goal scores are aggregated into a single
score using an arithmetic average. An alternative approach would be to use the geometric
average, which is non-compensatory, and represents the idea that high scores in one goal
should not compensate low scores in another, which is an alternative way to look at
sustainable development. Finally, nominal weights assigned at the goal level are all equal.
The effect of randomly varying these weights by +/-25% is investigated, to check modest
variations in the importance of individual goals.

To investigate the impact of varying these assumptions, a Monte Carlo experiment was
performed, which involved re-building the SDG Index 4000 times, each time with a
randomly-selected combination of assumptions 1-3. The overall results are shown in Figure
2.

The uncertainty in the rankings, given the assumptions tested, is mostly quite modest, but
some countries show particular sensitivity to changes. About 40% of countries have 90%
confidence intervals2 of ten places or less, with 10% having confidence intervals of five
places or fewer. The average confidence interval size is about 13 rank places, however,
this is over 162 countries in total, so does not represent a very large uncertainty in this
context. A small number of countries have wider confidence intervals (14% have intervals
wider than 20 places), with Singapore in particular having an interval of 57 places. The
ranking of Bosnia and Herzegovina is also more uncertain, with a confidence interval of 49
places. These stand-out cases are likely due to particularly uneven scores across indicators

2
A 90% confidence interval means that, given the uncertainties tested, the rank falls within this interval with
90% probability.

16
and goals, which mean that changes in the weighting and aggregation scheme have a
greater impact.

18
 0

LUX

20

ARE
40

BHR

60 AUS
CHN OMN
QAT
GRC
ISR
CYP
80
CUB
Rank

DOM

100 TJK
PAN
SGP
BIH
MNE

120
LSO

ZAF
140 BWA
NAM

YEM
160

180

Figure 2 – 90% confidence intervals of ranks in descending order of nominal rank. Selected countries with confidence intervals wider than 20 places are labelled.

Source: European Commission’s Joint Research Centre, 2019.

17
The overall picture is that the ranks of the SDG Index are fairly robust, and country ranks
can be stated to within around 13 places of precision, although some countries are
especially sensitive to the assumptions made. This information should be used to guide the
kind of conclusions that can be drawn from the index. For example, differences of two or
three places between countries cannot be taken as “significant”, whereas differences of 10
places upwards can show a meaningful difference. One can also observe from Figure 2 that
the confidence intervals are generally wider for mid-ranking countries, and narrower for
top and bottom-ranking countries.

The Monte Carlo results can also give an idea of sensitivity to the various assumptions.
Figure 3 shows the median ranks of the SDG Index for simulations with the full set of
indicators against those with the reduced set, and arithmetic against geometric mean. This
gives an idea of sensitivity of the rankings to these assumptions. Both plots show a
noticeable but fairly limited scatter, which implies that the assumptions are both
contributing fairly equally to the uncertainty, although the alternative geometric mean
assumption causes greater extreme rank shifts.

160 160
140 140
Median rank (reduced set)

Median rank (geomertic)

120 120
100 100
80 80
60 60
40 40
20 20
0 0
0 50 100 150 0 50 100 150
Median rank (full set) Median rank (arithmetic)

Figure 3 – Median ranks of SDG Index with full set of indicators against reduced set (left), and arithmetic mean
against geometric mean (right).

Source: European Commission’s Joint Research Centre, 2019.

To delve slightly further into the possibility of using a geometric average, Figure 4 shows
the nominal ranks of the SDG Index (i.e. the default modelling assumptions used by the
developers and featured in the final index) plotted against the nominal ranks with a
geometric mean applied at the goal level. This is different from Figure 3 in that the
uncertainty in the other assumptions is not considered. The results show that the impact
of changing to a geometric mean is fairly limited for many countries, with an average rank
shift of around four places. However, some countries do shift by a significant amount,
including Bosnia (-27), Singapore (-26), and Cuba (-18).

The JRC recommends to weigh up the possibility of using a geometric average: it may
better reflect the non-compensatory nature of sustainable development, but is more
difficult to communicate to stakeholders and comes with a fairly small change in rankings
for most countries. This possibility might be reflected on by the developers in future
versions of the SDG Index.

18
Figure 4 – Nominal ranks with arithmetic mean vs geometric mean at goal level. Selected countries with a large
rank change are labelled.

180
BDI
160 SWZ

140 BWA NAM

ZAF
120
Rank with geometric mean

100 BIH
SGP
TJK

80 CUB
ISR

60 BHR

40

20

0
0 20 40 60 80 100 120 140 160 180
Rank with arithmetic mean

Source: European Commission’s Joint Research Centre, 2019.

The overall implications of the uncertainty analysis are that the uncertainty in the rankings
is manageable, and allows meaningful conclusions to be drawn from the index, although
both the aggregation method and the set of indicators do cause a modest contribution to
the uncertainty. The full rankings, with confidence intervals, can be found in Annex II.

19
4 Communication on the SDG Index results

It is important to note that the SDG Index can be used as an overall aggregate score, but
should also serve as an access point to the underlying goals and indicators. The JRC
recommends the developers to derive more policy narratives and conclusions by delving
into the individual goals (i.e. first level of aggregation), rather than focusing exclusively on
the SDG Index score. The index score can indeed reveal patterns which do not directly
emerge by looking at the 17 goals separately, but an analysis at goal level can provide
more additional insights.
In fact, a detailed analysis of the countries’ ranking positions at SDG Index level and at
each goal level (Table 9) reveals that for 56% or more of the 162 countries included, the
SDG Index ranking and any of the 17 goals rankings differ by 10 positions or more. The
results suggest that the SDG Index ranking highlights aspects of countries’ efforts towards
sustainable development that do not emerge by looking into each one of the goals
separately. But at the same time, this result points to the value of examining individual
goals on their own merit in order to identify which goals are driving a country’s
performance, having into account that the overall index score allows full compensability.
In particular, SDG10, SDG12, SDG13, SDG14, SDG15 and SDG17 have more than 80% of
countries that differ by more than 10 positions from the overall SDG Index ranking. On the
other hand, SDG3 which presents the highest correlation with the overall index has the
lower number of countries with a shift of more than 10 positions (56%).

Table 9 – Distribution of differences between goals and SDG Index rankings.

Shifts with the

respect to SDG SDG1 SDG2 SDG3 SDG4 SDG5 SDG6 SDG7 SDG8 SDG9 SDG10 SDG11 SDG12 SDG13 SDG14 SDG15 SDG16 SDG17
Index

0 positions 2% 1% 3% 1% 1% 2% 3% 1% 4% 2% 1% 1% 0% 0% 1% 4% 1%
Less than 5
positions 17% 9% 20% 15% 9% 16% 19% 12% 22% 6% 17% 4% 3% 4% 4% 20% 7%
5 to 10
positions 14% 12% 24% 14% 12% 20% 18% 9% 15% 8% 8% 5% 3% 6% 7% 12% 12%
More than 10
positions 69% 78% 56% 71% 78% 64% 64% 79% 64% 86% 75% 91% 94% 91% 88% 68% 81%
11 to 20
positions 27% 23% 27% 31% 21% 27% 29% 25% 28% 19% 22% 10% 12% 11% 13% 21% 12%
21 to 30
positions 19% 19% 15% 15% 20% 19% 12% 17% 15% 9% 17% 7% 3% 7% 17% 16% 10%
More than 30
positions 23% 36% 14% 25% 38% 18% 22% 37% 20% 59% 36% 73% 78% 73% 59% 31% 59%

Source: European Commission’s Joint Research Centre, 2019.

Countries ranking first on the aggregated SDG Index can have significantly lower positions
on individual goals. This happens due to the presence of significant negative correlations
between SDG12 and SDG13 with any of the other goals in the SDG Index framework (see
section 2.6).
While there is a clear positive association between the SDG Index and most of the
underlying goals, the same does not held true for SDG12 and SDG13. From a statistical
point, the negative relationship between goals is a sign of trade-off, whereby some
countries that have poor performance on SDG12 and SDG13 have good performance on
all the other goals and vice-versa.

20
Figure 5 confirms the negative relationship between these two goals and the overall index
score. The top five countries are ranked among the bottom positions of SDG12 and SDG13.
For example, Denmark tops the list on the SDG Index, but is on the 143 th position on the
SDG12 ranking. On the other direction, Central African Republic which is at the bottom of
the SDG Index gets the second best position on SDG13.

Figure 5 - Relation between the goals 12, 13 combined, and the SDG Index.

Source: European Commission’s Joint Research Centre, 2019.

In addition to the SDG Dashboards where one perceives at a glance in which goals a
country is scoring better or worse as well as which goals present the greatest challenges,
the JRC would recommend to further explore how the statistical associations between goals
could be used to inform SDGs policies at global and national levels.
For instance, if the 17 SDGs are grouped into two groups: the environmental group on one
side (SDG12, SDG13, SDG14, SDG15) and all the other goals on the other side (SDG1,
SDG2, SDG3, SDG4, SDG5, SDG6, SDG7, SDG10, SDG11, SDG16, SDG17), one could look
at the countries located on the top right quadrant as the ones which have more balanced
profiles in terms of achieving both highest environmental and socio-economic performance
(Figure 6). This would be a complementary view to the index rankings.

21
Figure 6 – Relation between four environmental-related goals (SDG12, SDG13, SDG14 and SDG15) and all the
other goals in the SDGs framework.

Source: European Commission’s Joint Research Centre, 2019.

22
5 Conclusions

The JRC statistical audit delves into the extensive work carried out by the developers of
the SDG Index with the aim of suggesting improvements in terms of data characteristics,
structure and methods used. The analysis aims to ensure the transparency of the SDG
Index methodology and the reliability of the results. The present audit was preceded by a
JRC assessment on the 2018 edition, from which some suggestions related to data quality
issues were taken into account by the developers in the 2019 edition.

This report focused first on the assessment of the statistical coherence of the SDG Index
by carrying out a multilevel analysis of the correlations within and across the indicators
and goals. It was then followed by an assessment of the impact of key modelling
assumption on the SDG index ranking.

The methodology to calculate the SDG Index adopted by the developers included data
checking for outliers; normalisation using the min-max method in 1-100 scale (100 the
best score) including lower and upper bound setting, and; aggregation at all levels (i.e.
from indicators to goals and from goals to the overall index) by simple arithmetic average
and equal weighting.

The main challenge on the construction of the SDG Index lays on the inverted relationship
between socio-economic goals and environmental ones, in particular SDG12 (responsible
consumption and production) and SDG13 (climate action). Also, SDG 14 (life below water)
and SDG 15 (life on land) show no significant association with the SDG Index. The negative
relationship between goals is a sign of trade-off, whereby some countries that have poor
performance on SDG12 and SDG13 have good performance on all the other goals and vice-
versa. Upon these considerations, the JRC recommendation would be to focus on a
complementary analysis on the relationships between goals and to consider the option of
using the geometric average instead of the arithmetic average. The geometric average
could serve as an alternative aggregation method that is non-compensatory and fits with
the view that scores in different dimensions of sustainable development should not
compensate one another.

The uncertainty and sensitivity analyses carried out confirm that the uncertainty is
manageable and allows meaningful conclusions to be drawn from the SDG Index.
Nevertheless, both the aggregation method and the set of indicators do cause a modest
contribution to the uncertainty. A suggestion would be to guide the conclusions that can
be drawn from the SDG Index using the following information: differences of two or three
places between countries cannot be taken as “significant”, whereas differences of 10 places
can show a meaningful difference.

All things considered, the SDG Index is a noteworthy effort of synthetizing the 17 adopted
SDGs into a single figure. Overall, the ranks of the SDG Index are fairly robust. The index
is anchored on the 2030 Agenda for Sustainable Development adopted by all UN member
states and rigorously follows the same structure of 17 goals. The fact that the goals are
universal and highly diverse in nature makes the work of aggregating into a single number
quite challenging from a statistical point of view. The index is also complemented by
dashboards, which are a very communicative and neat way to show the performance of
countries at individual goal level. The SDG Index proposes a first-of-its-kind composite
measure to track progress on SDGs at national and global level, but it is fundamental that
communication of its results is accompanied by a deep understanding of its underlying
components and the relationships between them.

23
References

[1] E. Papadimitriou, A. R. Neves, and W. Becker, “JRC Statistical Audit of the

Sustainable Development Goals Index and Dashboards (Excel spreadsheet).” 2019.
[2] Sachs; J.; Schmidt-Traub; G.; Kroll; C.; Lafortune; Fuller, “SDG Index and
Dashboards Report 2018: Global Responsibilities,” New York, 2018.
[3] G. Lafortune, G. Fuller, J. Moreno, G. Schmidt-traub, and C. Kroll, “SDG Index and
Dashboards - Detailed Methodological paper,” 2018.
[4] E. Papadimitriou, A. R. Neves, and W. Becker, “Joint Research Centre Statistical Pre-
Audit of the 2018 Sustainable Development Goals Index and Dashboards,” 2019.
[5] United Nations, “Transforming Our World: The 2030 Agenda for Sustainable
Development, A/RES/70/1,” vol. 16301, no. October. United Nations General
Assembly, pp. 1–35, 2015.
[6] OECD and JRC, Handbook on Constructing Composite Indicators - Methodology and
User Guide. OECD Publications, 2008.
[7] W. Becker, M. Saisana, P. Paruolo, and I. Vandecasteele, “Weights and importance
in composite indicators: Closing the gap,” vol. 80, no. May, pp. 12–22, 2017.
[8] Inter-Agency and Expert Group on SDG Indicators (IAEG-SDGs), “Final list of
proposed Sustainable Development Goal indicators,” 2016.
[9] F. B. Bryant and P. R. Yarnold, “Principal-components analysis and exploratory and
confirmatory factor analysis,” in Reading and understanding multivariate statistics,
L. G. G. & P. R. Yarnold, Ed. Washington, DC, US: American Psychological
Association., 1995, pp. 99–136.
[10] R. A. Johnson and D. W. Wichern, Applied multivariate statistical analysis, 3rd. ed.
Prentice-Hall: Englewood Cliffs (N.J.), 1992.

24
Annex I – List of indicators included in the 2019 SDG Index
Goal ID Indicator
1a Poverty headcount ratio at $1.90/day (% population)
SDG1
1b Poverty headcount ratio at $3.20/day (% population)
2a Prevalence of undernourishment (% population)
2b Prevalence of stunting (low height-for-age) in children under 5 years of age (%)
2c Prevalence of wasting in children under 5 years of age (%)
SDG2 2d Prevalence of obesity, BMI ≥ 30 (% adult population)
2e Cereal yield (t/ha)
2f Sustainable Nitrogen Management Index
2g Human Tropic Level (best 2 - 3 worst)
3a Maternal mortality rate (per 100,000 live births)
3b Neonatal mortality rate (per 1,000 live births)
3c Mortality rate, under-5 (per 1,000 live births)
3d Incidence of tuberculosis (per 100,000 population)
3e New HIV infections (per 1,000)
Age-standardised death rate due to cardiovascular disease, cancer, diabetes, and chronic respiratory disease in populations age 30–70 years (per 100,000
3f
population)
SDG3 3g Age-standardised death rate attributable to household air pollution and ambient air pollution (per 100,000 population)
3h Traffic deaths rate (per 100,000 population)
3i Life Expectancy at birth (years)
3j Adolescent fertility rate (births per 1,000 women ages 15-19)
3k Births attended by skilled health personnel (%)
3l Percentage of surviving infants who received 2 WHO-recommended vaccines (%)
3m Universal Health Coverage Tracer Index (0-100)
3n Subjective Wellbeing (average ladder score, 0-10)
4a Net primary enrolment rate (%)
SDG4 4b Literacy rate of 15-24 year olds, both sexes (%)
4c Lower secondary completion rate (%)
5a Demand for family planning satisfied by modern methods (% women married or in unions, ages 15-49)
5b Ratio of female to male mean years of schooling of population age 25 and above
SDG5
5c Ratio of female to male labour force participation rate
5d Seats held by women in national parliaments (%)
6a Population using at least basic drinking water services (%)
6b Population using at least basic sanitation services (%)
SDG6 6c Freshwater withdrawal as % total renewable water resources
6d Imported groundwater depletion (m3/year/capita)
6e Percentage of anthropogenic wastewater that receives treatment (%)
7a Access to electricity (% population)
SDG7 7b Access to clean fuels & technology for cooking (% population)
7c CO2 emissions from fuel combustion / electricity output (MtCO2/TWh)

25
Goal ID Indicator
8a Adjusted Growth (%)
8b Prevalence of Modern Slavery (victimes per 1,000 pop)
SDG8 8c Adults (15 years and older) with an account at a bank or other financial institution or with a mobile-money-service provider (%)
8d Unemployment rate (% total labor force)
8e Fatal Accidents embodied in imports (fatal accidents per 100,000)
9a Population using the internet (%)
9b Mobile broadband subscriptions (per 100 inhabitants)
9c Logistics performance index: Quality of trade and transport-related infrastructure (1=low to 5=high)
SDG9
9d The Times Higher Education Universities Ranking, Average score of top 3 universities (0-100)
9e Number of scientific and technical journal articles (per 1,000 population)
9f Research and development expenditure (% GDP)
SDG10 10a Gini Coefficient adjusted for top income (1-100)
11a Annual mean concentration of particulate matter of less than 2.5 microns of diameter (PM2.5) (μg/m3)
SDG11 11b Improved water source, piped (% urban population with access)
11c Satisfaction with public transport (%)
12a Municipal Solid Waste (kg/year/capita)
12b E-waste generated (kg/capita)
12c Production-based SO2 emissions (kg/capita)
SDG12
12d Imported SO2 emissions (kg/capita)
12e Nitrogen production footprint (kg/capita)
12f Net imported emissions of reactive nitrogen (kg/capita)
13a Energy-related CO2 emissions per capita (tCO2/capita)
13b Imported CO2 emissions, technology-adjusted (tCO2/capita)
SDG13
13c People affected by climate-related disasters (per 100,000 population)
13d CO2 emissions embodied in fossil fuel exports (kg/capita)
14a Mean area that is protected in marine sites important to biodiversity (%)
14b Ocean Health Index Goal - Clean Waters (0-100)
SDG14
14c Percentage of Fish Stocks overexploited or collapsed by EEZ (%)
14d Fish caught by trawling (%)
15a Mean area that is protected in terrestrial sites important to biodiversity (%)
15b Mean area that is protected in freshwater sites important to biodiversity (%)
SDG15 15c Red List Index of species survival (0-1)
15d Permanent Deforestation, 5 year average annual %
15e Imported biodiversity threats (threats per million population)
16a Homicides (per 100,000 population)
16b Unsentenced detainees as a proportion of overall prison population
16c Proportion of the population who feel safe walking alone at night in the city or area where they live (%)
SDG16
16d Property Rights (1-7)
16e Birth registrations with civil authority, children under 5 years of age (%)
16f Corruption Perception Index (0-100)

26
Goal ID Indicator
16g Children 5–14 years old involved in child labour (%)
16h Transfers of major conventional weapons (exports) (constant 1990 US$ million per 100,000 population)
16i Freedom of Press Index
17a Government Health and Education spending (% GDP)
17b1 For high-income and all OECD DAC countries: International concessional public finance, including official development assistance (% GNI)
SDG17
17b2 Other countries : Government Revenue excl. Grants (% GDP)
17c Tax Haven Score (best 0-5 worst)

27
Annex II - Median ranks of countries with 95% confidence intervals
Countries ordered by nominal rank.

28
 Country Median rank Country Median rank

1 Denmark 1 [1, 1] 41 Ukraine 45 [39, 51]

2 Sweden 2 [2, 2] 42 Romania 41 [37, 45]
3 Finland 3 [3, 7] 43 Uruguay 44 [41, 51]
4 France 4 [4, 8] 44 Serbia 44 [40, 52]
5 Austria 9 [5, 11] 45 Argentina 44 [40, 47]
6 Germany 7 [5, 8] 46 Ecuador 48 [43, 55]
7 Czech Republic 9 [5, 13] 47 Maldives 42 [38, 51]
8 Norway 7 [3, 14] 48 Kyrgyz Republic 52 [46, 61]
9 Netherlands 7 [3, 11] 49 Israel 56 [43, 72]
10 Estonia 12 [6, 17] 50 Greece 54 [44, 66]
11 New Zealand 14 [10, 19] 51 Peru 54 [46, 64]
12 Slovenia 16 [11, 20] 52 Uzbekistan 55 [49, 62]
13 United Kingdom 8.5 [5, 15] 53 Algeria 50 [42, 57]
14 Iceland 16 [11, 24] 54 Vietnam 54 [49, 58]

15 Japan 15 [10, 19] 55 Russian Federation 52 [45, 58]

16 Belgium 15 [12, 20] 56 Cuba 67 [51, 80]
17 Switzerland 15 [9, 25] 57 Brazil 56 [47, 63]
18 Korea, Rep. 21 [14, 25] 58 Iran, Islamic Rep. 55 [47, 64]
19 Ireland 14 [9, 24] 59 Azerbaijan 63 [56, 69]
20 Canada 21 [16, 25] 60 Albania 64 [58, 70]
21 Spain 20 [17, 23] 61 Cyprus 59 [48, 73]
22 Croatia 20 [13, 25] 62 Fiji 64 [57, 71]
23 Belarus 23 [20, 26] 63 Tunisia 60 [51, 67]

24 Latvia 24 [21, 27] 64 Dominican Republic 69 [60, 84]

25 Hungary 26 [22, 29] 65 United Arab Emirates 57 [39, 73]
26 Portugal 27 [25, 28] 66 Singapore 76 [43, 100]

27 Slovak Republic 29 [24, 35] 67 Colombia 68 [61, 75]

28 Malta 26 [20, 32] 68 Malaysia 59 [50, 70]
29 Poland 30 [28, 33] 69 Bosnia and Herzegovina 77 [53, 102]

30 Italy 30 [28, 33] 70 North Macedonia 67 [60, 75]

31 Chile 33 [28, 36] 71 Tajikistan 85 [67, 95]
32 Lithuania 32 [30, 36] 72 Morocco 68 [59, 75]

33 Costa Rica 33 [29, 36] 73 Georgia 76 [71, 82]

34 Luxembourg 27 [14, 39] 74 Jamaica 73 [68, 82]
35 United States 35 [30, 37] 75 Armenia 75 [68, 84]

36 Bulgaria 34 [29, 38] 76 Bahrain 62 [49, 79]

37 Moldova 37 [33, 40] 77 Kazakhstan 74 [64, 81]
38 Australia 39 [35, 58] 78 Mexico 83 [73, 92]

39 China 43 [37, 61] 79 Turkey 77 [72, 84]

40 Thailand 40 [36, 43] 80 Bolivia 74 [68, 81]

29
 Country Median rank Country Median rank

81 Jordan 82 [75, 88] 122 Guatemala 121 [118, 123]

82 Nicaragua 83 [76, 89] 123 Syrian Arab Republic 123 [121, 126]
83 Oman 75 [65, 86] 124 Senegal 123 [117, 125]
84 Bhutan 81 [72, 87] 125 Kenya 123 [118, 126]
85 Trinidad and Tobago 85 [78, 90] 126 Rwanda 126 [122, 132]
86 Paraguay 82 [76, 87] 127 Cameroon 128 [123, 135]
87 Montenegro 95 [84, 107] 128 Tanzania 127 [125, 131]
88 Suriname 85 [76, 91] 129 Cote d'Ivoire 125 [120, 129]
89 El Salvador 89 [85, 99] 130 Pakistan 131 [125, 140]
90 Panama 86 [76, 98] 131 Gambia, The 128 [126, 131]
91 Qatar 83 [68, 93] 132 Congo, Rep. 134 [130, 141]
92 Egypt, Arab Rep. 91 [85, 96] 133 Yemen, Rep. 146 [132, 153]

93 Sri Lanka 96 [92, 106] 134 Mauritania 132 [128, 138]

94 Lebanon 95 [85, 104] 135 Ethiopia 136 [129, 143]
95 Sao Tome and Principe 97 [90, 104] 136 Mozambique 137 [133, 141]
96 Cabo Verde 96 [92, 101] 137 Comoros 137 [130, 147]
97 Philippines 97 [89, 102] 138 Guinea 140 [134, 145]
98 Saudi Arabia 93 [86, 101] 139 Zambia 141 [135, 148]
99 Gabon 100 [91, 108] 140 Uganda 137 [132, 141]
100 Mongolia 93 [88, 101] 141 Burkina Faso 136 [130, 144]
101 Turkmenistan 106 [94, 113] 142 Eswatini 150 [138, 154]
102 Indonesia 100 [96, 104] 143 Papua New Guinea 145 [140, 149]
103 Nepal 105 [100, 111] 144 Togo 142 [134, 147]
104 Ghana 101 [94, 108] 145 Burundi 152.5 [142, 161]
105 Mauritius 107 [102, 121] 146 Malawi 146 [135, 155]
106 Kuwait 102 [93, 110] 147 Sudan 147 [135, 154]
107 Honduras 106 [100, 110] 148 Djibouti 144 [136, 151]
108 Venezuela, RB 105 [96, 112] 149 Angola 145 [136, 151]
109 Belize 107 [99, 111] 150 Lesotho 150 [129, 155]
110 Myanmar 111 [105, 115] 151 Benin 146 [140, 151]
111 Lao PDR 111 [108, 114] 152 Mali 149 [146, 152]
112 Cambodia 111 [108, 114] 153 Afghanistan 154 [149, 158]
113 South Africa 117 [109, 131] 154 Niger 155 [153, 157]
114 Guyana 113 [110, 116] 155 Sierra Leone 151 [149, 155]
115 India 114 [109, 120] 156 Haiti 153 [146, 157]
116 Bangladesh 117 [114, 123] 157 Liberia 155 [154, 157]
117 Iraq 116 [110, 119] 158 Madagascar 158 [156, 159]
118 Vanuatu 116 [111, 121] 159 Nigeria 159 [152, 160]
119 Namibia 123 [114, 139] 160 Congo, Dem. Rep. 160 [159, 161]

120 Botswana 123 [116, 138] 161 Chad 160 [158, 161]
121 Zimbabwe 119 [116, 122] 162 Central African Republic 162 [162, 162]

30
GETTING IN TOUCH WITH THE EU
In person
All over the European Union there are hundreds of Europe Direct information centres. You can find the
address of the centre nearest you at: https://europa.eu/european-union/contact_en
On the phone or by email
Europe Direct is a service that answers your questions about the European Union. You can contact this
service:
- by freephone: 00 800 6 7 8 9 10 11 (certain operators may charge for these calls),
- at the following standard number: +32 22999696, or
- by electronic mail via: https://europa.eu/european-union/contact_en
FINDING INFORMATION ABOUT THE EU
Online
Information about the European Union in all the official languages of the EU is available on the Europa
website at: https://europa.eu/european-union/index_en
EU publications
You can download or order free and priced EU publications from EU Bookshop at:
https://publications.europa.eu/en/publications. Multiple copies of free publications may be obtained by
contacting Europe Direct or your local information centre (see https://europa.eu/european-
union/contact_en).
 KJ-1A-29776-EN-N

doi:10.2760/723763
ISBN: 978-92-76-08995-7