Author's personal copy

Journal of Environmental Management 91 (2010) 2301e2306

Contents lists available at ScienceDirect

Journal of Environmental Management

journal homepage: www.elsevier.com/locate/jenvman

Measuring sustainable development using a multi-criteria model: A case study

Antonio Boggia*, Carla Cortina
Department of Economics and Appraisal, University of Perugia, Italy

a r t i c l e i n f o a b s t r a c t

Article history: This paper shows how Multi-criteria Decision Analysis (MCDA) can help in a complex process such as the
Received 16 September 2009 assessment of the level of sustainability of a certain area. The paper presents the results of a study in
Received in revised form which a model for measuring sustainability was implemented to better aid public policy decisions
9 March 2010
regarding sustainability. In order to assess sustainability in specific areas, a methodological approach
Accepted 11 June 2010
based on multi-criteria analysis has been developed. The aim is to rank areas in order to understand the
Available online 13 July 2010
specific technical and/or financial support that they need to develop sustainable growth.
The case study presented is an assessment of the level of sustainability in different areas of an Italian
Keywords:
Sustainable development
Region using the MCDA approach. Our results show that MCDA is a proper approach for sustainability
Sustainability monitoring and assessment assessment. The results are easy to understand and the evaluation path is clear and transparent. This is
Multicriteria decision analysis what decision makers need for having support to their decisions. The multi-criteria model for evaluation
has been developed respecting the sustainable development economic theory, so that final results can
have a clear meaning in terms of sustainability.
Ó 2010 Elsevier Ltd. All rights reserved.

1. Introduction particularly emphasizes the multidimensionality of the assessment

process.” (Fusco Girard, 1993). A multidimensional evaluation
Sustainability is a multidimensional concept: economic, social process is based on the CSV which includes all the heterogeneous
and environmental aspects must be considered and integrated values coexisting in a resource. Therefore, the assessment is no
(Pope et al., 2004). The appropriate instrument for a multidimen- longer based on a single monetary indicator but on a set of indi-
sional representation is a suitable set of indicators that must be an cators, some of which are economic and others non-monetary. The
integral part of an assessment methodology to be used for the concept of incommensurability of values, considered a foundation
purposes of measuring sustainability (Ness et al., 2007; Moffat stone for ecological economics, is taken into account (Martinez-
et al., 2001). Improvements in the way the indicators are con- Alier et al., 1998). This is why multi-criteria assessment method-
structed and used are very important research issues (Munda and ologies assume a central role in the multidimensional evaluation
Nardo, 2009). process. Multi-criteria analyses are used to solve complex problems
The ecological economics approach lends itself particularly well by assessing all the variables, both individually and collectively,
to the interpretation and assessment of the many aspects involved assigning specific importance to each variable. Thus, multi-criteria
(Rees, 2003). The ecological economics approach, more complex methodologies were widely used to evaluate sustainability (Liu,
than the neoclassical model, is based on complexity 2007; Shmelev and Labajos-Rodrigues, 2009).
(Venkatachalam, 2007; Christensen, 1991; Costanza et al., 1991). In The aim of this study is to define an integrated methodology of
this context assessment procedures must be able to interpret and assessment and monitoring dealing with the integration of social,
evaluate many different aspects. It is necessary to link economic environmental and economic aspects involved in territorial
evaluations with biological, ecological and social evaluations as planning.
Complex Social Value does in its evaluation of public good. The The case study refers to Umbria, a region in central Italy. Taking
Complex Social Value (CSV) represents a paradigm alternative into account the intervention priorities described in a planning
to the Total Economic Value (TEV) but has been applied less. document of the Umbria Region (Preliminary Strategic Document
Milan Zeleny (1993), Peter Nijkamp and Henk Voogd (1989), Luigi 2007e2013), a set of environmental and socioeconomic indicators
Fusco Girard (1993) have developed the CSV analysis. “The CSV was determined. This set of indicators reflects sustainability on
a territorial level (in this case at municipality level); indicators are
used in a multi-criteria analysis model. Sustainable development is
* Corresponding author. considered achievable if it originates on the local level; a bottom-up
E-mail addresses: boggia@unipg.it (A. Boggia), ccortina@unipg.it (C. Cortina). approach from local to supra-national (Ravetz, 2000). Since multi-

0301-4797/$ e see front matter Ó 2010 Elsevier Ltd. All rights reserved.
doi:10.1016/j.jenvman.2010.06.009
 Author's personal copy

2302 A. Boggia, C. Cortina / Journal of Environmental Management 91 (2010) 2301e2306

criteria evaluation is multidimensional in nature, it allows us to a collaborative process, based on communication and participation
take into account economyeenvironment interactions. Weak or (Beratan et al., 2004).
strong sustainability concepts depend on the degree of compen- A sensitivity analysis of weights can be made inside the model,
sability allowed by the aggregation procedure (Martinez-Alier for example using different weighing profiles representing
et al., 1998). In this case the degree of compensability allowed is different political views of the sustainability issue, so that decision
quite low because the set of indicators was aggregated separately in makers can better understand the consequences of their decisions,
order to obtain two indexes at the territorial level: the environ- and which of the selected criteria can affect final results more.
mental index (EI) and the socioeconomic index (SE). The last step is The weighted sum methodology was applied to the data in order
the integrated interpretation, crossing the indexes, but not aggre- to obtain a classification of the alternatives.
gating them. This approach is based on the calculation of an overall value for
each alternative, obtained first by multiplying each value of the
2. Methodology attributes by its own weight and then by adding up the weighted
values of every alternative. Thus, the best alternative is obtained
Multi-criteria methods have been used in the past to evaluate as in (2):
sustainability at municipal level (Ferrarini et al., 2001). In our case,
the methodology is to be sought among the discrete quantitative max X
J


wj zji (2)
methods. These methods have the purpose of supplying informa- i ¼ 1; .; I
j¼1
tion on the ability of the alternatives to simultaneously reach the
objectives represented by the indicators used in the analysis. The final result of the analysis is a ranking of the territorial units;
The two basic steps, of a multi-criteria analysis, are: in this case municipalities.
A Geographic Information System (GIS) interface was applied to
1. the construction of an effects table X, with dimensions J
I, the multi-criteria model. The advantages of this application are
considering i (i ¼ 1, ., I) alternatives and j (j ¼ 1, ., J) indicators better management of aggregated data on the territorial level and
(criteria). For example, the value assumed by criterion j for each the possibility to identify new indicators using data contained in
of the I alternatives is represented by zji, zji0 , ., zjI. GIS data bases, which can easily be updated. Another advantage is
2. the implementation of a system of weights wj (j ¼ 1, ., J), that the possibility to get as an output the spatial representation of
supplies information on the importance attributed to the results.
different criteria, represented in (1). Social and economic dimensions have been analyzed together,
so that two procedures were developed separately: one for the

 socioeconomic profile and the second for the environmental
W ¼ w1 ; .; wj (1)
profile. In this way an index of socioeconomic performance and an
In the sustainability assessment model the set of the economic, index of environmental performance are separately obtained.
social and environmental indicators are the criteria used for the Finally, the socioeconomic data and the environmental data are
assessment (Lawn, 2006); the municipalities are the alternatives. crossed, which permit one to observe the level of sustainability for
The territorial analysis can be made on the local level as well as on each alternative (municipality).
the regional or national level.
In our case, the effects table represents the behavior of each 3. The set of indicators
territory towards the criteria used. Since the value of each criterion
in the matrix has its own specific unit of measurement, a stan- For years, EUROSTAT, the OECD, the World Bank and the Euro-
dardization process is necessary. In this way the transformation of pean Environment Agency have been developing environmental
the criteria into attribute values is obtained. and socioeconomic indicators that are shared with the scientific
The problem of standardization was confronted using two community. However, the indicators formulated by official sources
equations, one linear increasing and one linear decreasing. There- on a national scale rarely descend to a more detailed territorial
fore, all the values of the matrix are transformed into numbers reference, necessary to carry out an analysis at the local level.
between 0 and 1 with 1 considered the most desired value. Therefore, the individual local communities have the autonomy in
After careful evaluation, the next step is to establish a weight choosing the indicators that are most suitable for their situation.
for each attribute by constructing a vector of weights e W. To Indicators used to monitor areas of concerns must be sensitive to
attribute weights the SWING (von Winterfeldt and Edwards, 1986) management actions, therefore must be related to the critical issues
method has been used. A group of decision makers at regional at local level (Cornforth, 1999). In this way they can better represent
level was asked to consider an alternative (not real) having all the their own specific environmental and socioeconomic situation.
attributes at the worst level. The second step is to choose the first Hence the need to identify indicators at a local level; these indicators
attribute that the decision maker would like to move to its best must comply as much as possible with the three requisites estab-
level. To this attribute 100 points are assigned. The same procedure lished by the OECD that are also applied at the international level:
is continued considering the second most desirable attribute policy relevance, analytical soundness, measurability (OECD, 1993).
improvement, assigning points less than 100. The procedure is The following set of indicators has been determined at munic-
continued with all the attributes. At the end of the procedure the ipality level:
given points are normalized to sum up to one (Pöyhönen and
Hämäläinen, 2001). A. Environmental indicators
The guiding criterion for the decision makers in the SWING 1. CO2 emissions: total CO2 emissions;
process has been that of giving priority to those issues, both those 2. Artificial surface areas: urbanized areas of a municipality in
regarding the environment and socioeconomic ones, that at the relation to the total area;
present moment are most debated at the local level, but also at the 3. Fragmentation index: level of fragmentation of a territory due
international level, and that, therefore, represent the most critical to infrastructures and urbanization;
points for which answers must be found in a short time. This goal 4. Electric power use: consumption of electricity for domestic
could be better achieved if decision makers are able to implement use per consumer;
 Author's personal copy

A. Boggia, C. Cortina / Journal of Environmental Management 91 (2010) 2301e2306 2303

5. Waste separation: percentage of differentiated waste environmental, using a quantitative multi-criteria methodology.
collection; Nevertheless considering all indicators with just one index of
6. Drinking water use: amount of water used per capita per sustainability instead of analyzing the two separate indexes, may
municipality; result in a loss of information. This is precisely what happened in
7. Total potential loads: it is an index made up of four indicators the UN Dashboard of Sustainability, created in the framework of
in relation to the unit of surface area of the single munici- the United Nations Commission on Sustainable Development
palities (COD, BOD, N and P); (UNCSD), subsequently improved by a small group of researchers
8. Certified firms: the ratio between the number of companies led by the International Institute for Sustainable Development
with an environmental certification (ISO 14001 and EMAS) (Canada), and presented at the World Summit in Johannesburg in
and the other companies; 2002. The Dashboard of Sustainability combining the economic,
9. Certified public institutions: registered environmental social and environmental aspects provided a picture of the level of
management processes for municipality governments sustainable development at national, regional, provincial and
(Agenda 21, environmental accounting, environmental municipal levels. In this way it is possible to obtain an overall
certification); index of sustainability called the ESI (Environmental Sustainability
B. Socioeconomic indicators Index). Due to extreme aggregation of the information, the inter-
1. Population density: the ratio between the resident pop- pretation and the understanding of the results is very difficult; as
ulation and the territorial surface area for each municipality; a matter of fact the “Dashboard” has been defined a “black box”.
2. Unemployment rate: the percentage of the population Because there is no way of knowing exactly what happens inside
seeking jobs out of the total active population; the box: in other words how the data is processed. Our model on
3. Women’s unemployment rate: the percentage of women the other hand is completely transparent.
seeking jobs out of the total active population; In the WCED report on “Our Common Future” (1987) or Brundt-
4. Work-related accident: the ratio of injured people to those land Report, sustainable development is defined as “development
employed; which meets the needs of the present without compromising the
5. Index of higher education: the ratio of people over 17 with ability of future generations to meet their own needs”. Those who
high school or university diploma to the total population have studied this issue have concluded that sustainable and lasting
over 17; development is possible only if the planning, organization and
6. Index of tourist attraction: the ratio of tourists to the total management of resources are based not only on the traditional
resident population; factors of “fixed capital” and “work” but also on natural capital. In
7. Index of demographic dependence: the ratio between the formulating his economic theory of sustainable development, Her-
population from 0 to 14 years of age plus the population over man Daly, pointed out that there are two principles of sustainability
65 years of age and the population from 15 to 64 years of age; in the management of resources (Daly, 1996):
8. Active businesses: the ratio between the number of active
businesses and the residents; 1. the speed at which the resources are used up must be equal to
9. Available income: expressed in thousands of euros per their capacity to be regenerated;
inhabitant, measures the income that families are able to 2. the speed of production of waste must be equal to the capacity
spend after paying taxes. of absorption by the ecosystems into which the waste is put.

The data used for this case study refer to only one year, but the The capacity to regenerate and absorb must be treated as
model is dynamic because it can be updated, data series can be used so “natural capital”: if this capacity cannot be maintained, the capital
that trends can be understood and new indicators can be introduced. will be used up and sustainability is no longer possible.
Daly himself added that there were two ways of keeping total
4. Sustainability analysis capital intact:

Having identified the set of socioeconomic and environmental 1. Weak sustainability: this means keeping the sum “natural
indicators already mentioned for each municipality in Umbria, the capital þ capital produced by man” at a constant value. The
following phases are listed below: material capital and the natural capital can replace each other.
A “welfare package” consisting of a constant sum of material
1. quantification of environmental indicators ia1, ia2, ., ian; capital plus natural capital must be handed down to the future
2. quantification of socioeconomic indicators ise1, ise2, ., isen; generations.
3. aggregation of the environmental indicators into an environ- 2. Strong sustainability: this means keeping each component at
mental performance index EI, for each area studied, using a constant value. The material capital and the natural capital
a quantitative multi-criteria methodology; are not interchangeable. Both capitals must be maintained
4. aggregation of the socioeconomic indicators into a socioeco- intact, since the productivity of one depends on the availability
nomic performance index SE, for each area studied, using of the other. Replacing the natural capital with material capital
a quantitative multi-criteria methodology; is only possible to a limited extent. The earth and its resources
5. ranking of the municipalities, in relation to the environmental are assigned to each generation as trustees, and each genera-
index; tion has a duty to leave an “intact” nature (constant natural
6. ranking of the municipalities, in relation to the socioeconomic capital) to the next generation, whatever the level of well-
index; being reached may be.
7. crossing the environmental with the socioeconomic results and
classification of local areas in relation to the level of sustain- Aggregating environmental and socioeconomic information in
ability achieved. a sustainability index, obtained as the result of a multi-criteria
analysis, is very close to the concept of weak sustainability, due to
To develop step 7, a classification of local areas could be the compensation that is generated between the socioeconomic
obtained by aggregating all the indicators, socioeconomic and and the environmental aspects. That’s why we did not aggregate
 Author's personal copy

2304 A. Boggia, C. Cortina / Journal of Environmental Management 91 (2010) 2301e2306

together environmental and socioeconomic data. The contrast total capital must be preserved for the future generations, without
between strong and weak sustainability is under discussion replacing one component with another.
(Cabeza Gutés, 1996; Neumayer, 2004), but the concept of strong The aim of our sustainability analysis is to offer an integrated but
sustainability in our opinion is closer to true sustainability (van den not aggregated synthesis of the results obtained at the level of the
Bergh, 2007). Although social, economic and environmental single indexes (environmental and socioeconomic) so as to identify
aspects are integrated, they must retain their own independence. homogeneous local areas in Umbria on the basis of the indicators
Thus, the final assessment approach is based on crossing the values adopted. The areas illustrated below are the result of crossing the
of the environmental indexes with those of the socioeconomic two EI classes and the two SE classes (Fig. 1). In this evaluation the
indexes. Interpreting them after crossing makes it possible to principle of sustainability is understood as the search for a balance
investigate where a municipality with a given sustainability soci- between socioeconomic development and management of the
oeconomical score is ranked in terms of environmental sustain- environmental resources. The areas that are formed are therefore
ability, and vice versa. aggregations on the basis of which it is possible to record infor-
mation. We can observe situations of “balance” or of “prevalence”
5. Results and discussion of one aspect over another. The map of the Umbria region in Fig. 2
shows to which of the four classes identified each of the munici-
In Fig. 1 the crossing of the EI index and SE index for each palities belongs.
municipality is shown. Area A (SE class 1 and EI class 2) includes the municipalities with
This was done by plotting an axis parallel to the x axis and one a lower than average SE index and a higher than average EI index.
parallel to the y axis at the mean value of the SE index and the These municipalities have an unbalanced development in which
EI index respectively. In this way, the result consists of a diagram the environmental component prevails over the socioeconomic
showing on the x-axis two classes of the socioeconomic index component. There are 21 municipalities in this area, accounting for
(above and below the mean) and on the x-axis two classes of the about 13% of the surface area of the region and about 8% of its
environmental index (above and below the mean) respectively, on population.
which the municipalities are plotted. The position of the munici- Area B (SE class 2 and EI class 2) includes those municipalities
pality in one or another of the four quadrants indicates the level of with a higher than average SE index and a higher than average EI
sustainability. Since the values of the environmental index and of index. These are the municipalities in which the environmental and
the socioeconomic index are between 0 and 1, there are no negative socioeconomic components are not only evenly distributed but also
values. Furthermore from this allocation on the graph it is obvious well developed. This situation should summarize the results to be
that one can trace back both to the index and then to the basic data. aimed at in seeking to achieve development defined as sustainable.
The values of the EI index and the SE index that each munici- This group includes 38% of the municipalities in the Umbria region.
pality have obtained are linked to the level of sustainability ach- The overall surface area involved amounts to approximately 46% of
ieved, according to the goal set by this study. The key to understand the total while the population involved is about 50% of the whole
this goal is precisely the concept of strong sustainability mentioned population of the region.
above. According to this concept: the natural capital and the capital Area C (SE class 2 and EI class 1) includes the municipalities with
produced by man are not interchangeable and, therefore, even if a higher than average SE index and a lower than average EI index.
the results obtained from the environmental point of view are These are municipalities in which development is unbalanced, and
excellent, one does not move in the direction of sustainability if the where the socioeconomic component prevails over the environ-
socioeconomic performance levels are poor, and vice versa. The mental component. 16 municipalities fall within this area. Together

A B

D C

Fig. 1. Crossing environmental and socioeconomic indexes.

 Author's personal copy

A. Boggia, C. Cortina / Journal of Environmental Management 91 (2010) 2301e2306 2305

Fig. 2. Spatial representation of results.

they account for about 21% of the surface area and about 32% of the that some kind of information is associated with each object in the
population of the Umbria region. universe, expressed using suitable attributes that describe the
Area D (SE class 1 and EI class 1) includes the municipalities objects considered. Objects characterized by the same description
with a lower than average SE index and a lower than average EI are indiscernible with reference to the available information. The
index. They are the municipalities in which the environmental and relationship of indiscernibility generated in this way constitutes the
socioeconomic components are equally distributed but not highly mathematical basis of the rough-set theory. Greco et al. (1996)
developed. 20 municipalities fall within this area, accounting for proposed a new rough-set approach for multi-criteria ranking
20% of the surface area and about 9% of the regional population. problems in which, unlike the original approach, the approxima-
The model is being studied and improved constantly. The next tions are built up using relationships of dominance rather than
step will be a review of the multi-criteria methodology. In this indiscernibility (Dominance-Based Rough Set Approach, DRSA).
respect, implementation of the methodology according to the This enables the ordinal properties of the attributes (criteria)
Dominance-based Rough Set Approach (DRSA) (Slowinski et al., considered to be taken into account explicitly.
2009) is currently under way. This methodology will enable the
difficult phase of attribution of weights to be avoided, and will 6. Conclusion
enable results to be obtained from the description and evaluation of
a “desired” example proposed by the decision maker. The rough-set Definition of the sustainability diagram enables the 92 munici-
theory introduced by Pawlak (1982) is based on the assumption palities of the Umbria region to be positioned according to a neutral
 Author's personal copy

2306 A. Boggia, C. Cortina / Journal of Environmental Management 91 (2010) 2301e2306

distribution, defined by a system of multiple variables. Our results References

show that the diagram enables the existing dynamics to be read.
Critical aspects, emergencies and the potential of the local area can Beratan, K.K., Kabala, S.J., Loveless, S.M., Martin, P.J.S., Spyke, N.P., 2004. Sustain-
ability indicators as a communicative tool: building bridges in Pennsylvania.
also be identified by the decision maker. The model is also suitable Environmental Monitoring and Assessment 94, 179e191.
for simulating alternative scenarios for the purpose of evaluating Cabeza Gutés, M., 1996. The concept of weak sustainability. Ecological Economics 17,
the effects of activities aimed at improving critical environmental 147e156.
Costanza, R., Daly, H.E., Bartholomew, J.A., 1991. Goals, agenda and policy recom-
and/or socioeconomic situations highlighted thanks to the calcu- mendations for ecological economics. In: Costanza, R. (Ed.), Ecological
lation of the single indicators. Economics: the Science and Management of Sustainability. Columbia University
Our approach does not allow for interchangeability between Press, New York.
Christensen, P., 1991. Driving forces, increasing returns and ecological sustainability.
natural capital and that produced by man, and does not permit the In: Costanza, R. (Ed.), Ecological Economics: the Science and Management of
establishment of a “black box” type situation, because never in the Sustainability. Columbia University Press, New York.
calculations environmental indexes are aggregated with the Cornforth, I.S., 1999. Selecting indicators for assessing sustainable land manage-
ment. Journal of Environmental Management 56, 173e179.
socioeconomic ones.
Daly, H.E., 1996. Beyond Growth: The Economics of Sustainable Development.
Since more compensation among sustainability dimensions is Beacon Press, Boston.
acceptable in the case of weak sustainability, and less compensa- Ferrarini, A., Bodini, A., Becchi, M., 2001. Environmental quality and sustainability in
the province of Reggio Emilia (Italy): using multi-criteria analysis to assess and
tion among various sustainability dimensions is possible in the case
compare municipal performance. Journal of Environmental Management 63,
of strong sustainability (Martinez-Alier et al., 1998), our approach 117e131.
tends to strong sustainability, although even social and economic Fusco Girard, L. (Ed.), 1993. Estimo ed economia ambientale: le nuove frontiere nel
indicators should be separated. The final goal of our model is the campo della valutazione. Franco Angeli, Milano.
Greco, S., Matarazzo, B., Slowinski, R., 1996. Rough approximation of a preference
assessment of sustainability under the strong approach, and further relation by dominance relations (ICS Research report 16/96, Warsaw University
development of our research will be towards the complete of Technology).
achievement of this objective. However, the model already over- Lawn, P., 2006. Sustainable development: concept and indicators. In: Lawn, P. (Ed.),
Sustainable Development Indicators in Ecological Economics. Edward Elgar,
comes the limits of the “Dashboard of Sustainability” approach, Cornwall, UK.
becoming a “glass box”. Liu, K.F.R., 2007. Evaluating environmental sustainability: an integration of
The Regional Board of Umbria received our model well, and is multiple-criteria decision-making and fuzzy logic. Environmental Management
39, 721e736.
starting to use it for resource allocation and funding in EU Martinez-Alier, Munda, O’Neil, 1998. Weak comparability of values as a foundation
programs. The model is being applied to the existing monitoring for ecological economics. Ecological Economics 26, 277e286.
system of sustainability of the EU policies. The attention is partic- Moffat, I., Hanley, N., Wilson, M.D., 2001. Measuring and Modelling Sustainable
Development. The Parthenon Publishing Group, Bristol, UK.
ularly focused on the Regional Operative Plan and the Social
Munda, G., Nardo, M., 2009. Noncompensatory/nonlinear composite indicators
European Fund, for the present implementation period. for ranking countries: a defensible setting. Applied Economics 41 (12), 1513e1523.
In this way the guiding principle of the “Policy Coherence and Ness, B., Urbel-Piirsalu, E., Anderberg, S., Olsson, L., 2007. Categorising tools for
sustainability assessment. Ecological Economics 60, 498e508.
Governance” contained in the EU Sustainable Development
Neumayer, E., 2004. Weak Versus Strong Sustainability. Edward Elgar, Cheltenham,
Renewed Strategy (EU Council, 10917/06) is taken into account. The UK.
Strategy underlines the important role of local and regional levels Nijkamp, P., Voogd, H., 1989. In: Fusco Girard, L. (Ed.), Conservazione e sviluppo: la
in delivering sustainable development and building up social valutazione nella pianificazione fisica. Franco Angeli, Milano.
OECD, 1993. Environmental Indicators for Environmental Performance Reviews.
capital, to create a high quality of life. Our model can be used as well OECD, Paris.
for guiding the process of approaches like Local Agenda 21 and Pawlak, Z., 1982. Rough sets. International Journal of Information & Computer
other processes with broad public participation. The Regional Board Sciences 11.
Pope, G., Annandale, D., Morrison-Saunders, A., 2004. Conceptualising sustainability
of Umbria promotes coherence between regional actions and assessment. Environmental Impact Assessment Review 24, 595e616.
European Union policies in order to improve sustainable Pöyhönen, M., Hämäläinen, R.P., 2001. On the convergence of multiattribute
development. weighting methods. European Journal of Operational Research 129, 569e585.
Ravetz, J., 2000. Integrated assessment for sustainability appraisal in cities and
In any case, the results, and therefore the measure of sustain- regions. Environmental Impact Assessment Review 20, 31e64.
ability, depend to a great extent on the indicators used. The choice Rees, W.E., 2003. Economic development and environmental protection: an
of indicators and the local-area reference scale are functional to ecological economics perspective. Environmental Monitoring and Assessment
86, 29e45.
the goals. The fact that the indicators can be updated makes our
Shmelev, S.E., Labajos-Rodrigues, B., 2009. Dynamic multicriteria assessment of
model dynamic and suitable for use also as a tool for periodic macro sustainability: case study of Austria. Ecological Economics 68 (10),
evaluation (Shmelev and Labajos-Rodrigues, 2009). It remains to 2560e2573.
Slowinski, R., Greco, S., Matarazzo, B., 2009. Rough sets in decision making. In:
be understood whether the possibility of changing the indicators
Meyers, R.A. (Ed.), Encyclopedia of Complexity and Systems Science. Springer,
on the basis of one’s goals is preferable to a standard set of indi- New York.
cators that could possibly be developed at least at European level. van den Bergh, J., 2007. Sustainable development in ecological economics. In:
This would definitely be a less flexible set, but capable of Atkinson, J., Dietz, S., Neumayer, E. (Eds.), Handbook of Sustainable Develop-
ment. Edward Elgar, Cheltenham, UK.
comparing distant and different situations with one another, for Venkatachalam, L., 2007. Environmental economics and ecological economics:
a more effective allocation of resources in the framework of where they can converge? Ecological Economics 61, 550e558.
sustainable development policies. For this latter aim, the assess- von Winterfeldt, D., Edwards, W., 1986. Decision Analysis and Behavioral Research.
Cambridge University Press, Cambridge.
ment takes on a central role: an effort is required starting from the WCED, 1987. Our Common Future (The Bruntland Report). Oxford University Press.
world of research, together with the specialized organizations and Zeleny, M., 1993. Alla ricerca di un equilibrio cognitivo: bellezza, qualità, armonia.
operators, in order to make the assessment of sustainability an In: Fusco Girard, L. (Ed.), Estimo ed economia ambientale: le nuove frontiere nel
campo della valutazione. Franco Angeli, Milano.
organized process.