An international comparison of the outcomes of environmental regulation

There is much discussion on the stringency of environmental regulations and the variability of industrial environmental performance in different countries. Industrial groups frequently claim that regulatory standards for the minimum levels of industrial environmental performance are too high and that this impacts on their competitiveness (cf Ottewell 2012), whilst environmental groups frequently claim that regulatory standards and the associated levels of industrial environmental performance are too low and that this impacts on public health and environmental quality (cf Friends of the Earth 2013). These debates often have a significant political impact—calls for greater levels of environmental protection regularly run into concerns about the impacts of regulation on employment, innovation and competitiveness. These concerns become particularly acute where globalization and liberalization mean that investment and industrial activity can more easily move towards less heavily regulated countries.

Holzinger et al (2011) argue that in recent decades there has been a convergence in the environmental standards being adopted by policy makers in developed economies. This convergence could be driven by the internationalization of environmental policy with countries agreeing to adopt similar international standards (Busch and Jörgens 2005), by policy learning and the diffusion of policy instruments and approaches from country to country (Jörgens 2004), or by the diffusion of private standards and new environmental technologies and management practices (Dolowitz and Marsh 1996). From an environmental perspective, it is hoped that there is convergence around higher environmental standards, with countries gradually adopting higher standards over time, but there are also concerns that the competitive implications of environmental policy are leading to a 'race to the bottom' and convergence around lower standards (Holzinger and Sommerer 2011).

Based on the results of large scale surveys of expert opinion, Holzinger et al (2011) argue that there has been convergence around an upward trend in environmental standards, and that there has been mobility in international rankings with some laggard countries which had lower environmental standards catching up with and sometimes over-taking the leaders that historically had the higher environmental standards. Over time, the EU has moved from a laggard to a leader and the above argument is often put forward in the EU, where the received wisdom is that the EU has become a leader on environmental protection and where concerns are frequently expressed that EU industry has been put into a competitive disadvantage by over-regulation (Scheelhaase et al 2010).

Despite the widespread importance of the debate on the stringency of environmental regulations and the variability of industrial environmental performance, there are very few robust evaluations that allow meaningful comparisons of the actual outcomes of environmental regulations in different countries to be made. One of the reasons for this is that data scarcity commonly restricts the ability to make 'like for like' comparisons of regulatory outcomes and industrial performance across countries and over time. Different data sets are available in different countries, there is a lack of consistent, continuous data over time, and it is hard to generate normalized measures of performance that enable meaningful comparisons to be made (see Gouldson and Sullivan 2007).

However, such evaluations of regulatory outcomes and industrial performance are critically important. Whilst we might expect regulatory standards to impact directly on industrial performance, it is all too evident that governments can adopt tough regulatory standards on paper and then fail to translate them into practice (see Gouldson and Murphy 1997, Jordan 1999). For standards to impact on environmental performance, and then on environmental quality and public health, they have to be implemented and enforced, and regulated companies need to comply with them. Whilst some companies can be expected to go beyond compliance (Gunningham et al 2004), for example when there are economic reasons for them to do so, when implemented and enforced we can expect regulatory standards to set a minimum level for industrial environmental performance. These levels of performance therefore reflect important 'regulatory realities', but, surprisingly, they are very rarely evaluated, particularly through international comparative research (Gouldson and Murphy 1997).

In this paper, we seek to address this gap by analyzing emissions of benzene from oil refineries in various Organisation for Economic Cooperation and Development (OECD) countries. We adopt this focus for a number of reasons. Refineries have been present in a relatively common form in many countries for many years, and the basic activities of refineries generate benzene emissions that have been heavily regulated in many countries since the 1970s. There is therefore a meaningful basis for both international and historical comparison. To conduct such an analysis, we require emissions data for a large number of regulated sites in a range of countries over a long period of time. Such emissions data is available through Pollution Release and Transfer Registers (PRTR). In order to develop normalized measures of performance that take account of varying scales of production, we also require site-specific data on the productive capacities of each site. This data is available for refineries through various publically available data sets on the energy industry. Whilst there is a basis for comparison and access to data for this particular sector and pollutant, extensive searches suggested that there are very few other sectors and pollutants where these requirements would have been met. Whilst of course we are aware that we cannot necessarily draw wider conclusions from such a focused analysis, we argue that the results could offer some rare insights that could be of great relevance in widespread and politically charged debates that are often characterized by a lack of evidence.

2.1. Scope and focus

2.2. Data sources, methods and assumptions

The main findings of the analysis are presented in figure 1. As can be seen, there is a very significant variation in the average normalized emissions levels in the different countries. Japan is by far the best performing country with average normalized emissions of no more than 0.016 kg bcd⁻¹ in any year, stable levels of average performance over time and a worst performing refinery emitting 0.072 kg bcd⁻¹ in 2010. Germany is the second best performing country overall with average normalized emissions of 0.018 kg bcd⁻¹ in 2010, with performance having improved from 0.085 in 2001 and 0.044 in 2007. The worst performing German refinery in 2010 emitted 0.041 kg bcd⁻¹. In 2010, Japan and Germany achieved average normalized emissions levels that were less than half those of any other country. Australia is the third best performing country, with average normalized emissions improving from 0.165 kg bcd⁻¹ in 2001 to 0.050 kg bcd⁻¹ in 2010. The worst Australian refinery in 2010 emitted 0.141 kg bcd⁻¹. The US is the fourth best performing country, with steady falls in average emissions per unit of capacity from 0.209 kg bcd⁻¹ in 1990 to 0.041 kg bcd⁻¹ in 2010. The worst performing US refinery in 2010 emitted 0.404 kg bcd⁻¹. Ranked seventh, eighth and ninth respectively are Canada, the EU-15 and the UK for 2010. These had average normalized emissions levels that were approximately 7, 8 and 14 times higher than those of Japan and Germany. The UK has the worst emissions value for a single refinery within the EU-15 for all years except 2001 (when a Swedish refinery had an emissions level of 1.736 kg bcd⁻¹). In 2010 the worst performing UK refinery emitted 1.502 kg bcd⁻¹. Table 2 presents these results in more detail (see also figure 3.2 in supplementary material).

Zoom In Zoom Out Reset image size

Overall it is clear that the best performing refineries are in Japan and the worst are in the UK—average normalized benzene emissions from UK refineries were approximately 37 times those of Japanese refineries in 2010 and the worst UK refinery in 2010 emitted almost 21 times more benzene per unit of capacity than the worst Japanese refinery. While Japan and more recently Germany stand out for their good performance, the UK stands out for its poor performance, helping to ensure that average normalized emissions rates for the EU-15 were three times those of the US in 2010, which in turn were 3.5 times those of Japan.

Clearly our results depend on both the reliability of the PRTR data underpinning the analysis. As stated above, PRTR data is collected through monitoring protocols approved by regulatory agencies, and any misreporting of data would normally trigger legal sanctions. It can therefore be seen to be quite robust, and it is certainly the best publically available data on industrial emissions, but some potential for variations in the scope and quality of the data remains.

Thereafter, a key question then emerges about the extent to which we can attribute any variations in industrial environmental performance to differences in the regulatory standards applied in different countries. If regulation sets minimum standards that companies generally comply with, then lower performance can only be possible with lower regulatory standards.

It is important to note though that higher performance above these minimum standards need not be because of higher regulatory standards—it could be because various other factors lead to 'beyond compliance' behaviours (Gunningham et al 2004). Such behaviours could be adopted because of management cultures and practices, or because of market pressures and technological opportunities. They may also emerge because of private regulations and voluntary codes and standards, or because of social regulation and civic or NGO pressure.

Whilst this could mean that some of the variations in industrial performance are the result of these non-regulatory pressures, we focus on a globalized industry that is populated by large multi-national corporations, and we consider variations in performance at the country rather than company level and only in OECD countries with similar levels of development. As a result, we suggest that these non-regulatory pressures are likely to be broadly similar across all of the countries included in the study. We therefore posit that there is likely to be a direct and observable relationship between industrial performance and regulatory standards, but we recognise that this is an aspect of the research that warrants further investigation in the future.

If we accept these points, then our findings are significant as they reveal significant variations in levels of emissions of a carcinogenic air pollutant from a widespread industrial sector. They are of particular relevance for the regulators and operators of refineries, and for the communities that live near to them. More broadly, of course we recognise the need to be cautious in the extent to which we draw any wider conclusions from a focused analysis. However, we argue that the insights generated could be of great relevance in widespread and politically charged debates about comparative standards in environmental regulation that are often characterized by a lack of evidence, and that they demonstrate the value of such comparative analyses and the need for further research with a similar approach.

A first broader observation is that the results suggest that at least in this case there has been a widespread but not universal convergence in levels of industrial environmental performance in different countries in recent years. Holzinger et al (2011) have argued that globalization is leading to a convergence in environmental regulations at a broad scale, and that cross-national diffusion of innovations in environmental policy is widespread, but that there is a lack of systematic knowledge as to whether this results in greater homogeneity of national environmental policies in the long run. Our study does not analyse the design or implementation of national environmental policies, but it does suggest that the outcomes of these policies may be converging, at least in the case of benzene emissions from refineries where we see strong evidence of convergence in emissions performance in recent years.

A second but closely related finding relates to mobility, i.e. the extent to which convergence also coincides with 'processes of catching-up and changing rank between leaders and laggards' (Holzinger et al 2011). Again, we see clear patterns emerging in our study, with most countries exhibiting steady performance over time, most notably the US, but also France and Australia. Only Germany, however, has so far managed to catch up with the leader, namely Japan. Canada also managed to overcome its laggard position and catch up and even surpass countries like the US and Australia. Since 2006, however, Canada appears to have been unable to keep pace and fell behind, but closer inspection of the data suggests this is due to increased emissions in one refinery. In addition, there is also evidence of 'worst first' regulation being enacted, i.e. bringing worst performing oil refineries into line with the best. This is the case in all the countries under study, apart from the UK and Canada (see figure 1).

A third finding is that despite some level of convergence and catching up, the results confirm that considerable variability can still exist in the outcomes of environmental regulation across different countries. Japan exhibits the best performance by a substantial margin, while Germany ranks second best and is steadily converging with Japan. The US shows a linear improvement over the years, with Australia, France, Italy and Canada converging over time with US standards; the latter though only until 2006. Overall, the results suggest that oil refinery benzene emissions in the UK are substantially higher than in other countries, that emissions across the EU-15 and Canada are twice the average of the US and Australia, which are in turn twice the average of Japan.

Relatedly, a fourth major—but rather unexpected finding—relates to the EU-15 being the main outlier, which contradicts the prevailing rhetoric of it having assumed since the 1990s the place of the US as a domestic regulatory 'hegemon' (Bach and Newman 2007: 828). Indeed, the EU is often seen as a far more 'innovative, aggressive and successful regulator' compared to the US (Rosenbaum 2007: 146). Domestic environmental policy in Europe has over time evolved into one of the most rapidly expanding areas of EU activity, with its legislative corpus being currently considered amongst the most advanced and progressive worldwide in a range of areas, from greenhouse gas emissions trading to recycling, biosafety and eco-labelling (Falkner 2007). In addition to advertising its environmental leadership credentials, the EU has even engaged in efforts to 'globalize' its environmental regulation by championing international agreements that would result in other jurisdictions adopting environmental regulations of a comparative nature (Kelemen and Vogel 2010). But our results reveal that there are some instances at least where the rhetoric does not match the reality—the outcomes of environmental regulation can be much lower in the EU than they are in other countries. Such an observation that the EU's rhetoric does not match the reality has been repeatedly made in the wider literature (cf Bretherton and Vogler 2006, Falkner 2007).

A final major and again closely related finding is that EU performance is particularly skewed by the exceptionally poor performance of the UK. Closer inspection of UK refinery performance reveals that in 2009/10 three of the nine refineries had normalized levels of performance that were close to the EU average, two emitted approximately 2 times the EU average, two emitted from 3 to 4 times the EU average and two emitted from 6.7 to 9.5 times the EU average (see figure 2). If we assume that there are no significant differences in levels of 'beyond compliance' behaviour in different countries, then this must mean that the minimum regulatory standards adopted for the majority of the refineries in the UK are substantially lower than those adopted elsewhere.

Zoom In Zoom Out Reset image size

In seeking to understand the reasons why normalized emissions levels from UK refineries are higher on average (and especially in some instances) than in other EU countries, we conducted a small number of expert interviews with regulators, industry and pressure groups² . The responses of the different interviewees were quite consistent, with each respondent suggesting that the results could be the consequence of three interacting issues. The first is that there may be some older refineries in the UK that have not been upgraded for some time and that have a legacy of under-investment in both process and pollution control technologies. The second is that regulators in the UK may have greater discretion in the way that they interpret and apply regulations, and that in exercising this discretion they may give greater emphasis to economic concerns than in other European contexts. The third is that given the first and second points, UK regulators may pay particular attention to ambient environmental quality standards when setting emissions limits, and only require substantial investment in upgrading industrial facilities if, for example, levels of benzene in local air quality exceed target levels as set out for example in the EU's air quality framework directive. Collectively, these issues could mean that higher emissions levels are permitted from refineries that are operating in contexts (i.e. on the coast where prevailing winds disperse emissions) where air quality limits are not exceeded, or at least where air quality monitoring does not regularly detect such breaches.

If we exclude the UK and run the data for the remaining EU member states we see a somewhat different picture, whereby the EU-15 is quite close to the emissions levels of the US and Australia (see figure 3). Such disparity among intra-EU emissions levels serves to further highlight a key problem in European environmental governance, namely that of ensuring that community policies are adequately translated into on-the-ground action (see e.g. Hunter and Smith 2005, Jordan and Tosun 2012, Leventon and Antypas 2012, Zhelyazkova 2013). Apart from harmonization and implementation deficits on the EU level, the UK's laggardness with respect to reducing benzene emissions could reignite long-running debates on whether the UK could be viewed as the 'dirty man of Europe' (Rose 1991, Börzel 2002, Revell 2005).

Zoom In Zoom Out Reset image size

As stated above, our findings are based on two key assumptions; first that PRTR data provides a sufficiently robust basis for a comparative analysis that compares emissions over time and between countries, and second that there is an observable link between regulatory standards and industrial performance. They are also based on a specific case that may or may not be more broadly representative. If we accept these assumptions and the value of the case, then the results could be significant in a number of ways. They suggest that there can be convergence in the environmental standards that are adopted and the levels of industrial performance that are realized in different countries over time, but that there are still instances where there are major variations in the outcomes of environmental regulation between OECD countries. They indicate that it is possible for countries to improve the outcomes of environmental regulation and to catch up with the best standards in the world, but they also reveal that it is still possible for some countries to adopt much lower standards than their major competitors. They suggest that it is possible for industry to gain a competitive disadvantage from being less heavily regulated, and for communities to be more heavily exposed to toxic emissions in some countries than in others. They reinforce claims that there is no guarantee that the EU's rhetoric about being a leader in international environmental policy is translated into practice, and that the outcomes of environmental policies within the EU can remain a long way from being harmonized. They also show that any claims that the UK might make that it is no longer the 'dirty man of Europe' are not always true.

More broadly, the paper demonstrates the value of comparative studies of the outcomes of environmental regulation across countries and over time. There is much debate on comparative standards in environmental regulation, but these debates are often characterized by a lack of evidence. Despite the potential for weaknesses in the data, we hope that this paper has demonstrated the value of such comparative analyses in generating such evidence and the need for further research with a similar approach.

We would like to acknowledge the assistance of Sato Misato of the ESRC Centre for Climate Chance Economics and Policy/Grantham Research Institute, London School of Economics and Political Science, for accessing and translating emissions data from the Japanese PRTR. We also acknowledge the helpful comments of the three anonymous referees.