A Multidisciplinary Perspective on COVID-19 Exit Strategies Fabian Dablander†1*, Tessa F. Blanken†1, Charlotte C. Tanis†1, Rosa Breed1, Luc E. Coffeng2, Daan Crommelin3,4, Wouter Edeling3, Pieter Gautier 5,6, Beatrice de Graaf7, Federica Gugole3, Frenk van Harreveld8,9, Dewi Jager1, Lotte Jensen10, Rina Knoeff11, Melle van der Linde1, Benjamin Sanderse3, Anniek de Ruijter12, Sake J. de Vlas2, Catrien Santing11, Bert Zwart3,13, The Science versus Corona Collective, Hans Heesterbeek14, Denny Borsboom1* 1Department of Psychological Methods, University of Amsterdam of Public Health, Erasmus MC, University Medical Center Rotterdam 3Centrum Wiskunde & Informatica 4Korteweg-de Vries Institute for Mathematics, University of Amsterdam 5Department of Economics, Vrije Universiteit Amsterdam 6Tinbergen Institute 7Faculty of Arts, Utrecht University 8Department of Social Psychology, University of Amsterdam 9National Institute for Public Health and the Environment 10Faculty of Arts, Radboud University, Nijmegen 11Faculty of Arts, University of Groningen 12Faculty of Law, University of Amsterdam 13Eindhoven University of Technology 14Department of Population Health Sciences, Utrecht University 2Department Abstract Lockdowns and associated measures imposed in response to the COVID-19 crisis inflict severe damage to society. Across the globe, scientists and policymakers study ways to lift measures while maintaining control of virus spread in circumstances that continuously change due to the evolution of new variants and increasing vaccination coverage. In this process, it has become clear that finding and analysing exit strategies, which are a key aspect of pandemic mitigation in all consecutive waves of infection, is not solely a matter of epidemiological modeling but has many different dimensions that need to be balanced and therefore requires input from many different disciplines. Here, we document an attempt to investigate exit strategies from a multidisciplinary perspective through the Science versus Corona project in the Netherlands. In this project, scientists and laypeople were challenged to submit (components of) exit strategies. A selection of these were implemented in a formal model, and we have evaluated the scenarios from a multidisciplinary perspective, utilizing expertise in epidemiology, economics, psychology, law, mathematics, and history. We argue for the integration of multidisciplinary perspectives on COVID-19 and more generally in pandemic mitigation, highlight open challenges, and present an agenda for further research into exit strategies and their assessment. *Corresponding authors: dablander.fabian@gmail.com; dennyborsboom@gmail.com †These authors contributed equally to this work. Note: This manuscript has been rejected after review at PNAS in the summer of 2021. After a lengthy review process at Royal Society Open Science, which ended in the summer of 2022, we withdrew the manuscript. We are not seeking further publication, and have uploaded it here for posterity. Introduction The COVID-19 pandemic is one of the greatest global crises humankind has faced since World War II. In response to this crisis, public health professionals, governments and scientists across the globe have sprung into action in an attempt to stop the pandemic or at least mitigate its consequences. Focusing on the latter, never before did so many scientists, from so many disciplines, contribute to solving the same problem at the same time, resulting in over 400,000 papers across an unprecedentedly broad range of scientific fields (1) . Despite this huge endeavour and the great advances that have been made in improving our understanding of COVID-19, a conspicuous feature of this monumental scientific effort is that it has exposed the limits of our scientific organization in dealing with this type of crisis (2). Featuring high levels of specialization, geared towards monodisciplinary studies, much of scientific research has progressed within the disciplinary compartments that have evolved over the centuries: epidemiologists study epidemiology (3), psychologists study psychology (4), economists study economics (5), legal scholars study laws and regulatory conditions at different levels of governance (6). While there have been several prominent exceptions (7, 8) and calls for more collaborations across disciplines (9, 10), our current scientific organization generally lacks mechanisms for stimulating and regulating multidisciplinary research efforts – and in the absence of such mechanisms, it is hard for different disciplines to meet. This is problematic, because the COVID-19 crisis clearly does not limit itself to the confines of any one scientific discipline (11). The pandemic, as well as the severe measures that it has elicited, has evolved into a highly complex and multifaceted phenomenon (12, 13). In fact, it is a poignant example of how the simplest of causal processes – a microbe traveling from body to body – can have profound effects throughout the fabric of society, triggering interactions between factors at various levels of organization, operating at various (temporal, social and spatial) scales (14, 15). The complexity of these patterns of interactions, and their feedback loops, is spectacular. Biological details of virus transmission processes (e.g., that it transmits through human contact) impact the structure of societies’ interventions (e.g., physical distancing laws), which generate large-scale consequences (e.g., cancelling a broad range of activities) with severe economic fallout (e.g., businesses going bankrupt); these in turn lead to individual consequences (e.g., job loss) that trigger psychological reactions (e.g., frustration and anxiety) that may lead to opposition against the regulations designed to protect us from the virus (e.g., anti-vaccine ideologies, opposition to physical distancing, ‘lockdown fatigue’) that impinge on the virological level (e.g., by facilitating virus spread through behavioral change) and may thereby generate upstream epidemiological consequences (e.g., a new wave of infections). Thus, causal processes, feedbacks and interactions cut across levels of societal organisation and span many disciplines. This can hamper effective responses to a crisis. Perhaps nowhere is this problem more conspicuously present than in the lockdowns imposed in response to the COVID19 crisis. Together with the virus itself and the psychological responses it triggers (16), lockdowns inflict severe damage to the social, legal, and economic fabric of society. Therefore, across the globe, scientists and policymakers have studied ways to lift lockdowns while maintaining control of virus spread (17, 18). But how should we organize this progress, and how 2 can we generate fruitful interaction across scientific disciplinary boundaries into the production and evaluation of adequate exit strategies? How should we even define ‘adequate’ and does this change with changing circumstances? It has become clear that finding and analysing exit strategies from lockdowns is not merely a matter of epidemiological modeling but requires input from many different disciplines; in fact, a monodisciplinary focus can render us myopic and put society at risk (19). The purpose of the current paper is to sketch possible answers to these questions by reflecting on the Science versus Corona project, a grassroots attempt to organize the scientific community in order to inform policy. The central idea behind Science versus Corona was to involve researchers from different scientific disciplines in a joint effort to construct exit strategies and to construct a projection of the effects of such strategies on different dimensions (e.g., epidemiological, economic, and psychological consequences) so that a larger palette of consequences can be assessed from a policy perspective. The paper is structured as follows. First, we offer a description of the Science versus Corona project. Second, we describe the project’s main results in terms of the constructed and modelled exit strategies. Third, we reflect on the lessons learned and discuss opportunities and pitfalls in setting up more mature versions of the approach. The Science versus Corona Project We founded Science versus Corona early on in the pandemic as a multidisciplinary platform designed to bring together experts to contribute their knowledge and skills in the fight against the coronavirus (see https://scienceversuscorona.com/). The philosophy behind Science versus Corona is simple: share knowledge, share skills, share data, and share code across disciplines. In addition, it serves as a nucleus where experts become familiar with different ways of thinking, the scientific language and approaches from fields fundamentally different from their own, but equally relevant for the joint challenges. Science versus Corona consisted of two projects that each played a role in this respect. The first project – Data versus Corona (DvC) – involved a group of data scientists that provided advice and assistance in the analysis of data related to the coronavirus by connecting data scientists to organisations or research groups in need. The second project – Strategies versus Corona (SvC) – was a platform where scientists from a variety of disciplines worked on exit strategies following Open Science principles. Both projects had their most intense phase from April 2020 to August 2020. Members of both projects participated on a voluntary basis. A total of 248 individuals signed up for the Science versus Corona platform. The current paper largely results from the SvC component, for which a total of 76 individuals signed up. The Strategies versus Corona project aimed to crowdsource the development of exit strategies by collecting ideas from the general public, transforming these ideas into scientific models that can simulate the way different exit strategies would unfold, and evaluating them using panels of experts from different disciplines. This development was organized in four phases: 1) idea generation, 2) modeling, 3a) qualitative evaluation, and 3b) quantitative evaluation and robustness. 3 During the idea generation phase, we placed an open call on April 10th 2020 on our website (https://strategiesversuscorona.com/en/) where both experts and laypeople could submit suggestions for exit strategies and criteria on which the strategies should be evaluated. These ideas were collected and curated (see https://scienceversuscorona.com/crowdsourcing-ideas-forexit-strategies/) to arrive at a selection of criteria and proposals for (components of) exit strategies. During the modeling phase, we placed an open call on April 23rd 2020 on our website (https://strategiesversuscorona.com/en/challenge/) where experts could collaborate and submit a worked out exit strategy. To facilitate collaboration and transform raw ideas into worked-out exit strategies, we organized a digital work environment in which scientists could form teams to develop exit strategies further using the Slack platform (https://slack.com/intl/en-nl/). The group of scientists evolved organically, which led to the representation of scientists from the following areas of expertise, as represented among the authors of the current paper: epidemiology, economics, psychology, mathematics, history, and law. For several of these fields we created discipline-specific channels in the evaluation phase, and assigned a field coordinator to oversee the evaluation of the strategies presented in this paper from their own discipline’s perspective. Note that the omission of a field does not imply that this field is unimportant; clearly, important areas of investigation are not represented among the authors of this paper (e.g., virology, medicine, sociology, political science) and should ideally be involved. The Anatomy of Exit Strategies To systematically evaluate exit strategies, we needed to define what is considered an exit strategy. An exit strategy can be considered from different perspectives, and we took as a starting point the situation in which many measures were implemented to mitigate the spread of the virus (consistent with The Netherlands at the time the platform was launched). Other starting points for defining exit strategies are possible, and defining these clearly is an important aspect of the analysis. Similarly, the new concept of ‘lockdown’ has also quickly grown into a multidimensional container-concept, with various combinations of measures giving various levels of restriction all being denoted ‘lockdown’. Here, we refer to a ‘lockdown’ as a set of measures imposed for a limited time period by the government that collectively severely restrict the freedom and normal (social and economic) activities of people in an area, region, or country with the aim to reduce contact opportunities and transmission events for an infectious disease. This is intended only as a first definition of the term, which is open to improvement as the science of exit strategies progresses. The definition shows that ‘lockdown’ is a multi-dimensional concept because there are inherently many different sets of measures, strengths of these measures, durations that can play a role; lockdowns can be, and indeed were, different across countries and points in time (20). When talking about exit strategies, we need to be aware that the type of exit strategy will depend on the type of lockdown. Generally speaking and in line with our starting point, an exit strategy is a protocol for alleviating or withdrawing measures that ultimately leads to ending the lockdown. We can define two different aspects in which exit strategies could differ: the interventions they contain (what and at what level of restriction/strength) and the architecture that organizes these interventions and their levels over time (when), place (where), and subpopulations (who). Exit strategies are thus defined as complex composites in which different 4 ingredients are combined to lift a lockdown. Any exit strategy should do so within certain criteria of adequacy; how to select, define, and operationalize these criteria is an important subgoal of the analysis of exit strategies. Following our definitions, all exit strategies contain sets of interventions, but the kinds of interventions implemented are often different. Although interventions may be categorized in many ways (e.g., medical, legal), from the point of exit strategies the most important subdivision is between lifting interventions, which (completely or partially) remove or alleviate an existing restriction (e.g., allow schools to open, remove travel restrictions), and supplementary interventions, which add a new component to the situation that did not exist before (e.g., quarantine and isolation after contact tracing, rapid testing, vaccination, curfew). We defined ten possible supplementary and eight possible lifting interventions. Note that this is not an exhaustive list and new supplementary interventions are likely to be developed as time progresses. Also, note that although interventions can be instated and lifted in a categorical fashion (e.g., open or close schools entirely), typically they admit variations in degree (e.g., allow schools to open different numbers of days) that will mitigate virus spread proportionally to the strength of the interventions. Whether an intervention is a lifting or a supplementary intervention depends on the composition of the lockdown, and our categorization was inspired by the first Dutch lockdown. Combining interventions, a spectrum of different exit strategies emerges. One extreme consists in strategies that lift all measures that severely restrict freedom completely from one moment to the next: radical opening. This would lead to a resurgence of the epidemic as long as group immunity is insufficient to curb spread (by natural infection or vaccination). A second class of options is then to have intermittent lockdowns as an exit strategy (i.e., give complete or partial freedom for a limited period and then reinstate the lockdown measures to some degree after a fixed period or when a certain threshold is exceeded in numbers of cases or health burden); this is how many countries have approached COVID-19, be it often involuntarily and without clearly planning ahead. Finally, there is a class of strategies that gradually relaxes measures and their strength. Examples of gradual relaxation include curfews that shift in time, changes in the numbers of individuals allowed to visit a household, in the number of people that can meet in a gathering, shopping conditions, et cetera. 5 Figure 1. The anatomy of exit strategies. Exit strategies are defined as complex composites combining interventions following a certain architecture that places these interventions in time, space, and subpopulations. To arrive at exit strategies one can select among interventions (shown at the top). Interventions are currently defined from a lockdown situation, and as such, can be subdivided into lifting interventions (to get out of the lockdown) or supplementary interventions (to add measures to the current 6 situation). The architecture places these interventions over time (e.g., implement interventions all at once), place (e.g., in the whole country), and subpopulations (e.g., only for people at high-risk). Together, the interventions and architecture form the exit strategy. For example, if all interventions are lifted at once, this exit strategy is called Radical Opening. Note: Contact tracing implies quarantine and isolation. In summary, an exit strategy combines a set of lifting and supplementary interventions that are distributed over time intervals, geographical locations, and subpopulations in an overall architecture; this structure is graphically represented in Figure 1. The exit strategy will depend on the type of lockdown currently in place and the criteria for which the lifting of the lockdown is deemed adequate. This conceptualisation allows us to define the intervention set and architecture independently, and as such allows algorithmic generation of novel exit strategies from a basis set of interventions. Components of exit strategies can then be combined to create new, more elaborate strategies. However, some interventions combine more naturally than others, for instance because their objectives align or because interventions executed support each other. The Evaluation of Exit Strategies The input from the different disciplines varied and were organized according to different questions. Historians were asked what lessons we can draw from the past about the notion of ‘exit strategies’; epidemiologists were asked how particular strategies would shape the outcome of the pandemic; computational analysts about the robustness of the model projections as key parameters were varied; economists about the economic consequences of the strategies; psychologists about their psychological consequences; and legal scholars about potential legal objections or consequences. In this section, we provide a summary of their contributions; for a schematic overview, see Figure 2. 7 Figure 2. Schematic illustration of the assessment of (selected) exit strategies. Epidemiologists simulated different types of exit strategies (top panels). A selection of those have subsequently been subjected to a thorough uncertainty and sensitivity analysis by computational analysts (bottom left panels, adapted from (21)) and to qualitative evaluation by economists, psychologists, and legal scholars (bottom right). Historians provided insightful historical context. The history team reflected on the similarities and differences between the current pandemic to past ones, focusing on the question: how were exit strategies conceptualised in the past? They noted that, in addition to being the first pandemic that can be followed in real-time on a global scale, measures to curb pandemics in the past were undertaken by sectorial institutions with limited scope – no coordinated mitigation efforts beyond city or regional level and no large-scale socio-economic measures were ever deployed before the 21st century. In other words, exit strategies beyond the local scale and as defined in this paper, did not exist before COVID-19. Adopting a wider historical perspective, the history team focused on specific elements that would still shed light on aspects of modern exit strategies, namely (a) the transformation of medical crises into political crises and chaos via protest and rebellion; (b) normalization and adaptation to specific medical interventions; and (c) temporally and locally limited mobilization and demobilization of citizen’s resources around specific aspects of the crisis. The full report can be found in Appendix A1. 8 The epidemiology team implemented a number of different exit strategies in an individual-based SEIR model (22) and developed an online app that allows users to interactively explore different parameter settings (23). These strategies were Radical Opening, which involves a sudden lift of all mitigation measures to return to business-as-usual; Phased Lift of Control, which combines a sequential asymmetric opening in some geographical units while continuing restrictive measures in other areas to keep infection levels low there, with the goal of slowly building herd immunity without overwhelming the healthcare system; Intermittent Lockdown, which alternates between implementation and lifting of interventions according to a fixed or an adaptive schedule; contact tracing based isolation and quarantine measures, which we collectively refer to as Contact Tracing, which aims to identify and isolate infectious cases and to identify and quarantine contacts who are at risk of having been infected; Protecting the Vulnerable, which implements targeted policies to isolate the vulnerable; and Punctuated Pragmatism, which consists of punctuated decision moments at which the situation is assessed and measures are either implemented or lifted on an ad hoc basis with shifting criteria. The last two strategies could not be assessed in silico, as their definition was insufficiently precise. The quantitative epidemiological assessment of the other strategies clearly showed that Radical Opening would be a public health disaster; that although Phased Lift of Control and Intermittent Lockdown could, under some parameter settings, yield herd immunity without overshooting the healthcare system, this would take years. Moreover, because these two strategies aim to achieve herd immunity by natural infection, they also present a public health gamble given uncertainties about long-term effects and the increased probability of dangerous mutations, which have indeed occurred (24). This leaves (large-scale) Contact Tracing, which does not aim at natural herd immunity but to keep the epidemic at bay until herd immunity is reached through vaccination. The modeling suggested that aggressive testing, tracing, and isolating cases can keep the epidemic at bay (without additional interventions), but that this requires an extremely high level of effectiveness and population compliance. The full epidemiological assessment can be found in Appendix A2. The robustness analysis team took the computational model in which the exit strategies were implemented and systematically varied a number of parameters to quantify the uncertainty in outcomes for Phased Lift of Control, Intermittent Lockdown, and Contact Tracing. In particular, they studied how the maximum number of intensive care cases and the total number of intensive care patient-days in excess of capacity change as the parameters vary. The results are displayed in the shape of a cumulative distribution function (see the bottom left panel in Figure 2), which gives the probability (displayed on the vertical axis) that the quantity under study remains below (or at) any given threshold (horizontal axis). Intermittent Lockdown and Phased Lift of Control almost always resulted in a much higher maximum number of intensive care patients than Contact Tracing. But for all three strategies the maximum varies strongly, with some simulations that do not show any overburdening of the healthcare system, and others leading to large outbreaks. This reveals the need for extensive evaluation of the strategies before actual implementation. Lastly, the team found that, overall, non-policy-related uncertainties (for example regarding the basic reproductive number) were less important than behavioral uncertainties (for example compliance to isolation as part of contact tracing). A more extensive analysis with additional results and in depth discussion is reported in (21). A short overview of their assessment can be found in Appendix A3. 9 The quantitative epidemiological assessment of the exit strategies was augmented by qualitative assessments from researchers with a background in economics, psychology, and law. Economists stated that the increased economic output, that proponents of Radical Opening had suggested would result, is most likely illusory (8, 25) because economic activity reduces when hospitalizations increase; psychologists commented that it would erode trust in the government to protect its people, and likely lead to post-traumatic stress disorder in healthcare workers; legal scholars commented on the violation of the precautionary principle that holds that public authorities have to take action to prevent a potential public harm. Vivid Images from overcrowded hospitals and corpses stored in churches, such as we saw in Italy during the initial stages of the pandemic, would lead to outrage and erosion of trust. The economic effects of Phased Lift of Control would naturally be skewed, and economists commented on the possibility that citizens would migrate from a geographical area where a strict lockdown is enforced to areas where measures are lifted. In a similar vein, psychologists commented that people under lockdown may feel a strong sense of injustice because they do not receive the liberties that others do, while people where measures are lifted may feel like guinea pigs. Legal scholars noted that (from a European perspective) such asymmetric measures can potentially infringe on constitutionally protected rights to equality and freedom. Contact Tracing could be highly cost effective, economists pointed out, because – if effective – it would allow for substantially more economic activity compared to a situation where broad lockdown measures are in place. Psychologists commented on the importance of people’s trust in the government, and highlighted potential recall biases that may compromise the efficiency of tracing. Legal scholars noted that contact tracing presents a challenge to data protection and the right to privacy. Yet, contact tracing can be justified if it is solely aimed at protecting public health and does not go beyond what is needed to curb virus spread. For Intermittent Lockdown, legal scholars noted that, besides the challenges of upholding constitutional principles such as precaution, proportionality, solidarity, and fundamental rights, it would be difficult especially for the Intermittent Lockdown exit strategy to ensure the principle of legal certainty. Legal certainty holds that the law must be clear, accessible, predictable, and executable by citizens. The ad hoc manner in which countries have alternated between implementing and lifting lockdowns has given insight into this challenge, legal scholars noted, resulting in a (perceived) loss of governmental legitimacy and compliance by citizens. Protecting the Vulnerable would imply targeted lockdowns, and economists drew on research to suggest such targeted lockdowns (or targeted policies such as special supermarket hours for the vulnerable) can substantially decrease the death rate for a given level of economic activity. Psychologists noted that such differential treatment may be interpreted as discriminatory, especially because – and in contrast to Phased Lift of Control – whether a person is vulnerable to COVID-19 is unobserved and must be inferred through proxies such as age or medical status. These can be poor indicators. Citizens designated as vulnerable may further experience considerable psychological stress, depression and feelings of loneliness if isolated for prolonged periods, psychologists noted. Legal scholars noted that whether or not this strategy is discriminatory depends on the specifics of the measures. Punctuated Pragmatism is a continuously evolving strategy, and economists suggested that a gradual lifting of measures allowed businesses to cut some of their losses. Psychologists noted 10 that the strategy may be interpreted by some as being inconsistent (since future decisions might invalidate past ones), which can lead to eroded trust in the government, health agencies, and science. Legal scholars observed that the most important downside is the lack of legal certainty; compliance and effectiveness of legal rules is reduced in a continuously changing regulatory landscape. These discipline-specific comments supplement the epidemiological assessment of the exit strategies in Appendix A2. Discussion We have found that both citizens and professional scientists from various backgrounds recognized the importance of a multidisciplinary perspective on COVID-19 exit strategies. Here, we want to discuss two aspects of the Science versus Corona project and multidisciplinary initiatives more broadly: (a) a framework through which one might reason about multidisciplinary evaluation of exit strategies and (b) the process and structures which facilitate disciplines coming together. Systematic Evaluation of Exit Strategies Preliminary findings of the Science versus Corona project identified various dimensions on which putative exit strategies could be evaluated from the points of view of epidemiology, psychology, economics, and law. These exit strategies could then be modeled at the epidemiological level, based on the work of (22). Projected problems in the implementation of exit strategies, as well as consequences of putative exit strategies, could subsequently be reflected upon by scholars from other scientific disciplines. In our view, this organization exhibits the fundamental structure that a multidisciplinary evaluation of exit strategies could have and is illustrated further in Figure 3. Figure 3. Process proposal for systematic evaluation of exit strategies. Multiple disciplines contribute to strategy formation, after which disciplinary teams create an integrated model of different strategies’ consequences. These are weighed by importance and uncertainty, which creates input for decision processes in policy. 11 First, the anatomy of exit strategies involves the allocation of interventions distributed in time and space, and to evaluate the feasibility and costs of these interventions they must be analysed from multiple angles. This process is sketched under Exit Strategy Formation in Figure 3. Second, to judge the epidemiological course of the virus under a given exit strategy, epidemiological models are crucial; hence, epidemiological modeling forms the heart of every exit strategy evaluation. Ideally, epidemiological models are integrated with economic and psychological models to provide a more holistic assessment of the consequences of different exit strategies. This is illustrated as Integrated Modeling in Figure 3. These different consequences are then weighted in terms of their importance, a challenging task that goes beyond a scientific assessment and must ultimately reside with elected leaders or the public. Importantly, the models should themselves be thoroughly scrutinized and uncertainty in the projected outcomes, relative to the current state of knowledge, should be quantified, as was done through robustness analysis in our project. These two aspects are visualised in the Assessment of Cost and Uncertainty part in Figure 3. In the Decision Process step, the different exit strategies and their consequences are compared and a choice about which exit strategy to pursue is taken. This again goes beyond a purely scientific assessment and ultimately resides with the public or those that were elected to represent it. This decision step is followed by close monitoring of the situation and then feeds back into an adjustment of the modeling upon receiving new information. In the current paper, modeling of exit strategies was limited to the epidemiological model described in (22), which formed the core of the quantitative evaluation. Our goal is not to argue for the use of this particular model – or any other model, for that matter – but rather to illustrate that some form of epidemiological modeling is at the core of modeling exit strategies. In practice, a suite of different modeling approaches and frameworks may be used, and also should be used. Ideally, one would then extend the (particular) epidemiological model or suite of models with components for e.g. the behavioral, psychological, social, legal and economic consequences of the exit strategies. The possibilities for setting up such joint models were investigated during the Science versus Corona project but proved a daunting task. We encountered four important classes of problems that hamper model integration. First, in some areas, such as psychology, we lack validated mathematical models to describe behavior relevant to the epidemic in sufficient detail, which means that psychological theories or hypotheses cannot be unambiguously connected to the mathematical modeling frameworks offered by fields such as epidemiology and economics. Second, while in the latter areas we do have mathematical models, we lack strong theory that could inform a choice on how to connect their respective domains: various alternatives will result in very different evaluations, and we currently have no way of deciding which is best. Naturally, uncertainty at this level of the qualitative patterning of effects is very hard to quantify using robustness analysis. Here the robustness analysis is limited to the assessment of the impact of parameter uncertainty (i.e., uncertainty conditional on a model), but structural uncertainty (i.e., uncertainty across models) should be quantified as well and this is a much more difficult task (7, 26, 27). Third, as the robustness analysis shows, even the study of an epidemic model by itself is a complicated project (21) and it is unclear whether the robustness of multidisciplinary models could be subjected to an adequate analysis, due to its sheer complexity and the socio-cultural differentiations in any given situation. A final challenge, which is specific to integrating different disciplines, involves combining the outcomes of all disciplines in such a way that they can be incorporated into the same model: while the evaluation of exit strategies may be 12 multidimensional, the fact that we can only choose one means we have to decide on the tradeoffs between different kinds of problems, for instance by assigning utilities to different outcomes. Such trade-offs require us to weigh costs and benefits of different kinds, which is not trivial; for instance, how do we define the trade-off between health and economic damage? Any resulting model must necessarily be a (gross) simplification whose primary value is heuristic (28); it should be used to explore questions rather than confidently assert answers (29). Because creating such a model forces one to think deeply about trade-offs between different consequences, one may well find that the process of creating the model is more helpful than the model itself. Although we will never know how much better society would have fared if modeling of exit strategies had been systematically addressed from a multidisciplinary perspective, it is in our view likely that this would have improved decision making. As riots broke out in response to the second lockdown in The Netherlands (in accordance with the image that transpired from our historical research), the consequences of a sustained pattern of improvisation loomed large over governments worldwide – without a vaccine, the absence of a clear strategy to deal with a prolonged dominance of COVID-19 could very well have led to chaos. In addition, several identifications of problems with exit strategies proved accurate; for instance, the worry that the Dutch strategy of punctuated pragmatism would be perceived as inconsistent by the public was borne out, with public trust in government authorities declining severely towards the end of 2021 (30). Process and Organization There are several takeaways from the Science versus Corona project that may inform future initiatives. First, the pace of the project needs to be aligned with the pace of the crisis. This proved difficult in our case. For instance, by the time our teams were finalizing epidemiological models assessing how to exit the first wave efficiently, the second wave of COVID-19 was already underway in The Netherlands. This limited the ability of Science versus Corona to inform policy in the first year of the pandemic, and also illustrates a fundamental problem in organizing scientific responses to global disasters such as the COVID-19 outbreak: the standard organization of science in terms of separate disciplines working autonomously – i.e., determining their own goals and research directions – is slow to adapt to a crisis that spans the whole of society, which hampers our efficiency in integrating approaches across disciplines. As a result, events in the crisis unfolded more rapidly than the science that could model them, and governments had to respond with far-reaching measures in the absence of adequate projections of policies. In this situation, many governments developed exit strategies that we have come to define as Punctuated Pragmatism: periodic evaluations of eclectically chosen parameters that were used to define ad hoc policies that were mostly challenged for their legal constitutional basis rather than their epidemiological or economic soundness. Hence, such policies were chosen without the benefit of well-studied formal models or robustness analyses that could help chart possible future scenarios that might unfold in response. Naturally, the Science versus Corona project was also small relative to the size of the problem; many scientists worked on Science versus Corona largely in their spare time and the resulting evaluative framework remains largely qualitative and represents the points of view of a selection 13 of researchers from scientific disciplines, rather than being a definitive representation of opinions in these disciplines as a whole. Clearly defining the task and the outcome may have resulted in higher engagement. Contrasting the more targeted approach of Data versus Corona with the more open-ended challenge of Strategies versus Corona, we found that the former led to higher engagement. Data versus Corona required a specific request from an help-seeking party which was broadcasted to a pool of volunteers, who could then sign up. This was a similar design as also pursued by for example CrowdFight (https://crowdfightcovid19.org/). In contrast, the development of exit strategies pursued by the Strategies versus Corona initiative was a much broader, less well-defined task and consequently led to less engagement. Of course, developing exit strategies simply is more difficult and broader than providing data analytic help. The second takeaway is that dedicated funding and infrastructure thus needs to be made available to allow researchers from different disciplines to engage in such open-ended work. Our initiative did in fact lead to a large grant that combined researchers from different disciplines, but this grant was not specifically tailored to the development of exit strategies, but to COVID-19 research more broadly. If the world is to organize its scientific resources such that they can be quickly and easily mobilized, we need a new scientific structure that is able to immediately coordinate our intellectual resources across disciplines as soon as disaster strikes. One might coin this putative alternative organization of science crisis science (31). In a nutshell, organizations at the national and worldwide levels should develop institutions and protocols to quickly mobilize multidisciplinary scientific input. The Science versus Corona project we reported on in this paper provides a small-scale pilot that asserts the need for such multidisciplinary input in the context of the COVID-19 crisis. We hope that it may serve as an inspiration for future initiatives. Finally, in order for the fruits of multidisciplinary scientific initiatives to have societal impact, they need to be closely connected to the policy-making process. However, in The Netherlands, the policy development process featured very limited participation of scientists outside of official policy institutes. For instance, epidemiological models used in actual policy formation were not made available for the study of exit strategies by third parties, such as Science versus Corona, which limited the ability of the project to align its projections to the models used by the government. Therefore, next to securing sufficient funding and infrastructure to connect multiple disciplines, future scientific organizations that aim to deal with crises like the COVID-19 pandemic should take care to develop an active interface with policy-makers. Conflict of Interests: None. Data: The code for the computational model is available from https://gitlab.com/luccoffeng/virsim. The code to interactively explore exit strategies is available from https://github.com/fdabl/Covid-Exit. The code for uncertainty quantification and sensitivity analysis is available from https://github.com/FGugole/UQ_covid19. Only simulated data derived from these scripts have been used in this manuscript. Funding: The Science versus Corona initiative was funded by NWO Fast Track grant 440.20.032. FD was supported by ZonMw project 10430022010001. 14 References 1. World Health Organization, Global research on coronavirus disease (COVID-19) (2020) (June 3, 2021). 2. J. Brainard, Scientists are drowning in COVID-19 papers. Can new tools keep them afloat. Science 13 (2020). 3. M. Lipsitch, D. L. Swerdlow, L. Finelli, Defining the epidemiology of Covid-19—studies needed. N. Engl. J. Med. 382, 1194–1196 (2020). 4. B. Pfefferbaum, C. S. North, Mental health and the Covid-19 pandemic. N. Engl. J. Med. 383, 510–512 (2020). 5. J. M. Barrero, N. Bloom, S. J. Davis, “COVID-19 is also a reallocation shock” (National Bureau of Economic Research, 2020). 6. L. O. Gostin, L. F. Wiley, Governmental public health powers during the COVID-19 pandemic: stay-at-home orders, business closures, and travel restrictions. JAMA 323, 2137–2138 (2020). 7. W. Edeling, et al., The impact of uncertainty on predictions of the CovidSim epidemiological code. Nat. Comput. Sci. 1, 128–135 (2021). 8. A. Pichler, M. Pangallo, R. M. del Rio-Chanona, F. Lafond, J. D. Farmer, Production networks and epidemic spreading: How to restart the UK economy? (2020) https:/doi.org/10.2139/ssrn.3606984. 9. E. J. Murray, Epidemiology’s time of need: COVID-19 calls for epidemic-related economics. J. Econ. Perspect. 34, 105–20 (2020). 10. J. J. Van Bavel, et al., Using social and behavioural science to support COVID-19 pandemic response. Nat. Hum. Behav. 4, 460–471 (2020). 11. J. P. Sturmberg, C. M. Martin, COVID-19–how a pandemic reveals that everything is connected to everything else. J. Eval. Clin. Pract. (2020). 12. C. Ansell, E. Sørensen, J. Torfing, The COVID-19 pandemic as a game changer for public administration and leadership? The need for robust governance responses to turbulent problems. Public Manag. Rev., 1–12 (2020). 13. D. Krakauer, G. West, The Complex Alternative: Complexity Scientists on the COVID-19 Pandemic (SFI Press, 2021). 14. H. Heesterbeek, et al., Modeling infectious disease dynamics in the complex landscape of global health. Science 347 (2015). 15. R. Solé, S. F. Elena, Viruses as complex adaptive systems (Princeton University Press, 2018). 15 16. M. Pierce, et al., Mental health before and during the COVID-19 pandemic: a longitudinal probability sample survey of the UK population. Lancet Psychiatry 7, 883–892 (2020). 17. L. López, X. Rodó, The end of social confinement and COVID-19 re-emergence risk. Nat. Hum. Behav. 4, 746–755 (2020). 18. N. W. Ruktanonchai, et al., Assessing the impact of coordinated COVID-19 exit strategies across Europe. Science 369, 1465–1470 (2020). 19. N. Chams, et al., COVID-19: a multidisciplinary review. Front. Public Health 8, 383 (2020). 20. T. Hale, et al., A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker). Nat. Hum. Behav. 5, 529–538 (2021). 21. F. Gugole, et al., Uncertainty quantification and sensitivity analysis of COVID-19 exit strategies in an individual-based transmission model. medRxiv (2021). 22. S. J. de Vlas, L. E. Coffeng, Achieving herd immunity against COVID-19 at the country level by the exit strategy of a phased lift of control. Sci. Rep. 11, 1–7 (2021). 23. F. Dablander, L. E. Coffeng, Covid-Exit v1.0 (Zenodo, 2020) https:/doi.org/10.5281/zenodo.4055228 (June 3, 2021). 24. J. P. Moore, P. A. Offit, SARS-CoV-2 vaccines and the growing threat of viral variants. JAMA 325, 821–822 (2021). 25. R. Chetty, J. Friedman, N. Hendren, M. Stepner, The economic impacts of COVID-19: Evidence from a new public database built from private sector data. Oppor. Insights (2020). 26. R. C. Smith, Uncertainty quantification: theory, implementation, and applications (Siam, 2013). 27. W. Edeling, P. Cinnella, R. P. Dwight, Predictive RANS simulations via Bayesian modelscenario averaging. J. Comput. Phys. 275, 65–91 (2014). 28. N. Oreskes, K. Shrader-Frechette, K. Belitz, Verification, validation, and confirmation of numerical models in the earth sciences. Science 263, 641–646 (1994). 29. A. Saltelli, et al., Five ways to ensure that models serve society: a manifesto. Nature 582, 482–484 (2020). 30. G. Engbersen, et al., De laag-vertrouwensamenleving: de maatschappelijke impact van COVID-19 in Amsterdam, Den Haag, Rotterdam & Nederland. (2021). 31. R. R. Colwell, G. E. Machlis, Science during crisis. Science 364, 5–5 (2019). 32. D. Brockmann, et al., Coronavirus-Pandemie – Die Krise nachhaltig überwinden: Dritte Adhoc-Stellungnahme zur COVID-19-Pandemie (2020). 16 33. G. P. Van de Ven, A. M. A. J. Driessen, W. Wolters, J. Wasser, Niets is bestendig. De geschiedenis van de rivieroverstromingen in Nederland (1995). 34. S. K. Cohn Jr, Lust for liberty: the politics of social revolt in medieval Europe, 1200-1425 (Harvard University Press, 2008). 35. B. de Graaf, Taming Pandemics in International Relations: Why Do States Cooperate in Transboundary Crises? An Applied History Perspective. J. Appl. Hist. 2, 36–61 (2020). 36. R. Knoeff, L. Jensen, B. De Graaf, C. Santing, Dancing with death. A historical perspective on coping with Covid-19. Risk Hazards Crisis Public Policy (2021). 37. B. Dolan, Unmasking History: Who Was Behind the Anti-Mask League Protests During the 1918 Influenza Epidemic in San Francisco? Perspect. Med. Humanit. 5 (2020). 38. J. Henderson, The invisible enemy: Fighting the plague in early modern Italy. Centaurus 62, 263–274 (2020). 39. G. F. Gensini, M. H. Yacoub, A. A. Conti, The concept of quarantine in history: from plague to SARS. J. Infect. 49, 257–261 (2004). 40. A. Chase-Levenson, Early nineteenth-century Mediterranean quarantine as a European system. Quar. Local Glob. Hist. Ed. Bashford, 35–53 (2016). 41. A. Robarts, “Nowhere to Run to, Nowhere to Hide? Society, State, and Epidemic Diseases in the Early Nineteenth-Century Ottoman Balkans” in Plague and Contagion in the Islamic Mediterranean, (ARC, Amsterdam University Press, 2017), pp. 221–242. 42. S. K. Cohn Jr, Epidemics: Hate and Compassion from the Plague of Athens to AIDS (Oxford University Press, 2018). 43. S. K. Cohn, The black death: the end of a paradigm (Routledge, 2017). 44. R. J. Evans, Epidemics and revolutions: cholera in nineteenth-century Europe. Past Present, 123–146 (1988). 45. The British Academy, The COVID Decade: understanding the long-term societal impacts of COVID-19 (June 3, 2021). 46. A. de Waal, New Pandemics, Old Politics: Two Hundred Years of War on Disease and its Alternatives (John Wiley & Sons, 2021). 47. M. S. Eichenbaum, S. Rebelo, M. Trabandt, “The macroeconomics of epidemics” (National Bureau of Economic Research, 2020). 48. M. Farboodi, G. Jarosch, R. Shimer, “Internal and external effects of social distancing in a pandemic” (National Bureau of Economic Research, 2020). 49. , Art. 15 ECHR (June 4, 2021). 17 50. , CJEU Case Pfizer, 2002 (2002) (June 4, 2021). 51. , ECHR Case Oneryildiz, 2004 (2004) (June 4, 2021). 52. S. Park, G. J. Choi, H. Ko, Information technology–based tracing strategy in response to COVID-19 in South Korea—privacy controversies. JAMA 323, 2129–2130 (2020). 53. H. Cho, D. Ippolito, Y. W. Yu, Contact tracing mobile apps for COVID-19: Privacy considerations and related trade-offs. ArXiv Prepr. ArXiv200311511 (2020). 54. M. Siegrist, G. Cvetkovich, Perception of hazards: The role of social trust and knowledge. Risk Anal. 20, 713–720 (2000). 55. F. van Harreveld, B. Rutjens, The impact of threat to personal control on trust in experts and non-experts communicating about risk (2020) https:/doi.org/10.31234/osf.io/8nq47. 56. B. T. Rutjens, S. J. Heine, R. M. Sutton, F. van Harreveld, “Attitudes towards science” in Advances in Experimental Social Psychology, (Elsevier, 2018), pp. 125–165. 57. M. C. Anderson, S. Hanslmayr, Neural mechanisms of motivated forgetting. Trends Cogn. Sci. 18, 279–292 (2014). 58. C. O. F. R. O. T. E. Union, Charter Of Fundamental Rights Of The European Union (OJ C 364/01) (2000). 59. , Convention for the Protection of Human Rights and Fundamental Freedoms (Rome 4-111950) (June 4, 2021). 60. H. van Kolfschooten, A. de Ruijter, COVID-19 and privacy in the European Union: A legal perspective on contact tracing. Contemp. Secur. Policy 41, 478–491 (2020). 61. , Decision No 1082/2013/EU of the European Parliament and of the Council of 22 October 2013 on Serious Cross-Border Threats to Health and Repealing Decision No 2119/98/EC (O.J. L 293/1 5-11-2013) (June 4, 2021). 62. , Regulation (EU) 2016/679) of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95/46/EC (General Data Protection Regulation) (OJ L119/1, 4-5-2016) (June 3, 2021). 63. I. Braithwaite, T. Callender, M. Bullock, R. W. Aldridge, Automated and partly automated contact tracing: a systematic review to inform the control of COVID-19. Lancet Digit. Health (2020). 64. S. Elbe, Pandemics on the radar screen: health security, infectious disease and the medicalisation of insecurity. Polit. Stud. 59, 848–866 (2011). 65. T. Hanrieder, C. Kreuder-Sonnen, WHO decides on the exception? Securitization and emergency governance in global health. Secur. Dialogue 45, 331–348 (2014). 18 66. , ECHR, Article 8(2) (June 4, 2021). 67. , CFREU, Article 52 (June 4, 2021). 68. M. Chambon, J. Dalege, J. E. Elberse, F. van Harreveld, A psychological network approach to attitudes and preventive behaviors during pandemics: A COVID-19 study in the United Kingdom and the Netherlands. Soc. Psychol. Personal. Sci., 19485506211002420 (2021). 69. J. Grogan, N. Weinberg, Principles to Uphold the Rule of Law and Good Governance in Public Health Emergencies - Policy Brief August 2020. Recon Project, 29. (2020) (June 3, 2021). 70. H. C. Kraemer, Evaluating medical tests: Objective and quantitative guidelines (Sage publications Newbury Park, CA:, 1992). 71. D. Acemoglu, V. Chernozhukov, I. Werning, M. D. Whinston, “A multi-risk SIR model with optimally targeted lockdown” (National Bureau of Economic Research, 2020). 72. L. Brotherhood, P. Kircher, C. Santos, M. Tertilt, “An Economic Model of the Covid-19 Epidemic: The Importance of Testing and Age-Specific Policies” (Social Science Research Network, 2020) (June 14, 2021). 73. M. Bognanni, D. Hanley, D. Kolliner, K. Mitman, “Economics and Epidemics: Evidence from an Estimated Spatial Econ-SIR Model” (IZA Discussion Papers, 2020). 74. , ECtHR, A. and Others v. the United Kingdom(GC), 3455/05, 19 February 2009, §§ 182190 (June 4, 2021). 75. , ECtHR, Cyprus v. Turkey(GC), 25781/94, 10 May 2001, §§ 312-315. (2001) (June 4, 2021). 76. Rijksoverheid, Coronavirus dashboard (June 3, 2021). 77. D. Adam, Simulating the pandemic: What COVID forecasters can learn from climate models. Nature 587, 533–534 (2020). 19 Appendix A1: The History of Exit Strategies This COVID-19 crisis is the first pandemic crisis in history where the development of the pandemic in its epidemiological and medical statistics can be followed in real time (32). It is also the pandemic that can be considered most democratic, in the way discussion of these statistics, official analyses and regulations unfolds globally and online. Millions of ordinary citizens now present themselves as experts on virological and epidemiological matters. Remarkably, it is also the first time in history that global and national lockdown measures are deployed across the scope of the public domain – and that the way to project and manage a scaling down of these measures and to exit lockdown regulations is equally put on the national and global agenda, albeit hitherto not successfully handled on that level (32). In this brief historical exposé we will discuss how previous pandemic crises dealt with the question of ending medical, hygienic, and other pandemic related measures: how did they execute exit strategies in history? The term ‘exit strategies’ only surfaced in the 1980s, albeit marginally, and was used in the realm of business investment and socio-economic investment plans. For a historical look at ‘exit strategies’ pertaining to the roll back of formal health regulations surrounding medical crises, we have to resort to other terms and definitions. Most importantly, up until the post-war period, or even up until the late 20th century, governments did not intervene in epidemiological crises with large-scale policies and intervention packages. Measures were undertaken locally, regionally, sometimes transnationally as well, but they were undertaken and carried out by sectoral institutions that had a limited scope of action: local or regional health boards, quarantine stations or port authorities for example. The measures pertained to direct medical and epidemiological questions; up until the 21st century, no large scale, national or international socio-economic measures were deployed that curtailed the public domain, put national economies in a lockdown and limited transportation and production schemes. Vice versa, exit strategies were equally locally or regionally restricted, and less strictly nationally engineered in scope and nature. This lack of national or internationally engineered (exit) strategies does not only pertain to health and medical crises but can also be identified in other situations of disaster management. For example, for the Netherlands, the handling of large scale floods required the organization of evacuations and direct emergency help. Most of these emergency measures, including the prevention of the outbreak of diseases after floods, were proclaimed by local governments, and funds were raised by citizens’ committees (33). Only in the 20th century a ‘National Disaster Fund’ was established to coordinate fundraising and distribution of aid. Also, large scale reconstruction programmes were issued by the national government, such as the Afsluitdijk or the huge Delta Works to protect large areas of land from the sea. But such reconstruction projects cannot be considered ‘exit strategies’. That is why historical research into ‘exit strategies’ needs to be less focused on concrete, intentional ‘exit’ programmes and nationally engineered ‘strategies’, but has to take into account a wider array of actions to cope and ‘learn to live with the crisis’ and mitigate its impact (34–36). If we adopt this broader historical perspective on ‘exit strategies’, and take a quick scan through history, we can discern three types of – what we anachronistically still delineate as - ‘exit 20 strategies’, where measures and interventions taken to mitigate or manage an epidemic crisis were smoothed out, rolled back or upended: 1) transformation of medical crises and its regulations via protest and rebellion into political crises, chaos, and an dissolution or overruling of previous measures; 2) normalization of and adaptation to specific medical interventions; 3) temporally and locally limited mobilization and demobilization of citizen’s resources around specific aspects of the crisis. We discuss these in turn. Protest and rebellion As Leopoldina, the German National Academy of Sciences, remarked, if measures are considered ‘clear, unequivocal and sensible’, citizens will accept and obey them (32). If they are considered haphazard, illogical, or even worse, unfairly distributed, they will rally against such measures. This pattern is compounded by historical evidence. Epidemics and pandemics in history have oftentimes led to rebellion and upheaval. As the social historian Samuel Cohn has demonstrated, epidemics exacerbate social inequalities. For example, in late medieval Europe, outbreaks and spread of the bubonic plague led to widespread and longlasting waves of rebellion. The majority of the uprisings pitted ordinary people against their overlords and were fought over citizenship rights, access to governmental offices. Interestingly, not immediate plight and hunger, but anger and frustration over the way the ravage was handled, triggered the revolts. Popular revolts increased after the Black Death, and from the mid-1350s to the early fifteenth century conflicts arose over demands for liberty and equality. The way feudal lords and princes handled the impact of the Black Death, by increasing royal taxes and allowing corruption to rise, frustrated populations throughout Europe. Remarkably, revolting protagonists did not come from the poorest classes trying to get more grain, they were merchant elites and people of a ‘middling sort’, triggered in their rebellion by a postplague rise in royal taxes. They saw these measures as unfair and corrupt, and attacked the town councilors and royal officers who supported these and carried them out (34). Throughout history these rebellions typically surfaced and resurfaced in epidemic times. Closer to our times and type of pandemic, we could also point to the emergence of the ‘antimask-Leagues’ during the time of the Spanish Flu, in 1918-9. Here as well, not the initial ordinance issued by, for example, the local Health Board of San Francisco to require every citizen to wear a mask caused protest and dissent. But only when the ordinance was annulled in November 1918, but then re-issued again in January 1919, after a new outbreak of flu cases was reported, citizens were galvanized to mobilize a front against these regulations. The more so, since diverging reports emerged in the media on the benefits of masks, the authorities themselves, after the first lifting of the ordinance, had stressed the advantages of clear air and sunshine, and offenders to the ordinance were quite rigorously punished. Interestingly, the premature attempt to exit the first social distancing and mask measures, and the swift deployment of the second ordinance, including the enforced compliance, created the unrest and protest that in February 1919 made the Board to rethink its policy again and abort the measure. From this example, it shows that measures that are deployed too haphazardly and erratically and that try to radically alter social behavior overnight, are near impossible to maintain. As (37) states, both the issuing of new measures and the timely way to exit them, depend on sound and well thought through attempts ‘to persuade’ and reason, rather than to enforce them. 21 Normalization and adaptation With the aforementioned lack of clearly articulated and engineered national or international ‘exit strategies’ in history in mind, we can still point to the way authorities deployed measures to mitigate pandemic crises in their times and ages. An important example of concrete measures and programmes intended to promote ‘social distancing’ and control the spread of diseases is the instrument maintained by port and border authorities to erect lazarets, create quarantine stations and to monitor incoming streams of people and goods. Already during the early times of the Black Death, port cities around the Mediterranean experimented with lazaretts and quarantine stations where incoming seamen were detained and observed, from ships that were suspected of carrying the plague (38). Town and clerical administrations proclaimed official norms for distancing and for instance prescribed white sticks to measure the right distance one should keep from plague victims (34). Even without today’s knowledge of the origins of zoonotic diseases, their natural history or epidemiological vectors, cities created health boards and port authorities to impose travel restrictions on vessels, issue health bills, and imposed restrictions on their populations and streams of tradesmen. Such anti-epidemic infrastructures were kept in place and scaled back once a virulent epidemic was over. These infrastructures were continuously improved, fine grained and expanded with the increase of more practical and scientific knowledge (39). In the early 19th century, pandemics caused or triggered by the streams of mobilizing and demobilizing coalition troops and displaced persons (in and after the Napoleonic wars, and thereafter due to the Russian-Ottoman wars), were managed on an unprecedented transnational scale. Due to the increased cooperation and allied exchange of information, envoys, diplomats and emissaries created a system of transnational information management and early warning systems to report on new outbreaks. Furthermore, the Russian empire created a sophisticated anti-epidemic infrastructure along the European borderlands, in the Danubian principalities and the Balkan. Checkpoints in the mountain passes were constructed where crossing travelers were forced to undergo a period of medical observation; fortress towns and monasteries were transformed into lazarets and facilities for the detention and observation of individuals afflicted with plague. Travel restrictions and quarantine-like measures became standard procedures along the commercial arteries in the region. A veritable Danubian cordon sanitaire, with main and secondary quarantine stations was created, staffed by learned doctors, nurses and interpreters (next to the military and policemen). When the Russian forces withdrew in the mid-1830s, the Ottoman authorities took over this quarantine line, and even expanded many of the secondary stations. In short, measures once intended to curtail immediate outbreaks and threat to occupation forces, were normalized and were adapted into standard practices and permanent infrastructures of disease control (40, 41). Limited mobilization and demobilization of citizen’s resources So far, we have discussed the prerogative of executive authorities and the increasing role of nation states in issuing ordinances and regulations to prevent, control and mitigate epidemic and pandemic crises. We have also discussed the way national and local authorities in pre-welfare societies and in situations of limited national health control applied and handled measures that we could somewhat anachronistically consider ‘exit strategies’. Mostly, there were no such strategies, but measures and programmes were either normalized, institutionalized, and adapted 22 to persisting threats and circumstances, or they led to protest and rebellion if citizens considered them unfair and not reasonable. Interestingly, history also abounds with examples of temporary initiatives taken up by citizens. This mobilization, and demobilization, of citizen’s resources remains an under-researched topic; it cannot properly be considered an ‘exit strategy’, since there is no overarching strategy of planning and control where to exit from. Yet, history shows that as a ‘coping strategy’, to mitigate the impact of a crisis and facilitate and enable the turn from crisis to normalcy, and to restore a form of social balance, these citizen’s initiatives are of utmost importance (42). Take for example initiatives deployed in 1826, after the outbreak of the ‘Groninger Fevers’, a malaria epidemic that killed 10 percent of the city’s population (2,844 people). In the media, a group of citizens announced a charitable action to collect money and organize practical help to facilitate the ill, poor and recuperating victims of the epidemic; to ‘liberate them from the humiliation of total poverty; and keep them as useful part of the civil society’ (Groninger Courant, 1826). This ‘People’s Illness-committee’ (‘Volksziektecommissie’) lasted for two years, and explicitly terminated its charitable activities when the epidemic was over, since it intentionally did not want to transform itself into an institutionalized welfare organization. It was intended as a humanitarian intervention for direct emergency relief, out of private, charitable concerns. The initiative was considered so successful that in 1848 a Cholera committee and in 1856 a Typhus committee were created as well – these organizations lasted longer, but were also terminated once the epidemic was considered over. Interestingly, although emergency relief to people in need was stated as its main aim, the committees in their deliberations also made clear that the maintenance of public order and a healthy, balanced civil society was a no less important goal. These private initiatives in the 19th century (before the introduction of social welfare measures) felt that in addition to Christian, diaconical care for the poor and residual governmental measures, a social security gap had to be filled, that was left empty after the dissolution of the guilds. Yet, these private ‘stop gap measures’ were explicitly designed as temporary relief measures; their termination depended on the decrease of the epidemic outbreak. Concluding from history In sum, exit strategies as intentional, nationally designed and engineered programs did not exist before the late 20th century. Yet, local, regional, and imperial authorities did engage in creating, implementing, and monitoring specific interventions geared towards controlling the outbreak of epidemics and pandemics. Many of these measures were based on best practices from the past; the late medieval script of quarantine measures, cordon sanitaires, social distance measures and observation and contact tracing posts were developed over the ages and honed to a detailed and fine grained perfection in the late 19th century, before vaccines were developed. Normalization and adaptation of such anti-epidemic infrastructures became part and parcel along the Mediterranean, around the main European land and sea routes, across the borderlands and commercial arteries between Europe and Asia, for example. A recurring theme was the management of unrest and rebellion that frequently arose in times of crises. Rebellion was oftentimes triggered by measures considered to be unjust and unfairly distributed, or by measures that were issued too haphazardly and enforced too stringently (34). In the 1840s, measures to combat cholera for example addressed underlying structures of injustice and inequality far more seriously than during the epidemic of the 1830s; this translated into a far better maintenance of public order and consent during the 1840s crisis (43, 44). In fact, we 23 believe that the Netherlands escaped large-scale and violent riots against Cholera measures, precisely because of these local and social initiatives that answered not only medical, but also social-economic needs. In COVID-19 times, we can see these patterns resurfacing: cities that lifted lockdowns too early, are struggling to enforce regulations once again (Melbourne). Places where epidemics highlight inequalities, will experience a higher chance in outbursts of social protests (Black Lives Matter protests in US cities). And, as also the British Academy has recently argued, communities with strong local bonds, capacities and channels of interconnectedness, are best able to respond to the crisis (36, 45). If we can learn anything in terms of exit strategies from history, it may well be these two insight: (a) social reaction to crisis and crisis measures very much reflects the way in which a crisis is handled transparently, consistently and in maintaining a fair and just distribution of the pain and suffering caused both by the pandemic and the measures to regulate it. Only under these conditions will ‘exit strategies’ function as means to mitigate the impact of a crisis and enable a transformation into an acceptable ‘new normal’ (46). And (b) it is of vital importance to invest in bottom-up initiatives that strengthen a society’s resilience. This means investments in the cultural sector, for unlike any other, this sector strengthens the social and cultural coping mechanisms of any society. Appendix A2: Assessment of Exit Strategies The historical survey of exit strategies shows that the very concept of a cohesive exit strategy, coordinated at national and supranational levels, is historically novel. Yet, history shows that we should never forget local cultures, interests and initiatives, including the social protests and rebellion against governmental measures that arise out of them. Understanding these cultural settings is crucial for the success of any exit strategy, as has been demonstrated above. With those lessons in mind, and with the realization that the application of centralized, coordinated systems of interventions is a very recent development, the advancement of a system to evaluate exit strategies is of paramount importance. This is what we set out to do in this section. In accordance, this section describes different exit strategies that result from organizing the interventions discussed in the previous sections over time segments, geographical locations, or subpopulations. The selection of exit strategies discussed in this section is based on the contributions we received during the crowdsourcing projects, and some of these exit strategies have already been formulated in a quantitative model (22), which facilitates their evaluation. In addition, in the context of the Science versus Corona project, (23) have constructed an app, available at https://scienceversuscorona.shinyapps.io/covid-exit/, that one may use to project the effect of different scenarios and to study the results below in greater detail. This section discusses the overall patterns that are generated by exit strategies. For a subset of promising exit strategies, we also examined robustness, which will be discussed in Appendix A3. Radical Opening The Radical Opening architecture involves a sudden lift of the lockdown to return to business-asusual. Radical Opening can be implemented directly, by revoking COVID-19-related prohibitive regulations. Radical Opening is a practically feasible strategy as there are no practical obstacles to its implementation. 24 To assess the projected epidemiological course of the model, Radical Opening has been implemented in the model of (22). In this implementation, which is based on the Dutch situation, it is assumed that society goes back to normal, i.e., parameters like the rate of transmission take the values they had before the lockdown. The most salient consequence of Radical Opening is a large uncontrolled outbreak of the virus, as illustrated in Figure A1. As is shown in the figure, maximum intensive care (IC) capacity is reached in as little as 30 days after Radical opening is implemented (sooner or later if prevalence of infection is higher or lower at the time of implementation, respectively). At the peak of the outbreak, the IC capacity is exceeded by 1000 new cases a day for a period of two weeks, who will die if not provided with the required IC attention. Hospitals will be unable to provide adequate healthcare, leading to a backlog in standard healthcare and screening programs (e.g., for breast cancer), which we expect to lead to secondary deaths from other causes over an extended period of time. In addition to a high death toll, the long-term consequences of even a mild infection with the coronavirus remain unclear. While it might be intuitive to think that such an opening would have immediate positive economic consequences, recent modeling work that incorporates economic behavior into the compartmental models, like (47) and (48) find sharper recessions (because economic activity drops when the healthcare systems gets overloaded) and fewer deaths. They also show that radical openings are welfare-decreasing because people do not fully internalize that they may infect others. 25 Figure A1. Shows per million numbers of prevalent infectious cases, number of new cases in IC per day, and number of cases present in IC (with the horizontal dashed line indicates the IC capacity), as well as percentage of people having experienced the infection (horizontal dashed line indicates the theoretical required level for herd immunity, if immunity is lasting). Radical Opening is implemented at time point 0 (rightmost vertical dotted line). To investigate model predictions further, the reader may consult (18)’s online app at https://scienceversuscorona.shinyapps.io/covid-exit/. 26 The psychological consequences of the Radical Opening strategy seem bleak, too. The overload of the IC’s would lead to hospital personnel being forced to withhold care for COVID-19 victims as well as other cases. Many would die as a result. This experience would likely result in an increase of PTSD and associated trauma-related conditions in hospital personnel. The shock of seeing so many die would likely lead to widespread fear of contamination in the general population, would erode trust in government and science, and would make it harder to get people to adhere to future policies. These psychological consequences clearly have considerable ramifications for the economy as well. For the legal analyses here the rules that are referred to, are mostly those in the realm of European law more widely – that is, including the ECHR framework of which 47 national states are a member. However, these rules are referred to as an illustration of legal principles that are accepted more generally across legal systems in the world. Legally, restrictive state measures to prevent the loss of life in case of a public health emergency, may justify a derogation from the general rule that fundamental rights must be upheld (49). This does not mean that the rule of law can be set aside in an emergency. Public health measures need to be in accordance with the law, and all executive acts and actions must be founded on constitutional provisions. Radical opening in this regard would require a lifting of the current measures in place, but otherwise, there are no immediate legal limitations for an all-out re-opening of society. However, the question is how such a course of action would pan out in light of more general legal principles such as precaution, proportionality, solidarity, and fundamental rights that are generally leading in public health regimes. The precautionary principle entails that even in the absence of absolute scientific sureties, public authorities can, and under circumstances are required, to prevent the loss of life (50, 51). In case of a radical opening, given what we know about its potential impact on the loss of human life, not only the principle of precaution would nudge governments into action, the principle of proportionality in a second step of analysis might also work to balance between potentially competing interests and trade offs. Precaution comes at a cost, the principle of proportionality means that although public authorities need to take action to prevent the loss of life and protect public health, this needs to be balanced out by the economic and other impacts on the welfare and wellbeing of citizens. In this regard we see that in a radical opening none of the other public aims will likely be served well. Radical opening would also be detrimental in light of the principle of solidarity as the risk of poor health is not shared evenly, but will fall on those most vulnerable. At the same time, as we have seen in many countries, the lock-down has created deep impacts in fundamental freedoms and rights. In this regard however, protection of public health can justify limitations in fundamental rights protections, but not more than needed to justify the public health aim. A massive COVID-19 outbreak would likely lead the government to instate a total lockdown until the virus is under control. This lockdown would incur further massive economic damage and, as a result, could increase social unrest, especially if the lockdown measures are considered unclear, and unjustly distributed (see also Appendix A1). If economic damage, social unrest, and psychological consequences are coupled in a process of mutual reinforcement, a negative spiral could arise. Although we have not modeled this process explicitly, research done on historical episodes of pandemics and protest (see above) suggest that the consequences of such a feedback loop may be catastrophic and lead to long-term destabilization of society. 27 Phased Lift of Control The Phased Lift of Control exit strategy combines a sequential asymmetric opening in some geographical units, while continuing restrictive measures such as a lockdown or contact tracing in other areas to keep infection levels low there. Thus, its architecture combines timing with geography, in that interventions are heterogeneously distributed over regions and over time. The goal is to arrive at herd immunity in the whole country without exceeding healthcare capacity at any point in time. Of course, as for the Radical Opening strategy, aiming for herd immunity presents a public health gamble due to the currently unknown long-term consequences of an infection. In contrast to Radical Opening, the Phased Lift of Control strategy, if implemented and adhered to correctly, would not lead to an overburdened healthcare system (see Figure A2). However, as (22) point out, this strategy is only feasible if a number of requirements are met. First, it must be possible to shield vulnerable cases until the local area in which the lockdown is lifted has achieved herd immunity. Second, the country must have sufficient infrastructure to allow the transportation of IC patients across local areas. Third, control is sufficient to avoid outbreaks in parts of the country where measures have not been lifted. Economic consequences that arise from a phased lift of control are likely to be skewed across geographical regions. Areas that are open will generate more commercial activity and as such will generate more jobs. Even for a small country such as the Netherlands, the contrast between the area that opens first and the area that opens last is large (the time difference between the first and last area to open up can be in the order of years). Unless phased lift of control is flanked by travel restrictions, this exit strategy may lead to migration from areas that are in lockdown to areas that are not. Psychological consequences are also likely to result from the skewness of policy across regions. It is likely that people who live under stronger restrictions will experience feelings of injustice and will be less compliant. At the same time, citizens in areas without restrictions may feel that they are being experimented on like guinea pigs, which may lead to resentment. Thus, this exit strategy is likely to increase polarization in societies where it is implemented, and may lead to increased dissatisfaction with government policies. An important factor in this respect is that the position of a region in the sequence of openings is arbitrary. Arbitrary decisions with adverse effects are not easy to cope with, as they are intrinsically prone to disadvantage groups based on characteristics that are irrelevant to the problem to be solved. Legally, from an in-state (European) perspective such measures would infringe constitutionally protected rights to equality and freedom. As outlined, measures to protect public health can form a legitimate argument to (temporarily) limit these rights, but for their legal legitimation these measures must be, among other things, proportionate to their aim. If measures are clearly not sufficient to reach the objective for which they are taken, they might be deemed illegitimate infringements of fundamental rights and freedoms. A phased lifting of the measures would then need to be well-reasoned and the potential uneven impact - such as the impact on business continuity and accessible mental health care - would need to be alleviated by government support, so as to ensure equal treatment and proportional intervention to the extent possible, given the public health objectives. 28 Figure A2. Shows per million numbers of prevalent infectious cases, number of new cases in IC per day, and number of cases present in IC (with the horizontal dashed line indicates the IC capacity), as well as percentage of people having experienced the infection (horizontal dashed line indicates the theoretical required level for herd immunity, if immunity is lasting) for a Phased Lift of Control as reported in De Vlas & Coffeng (2021). Assumes 20 phases with a duration of 45 days; reduction in transmission during control periods to 25%; reduction in interregional transmission during lift periods to 50%. To investigate this scenario further, the reader may consult the online app at https://scienceversuscorona.shinyapps.io/covid-exit/ or (17). 29 Contact Tracing Contact tracing aims to identify and isolate infectious cases and to identify and quarantine contacts who are at risk of having been infected. Implementation at large scale requires a large workforce of manual contact tracers, possibly with contact tracing apps aiding the process, as implemented in many countries over the course of 2020. From an epidemiological standpoint, the success of this exit strategy depends on three parameters: (1) the delay between a person becoming infectious and being identified and isolated (if at all), (2) the probability of an infected contact of an infectious person being identified and quarantined before they turn infectious, (3) the quality of isolation and quarantine and their effects on transmission. Figure A3 illustrates a scenario of this exit strategy using the model by (22). In The Netherlands, where contact tracing was the major control mechanism over the summer of 2020, the strategy ultimately did not manage to achieve these goals, as was the case in many other countries. 30 Figure A3. Shows per million numbers of prevalent infectious cases, number of new cases in IC per day, and number of cases present in IC (with the horizontal dashed line indicates the IC capacity), as well as percentage of people having experienced the infection (horizontal dashed line indicates the theoretical required level for herd immunity, if immunity is lasting) for Contact Tracing (assuming no other interventions are implemented from time point 0 onwards) as calculated for a 2 day delay in detection of infectious cases, 60% of potentially infected contacts being traced, and a 70% reduction in contact rates of identified infectious cases and the identified fraction of their contacts. For other parameterizations, see https://scienceversuscorona.shinyapps.io/covid-exit/. 31 If successfully implemented, the Contact tracing exit strategy does not lead to herd immunity in principle, as shown in the bottom panel of Figure 3. Instead, this policy needs to remain in place until a vaccine is administered to a large enough group of people. In addition, the fact that the proportion of people who have recovered from COVID-19 is kept low through contact tracing implies that large outbreaks remain possible at any time. A significant advantage of contact tracing over lockdowns is that it aims to keep only (potentially) infected people at home. As a result, from an economic point of view, contact tracing is probably a very cost effective way to reduce disease transmission as substantially more economic activity is possible than when society isolates all citizens in a total lockdown. Should legal or practical constraints prevent contact tracing from being feasible at a large scale, it can still be part of an exit strategy that aims for an optimal path towards herd immunity at the lowest possible costs, which would require an reproductive number close to 1 (48). From a psychological point of view, contact tracing is often seen as a strategy that involves a trade-off between security and privacy concerns (52), both of which relate to fundamental human motivations. On the one hand people strive for a sense of safety and security, but on the other hand they also strive for autonomy and control. Apps that enable contact tracing ideally offer individuals information on their exposure without requiring them to trust any party with their private location or medical information (53). Successful contact tracing also requires citizens to be willing and able to cooperate with the authorities. Trust in authorities is known to increase in the face of risks, especially when individual knowledge about the risk is low (54). In the present context, there may be fertile ground for government-imposed contact tracing. It appears, however, that trust in authorities is especially enhanced when personal control over the risk is low and the relevant authority communicates certainty and thus provides compensatory control (55). For contact tracing to be successful, trust in authorities is key and there are several factors in the present context that pose a threat to such trust. For example, trust in authorities generally becomes harder to achieve in a polarized environment. In many countries we see a rise in polarization with regard to the response to the pandemic. Relatedly, trust in scientists appears to be eroding (56), which also threatens trust in governments that rely on scientific data to inform policy making. Determining trust and the associated willingness to adopt tracing apps among different groups in society is therefore of considerable importance. At a more micro-level, there are various psychological factors at play as well. The success of contact tracing also relies on people adequately recalling where they were and who they have met. Psychology teaches that memory capabilities have severe limits and that people can be driven towards forgetting information that does not suit their motives (57). One could argue that the psychological threat of exposure to the corona virus is such that people may be susceptible to such motivated forgetting. The psychological factors that may impair people’s ability to adequately report when they were where are too numerous to describe here, but when employing a test-and-trace policy, it is important to be aware of the intrinsic cognitive limitations that play a role. Legally, contact tracing, both within the state and across borders is a limitation of data protection and the right to privacy and can only be justified if it is aimed at protecting public health and 32 does not go beyond what is needed to curb the spread of the virus. Contract tracing involves the right to privacy and the contacts of those who have been exposed to a dangerous pathogen. The right to privacy is strongly protected in fundamental rights instruments and also in national constitutions (58, 59). Moreover, since 2018, the EU has the strictest data protection regime in the world under the General Data Protection Regulation (GDPR) (60). In European law, contact tracing involves personal data and health data for the purpose of contact tracing (61, 62). Countries of the European Union are required to share information on contact-tracing via an electronic information system: The Early Warning and Response System (EWRS) (60) with due regard to data protection, particularly also for medical information. In individual states in the world there are also further systems through which contact-tracing information is gathered (63). In the current circumstances, when there is a serious threat to health, limitations to individual freedoms can be temporarily limited to a large degree (64, 65). Privacy in this regard is not an absolute right (66, 67). Intermittent Lockdown An intermittent lockdown alternates between implementation and lifting of interventions according to a fixed or adaptive schedule. In a fixed schedule, a country could open for e.g. five days (allowing the virus to spread) and subsequently close for e.g. twentyten (restricting secondary infections arising from the opening part. Such scheduled lockdowns could also be instantiated for different subpopulations. For example, rules could be instantiated that allow anyone under the age of forty to go out from Monday until Thursday, while they have to stay at home during the other days of the week. For citizens over the age of forty, the reverse would hold. A second way of organizing an intermittent lockdown is by making the lockdown schedule dependent on the development of the virus. Such an adaptive schedule results in lockdowns whenever the value of an indicator (e.g., the number of infections, the number of hospital uptakes, etc.) rises to a particular level. When timed correctly, this can in theory avoid the healthcare system becoming overburdened. In practice, however, the resulting trends in IC cases are sensitive to the timing and duration of lockdown. There is thus a risk of overshooting or undershooting the maximum healthcare capacity. Figure A4 represents these scenarios graphically. 33 Figure A4. Shows per million numbers of prevalent infectious cases, number of new cases in IC per day, and number of cases present in IC (with the horizontal dashed line indicates the IC capacity), as well as percentage of people having experienced the infection (horizontal dashed line indicates the theoretical required level for herd immunity, if immunity is lasting) for Intermittent Lockdown. To investigate other situations, see https://scienceversuscorona.shinyapps.io/covid-exit/. The intermittent lockdown with an adaptive schedule is at least a partial description of the response to COVID-19 in many countries during the course of 2020, when societies reinstated lockdowns when infection numbers and hospital uptakes rose. For instance, in The Netherlands, 34 the development of the epidemic was characterized by a release phase that was arguably too light and endured too long, giving rise to a second wave. The fact that lockdowns have been repeatedly lifted in many countries has allowed us to learn about the way an intermittent lockdown could operate. From an economic point of view, the lifting of lockdown measures in many countries over the summer of 2020 has allowed companies that suffered most from the lockdown (e.g., restaurants) to partially recover and make up for some of their losses. We have also seen that compliance is more difficult to uphold during later lockdowns as compared to the first. There are numerous reasons for this that are difficult to completely disentangle. One important reason is that for large groups in society, fear of infection with the virus has reduced considerably. Research into psychological factors influencing compliance with social distancing measures (68) has shown that fear for infection is a key predictor and as treatment of COVID-19 has become more effective and as it has become more clear who are at risk and who are less so, groups that feel less vulnerable will be less motivated to comply with lockdown measures. There is of course an ironic effect at play here in that fear of the virus likely also has subsided as a result of the lockdowns curtailing the number of infections, subsequently eroding the support for such lockdowns. A second reason for reduced compliance during later lockdowns is that we have seen a reduction in trust in the government and its policies. In the Netherlands, protests against lockdown measures occurred frequently and the sentiment at these protests was often intense and soaked in conspiracy theories. Although it is impossible to firmly establish that this decline in trust was the result of the intermittent lockdowns in the Netherlands, we have argued that trust in the government is likely to suffer from policies that can be seen as haphazard and inconsistent. Legally, besides ensuring precaution, proportionality, solidarity and upholding fundamental rights to the success of an intermittent lockdown would be dependent on a state's ability to do so in keeping with the principle of legal certainty. Legal certainty is the principle that holds that law must be clear, accessible, not vague in its meaning, predictable and executable by citizens. When rules are subject to constant change, or are inconsistently interpreted, it becomes too difficult for citizens to follow the rules. The principle of legal certainty is enshrined in the rule of law principles such as those based on Article 2 of the Treaty on the European Union, but also as part of the standing case law of the European Court of Human Rights. The downside of an intermittent lockdown would be the difficulty for public authorities to ensure legal certainty. The success and legal feasibility would depend on the clarity of the plan, the transparency, coordination and public organization that can be brought to bear, and of course the communication of the rules to the public (69). In practice, we have already seen some more adhoc practices of intermittent lockdowns across the world, with added complexities of legal exceptions, and much confusion about the conditions under which lockdowns are leveled up or down. The loss of legal certainty here comes down to a loss of legitimacy and potential loss for compliance. This would have to be prevented by ensuring legal certainty; through at least a clearly marked roadmap with transparent and well-reasoned arguments for leveling up or down also allowing citizens and business to ensure compliance and prepare their affairs for the changing public health circumstances. 35 Protecting the Vulnerable This exit strategy does not distribute interventions over time, but over subpopulations defined in terms of vulnerability. It aims to protect those people who have a relatively high risk of mortality or complications due to COVID-19. Protective measures would be in place specifically for this group, while people with relatively low risk of mortality or complications due to COVID-19 would live life as normal as possible. The group of people with relatively low risk could build herd immunity, without risking lives of people in the relatively high risk group. In addition, the strategy must be implemented with a strict separation between groups, because allowing low-risk groups to roam freely without restrictions will automatically increase the probability of infection for high-risk groups. Moreover, due to the unknown long-term medical effects of COVID-19, building herd immunity in low-risk groups involves a public health gamble governments may not be willing to take. The categorization of belonging to a high versus low risk group needs to be well-defined before implementing this strategy. This is problematic, as it is hard to decide who is vulnerable and who is not; in fact, these allocations are probabilistic and as such will generate false positives and negatives (70). Simply drawing an age line may be insufficient, as age is not perfectly correlated to vulnerability. Also, allocations based on e.g. statistical models may not be perceived as fair: some people may see themselves as vulnerable, but would not be categorized as such, and vice versa. Thus, a government agency will likely have to decide whether an individual is to be categorized as vulnerable or not. The public’s sense of safety and trust in authorities and science may be undermined in this process if people become seriously ill who previously were deemed not at risk. Analyses suggest that targeted lockdowns can substantially reduce the death rate for a given level of economic activity. (71) consider different risk groups in a SIR model without behavioral responses and derive the optimal policy function. For their calibration, they find that optimal semi-targeted policies reduce death rate substantially compared to optimal uniform policies for a given decline in GDP (24%). For instance, special super market hours when only the elderly can shop, reduced death rate from 1.83% to 0.71%. Note that while there is a lot of model uncertainty, the finding that targeted policies can have potentially large effects is quite robust to different specifications and continues to hold in combination with other policies like testing and tracing. A strict and long lockdown for the high-risk group both reduces infections and enables less strict lockdowns for the lower-risk groups. (72) allow for behavioral responses but do not solve for the optimal policy. They argue that it is key to test the young to reduce the negative externality they impose on the old. In their model, general lockdown policies (also for the young) have the disadvantage that they delay herd immunity and thereby increase the cumulative exposure over time for the old. They find that targeted lockdowns for the old can keep the death rate low at modest economic costs. However, despite the lower death rate this policy is welfarereducing for the old. When a vaccine is available, the best policy is to keep infections low and prevent overshooting. Vaccines allow us to reach herd immunity without costly infections and hospitalizations (73). Psychologically, the effects of this exit strategy are related to the differential treatment of subpopulations, similar to the effects of phased lift of control, which produces a similar mechanism except that subpopulations are geographically defined. There is an important difference, however. Geographical location is an objective characteristic that can be determined 36 with certainty, while vulnerability is an abstract concept that has no deterministic relations to observable features. Since vulnerability itself is hard to measure directly, differential treatment will have to be based on an observable proxy, like age or medical status. Differential treatment of age will come down to discriminatory practice, as e.g., older citizens will have to be restricted in their activities, and will likely be perceived as unfair by a part of the population. In addition, vulnerable citizens may be isolated for prolonged periods of time, leading to considerable psychological distress and feelings of loneliness. The principle of non-discrimination is key when it comes to determining the potential of an exit strategy that would use vulnerability as a benchmark for rule making. Public health rules cannot discriminate without a clear justification between persons, even in the context of a public health emergency (74) If this discrimination amounts to degrading treatment, the emergency situation cannot be used to justify the infringement of the states prohibition to use inhuman and degrading treatment (75). This does not mean that using vulnerability as a deciding criterion in an exit strategy is necessarily discriminatory, but it depends on the specificity of the measures. If a group is deemed vulnerable based on indicators such as age and health status and is limited in their freedom or equal access to public services, this would entail discrimination based on age or ability. However, if these measures would only create particular privileges for these groups, such as safe moments to go grocery shopping and go to the library, this would not immediately create a legal problem of discrimination. So for instance if elderly were prohibited to shop at certain hours of the day, this would entail discrimination based on age. However, if there were special hours of the day made available for safe shopping for the elderly, this is most likely fully in line with anti-discrimination laws, particularly given that the public health emergency warrants justifications of un-equal treatments. Punctuated pragmatism Over 2020, during the exit process from the first lockdowns, it has become apparent that governments rarely follow consistent strategies that are defined a priori, as is the case for the strategies discussed above. Instead, governments have followed the strategy of punctuated pragmatism. This strategy consists of punctuated decision moments, i.e., decisions that are spaced in time (about three weeks in the Dutch situation). At each decision moment, lifting interventions have been considered on the basis of a diverse set of criteria that varied with the political and economic tide and the level of scientific research into the transmission and etiology of COVID-19. The punctuated pragmatism strategy is typically guided by pre-specified indicators (e.g., intensive-care uptakes) and imposes and lifts regulations depending on a specific threshold with respect to these. As such, punctuated pragmatism is crucially dependent on adequate monitoring and interpretation of these indicators. An example is the Corona Dashboard produced by the Dutch Government (76). This dashboard gives an overview of some of the most important indicators of the present infection rate in the Netherlands. Signal thresholds are established based on expert consensus, and when one of these thresholds is reached, the government deliberates whether it is necessary to take new (temporary) measures to slow down the spread of COVID-19. The dashboard also contains regional information, which makes it possible to impose measures locally rather than nationally. 37 Economic effects of punctuated pragmatism are likely positive in the short run, if the strategy is effective. Over the summer of 2020, when many governments lifted lockdowns gradually, economic activity increased markedly and many businesses had the opportunity to cut at least some of their losses. However, for many countries, the strategy featured overshooting in the lifting of interventions, as too much activity was allowed, resulting in a second wave. This may be due to signal values for the lead indicators being chosen inadequately, to measurement error in the indicators, or to leakage in the effectiveness of interventions (e.g., through reduced compliance). It is clear that few countries were effective in their implementation of punctuated pragmatism and thus the economic benefits of this strategy should be assessed with care. Psychologically, punctuated pragmatism is a sensible exit strategy as long as it succeeds. Because governments do not stick to a priori determined decision criteria, but develop new criteria and decisions as events unfold, they can provide continuous updates with respect to the rationale for their interventions. In addition, they can backtrack on interventions that seem to have adverse effects. This is likely to be valued as worthwhile by a part of the population, but it can be viewed as inconsistent and unclear by another part. In addition, failure to successfully manage the epidemic (as has been manifested in many countries) can erode public trust in the government, health agencies, and science. Legally the downsides and conditions for success here are similar to those for an intermittent lockdown. Legal certainty is the major downside of an approach of punctuated pragmatism. However, this can be remedied by the modes of implementation and the level of planning ahead that governments are able to muster. Appendix A3: Robustness Analysis Several of the exit strategies discussed above have been implemented in a computational model (the virsim model by (21)), enabling a quantitative analysis of these strategies. However, the model and its strategy implementations have uncertainties, e.g. concerning the choice of specific numerical values for model parameters. In this section we use mathematical tools to analyze how robust the model-simulated impact of a strategy is under such uncertainties. For the analysis of robustness we choose plausible probability distributions for model input parameters that are considered uncertain. We then aim to characterize the range of the model simulation output for the selected exit strategies (more precisely, the probability distribution of the output), given these input probability distributions. This type of analysis is known as Uncertainty Quantification (UQ) (26). For a recent UQ study of a COVID-19 epidemiological model, see for example (7) and (77). Starting from this UQ perspective on model uncertainties, a (model-based) quantitative analysis of robustness can be approached in the following way: one chooses a so-called Quantity of Interest (QoI), i.e., an important indicator or quantity that is part of the model output, as well as a threshold for this QoI. The model input uncertainties lead to model output uncertainties and hence also to uncertainty in the QoI. We analyze the exit strategies Intermittent Lockdown, Phased Lift of Control, and Contact Tracing; see S1 Appendix in (21) for a complete description of the model input parameters and their uncertainties. 38 An example of a QoI is the peak number of COVID-19 patients in intensive care (IC) units (maximum taken over a time span of, say, two years). If we set a threshold of e.g. 109 IC patients per million inhabitants (the maximum IC capacity for the Netherlands as estimated in (22)) we may call an exit strategy robust when, given all the model parameter uncertainties, the chance that the peak remains below 109 in the model is at least 95%. Determining the probability distribution of the QoI must be done by estimation based on model simulations. It is rarely the case that these properties can be determined exactly using purely theoretical/analytical methods. Instead, simulation-based estimation gives an approximation of the percentile, or other properties of interest. The computational cost of the required simulations can be very high, so that substantial computer resources may be needed to perform the simulations for a comprehensive analysis of uncertainties. It must be emphasized that the robustness analysis as sketched above takes the computational model (in this case, the virsim model) as given. It does not involve validation or verification of this model, i.e., no analysis of the validity or correctness of the model as a computational representation of real-world phenomena were performed. Similarly, it does not include analysis of potential errors in model form or model structure. Thus, the computational model is assumed to be valid and correctly implemented, and the only sources of uncertainty are assumed to be the model input parameters. We refer the reader to (21) for the complete analysis. There, an overview of the computational model, of the key UQ concepts, and of the types of uncertainties that might arise in a modelbased analysis, are provided at the beginning of the section Methods and model. This is followed by a discussion on the chosen QoIs and the considered uncertainties in the input parameters for each strategy, and by a description of the computational UQ framework employed. The Results section in (21) provides a more extensive analysis and interpretation of the model outcomes than what has been reported here. It includes the study of the cumulative distribution function of the QoIs and of their sensitivities to the uncertainties in the input parameters. It is also explored if there is a safe operating space, a region in parameter space with combinations of parameter values that lead to favourable model-predicted outcomes. Lastly, the benefits and limitations, together with some remarks on how to scale up such analysis in order to be executed in a timely fashion (and potentially inform policy-makers) are debated in the Discussion section. Additional details on the technical aspects of the analysis are provided in the Appendices of (21). 39