Scenario models for COVID-19 aim to bound the uncertainty around future outcomes over some time frame. Each potential scenario is composed of one or more variables (which could be, for example, biological parameters such as the rate of immunity waning, or assumed future changes in contact behaviours) deemed plausible. The scenario models encode the influence of these variables on COVID-19 outcomes, and results from scenario models show possible outcomes within the given timeframe assuming that present causal relationships hold in the future. The definition of scenario variables and values depends on the intended use of scenario results. One purpose of scenario modelling is to support comparisons of possible actions taken in the present with long term effects. Separately, scenarios can also be used to understand the sensitivity of future outcomes to varying parameter conditions.
Each single model provides a framework for estimating the upper and lower limits of uncertainty in COVID-19 outcomes corresponding to each scenario. However, there is no objectively best framework for modelling the transmission of SARS-CoV-2 and resulting dynamics of COVID-19, meaning the true range of uncertainty may be most appropriately expressed by results from multiple alternative models.
We created the European Scenario Hub to bring together scenario modellers for the purpose of better understanding possible COVID-19 futures in a way that informs short-term policy strategies for managing COVID-19 across Europe. This follows previous work to produce collaborative forecasts and scenarios in Europe and the United States (https://covid19forecasthub.org/, https://github.com/KITmetricslab/covid19-forecast-hub-de, https://covid19scenariomodelinghub.org/). The following text aims to ensure transparent principles and criteria to guide all participants in the Hub and interpretation of Hub results.
The short-term goals of the European Scenario Hub are:
- Creation and visualisation of open access scenario projections
- Collaboration among hub participants (that improves the quality of projections)
- Identifiable potential for use of results in policy decision-making
- Ethical communication of scenario projections to the public;
- Secondary analysis of scenario hub projections including
- Degree of agreement between different models for the same scenario
- Understanding the impact of choices of model structure and complexity on projected futures
- Methods for combining scenario results
Long term goals include:
- Creating an open database of standardised, comparable scenario projections that are easily accessible for secondary analyses by the wider research community
- Identifying benefits or drawbacks of collaborative improve real-time scenario modelling with implications for future work
- creating a self-sustaining community of European scenario modellers
This protocol addresses the first four of the short-term aims of the Scenario Hub.
We are creating scenarios at 4-8 week intervals with each set of scenarios (a round) projecting over a future 3-12 months. Our framework for setting scenarios in each round is usually a 2x2 matrix composed of two variables, each with optimistic and pessimistic parameter values for its influence on COVID-19 outcomes.
We usually include one variable that is biologically intrinsic to COVID-19, and one variable that can be influenced extrinsically by national-level policy action. For each variable we identify one or more observable parameters that represent the causal pathway from the variable to the outcome.
For each scenario round, we collaborate with the European Centre for Disease Prevention and Contorl (ECDC) to define the relevant variables, based on their interactions with national policy-makers and assessment of possible policy options. We will draw on the epidemiology of COVID-19 in Europe to suggest themes for the biologically intrinsic variable. After an initial consultation with the ECDC, we share and seek suggestions for the scenario variables from participating teams.
Each of the two variables can be expressed using one or multiple parameters representing an observable value on the causal pathway between the variable and outcome. We can explore the uncertain impact of each parameter by identifying the two most plausible parameter values that would create either the best possible (optimistic) or worst plausible (pessimistic) COVID-19 outcome.
Modelling teams do not have to include the underlying variables in their model, but the model structure should be able to include each explicit parameter value. Additional implicit parameters may be relevant to each variable but not quantitatively specified in the scenario definition. Whether these parameters are included in a model is at each teams’ discretion.
For each scenario round, after defining the variables we will collaborate with teams to define which parameters are explicitly specified and at what value. Teams must be able to include the explicit parameters, and we do not specify any other restrictions on the type or structure of models that teams use.
We focus on weekly incident numbers for any of the three following COVID-19 outcomes: cases, hospitalisations and deaths. We collate simulated trajectories from contributing scenario models as samples that are equally likely given the conditions laid out in the identified scenarios described above. We collect metadata from teams in the form of a structured abstract. This may vary slightly with each round of scenarios. See the technical Wiki for more details on the submission process and requirements.
We synthesise results both by exploring differences between and within models, and by creating ensembles from the combination of all the models. Different ensemble choices will be considered and differences between outcomes presented by different models discussed with the modellers in order to avoid hiding differences from structural or parametric assumptions by taking the ensemble.
For each scenario round, we will create a standardised visualisation allowing all models’ scenario projections to be compared. This will be presented on the website and we will follow this with a narrative summary of model results, including information taken from model abstracts to demonstrate differences between teams’ approach to and results from scenarios.
We aim to choose plausible scenarios that can inform policy but do not expect any of them to be an exact characterisation of the future. We therefore do not expect data to align with any one model result. Moreover, observed data would not align with any scenario output if the scenarios excluded a parameter that operated within the relevant timeframe and had substantial confounding effects across all of the scenarios.
Originally set up in collaboration with the Epiforecasts team the hub is now maintained by the European Centre for Disease Prevention and Control.