Hostname: page-component-7c8c6479df-24hb2 Total loading time: 0 Render date: 2024-03-27T10:25:46.079Z Has data issue: false hasContentIssue false

Disease prioritization: what is the state of the art?

Published online by Cambridge University Press:  16 April 2015

V. J. BROOKES*
Affiliation:
Faculty of Veterinary Science, The University of Sydney, Camden, NSW, Australia
V. J. DEL RIO VILAS
Affiliation:
Pan-American Centre for Foot-and-Mouth Disease (PANAFTOSA), Pan American Health Organisation, Rio de Janeiro, Brazil
M. P. WARD
Affiliation:
Faculty of Veterinary Science, The University of Sydney, Camden, NSW, Australia Marie Bashir Institute for Infectious Diseases and Biosecurity, The University of Sydney, Camperdown, NSW, Australia
*
*Author for correspondence: Dr V. J. Brookes, School of Animal and Veterinary Science, Charles Sturt University, Wagga Wagga, NSW, Australia. (Email: viki.brookes@bigpond.com)
Rights & Permissions [Opens in a new window]

Summary

Disease prioritization is motivated by the need to ensure that limited resources are targeted at the most important problems to achieve the greatest benefit in improving and maintaining human and animal health. Studies have prioritized a range of disease types, for example, zoonotic and foodborne diseases, using a range of criteria that describe potential disease impacts. This review describes the progression of disease prioritization methodology from ad hoc techniques to decision science methods (including multi-criteria decision analysis, conjoint analysis and probabilistic inversion), and describes how these methods aid defensible resource allocation. We discuss decision science in the context of disease prioritization to then review the development of disease prioritization studies. Structuring the prioritization and assessing decision-makers' preferences through value trade-offs between criteria within the decision context are identified as key factors that ensure transparency and reproducibility. Future directions for disease prioritization include the development of validation techniques, guidelines for model selection and neuroeconomics to gain a deeper understanding of decision-making.

Type
Review
Copyright
Copyright © Cambridge University Press 2015 

INTRODUCTION

Prioritization in any context is used to establish the most attractive alternative, or group of alternatives, as a decision aid for resource allocation. In human and animal health, resources need to be allocated to preventive activities such as biosecurity, vaccination, public education and routine health screening, early warning of disease incursions through surveillance, and instigation of control measures in the case of a disease incursion, among other needs. The choices for resource allocation are effectively limitless; better understanding of epidemiology and pathophysiology have both increased the potential choices for disease prevention and control, and expanding human populations, increased trade and travel, and the emergence and re-emergence of pathogens are further increasing options for resource allocation. However, resources are constrained by finite capital, as well as by the availability of people and the time required to deliver effective prevention and control methods or to undertake research. Therefore, disease prioritization in animal and human health is motivated by the need to ensure that the allocation of limited resources is targeted to achieve the greatest benefit in improving and maintaining human and animal health.

Defining disease importance is difficult. Diseases can have social, economic and environmental impacts, and most diseases cause a variety of impacts that span these broad categories. In addition, impacts vary in scale and are often intangible, and the perception of importance of each impact varies between decision-makers [Reference Wilson, Ward and Garner1]. Economic impacts include the cost of prevention and control and lost productivity due to disease and its perceived risk (e.g. SARS was estimated to cost between US$ 30–100 billion across many sectors, but particularly travel and tourism [Reference Smith2]). Social costs include impacts on individuals and communities – both through the psychological and social impacts of disease burden and loss – and also through altered lifescapes as was seen in the UK foot-and-mouth-disease (FMD) outbreak in 2001 [Reference Mort3]. Environmental impacts include altered ecological balance due to changes in species distribution and abundance [Reference Daszak4]. These areas of impact also overlap – for example, economic impacts can lead to social impacts – which further complicate comparison of disease impacts. To determine the most important diseases for resource allocation, prioritization should not only define and compare the impact of diseases, but also the importance of those impacts to relevant stakeholders, of which there can be multiple groups. In addition, the prioritization process must be transparent and reproducible – and therefore auditable – and provide valid and robust results to direct decision-makers in the development of defensible animal and human health policy [Reference Giesecke5]. Disease prioritization is therefore a complex decision process; the overall objective is important – we want maximum benefit to animal and human health – and there are multiple, and often conflicting, criteria that not only have different scales or are intangible, but also differ in value depending on the perceptions of the decision-maker.

Methods from the discipline of decision science have been used extensively to aid complex decision-making in fields such as information technology and engineering, but have had relatively limited application in health sciences [Reference Bragge, Ehrgott, Naujoks, Stewart and Wallenius6]. However, multi-criteria decision analysis (MCDA), a family of methods from decision science, has been recently used for disease prioritization [Reference Del Rio Vilas7Reference Havelaar9]. MCDA methods are valuable tools for decision-making in animal and human health, because they account for varying scales and perceptions of impacts using a structured, transparent approach, and aim to give consistent, reproducible results. This review describes the development of disease prioritization as a methodology by illustrating its progression towards methods from the discipline of decision science, particularly MCDA. As a foundation, the requirements of disease prioritization in the context of decision theory are discussed. This is followed by an overview of decision analysis and MCDA relevant to disease prioritization. The development of disease prioritization applications from ad hoc methods to a standardized format is then reviewed. Finally, recent developments and how they aid defensible resource allocation, as well as future directions in disease prioritization are examined.

DECISION THEORY AND THE REQUIREMENTS OF DISEASE PRIORITIZATION

Decision theory forms the basis of decision science and can be divided into three areas: normative, descriptive, and prescriptive [Reference Anand, Pattanaik and Puppe10]. Each of these divisions has important implications for disease prioritization. First, the results of prioritization studies must be defensible, which requires that the method is transparent and logical. This relates to normative decision theory, the basis of which is rational choice. For a decision to be considered rational, an individual should prioritize the disease that maximizes their ‘expected utility’ [Reference Bernoulli11]. ‘Utility’ is a measure of disease importance comprising both the scale and value of impacts to that individual (we note here that for disease we can also refer to this as ‘dis-utility’), and ‘expected’ refers to the individual's estimate of the probability of occurrence of the disease. Normative decisions comply with a set of logical rules, or axioms (Table 1) and can be described by a utility function [Reference Ramsey12, Reference von Neumann and Morganstern13]. Equation (1) shows an example of a utility function in which an individual's expected utility (U) for disease (i) is described as a function of the sum of all the values of the possible overall impacts of the disease (possible disease states, u ij ) and the probability of occurrence of each of those possible disease states (p j ). The axioms that describe rational choice – in particular, completeness and transitivity – have traditionally been used as the ‘gold standard’ to define rational choice, against which decisions are judged as logical; normative decision theory is the dominant underlying theory in decision science, particularly in economics [Reference Jallais, Pradier and Teira14]. In the context of disease prioritization actions need to be defensible, particularly in the face of disease events, when questions are asked such as, ‘given the impact of this event, were the actions justified and should resources have been directed differently?’ Using a decision-aid based on normative theory – therefore complying with the axioms of rational choice – gives rational results, inherently confirming that decisions were logical and that the process was transparent and therefore, auditable [Reference Savage15].

(1) $${U_i} = \sum\limits_{\,j = 1}^n {{\,p_j}{u_{ij}}}. $$

Table 1. Axioms of rational choice associated with normative decision theory, and their relevance to disease prioritization

The second requirement of disease prioritization is that the results should be valid, meaning that they should accurately reflect the priorities of the decision-makers. Normative decision theory would be sufficient to validly describe priorities if decision-makers generally made normatively rational decisions by considering the overall utility of each disease. However, it has been demonstrated that decision-makers systematically violate the axioms of rational choice, and consequently their decisions cannot be validly described with a utility function [Reference Tversky and Kahneman16Reference Edwards20]. Normative decision models are static and isolated in their decision context; their inability to account for temporal and complex social constraints can result in apparently irrational choices. Moreover, differences in the way the brain reacts to various types of information (e.g. risk and ambiguity) as well as differing decision-making heuristics can also affect choice [Reference Dieckmann, Dippold and Dietrich21, Reference Payne, Bettman and Johnson22]. For example, Hsu et al. [Reference Hsu23] demonstrated that aversion to choice under ambiguity (epistemic uncertainty) generally overrides choice with stated probabilities and suggest that there is a neural basis for choice. Einhorn [Reference Einhorn24] proposed that in some contexts, people use simplifying heuristics to make decisions instead of considering the overall utility of each alternative. Making sense of the apparently illogical choices that humans make is the field of descriptive (or behavioural) decision theory. An example of a descriptive theory of decision-making is prospect theory, developed by Tversky and Kahneman [Reference Tversky and Kahneman16, Reference Kahneman and Tversky25] to account for the psychological effects of framing, loss aversion and anchoring on rational decision-making. Complying with descriptive decision theory is not just important to produce valid results in prioritization, it is also important for designing decision support frameworks to ensure that decision-makers’ preferences are represented as accurately as possible. However, in all decision processes there is a conflict between descriptive and normative decision theory in producing valid results that are also rational, and therefore, defensible.

Prescriptive decision theory attempts to connect the mathematical ideals of normative theory with the psychology of descriptive theory. Throughout the last century many prescriptive theories have been suggested, ranging from ones that follow mainly normative rules – such as von Neumann & Morgenstern's [Reference von Neumann and Morganstern13] game theory that accounted for strategic interaction (including social constraints) in decision making – to others that relax or vary the definitions of the axioms of rational choice to account for choice consistent with descriptive theory. Examples include Simon's [Reference Simon26] concept of bounded rationality to account for constraints in decision-making and models developed by Einhorn [Reference Einhorn24] to describe choice using simplified decision-making heuristics. Recently, it has been suggested that full understanding of decision processes will only be reached through understanding the biological basis of decision-making, and that consequently, the fields of economics, psychology and neuroscience are converging to achieve this goal in a new branch of decision theory called neuroeconomics [Reference Glimcher and Rustichini27, Reference Wilson28]. Until such a state of understanding is reached, there needs to be a balance in achieving validity and rational defensibility through current methods; decision-makers should be aware of this limitation in all decision contexts, not just disease prioritization. The next section gives an overview of available techniques for decision analysis, which aim to convert decision theory into practical decision-making.

DECISION ANALYSIS

Decision analysis is applied prescriptive decision theory [Reference Howard29]. Formal methods for complex decision analysis include decision-trees, monetary-based methods (e.g. cost-benefit analysis), and MCDA [Reference Dodgson30]. MCDA comprises a large group of methods that provide a framework to compare alternatives with diverse and often intangible impacts (broader than monetary-based techniques) by incorporating value judgements on multiple, conflicting criteria with incommensurate scales [Reference Triantaphyllou31]. Consequently, MCDA has become popular for disease prioritization [Reference Del Rio Vilas7Reference Havelaar9] and we focus on MCDA in this section.

MCDA methods can be divided into multi-attribute decision-making (MADM) and multi-objective decision-making (MODM) [Reference Triantaphyllou31]. MODM methods are designed to find optimal criterion values for decision problems in which a solution is required from a range of continuous objectives [Reference Cohon32]. They are commonly used in fields such as engineering and environmental science, in which resource allocation might be based on finding the optimal combination of a range of resources that have constraints (e.g. cost and site size for hazardous waste disposal [Reference Alumur and Kara33, Reference Giannikos34]). By contrast, MADM methods are used to rank or group alternatives from a finite set of discrete decision alternatives based on comparisons between criteria; therefore, MADM methods are suited to disease prioritization. Diseases are the decision alternatives, and disease attributes that describe disease impacts – such as incidence or severity of clinical signs – are the criteria on which prioritization is based.

Normative rationality dominates prescriptive decision theory, including MADM. This logical approach is achieved by systematic structuring of the decision problem, and is described in detail by von Winterfeld [Reference von Winterfeldt35] and Keeney [Reference Keeney36]. Figure 1 shows a representation of the steps of decision analysis for disease prioritization, modified from Keeney [Reference Keeney36]. In the case of MADM, the underlying structure of the decision process is the performance matrix (Fig. 2) created in steps 1 and 2 of decision structuring (Fig. 1) [Reference Dodgson30, Reference Triantaphyllou31]. Diseases for prioritization are identified, criteria by which the disease impacts can be described are selected, and criterion measurements are collected for each disease. For example, the criterion ‘case-fatality rate in humans’ might be estimated to be 30% for Japanese encephalitis [Reference Brookes8]. Criterion weights are described separately by evaluating the trade-offs that decision-makers’ are prepared to make between criteria [Reference Keeney37]. For example, the weight of the criterion ‘case-fatality rate in humans’ could be 0.75, following evaluation of ‘case-fatality rate in humans’ in the context of all other criteria relevant to the prioritization. Criterion weights are not disease-specific. The weights allow comparison of criteria with incommensurable scales, and have previously been described as ‘scaling constants’ [Reference Del Rio Vilas, Montibeller and Franco38]. Objective measurements for the criteria for each disease are then aggregated with criterion weights according to the MADM method. Separation of objective measurements of criteria from their subjective weights gives transparency and reproducibility to the process, and also can remove cognitive bias that decision-makers might have when directly evaluating the importance of diseases.

Fig. 1. Schematic representation of the steps of decision analysis for disease prioritization. (Modified from Keeney [Reference Keeney36].)

Fig. 2. Performance matrix for disease prioritization. D, Disease identity, C, criterion; w, weight for criterion; a, measurement for each criterion for each disease. (Modified from Brookes et al. [Reference Brookes8].)

Although all MADM methods follow this structured approach, there are different techniques to evaluate decision-makers’ preferences and aggregate information, and choosing the most appropriate MADM method is not straightforward – in fact making a decision about which decision-making model to use is known as the decision-makers’ paradox [Reference Triantaphyllou and Mann39, Reference Guitouni and Martel40]. Axioms of rational choice are relevant throughout the decision process shown in Figure 1; steps 1 and 2 require that at least two decision alternatives can be defined for which decision-makers can state preferences (axiom of completeness), and steps 3 and 4 maintain that decision-makers preferences should be transitive, monotonic, and independent of irrelevant alternatives. Compliance with normative axioms at steps 3 and 4 is relaxed for methods that follow descriptive theory. The aim in selecting a MADM method should be to match the expected heuristics of the decision-makers, and the most appropriate MCDA method will give the best compromise between normatively rational and descriptively valid results. However, in practice the selection of a MADM method is often based on the decision analyst's familiarity with techniques, the time available, and the perceived complexity of the decision problem [Reference Guitouni and Martel40]. MADM methods include elementary methods (pros-and-cons analysis, maximax and maximin methods, conjunctive and disjunctive methods, lexicographic methods), outranking methods and multi-attribute utility theory (MAUT) or multi-attribute value theory (MAVT) based methods.

Elementary methods reflect simple heuristics using a descriptive approach to decision-making, such as those suggested by Dieckmann et al., Payne et al., and Einhorn [Reference Dieckmann, Dippold and Dietrich21, Reference Payne, Bettman and Johnson22, Reference Einhorn24]. They are non-compensatory in that the primary criterion of concern cannot be outweighed by combinations of other criteria, and the criteria are not weighted. For example, when using lexicographic methods, one criterion is chosen as more important than all the others and initial ordering is based on this primary criterion. Subsequent ordering of choices depends on a secondary criterion, and so on, until all the alternatives have been separated and ordered. Outranking methods are also predominantly descriptive, and are considered to be a more sophisticated extension of elementary methods because they evaluate preferences of decision-makers to assign weights to criteria, reflecting more complex heuristics; there are many methods within this group – also termed the ‘European’ or ‘French school’ of decision analysis – such as ELECTRE (Eliminating Choice and Translating Reality [Reference Roy41]) and PROMETHEE (Preference Ranking Organization Method for Enrichment Evaluations [Reference Brans and Vincke42]).

The MAUT and MAVT group of MADM methods follow a normative approach. MAUT methods for use in decision analysis were developed by Keeney & Raiffa [43] and aggregate information according to a utility function [described in equation (1)]. MAVT methods are a simplification of MAUT methods in that they do not explicitly incorporate decision-makers’ estimates of the probability of occurrence of all possible outcomes of an alternative. MAVT methods have been most commonly used for disease prioritization; subjective weights and objective measurements for the criteria are aggregated for each disease according to a value function. An example of a weighted-sum value function is given [equation (2)] in which the value (D) of a disease (i) is a function of the sum of the weights (w j ) for the criteria combined with criterion measurements (c ij ) for each disease. Examples of the use of MAVT methods include prioritization of emerging zoonoses in The Netherlands [Reference Havelaar9], zoonoses and diseases of food-producing animals [Reference Humblet44], ranking emerging threats in the UK [Reference Del Rio Vilas7] and prioritization of exotic diseases of pigs in Australia [Reference Brookes8]. These methods are considered to be compensatory; diseases are ordered or grouped according to their overall utility, and disease x can be prioritized over disease y based on the aggregated utility or value of different combinations of criteria.

(2) $${D_i} = \sum\limits_{\,j = 1}^n {{w_j}{c_{ij}}}. $$

Elementary, outranking and MAUT or MAVT methods are not rigid categories. The analytical hierarchy process (AHP) is sometimes categorized as a MAUT method, but because it is based on measurement theory it falls outside the three main groups [Reference Saaty45]. The AHP is implemented in terms of evaluations of weights through direct pairwise comparison of criteria on a Likert-type scale of relative importance. This method has recently been used in a Japanese study to prioritize zoonoses [Reference Kadohira46]. Other methods, such as EVAMIX (EVAlutions with MIXed data), have been developed to include both outranking and MAUT/MAVT methods; EVAMIX was used by Mourits et al. [Reference Mourits, van Asseldonk and Huirne47] to evaluate control strategies for classical swine fever in the European Union.

Currently, there are no guidelines for selection of MADM methods for decision analysis in any field; there is a need to develop procedures to select methods appropriate to the decision context that most accurately reflect decision-makers’ priorities. Until procedures are developed, one of the most important requirements for decision analysis is development of methods to assess the validity of results, particularly when using normative-based methods such as MAUT or MAVT. This step is not included in Keeney's original paper on decision structuring [Reference Keeney36], but is mentioned in his later work on evaluating decision-makers’ trade-offs [Reference Keeney37]. Normative axioms dominate prescriptive theories, and validation to assess the accuracy of results will give an indication of how confident the analyst can be that subsequent recommendations reflect the decision-makers' values. Current methods used for validation are discussed in the section on future directions for disease prioritization.

THE DEVELOPMENT OF DISEASE PRIORITIZATION

This section examines the evolution of disease prioritization in the context of decision analysis. Disease prioritization is already a complex decision problem because the highest priority disease depends on comparison of multiple disease impacts, but complexity increases further because there are usually many decision-makers who represent groups of individuals with potentially diverse objectives [Reference Wilson, Ward and Garner1]. For example, one group might value environmental impacts more than economic costs, or countries or regions might have different values for the social impacts of disease; measuring the utility or value of these impacts across such diverse groups can result in a central measure that is not representative of any group preferences [Reference Brookes48]. This makes assigning utility or value to disease impacts more difficult. Initially, disease prioritization was implemented by simply considering the scale of disease impacts without their utility or value to decision-makers. In a prioritization of diseases in Ghana [Reference Nimo49], the metric ‘healthy days of life lost’ was calculated using objective measurements of disease impact in terms of severity, the average age at disease onset and death, the duration and the level of disability caused by the disease, disease incidence and case-fatality rate. As well as prioritizing diseases by impact on healthy days of life lost, this metric was used to assess the benefits of various health improvement procedures, to select those that could save the greatest number of healthy days of life relative to the cost of the procedure. Following this study, disease prioritization moved in two directions: further development of metrics to quantify disease burden [e.g. disability adjusted life years (DALYs), used by the World Health Organisation (http://www.who.int/healthinfo/global_burden_disease/metrics_daly/en/)], and development of an approach for disease prioritization structured according to decision-science methodology to reflect the value of disease to those directing the prioritization.

Disease prioritization to incorporate the value of disease impact to those directing the prioritization has developed at regional, national and industry levels. Initially, studies were not structured to separate disease criterion measurements and criterion weights. A Canadian study initiated in 1987 prioritized communicable diseases for national surveillance using a point scoring system in which decision-makers directly scored each disease against ordinal scales of severity for each criterion [Reference Carter50]. Similar scoring systems were also used in the 1990s to establish priorities for communicable disease surveillance in Europe [Reference Weinberg, Grimaud and Newton51] and resource allocation in England [Reference Rushdy and O'Mahony52], as well as for prioritization of non-foodborne zoonoses in France [Reference Valenciano53] and communicable diseases for surveillance in a repeat of the original 1987 Canadian study [Reference Doherty54]. Carter et al. [Reference Carter55] had noted that a rational system was needed for disease prioritization, and Rushdy & O'Mahony [Reference Rushdy and O'Mahony52] stated that prioritization should be an ‘open process of communication and consultation’. The overall benefit of these early studies was to introduce disease prioritization as a systematic process to direct resources through consultation between decision-makers.

In a commentary in The Lancet, Giesecke [Reference Giesecke5] noted that transparency was one of the main challenges in disease prioritization. Subsequently, prioritization studies introduced concepts from decision analysis to improve transparency by separating criterion measurements from criterion weights. Examples include three European studies: prioritization of foodborne zoonoses by Cardoen et al. [Reference Cardoen56], communicable disease prioritization by Balabanova et al. [Reference Balabanova57] (initiated by Krause et al. [Reference Krause58] and reviewed by Gilsdorf & Krause [Reference Gilsdorf and Krause59]), and animal and zoonotic diseases by Humblet et al. [Reference Humblet44]. As well as incorporating problem structuring, each of these studies aggregated information based on MAVT using a linear weighted sum value to give a score for each disease. Overall, this led to improved transparency by clearly separating decision-makers’ subjective opinions regarding the value of criteria from measurements for individual diseases, as well as reducing opportunity for cognitive bias that can arise when directly valuing diseases.

In addition to structuring prioritization studies to separate criterion measurements and weights, it is essential to evaluate decision-makers preferences to validly reflect their priorities (step 3, Fig. 1). In a review of common mistakes made during this step, Keeney [Reference Keeney37] highlighted the importance of evaluating preferences by asking decision-makers to make trade-offs between criteria, as well as describing some of the difficulties encountered at this stage. In particular, it was noted that not knowing the context of the trade-off (the scale and potential range of consequences) can lead to incorrect value judgements, potentially making the decision analysis results invalid. This is consistent with Steele et al. [Reference Steele60], who showed the importance of integrating the scale of criterion measurements with value judgements of the criteria; the authors demonstrated that the order of alternatives could be changed by simply changing the scale of criterion measurements while values judgements were held constant. Some aspects of the importance of correctly evaluating decision-makers’ preferences (as well as other challenges associated with disease prioritization) are also described by Del Rio Vilas et al. [Reference Del Rio Vilas, Montibeller and Franco38]. Ultimately, the results of decision analyses are valid only through good fortune if decision-makers’ preferences are elicited independently of scale and context; without validation, the accuracy of the results of these prioritizations cannot be assessed.

In a study to develop a framework for prioritization of emerging threats and vulnerabilities for the Department for the Environment, Food and Rural Affairs (Defra) in the UK, Del Rio Vilas et al. [Reference Del Rio Vilas7] further advanced prioritization in health settings by using established decision analysis methods to both structure the prioritization and also evaluate trade-offs. Normative rationality was provided by the MADM structure of the prioritization that separated disease measurements from decision-makers' values for criteria, and aggregated information using a linear weighted sum model based on MAVT. Swing-weighting was used to quantify disease experts' subjective preferences for the importance of criteria. This is an established method to directly weight criteria [Reference von Winterfeldt and Edwards61], and ensured that trade-offs were evaluated within the context of the decision problem and the scale of the criteria. Swing-weighting has been used previously in animal health for prioritization of control measures for contingency planning for an FMD incursion [Reference Mintiens and Vose62]. Evaluation of decision-makers’ preferences in this way is more likely to accurately reflect decision-makers' preferences and the results are more likely to be repeatable within the same group. MADM based prioritizations using swing-weighting are suited to small groups of decision-makers in the way that the Delphi technique is suited to small group expert-opinion elicitation. While the methods for traditional MCDA are repeatable within groups and reproducible across groups, reproducibility of results across groups are dependent on the selection of decision-makers in the context of the disease prioritization.

Both of these previous studies used linear weighted sum models to represent decision-maker heuristics [equation (2)] [Reference Del Rio Vilas7, Reference Mintiens and Vose62]. This model is commonly used because MAUT- or MAVT-based models with dependent or nonlinear preferences between criteria are complex to implement [Reference Dodgson30]. Although the linear weighted sum model is straightforward, an assumption of independent preferences between criteria is essential, which is restrictive and might not validly describe decision-makers’ heuristics; recent disease prioritization studies have used techniques to circumvent this assumption. These studies used probabilistic inversion [Reference Brookes8, Reference Havelaar9, Reference Brookes48] and conjoint analysis [Reference Ng and Sargeant63, Reference Ng and Sargeant64] to indirectly derive criterion weights. Decision-makers ranked groups of constructed disease scenarios that were designed to force trade-offs between different criteria. Variation in decision-makers' preferences was modelled and the studies using conjoint analysis quantified this variation to allow generalization of the results to the wider population; further development is required to allow this with probabilistic inversion. Each study assessed whether decision-makers' heuristics were normatively rational by checking that choices were transitive [Reference Brookes8, Reference Brookes48], and maximized utility [Reference Ng and Sargeant63, Reference Ng and Sargeant64]. Also, the online surveys used to elicit preferences for criteria presented realistic disease scenarios using non-technical terminology, which enabled participation of decision-makers who were not disease experts [Reference Brookes8, Reference Brookes48, Reference Ng and Sargeant63, Reference Ng and Sargeant64]. The main advantage of these methods is that they enable out-of-sample validation in which the fitted model can be used to predict decision-makers' rankings of a set of disease scenarios other than those used to fit the model [Reference Neslo and Cooke65]. Assessing the validity of models allows assessment of the accuracy of predictions, and further aids defensible resource allocation in addition to the systematic, transparent and reproducible methodology provided by decision analysis. A disadvantage of probabilistic inversion and conjoint analysis is that they rely on statistical methods to produce valid results and therefore, depending on model complexity, potentially require large numbers of participants, making them unsuitable for use in small-group decision-making.

FUTURE DIRECTIONS FOR DISEASE PRIORITIZATION

To date, out-of-sample validation has shown that disease prioritization models have variable validity [Reference Havelaar9, Reference Brookes48, Reference Neslo and Cooke65Reference Teck67]. It is unclear whether limited validation occurs because validation methods are inadequate, or whether the models are poor representations of decision-makers’ heuristics; it is difficult to develop validation methods given this structural uncertainty. Currently, MADM methods are selected on an empirical basis, and most disease prioritizations have used MAVT-based methods with a linear weighted sum model. There is a requirement to develop systematic science-based processes to select models for prioritization to improve our ability to accurately describe decision-makers' heuristics, and concurrently allow development of validation techniques, particularly for disease prioritizations such as MAUT-/MAVT-based methods which are currently not validated. At a minimum, validation provides an indication of the accuracy of model predictions. Concurrent development of model selection and validation should result in more predictive models that are therefore more valuable decision tools.

Sources of variability in disease prioritization results include heterogeneity in decision-makers’ values, as well as uncertainty and variability associated with disease impacts. In the case of MAUT methods, there is also variation in the decision-makers’ expected probability of occurrence of disease. Sensitivity analysis can be used to examine the robustness of prioritization results to variability in inputs. Similar to other methods for disease investigation (such as risk analysis and disease modelling), prioritization is subject to the same challenges in uncertainty and variability in disease measurements. Uncertainty and variability associated with disease impacts has been accommodated in disease prioritizations by including multiple disease scenarios or ranges of uncertainty or variability across levels of criterion measurements. Disease measurements can change over time, and should be updated as new information arises, especially if the results from the prioritization are highly sensitive to variation in criterion measurements. However, unlike other methods for disease investigation, prioritization results are dependent on, and can also be sensitive to, decision-makers subjective values. Subjective values are likely to be dynamic within individuals – changing as experiences change – and heterogenous throughout groups of decision-makers, particularly if the scope of the decision problem is wide. Scope in this context has two sources: the breadth of the problem (e.g. prioritization of diseases affecting more than one species has broader scope) and the diversity of the decision-makers (e.g. a prioritization study that includes multiple groups of stakeholders also has broader scope). Variation in decision-makers' preferences has been modelled using conjoint analysis and probabilistic inversion [Reference Brookes8, Reference Havelaar9, Reference Ng and Sargeant63, Reference Ng and Sargeant64]. In a study in which pig producers prioritized both zoonoses and diseases that only affect livestock it was found that producers preferences were divided by these disease types and that each group of stakeholders’ preferences were better represented using a model specific to each group [Reference Brookes48]. Although the feasibility of representing multiple stakeholder groups is dependent on the prioritization context, decision-makers should be aware of the challenges associated with increasing the scope of disease prioritizations – it is possible to produce results that are not representative of any stakeholder group or disease type [Reference Brookes8]. In addition, decision-makers’ preferences should be re-assessed periodically to ensure that preferences are current. What constitutes an adequate time-frame for re-assessment is again dependent on the prioritization context. Doherty et al. [Reference Doherty54] repeated a prioritization of diseases in Canada after a period of 10 years and found changes in priorities [Reference Carter50] – appropriate time-frames will become clearer as disease prioritizations become more commonly used and repeated.

The final potential source of variability, the probability of disease occurrence, has not been explicitly modelled in disease prioritizations that have been structured according to decision analysis methods. Utility functions, by their definition, include a measure of probability through evaluation of decision-makers’ expected probability of occurrence of either individual or grouped criteria, or the overall probability of occurrence of the decision alternative [Reference Keeney and Raiffa43]. However, in the context of disease prioritization, utility functions (MAUT-based methods) have been simplified to value functions (MAVT-based methods) in which the outcome is simply a measure of impact for a particular disease. If MAUT methods are to be developed for disease prioritization it is essential that the sources of variation are explicitly modelled, for example by producing a two- or three-dimensional output. The results of prioritization are potentially misleading if the probability of occurrence, uncertainty of impacts and variation in decision-makers' preferences are not differentiated. Ultimately, however, disease prioritization is a decision aid that must provide decision-makers with a foundation from which they can make informed, defensible choices. Given the many pathways by which a disease can occur, and the associated probabilities of each step of a pathway, adding complexity through inclusion of the probability of disease occurrence as a utility model is unlikely to aid communication between decision analysts and decision-makers. Human and animal health have well established methods for risk assessment – for example the framework for import risk analysis for animals and animal products [68]. Prioritization can be considered the first step in a chain of investigations that use a range of methods (e.g. risk analysis and disease modelling) to identify the highest impact diseases that have a perceived high probability of occurrence given those impacts, for which resource allocation will make a valuable contribution in reducing the actual or potential disease burden. Whether for exotic or endemic diseases, resource allocation needs to achieve the greatest benefit in reduction of potential or existing disease burden. This requires a secondary process beyond prioritization. MODM methods can ensure that resources are allocated to the optimum combination of prevention and mitigation strategies given the potential benefits and constraints – such as economic cost, welfare implications, and time and human resources required for implementation – of each strategy [Reference Dodgson30]. There is potential to add MODM to the range of tools available for decision-making in the context of disease prioritization.

CONCLUSION

MADM methods from decision analysis have gradually been applied to disease prioritization with the objective of improved transparency of method and validity of results. The key points in applying these methods are structuring the prioritization to separate objective measurements from subjective values for the disease criteria, and assessing decision-makers' preferences through value trade-offs between criteria within the decision context. Ideally, prioritization results should be both rational and valid. However, normative and descriptive decision theories conflict and there is a balance in prescriptive decision analysis in achieving rationality and validity. Currently, selecting an appropriate MADM method is as much art as science, and generally prioritization at regional, national and industry levels has taken a normative approach, using MAVT based on a linear weighted sum models for aggregation of information. This approach is logical and consistent, and provides results that are rational, and therefore auditable and defensible. Recently, weighting of criteria using probabilistic inversion has enabled validation of results to provide an indication of the accuracy of predictions, and conjoint analysis has enabled assessment of the reproducibility of results across similarly representative samples from the same group of stakeholders. However, further research to select the most appropriate MADM method and aggregating function is required and decision-makers should be aware of the limitations of prioritization in validly reflecting decision-makers’ heuristics. Structuring prioritization studies to separate objective disease measurements from subjective values of disease impacts is a step that is now consistently applied. However, evaluating preferences by assessing the trade-offs that decision-makers are prepared to make between criteria is not consistently undertaken. It should be noted that criterion weights are not direct measures of importance of criteria and cannot be evaluated as such – evaluation of criterion weights within the decision context is required to scale incommensurate criteria to their relative comparative values and produce a valid overall disease score. It is an essential step and should not be seen as adding unnecessary complexity to the process. Although MADM methods might be more difficult to explain to decision-makers in the short term, they are likely to become more accepted through use, and – of greater importance – decision-makers can be more confident that subsequent resource allocation reflects current disease priorities.

ACKNOWLEDGEMENTS

The first author was funded as a PhD student by Australian Pork Limited (project no. 2010/1000.381).

DECLARATION OF INTEREST

None.

References

REFERENCES

1. Wilson, SJ, Ward, MP, Garner, MG. A framework for assessing the intangible impacts of emergency animal disease. Preventive Veterinary Medicine 2013; 111: 194199.Google Scholar
2. Smith, RD. Responding to global infectious disease outbreaks: lessons from sars on the role of risk perception, communication and management. Social Science and Medicine 2006; 63: 31133123.CrossRefGoogle ScholarPubMed
3. Mort, M, et al. Psychosocial effects of the 2001 UK foot and mouth disease epidemic in a rural population: qualitative diary based study. British Medical Journal 2005; 331: 12341237.CrossRefGoogle Scholar
4. Daszak, P. Emerging infectious diseases of wildlife – threats to biodiversity and human health. Science 2000; 287: 17561756.Google Scholar
5. Giesecke, J. Choosing diseases for surveillance. Lancet 1999; 353: 344344.CrossRefGoogle ScholarPubMed
6. Bragge, J, et al. Bibliometric analysis of multiple criteria decision making/multiattribute utility theory. In: Ehrgott, M, Naujoks, B, Stewart, TJ, Wallenius, J, eds. Multiple Criteria Decision Making for Sustainable Energy and Transportation Systems: Proceedings of the 19th International Conference on Multiple Criteria Decision Making, 2010, pp. 259–268.Google Scholar
7. Del Rio Vilas, VJ, et al. An integrated process and management tools for ranking multiple emerging threats to animal health. Preventive Veterinary Medicine 2013; 108: 94102.CrossRefGoogle ScholarPubMed
8. Brookes, VJ, et al. Building a picture: prioritisation of exotic diseases for the pig industry in Australia using multi-criteria decision analysis. Preventive Veterinary Medicine 2014; 113: 103117.Google Scholar
9. Havelaar, AH, et al. Prioritizing emerging zoonoses in the Netherlands. PLoS ONE 2010; 5: e13965.CrossRefGoogle ScholarPubMed
10. Anand, P, Pattanaik, P, Puppe, C. The Handbook of Rational and Social Choice: Oxford: Oxford University Press, 2009.CrossRefGoogle Scholar
11. Bernoulli, D. Exposition of a new theory on the measurement of risk. Econometrica 1954; 22: 2336.Google Scholar
12. Ramsey, FP. Truth and probability (1926). The Foundations of Mathematics and Other Logical Essays 1931, pp. 156–198.Google Scholar
13. von Neumann, J, Morganstern, O. Theory of Games and Economic Behavior. Princeton, NJ: Princeton University Press, 1944.Google Scholar
14. Jallais, S, Pradier, PC, Teira, D. Facts, norms and expected utility functions. History of the Human Sciences 2008; 21: 4562.CrossRefGoogle Scholar
15. Savage, LJ. The Foundations of Statistics. New York Courier Dover Publications, 2012.Google Scholar
16. Tversky, A, Kahneman, D. Rational choice and the framing of decisions. Journal of Business 1986; 59: S251S278.Google Scholar
17. Ellsberg, D. Risk, ambiguity and the savage axioms. Quarterly Journal of Economics 1961; 75: 643669.Google Scholar
18. Allais, M. The behaviour of rational man given risk: critique of the postulates and axioms of the American school [in French]. Econometrica: Journal of the Econometric Society 1953: 503–546.Google Scholar
19. Simon, HA. Administrative Behavior: a Study of Decision-making Processes in Administrative Organization. New York: Macmillan Co., 1947.Google Scholar
20. Edwards, W. The theory of decision making. Psychological Bulletin 1954; 51: 380417.CrossRefGoogle ScholarPubMed
21. Dieckmann, A, Dippold, K, Dietrich, H. Compensatory versus noncompensatory models for predicting consumer preferences. Judgment and Decision Making 2009; 4: 200213.CrossRefGoogle Scholar
22. Payne, JW, Bettman, JR, Johnson, EJ. Adaptive strategy selection in decision making. Journal of Experimental Psychology: Learning, Memory, and Cognition 1988; 14: 534.Google Scholar
23. Hsu, M, et al. Neural systems responding to degrees of uncertainty in human decision-making. Science 2005; 310: 16801683.CrossRefGoogle ScholarPubMed
24. Einhorn, HJ. Use of nonlinear, noncompensatory models as a function of task and amount of information. Organizational Behavior and Human Performance 1971; 6: 127.CrossRefGoogle Scholar
25. Kahneman, D, Tversky, A. Prospect theory – analysis of decision under risk. Econometrica 1979; 47: 263291.Google Scholar
26. Simon, HA. A behavioral model of rational choice. Quarterly Journal of Economics 1955; 69: 99118.Google Scholar
27. Glimcher, PW, Rustichini, A. Neuroeconomics: the consilience of brain and decision. Science 2004; 306: 447452.Google Scholar
28. Wilson, EO. Consilience: the Unity of Knowledge. New York: Vintage, 1998.Google Scholar
29. Howard, RA. Decision Analysis: Applied Decision Theory. Stanford Research Institute, 1966.Google Scholar
30. Dodgson, J, et al. Multi-criteria Analysis: A Manual. Department for Communities and Local Government: London, 2009.Google Scholar
31. Triantaphyllou, E. Multi-criteria Decision Making Methods: A Comparative Study. Dordrecht, Boston, MA: Kluwer Academic Publishers, 2000.Google Scholar
32. Cohon, JL. Multiobjective Programming and Planning: New York: Courier Dover Publications, 2004.Google Scholar
33. Alumur, S, Kara, BY. A new model for the hazardous waste location-routing problem. Computers & Operations Research 2007; 34: 14061423.Google Scholar
34. Giannikos, I. A multiobjective programming model for locating treatment sites and routing hazardous wastes. European Journal of Operational Research 1998; 104: 333342.Google Scholar
35. von Winterfeldt, D. Structuring decision problems for decision analysis. Acta Psychologica 1980; 45: 7193.Google Scholar
36. Keeney, RL. Decision-analysis – an overview. Operations Research 1982; 30: 803838.Google Scholar
37. Keeney, RL. Common mistakes in making value trade-offs. Operations Research 2002; 50: 935945.Google Scholar
38. Del Rio Vilas, V, Montibeller, G, Franco, L. Prioritisation of infectious diseases in public health: feedback on the prioritisation methodology, 15 July 2008 to 15 January 2009 [Letter]. Eurosurveillance 2011; 16.CrossRefGoogle ScholarPubMed
39. Triantaphyllou, E, Mann, SH. An examination of the effectiveness of multi-dimensional decision-making methods: a decision-making paradox. Decision Support Systems 1989; 5: 303312.Google Scholar
40. Guitouni, A, Martel, J-M. Tentative guidelines to help choosing an appropriate MCDA method. European Journal of Operational Research 1998; 109: 501521.Google Scholar
41. Roy, B. The outranking approach and the foundations of electre methods. Theory and Decision 1991; 31: 4973.Google Scholar
42. Brans, JP, Vincke, PH. A preference ranking organization method – (the promethee method for multiple criteria decision-making). Management Science 1985; 31: 647656.CrossRefGoogle Scholar
43. Keeney, RL, Raiffa, H. Decisions with Multiple Objectives. New York: Cambridge University Press, 1993.CrossRefGoogle Scholar
44. Humblet, M-F, et al. Multidisciplinary and evidence-based method for prioritizing diseases of food-producing animals and zoonoses. Emerging Infectious Diseases 2012; 18.Google Scholar
45. Saaty, TL. An exposition of the AHP in reply to the paper remarks on the analytical hierarchy process. Management Science 1990; 36: 259268.Google Scholar
46. Kadohira, M, et al. Stakeholder prioritization of zoonoses in Japan with analytic hierarchy process method. Epidemiology and Infection 2015; 143: 14771485.Google Scholar
47. Mourits, MCM, van Asseldonk, MAPM, Huirne, RBM. Multi criteria decision making to evaluate control strategies of contagious animal diseases. Preventive Veterinary Medicine 2010; 96: 201210.Google Scholar
48. Brookes, VJ, et al. Identifying and measuring stakeholder preferences for disease prioritisation: a case study of the pig industry in Australia. Preventive Veterinary Medicine 2014; 113: 118131.CrossRefGoogle ScholarPubMed
49. Nimo, VA, et al. A quantitative method of assessing the health impact of different diseases in less developed countries. International Journal of Epidemiology 1981; 10: 7380.Google Scholar
50. Carter, A, National Advisory Committee on Epidemiology Subcommittee. Establishing goals, techniques and priorities for national communicable disease surveillance. Canadian Journal of Infectious Diseases 1991; 2: 3740.Google Scholar
51. Weinberg, J, Grimaud, O, Newton, L. Establishing priorities for European collaboration in communicable disease surveillance. European Journal of Public Health 1999; 9: 236240.CrossRefGoogle Scholar
52. Rushdy, A, O'Mahony, M. PHLS overview of communicable diseases 1997: results of a priority setting exercise. Communicable Disease Report. CDR Weekly 1998; 8 (Suppl 5): S112.Google Scholar
53. Valenciano, M, Working Group. Setting priorities for non-foodborne zoonoses. Medecine et Maladies Infectieuses 2001; 31: 302S304S.Google Scholar
54. Doherty, JA. Establishing priorities for national communicable disease surveillance. Canadian Journal of Infectious Diseases 2000; 11: 2124.Google Scholar
55. Carter, AO. Setting priorities: the Canadian experience in communicable disease surveillance. Morbidity and Mortality Weekly Report 1992; 41: 7984.Google ScholarPubMed
56. Cardoen, S, et al. Evidence-based semiquantitative methodology for prioritization of foodborne zoonoses. Foodborne Pathogens and Disease 2009; 6: 10831096.Google Scholar
57. Balabanova, Y, et al. Communicable diseases prioritized for surveillance and epidemiological research: results of a standardized prioritization procedure in germany, 2011. PLoS ONE 2011; 6.Google Scholar
58. Krause, G, Prioritization Working Group. How can infectious diseases be prioritized in public health? A standardized prioritization scheme for discussion. EMBO Reports 2008; 9: S22S27.Google Scholar
59. Gilsdorf, A, Krause, G. Prioritisation of infectious diseases in public health: feedback on the prioritisation methodology, 15 July 2008 to 15 January 2009. Eurosurveillance 2011; 16: 1521.Google Scholar
60. Steele, K, et al. Uses and misuses of multicriteria decision analysis (MCDA) in environmental decision making. Risk Analysis 2009; 29: 2633.Google Scholar
61. von Winterfeldt, D, Edwards, W. Decision Analysis and Behavioral Research: Cambridge: Cambridge University Press, 1986.Google Scholar
62. Mintiens, K, Vose, D. Multi-criteria decision analysis for evaluating control options during FMD outbreaks. Society for Veterinary Epidemiology and Preventive Medicine, Glasgow, 2012.Google Scholar
63. Ng, V, Sargeant, JM. A quantitative and novel approach to the prioritization of zoonotic diseases in North America: a public perspective. PLoS ONE 2012; 7: e48519.CrossRefGoogle Scholar
64. Ng, V, Sargeant, JM. A quantitative approach to the prioritization of zoonotic diseases in North America: a health professionals' perspective. PLoS ONE 2013; 8: e72172.CrossRefGoogle Scholar
65. Neslo, REJ, Cooke, RM. Modeling and validating stakeholder preferences with probabilistic inversion. Applied Stochastic Models in Business and Industry 2011; 27: 115130.Google Scholar
66. Flari, V, et al. Expert judgment based multi-criteria decision model to address uncertainties in risk assessment of nanotechnology-enabled food products. Journal of Nanoparticle Research 2011; 13: 18131831.Google Scholar
67. Teck, SJ, et al. Using expert judgment to estimate marine ecosystem vulnerability in the California current. Ecological Applications 2010; 20: 14021416.Google Scholar
68. World Organisation for Animal Health. Handbook on Import Risk Analysis for Animals and Animal Products, 2nd edn. Paris: The World Organisation for Animal Health (OIE), 2010.Google Scholar
Figure 0

Table 1. Axioms of rational choice associated with normative decision theory, and their relevance to disease prioritization

Figure 1

Fig. 1. Schematic representation of the steps of decision analysis for disease prioritization. (Modified from Keeney [36].)

Figure 2

Fig. 2. Performance matrix for disease prioritization. D, Disease identity, C, criterion; w, weight for criterion; a, measurement for each criterion for each disease. (Modified from Brookes et al. [8].)