Hostname: page-component-7c8c6479df-ph5wq Total loading time: 0 Render date: 2024-03-29T10:44:05.930Z Has data issue: false hasContentIssue false

Comparing listeriosis risks in at-risk populations using a user-friendly quantitative microbial risk assessment tool and epidemiological data

Published online by Cambridge University Press:  28 March 2016

L. E. FALK
Affiliation:
Public Health Agency of Canada, Guelph, Ontario, Canada University of Toronto, Toronto, Ontario, Canada
K. A. FADER
Affiliation:
Public Health Agency of Canada, Guelph, Ontario, Canada University of Guelph, Guelph, Ontario, Canada
D. S. CUI
Affiliation:
Public Health Agency of Canada, Guelph, Ontario, Canada University of Waterloo, Waterloo, Ontario, Canada
S. C. TOTTON
Affiliation:
Public Health Agency of Canada, Guelph, Ontario, Canada
A. M. FAZIL
Affiliation:
Public Health Agency of Canada, Guelph, Ontario, Canada
A. M. LAMMERDING
Affiliation:
Public Health Agency of Canada, Guelph, Ontario, Canada
B. A. SMITH*
Affiliation:
Public Health Agency of Canada, Guelph, Ontario, Canada
*
*Author for correspondence: Mr B. A. Smith, 206-160 Research Lane, Guelph, ON, N1G 5B2, Canada. (Email: ben.smith@phac-aspc.gc.ca)
Rights & Permissions [Opens in a new window]

Summary

Although infection by the pathogenic bacterium Listeria monocytogenes is relatively rare, consequences can be severe, with a high case-fatality rate in vulnerable populations. A quantitative, probabilistic risk assessment tool was developed to compare estimates of the number of invasive listeriosis cases in vulnerable Canadian subpopulations given consumption of contaminated ready-to-eat delicatessen meats and hot dogs, under various user-defined scenarios. The model incorporates variability and uncertainty through Monte Carlo simulation. Processes considered within the model include cross-contamination, growth, risk factor prevalence, subpopulation susceptibilities, and thermal inactivation. Hypothetical contamination events were simulated. Results demonstrated varying risk depending on the consumer risk factors and implicated product (turkey delicatessen meat without growth inhibitors ranked highest for this scenario). The majority (80%) of listeriosis cases were predicted in at-risk subpopulations comprising only 20% of the total Canadian population, with the greatest number of predicted cases in the subpopulation with dialysis and/or liver disease. This tool can be used to simulate conditions and outcomes under different scenarios, such as a contamination event and/or outbreak, to inform public health interventions.

Type
Original Papers
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © Cambridge University Press 2016

Introduction

Listeria monocytogenes causes few cases of foodborne illness relative to other pathogens [Reference Thomas1], but infection can lead to mortality, particularly in newborns and immunocompromised patients [Reference Roberts and Wiedmann2Reference Scallan4]. L. monocytogenes infection is the third leading cause of death attributed to foodborne illness in the United States [Reference Scallan4]. Clinical signs of invasive listeriosis include meningitis, encephalitis, premature labour, miscarriage, abortion, and stillbirth [Reference Roberts and Wiedmann2, Reference Clark5].

Some ready-to-eat (RTE) delicatessen meats that have not been dried or salted, such as ham, roast beef, and turkey breast, and hot dogs (frankfurters), can support the growth of L. monocytogenes at refrigeration temperatures. A 2008 outbreak of invasive listeriosis in Canada involving RTE meats resulted in 57 illnesses and 22 deaths [Reference Weatherill6]. Following this outbreak, the ‘Policy on L. monocytogenes in Ready-To-Eat (RTE) Foods’ was updated to improve control in high-risk foods [7]. Nevertheless, there is a need to evaluate potential risks to vulnerable populations as a preventive measure, and in the case of a future confirmed or suspected contamination event.

Quantitative microbial risk assessment (QMRA) is a predictive method that can be used to provide estimates of the number of human illnesses resulting from consumption of pathogens in food [Reference Smith, Doores and Tortorello8]. When pathogenic bacteria such as L. monocytogenes are detected in a food product or environment at a manufacturing or retail facility, it is sometimes necessary to estimate potential risks to consumers within a short time-frame. Performing a detailed QMRA is highly resource-intensive and often difficult to accomplish considering factors such as availability of information and time sensitivity. In some cases, conditions are more amenable to a simplified model [Reference Evers and Chardon9]. An easy-to-use, pre-populated model would be more practical in scenarios where time-sensitive information is required, or modelling expertise is limited or unavailable. It could be used to inform health officials on relative risks under different scenarios of contamination levels and prevalence, either in a confirmed or suspected contamination scenario or during ‘peacetime’ (when no known outbreak is suspected) to explore potential future scenarios and intervention efforts.

Integration of regional prevalence of risk factors for listeriosis into a user-friendly model would allow for comparison of risks in regions and vulnerable subpopulations. Dose-response models that incorporate L. monocytogenes strain variability have recently been developed for selected populations vulnerable to listeriosis [Reference Pouillot10], and could be integrated with demographic information to derive population-level estimates of risk. Ross et al. [Reference Ross11] have estimated the prevalence of several at-risk subpopulations in Australia, but similar estimates have not yet been compiled for Canada.

The study objective was to develop a user-friendly QMRA model consisting of probabilistic inputs to produce estimates of the predicted number of cases of invasive listeriosis associated with the consumption of pre-packaged RTE delicatessen meats or hot dogs contaminated with L. monocytogenes. The goal was to create a stand-alone model to be used by public health officials, if necessary, but also readily modifiable by risk analysts if required.

Materials and Methods

Model overview

The model produces estimates of the risk of listeriosis as a probability density function for the total number of cases predicted to occur. The median number of cases are estimated in the total population and in each of ten vulnerable subpopulations. Treatment of the product is considered from retail display to consumption. There are three main components of the model: exposure assessment, hazard characterization, and risk characterization (Fig. 1).

Fig. 1. Conceptual risk assessment model for L. monocytogenes in ready-to-eat delicatessen meats and hot dogs.

Processes considered within the exposure assessment component include: (i) growth during retail display, transport to home, home storage, and ambient counter-top storage, (ii) cross-contamination during kitchen preparation, and (iii) inactivation during reheating. The number of potentially contaminated portions at consumption is divided into nine categories based on contamination at retail (yes/no), type of cold storage (refrigerator/freezer), cross-contamination (yes/no), and reheating (reheated/unheated). The consumed doses are calculated for each contaminated portion category. Portions are distributed in healthy and at-risk subpopulations based on the prevalence of risk factors within each province or territory in Canada, or Canada-wide, depending on the location selected. Therefore, the proportion of product consumed by each vulnerable subpopulation is equivalent to the respective risk factor prevalence in the region.

The hazard characterization component is used to derive subpopulation-specific dose-response models. These are integrated with the results of the exposure assessment to estimate the probability of listeriosis per portion for each subpopulation of interest. Finally, based on the number of portions consumed by each subpopulation, the number of listeriosis cases is predicted in the risk characterization component (Fig. 1).

The model was implemented in @Risk 6·3 (Palisade Corp., USA), an add-in for Microsoft Office Excel (Microsoft Corp., USA), and can be obtained by request from the corresponding author. Monte Carlo analysis with Latin Hypercube Sampling is used throughout the simulation process. The model is divided into six worksheets. Users can operate the model through the Model Input and Summary sheet, where basic information must be entered, and summary results are displayed. The user must provide, at a minimum, the type of food product (RTE beef, ham, or turkey delicatessen meat, or hot dogs, with or without growth inhibitors), the number of portions potentially contaminated (e.g. the number of servings produced in one batch or set of batches at a processing facility), the prevalence and initial contamination level of the product at retail, and the geographical location (Canada-wide, or by province or territory) where the food is distributed and consumed. Once the user has entered the minimum required data, the simulation can be run. The model presents the calculated risk as a probability density function for the total number of cases predicted to occur from the potentially contaminated portions, in the population of interest. Summary statistics are also provided.

Since the model is pre-populated with data, modifications to other worksheets are possible, but not mandatory for operation. Inputs can be customized by entering three parameters to create a triangular distribution: minimum, most likely, and maximum values. While this distribution has no theoretical basis, its parameters can be intuitively defined when little data are available and the uncertainty/variability of input variables can be quickly and easily captured [Reference Vose12]. In addition, an individual with limited statistical experience can generally understand how the distribution's shape has been derived, making it an appropriate choice for a user-friendly model. A description of the other worksheets is provided in the Supplementary material (S1).

Model details

A list of default input parameters and brief descriptions are reported in Table 1. Full descriptions can be found in the Supplementary Material (S2). Key aspects and calculations for the QMRA are provided below, and these calculations remain constant in the model regardless of user input values.

Table 1. Input parameters for the exposure assessment component

Exposure assessment

Growth models and storage conditions

The model incorporates six sets of growth parameters specific to L. monocytogenes on RTE beef, ham, and turkey delicatessen meats, each with or without growth inhibitors [Reference Pradhan17]. The growth parameters for RTE beef delicatessen meats were adopted for hot dogs in the absence of other suitable data. Growth inhibitors include lactate and diacetate, both of which are permitted food additives in Canada [18]. Growth is modelled throughout retail display, transport to home, consumer refrigerated storage, and counter-top storage. Time elapsed between production and retail display is considered to determine whether the initial lag phase has been exceeded and growth has occurred (Supplementary material, S2.17). Growth rates are adjusted for storage temperatures at each stage based on the US FDA/FSIS [19] equation:

(1) $${\rm EG}{{\rm R}_T} = {\rm EG}{{\rm R}_5}/{\left( {6.18/T + {\rm} 1.18} \right)^2}$$

where EGR T represents the calculated exponential growth rate (log10 c.f.u./g per day), EGR5 represents the exponential growth rate at 5 °C and T represents the storage temperature (°C). The maximum concentration of L. monocytogenes is limited to uniform ranges between 4·5–5·5, 6–7, and 7·5–8·5 log10 c.f.u./g for temperatures <5 °C, 5–7 °C, and >7 °C, respectively, in accordance with previous risk models [Reference Pradhan17, 19]. Freezer storage temperature is not included because frozen foods do not support growth of L. monocytogenes [7].

Cross-contamination

The likelihood of cross-contamination by the consumer is based on the prevalence of using unwashed cutting boards during preparation of raw and cooked RTE foods [Reference Kusumaningrum20] (Supplementary material, S2.11). Where no cross-contamination occurs, the level of L. monocytogenes remains unchanged and is carried forward to the thermal inactivation module. For all other portions, cross-contamination is modelled in two stages, using distinct transfer rates: (1) transfer of L. monocytogenes from unheated meat to the preparation environment and (2) subsequent transfer from the preparation environment to either unheated or heated meat prior to consumption [Reference Goh21]. The latter stage reduces the overall amount of bacteria eventually consumed for unheated products, because a portion of bacteria are assumed to remain on the food preparation surface. The bacterial load transferred to the preparation surface is important to capture for reheated products such as hot dogs, as it is not subjected to inactivation during reheating, whereas the load remaining on the product can be inactivated. The estimates are defined by discrete uniform distributions based on transfer rates under several experimental conditions (Supplementary material, S2.12).

Thermal inactivation (reheating)

L. monocytogenes inactivation is modelled using the thermal inactivation equation [Reference Foerster, Figueroa and Evers22]:

(2) $$PS = {10^{ - \big[ {{t_{\rm h}}/D_{\rm ref\;} \times {{10}^{ - \left( {\textstyle{{{T_{\rm h}} - {T_{{\rm ref}}}} \over z}} \right)}}} \big]}},$$

where PS is the probability of survival, t h is the heating time, T h is the heating temperature, D ref is the reference decimal reduction time (the heating time required to kill 90% of the organisms), T ref is the reference temperature, and z is the temperature required for a 1 log reduction in the D value. Heating time and temperature are each defined by triangular distributions. Heating times are based on the manufacturer's cooking instructions [23] and heating temperatures are based on the US FDA/FSIS risk assessment thermal inactivation module for hot dogs [19] (Supplementary material, S2.14–15). The D ref, T ref, and z values are specific to the thermal inactivation of L. monocytogenes in beef frankfurters [Reference Huang14] (Supplementary material, S2.16). The probability of survival is used to approximate L. monocytogenes concentrations (c.f.u./g) remaining in reheated portions following thermal inactivation.

The model is set so that between 90% and 99·8% of hot-dog portions are reheated prior to consumption [19, Reference Cates24]. It is conservatively assumed that 100% of RTE delicatessen meats were consumed without a reheating step performed by the consumer.

Risk factors and comorbidity

Given that host risk factors such as pre-existing health conditions drive vulnerability to listeriosis, Canadian demographic data describing occurrence of select risk factors were collected (Supplementary material, S3) to allow for comparison of risks across subpopulations in Canada. Portions are divided in vulnerable subpopulations based on risk factor prevalence in the selected region. The risk factors include age ⩾65 years, pregnancy, and several other diseases/conditions which alter the immune system (cancer, organ transplant patient, etc.). The subpopulations used in the model are described in Table 2, and were selected for inclusion primarily because dose-response information was available. A literature search was conducted to identify prevalence estimates of risk factors across Canada and, where possible, within each province and territory. When available, national, provincial, and territorial estimates were preferentially captured from government websites and relevant health organizations. Missing data were sought through primary research articles. Where province- and territory-specific estimates were unavailable, national estimates were used.

Table 2. Subpopulation descriptions and their corresponding relative risk values and listeriosis dose-response model R parameters

ENT, Ear, nose and throat.

Previous listeriosis risk assessments estimated the proportion of susceptible individuals by summing prevalence estimates across risk factors [Reference Ross11, Reference Hitchins25, Reference Lindqvist and Westöö26]. However, this does not account for comorbidities, or occurrence of more than one risk factor for listeriosis in any given individual, and may lead to double counting. This model considers ten risk factors (in addition to the general population aged <65 years with no conditions). Disregarding the potential for multiple risk factors for listeriosis would lead to an overestimation of the vulnerable population (e.g. summing prevalence of individual risk factors without accounting for comorbidity results in approximately 40% of the population considered as vulnerable). Unfortunately, there is a paucity of data on the occurrence of multiple risk factors specific to listeriosis. As an approximation, Canadian comorbidity data available for a suite of health conditions were used to estimate the proportion of the Canadian population with either zero risk factors or at least one risk factor for listeriosis [Reference Broemeling, Watson and Prebtani27]. Using these data combined with age distributions from Statistics Canada [28], the modeled population was divided into five primary categories; <65 years with no conditions, <65 years with ⩾1 condition, ⩾65 years with no conditions, ⩾65 years with ⩾1 condition, and pregnant women. The latter four categories are all considered vulnerable to listeriosis. Population segments with ⩾1 condition were further divided by their primary condition as determined by the relative prevalence of risk factors in the chosen location. It was assumed that pregnant women do not have additional risk factors. Portions are divided in the subpopulations according to their prevalence as described above. Sex-specific population and prevalence estimates were used only in intermediate calculations to account for key variation in some risk factors (e.g. prostate cancer, pregnancy). Due to the absence of sex-specific data on all risk factors and sex-specific listeriosis risks, outputs are pooled for both sexes.

Hazard characterization

Exponential dose-response models of the general form below are used:

(3) $${P_{\rm listeriosis}} = {\rm} 1{\rm} -{\rm} {{\rm e}^{-RD}},$$

where P listeriosis is the probability of listeriosis, D is the dose consumed, and R is the fitted parameter specific to each subpopulation of interest. The general model was obtained from Ross et al. [Reference Ross11], which was based on an exponential model [Reference Buchanan29]. Eleven dose-response models are used, each with previously derived subpopulation-specific distributions of R parameters (Table 2) [Reference Pouillot10]. The R parameters capture variability in subpopulations, within subpopulations, and in L. monocytogenes strains [Reference Pouillot10]. The subpopulation-specific R parameters were derived from listeriosis relative risk estimates identified in a French population by Goulet et al. [Reference Goulet30] (Table 2).

Risk characterization

Integration of the exposure assessment and hazard characterization components yields the probability of listeriosis for each subpopulation/portion category combination, which is then multiplied by the number of portions consumed by each subpopulation to predict the number of listeriosis cases for each subpopulation. Due to the stochastic nature of the model and lack of truncation bounds on the distributions of R values used in the dose-response analyses, in some model iterations the R value for a given risk factor is less than the value for the same age group with no conditions. In these cases, the greater R value was implemented in the dose-response model for the vulnerable subpopulation. Although it is acknowledged that the dose-response functions account for strain variability, this adjustment was made as a conservative measure. Estimated cases are summed across all subpopulations to provide an estimate of risk for the entire population.

Simulations

Demonstrative sets of simulations were performed to compare products, growth inhibitor use, regions, and subpopulations, using hypothetical user inputs. Although it was preferred to simulate a historical contamination event, adequate data to sufficiently replicate a previous outbreak, such as the 2008 outbreak in Canada [Reference Weatherill6], were unavailable. For all simulations, 100 000 iterations were run with a fixed initial seed. Triangular distributions (minimum, most likely, maximum) were used to characterize the user inputs: number of potentially contaminated portions (80000, 100000, 120000), contamination prevalence (50%, 60%, 65%), and the L. monocytogenes contamination level at retail (0·04, 2·34, 254 c.f.u./g). Prevalence of contamination approximated conditions measured from previous L. monocytogenes outbreaks in RTE foods [31Reference Mead33]. The 5th, 50th, and 95th percentiles of the cumulative distribution of levels of L. monocytogenes in Australian processed meats in Ross et al. [Reference Ross34] were adopted. Spearman's rank-order correlation coefficients, as well as summary statistics such as median, 5th, and 95th percentiles of outputs, were derived from @Risk outputs.

RESULTS

Product and growth inhibitor comparison

Hypothetical contamination events were simulated to compare products and growth inhibitor use, assuming equal product distribution across Canada. The medians, 5th, and 95th percentiles of listeriosis cases predicted are presented in Table 3. The model predicted approximately two orders of magnitude fewer listeriosis cases for products with growth inhibitors compared to products without growth inhibitors. The median number of estimated listeriosis cases ranged from 0·0022 to 0·089 and 0·11 to 6·6 cases for products with and without growth inhibitors, respectively. Distributions of estimated listeriosis cases for each product are depicted on a logarithmic scale in Figure 2. This figure provides an indication of the relative differences in products, but note that the central tendencies are not constant across log transformations. While similar patterns in products are apparent, the model predicts a wide range of estimated listeriosis cases for each product scenario.

Fig. 2. Probability distributions of total estimated listeriosis cases for consumption of contaminated beef, ham, and turkey delicatessen meats, and hot dogs, with and without growth inhibitors in Canada. Distributions result from Monte Carlo simulations using Latin Hypercube Sampling with 100 000 iterations. Distributions are shown in a log scale, and central tendencies are not constant across log transformations. (See Table 3 for median estimates of listeriosis cases.)

Table 3. Median listeriosis cases and the 5th and 95th percentile estimates for each of the products following a simulated L. monocytogenes contamination event in Canada, as determined by Monte Carlo simulations using Latin Hypercube Sampling with 100 000 iterations. Relative differences as compared to the product with the greatest public health risk are provided to demonstrate the utility of the model for comparing various product and product treatment scenarios

The impact of model input variables on the number of cases was assessed through comparison of Spearman's rank order correlation coefficients. Input variables with the greatest coefficients for each scenario ranged from 0·30 to 0·43, were product dependent, and included consumer refrigerator storage temperature, consumer refrigerated storage time, exponential growth rate, retail storage temperature, and storage temperature prior to retail display. The latter was used to determine lag time for L. monocytogenes growth, and correspondingly, lag time was negatively correlated with public health impacts (−0·13 to −0·27).

Median L. monocytogenes contamination levels (c.f.u./g) were greater at all exposure points prior to consumption compared to initial levels at retail for all delicatessen meats (Fig. 3). Contamination levels on products increased from retail through counter-top storage, but not following cross-contamination and reheating. The greatest relative increase in L. monocytogenes levels occurred during cold home storage. The largest decrease in contamination levels occurred during reheating of hot dogs. The ranking of final contamination level in products corresponded with the median number of listeriosis cases (Table 3, Fig.3).

Fig. 3. Listeria monocytogenes median contamination levels throughout the retail-to-consumption pathway for beef, ham, and turkey delicatessen meats, and hot dogs with and without growth inhibitors. Contamination levels are based on weighted average medians across all portion types taken at the beginning and end of each step in the pathway, as determined using Monte Carlo simulations using Latin Hypercube Sampling with 100 000 iterations. C/C, Cross-contamination. Initial concentrations were equivalent across products, and final median concentrations are indicated. Each vertical line indicates the beginning of one stage and the end of the previous stage. Contamination levels at entry and exit of each stage are linked with straight lines for ease of visualization only, and do not indicate linear increases within stages.

Risk factor comparison

Simulation results for risks from turkey delicatessen meat with growth inhibitors in Canada were further studied to evaluate the distribution of model results by subpopulation. The median number of portions consumed, median probability of illness averaged across portion types, and median number of listeriosis cases, along with the 5th and 95th percentiles, for each subpopulation are presented in Table 4. The highest probability of illness was predicted in the haematological cancer risk factor subpopulation, followed by the transplant, dialysis/liver disease, and pregnant risk factor subpopulations. The relative proportion of contaminated product consumed by each subpopulation is compared with their respective number of simulated cases of listeriosis in Figure 4. Portions consumed directly reflect the relative prevalence of the risk factors in the population, as described previously. The largest number of listeriosis cases was predicted in the dialysis/liver disease risk factor subpopulation followed by the diabetes, non-haematological cancer, inflammatory disease, and haematological cancer risk factor subpopulations. Subpopulations with no risk factor (<65 years with no condition) accounted for only 1% of all listeriosis cases, despite consuming 63% of the simulated portions. A marginally greater number of cases were predicted in subpopulations aged ⩾65 years with no other risk factors, despite the relatively low prevalence of these individuals in Canada (approximately three quarters of individuals aged ⩾65 years across Canada were considered to have at least one other known risk factor in the model, and are therefore represented in the other risk-factor classifications). Individuals undergoing dialysis and/or with liver disease accounted for 21% of all listeriosis cases, but consumed only 2% of the contaminated portions from the simulated event.

Fig. 4. Relative percent of listeriosis cases in eleven subpopulations given consumption of contaminated turkey delicatessen meat with growth inhibitors in Canada, as determined using Monte Carlo simulation using Latin Hypercube Sampling with 100 000 iterations. Relative size of each subpopulation in Canada is also shown. Relative population size is equivalent to the relative number of contaminated portions consumed by each subpopulation. (See Table 4 for median estimates of listeriosis cases and portions.)

Table 4. Median portions consumed, probability of illness, listeriosis cases and respective 5th and 95th percentile estimates for the consumption of contaminated turkey delicatessen meats with growth inhibitors in each subpopulation included in the model, assuming a hypothetical contamination event. Probability of illness was determined as a weighted average across all portion types. Results were determined by Monte Carlo simulations using Latin Hypercube Sampling with 100 000 iterations

Two additional contamination event scenarios were tested using the model to compare differences in risks in subpopulations aged <65 years and ⩾65 years, such as what might happen if product is generally distributed to grocery stores, etc., or distributed solely to institutions comprised mostly of at-risk subpopulations such as long-term-care facilities. For these simulations, beef delicatessen meat with growth inhibitors was assumed to be distributed either generally across the province of Ontario, or solely to subpopulations aged ⩾65 years. The median number of resulting listeriosis cases in the general population and the ⩾65 years population was 9·1 × 10−2 (5th percentile: 1·9 × 10−4, 95th percentile: 7·1 × 102) and 1·7 × 10−1 (5th percentile: 3·1 × 10−4, 95th percentile 1·4 × 103), respectively. Therefore, the number of estimated listeriosis cases was 1·9 times greater in the population aged ⩾65 years compared to the general population.

DISCUSSION

Absolute risk values should be interpreted with caution for any given simulation due to uncertainty in model inputs. Rather, the purpose of this model is to compare risks under different scenarios, or for different products, interventions, regions, or risk factors. The model can be used to explore conditions and outcomes under different circumstances, such as simulating a contamination event and/or outbreak. The discussion below is provided in reference to the simulated hypothetical contamination events; while the intention of this paper is primarily to introduce the user-friendly model, a focus on the simulated results is prudent to provide an indication of how model results can be interpreted and used to inform public health activities.

Previous risk assessments have demonstrated that L. monocytogenes growth on RTE meats and resulting listeriosis cases are dependent on the product and its additives [Reference Ross11, Reference Pradhan17, Reference Zhang35]. Thus, it was important to include the ability to evaluate listeriosis risk in different RTE meats and growth inhibitor uses in the model. Consumption of turkey delicatessen meat without growth inhibitors resulted in the highest number of listeriosis cases in the hypothetical scenario (Table 2). As expected, presence of growth inhibitors led to fewer listeriosis cases for all RTE meats considered in this study; however, the relative reduction in cases due to growth inhibitor use differed by RTE meat type and initial contamination level. Use of growth inhibitors led to a 110-fold reduction in the median number of listeriosis cases for turkey delicatessen meat, followed by ham delicatessen meat, beef delicatessen meat, and hot dogs, with 78-fold, 56-fold, and 49-fold reductions, respectively, using the simulation inputs previously described. However, arbitrarily lowering the initial contamination levels to reflect a triangular (0·04, 1, 2 c.f.u./g) distribution resulted in corresponding 952-, 279-, 381-, and 116-fold reductions. Use of growth inhibitors had a greater relative impact on risks from beef compared to ham delicatessen meat at this lower contamination level scenario. Such differences highlight the importance of using event-specific inputs where possible, as general ‘rules of thumb’ might not apply across all possible contamination scenarios. The effects of growth inhibitors are dependent on several properties (e.g. temperature, pH, fat content), which may explain variation in products [Reference Abou-Zeid36Reference Shelef and Yang38].

Within each category of growth inhibitor use, hot dogs caused fewer listeriosis cases than delicatessen meats (Table 2). This is consistent with Ross et al. [Reference Ross11] and is attributed to differences in consumer storage and reheating practices. Of delicatessen meats, fewer cases were consistently estimated for ham. Median lag times for ham exceeded those for beef and turkey. Such extended lag times resulted in shorter growth periods and fewer predicted listeriosis cases. While the growth rates and lag times incorporated in this model were distributions based on several studies [Reference Pradhan17], recent studies on L. monocytogenes growth [Reference Zhang35, Reference Myers39] could be incorporated into the distributions based on user needs.

The importance of L. monocytogenes growth during storage is noted in several risk assessments [19, Reference Pouillot40Reference Yang42], and not surprisingly echoed in our results. The majority of L. monocytogenes growth occurred during consumer cold storage and although growth during retail storage was not as pronounced, correlation with model outputs remained high, especially in products without growth inhibitors.

Regional comparisons provide a framework for integration of demographic, epidemiological, and dose-response data to compare risks across regions for decision making purposes. Age demographics and the prevalence of listeriosis risk factors varied in the regions considered in the model, and therefore regional distribution of potentially contaminated product is important. However, regional prevalence data, although available (see Supplementary material, S3), were not derived using identical approaches in all cases. These data are incorporated in the model, but a summary of regional comparisons is not presented herein given differences in data derivation for the provinces and territories.

Generally, previous listeriosis risk assessments have summed national risk factor prevalence and used adjusted dose-response models to account for two subpopulations; the healthy and the vulnerable [Reference Ross11, 19]. This model builds on this approach by incorporating region-specific demographic data for several risk factors, accounting (albeit imperfectly) for comorbidities, and providing estimates of risk for ten vulnerable subpopulations alongside the general population. The model is amenable to modifications to allow for greater regional specificity in model components (e.g. portion sizes), and smaller-scale regional assessments (e.g. census subdivisions, municipalities, institutions, etc.). At present, identically derived provincial and territorial demographic data are lacking, and data are unavailable to tailor the model to an even smaller spatial scale. However, in the event of a suspected contamination event or outbreak, health authorities may be able to obtain product distribution and demographic information to allow for such modifications. The regional selection in the model allows for comparison of inter- or intra-regional risks for estimating listeriosis cases and targeting interventions such as public health messaging to vulnerable groups.

Individual-level risk and population-level risk of listeriosis differ in their computation and utility. Individual-level risk for example, is often represented by relative risk in epidemiology studies or probability of illness in QMRAs [19, Reference Ross34, 43]. The relative estimated probabilities of illness across vulnerable subpopulations included in this model align with relative risk values recently derived and used in the model [Reference Pouillot10]. Individuals with haematological cancer had the highest probability of listeriosis per portion, followed by transplant patients and individuals on dialysis/with liver disease (Table 4). These subpopulations represent high-risk individuals, and would benefit most from interventions at the primary care level (e.g. physician advice to at-risk patients to heat RTE foods to temperatures lethal to L. monocytogenes, or avoid specific higher-risk foods such as RTE delicatessen meats).

The model represents population-level risk as the number of estimated listeriosis cases. This estimate accounts for both individual-level risk, and risk factor prevalence in the population. Subpopulations with at least one risk factor accounted for 99% of all predicted listeriosis cases, which is consistent with epidemiological studies where 98% of individuals with listeriosis had at least one pre-existing condition [Reference Schuchat44]. The Canadian subpopulation on dialysis or with liver disease was associated with the most listeriosis cases in the model. From a risk assessment perspective, in outbreak situations and exploratory modelling, population-level risk is important as it identifies the populations where one might expect the most cases to occur. The subpopulation comparison indicated that 80% of cases are expected to occur in 20% of the population: those with liver disease and/or undergoing dialysis, diabetes, cancer, and inflammatory disease. By targeting interventions such as public health messaging at these specific risk factor groups, significant listeriosis burden could be reduced using a fraction of the resources required to target the entire population.

The model predicted that distribution of a contaminated batch of RTE meat to a population aged exclusively ⩾65 years (e.g. a long-term-care facility) led to nearly double the number of cases compared to distribution to a population comprising all ages. This reiterates the importance of both age, and risk factors that accumulate with age, in listeriosis outbreaks [Reference Goulet30, Reference Gerner-Smidt45Reference Rocourt47]. Individuals aged ⩾65 years are more likely to have at least one additional chronic, often immunosuppressive, condition [Reference Broemeling, Watson and Prebtani27], as was the case in the 2008 listeriosis outbreak [Reference Weatherill6]. This tool, had it existed at that time, could have been used to provide insights on risks posed to vulnerable populations to which different RTE meat products were distributed (including long-term and acute-care facilities), and support preventative strategies.

There are several key limitations to the model and its inputs. Some inputs to the model were not well-characterized in the literature or publicly accessible databases. Region-specific risk factor prevalence values were, in some cases, unavailable. In these cases, national prevalence data were used as a surrogate input. It is expected that regional variations in risk factor prevalence exist, and should be implemented in the model when data are available. For some inputs, specifically demographic data for cancer, comparable data across regions were not available (e.g. 10-year vs. lifetime prevalence values). Additionally, the risk factors and relative risk estimates included in the model were originally derived for France [Reference Goulet30], and could lack generalizability. Since the relative risk factors were derived based on observed cases of listeriosis, they inherently account for differences in general consumption patterns in vulnerable subpopulations. However, consumption patterns of RTE meats in at-risk groups could differ in Canada. Relative risk values for listeriosis should be parameterized using Canadian data and implemented in risk models.

While the methodology used to account for comorbidity improves, in our opinion, upon previous L. monocytogenes QMRA models, it is based on comorbidity information for a small subset of conditions [Reference Broemeling, Watson and Prebtani27]: arthritis, cancer, chronic obstructive pulmonary disease, diabetes, heart disease, high blood pressure, and mood disorders. Only some of these conditions were identified as risk factors for listeriosis, and other risk factors were not included in comorbidity estimates. Nevertheless, using these data provided a generic indication of occurrence of comorbidities in the Canadian population, which is preferable to simply summing the prevalence of all conditions (which will overestimate the total vulnerable population as approximately 40%) and ignoring co-occurrence of risk factors to listeriosis altogether. Estimates of the total vulnerable population calculated in the model ranged from 20–39%, depending on region considered. The World Health Organization (WHO) and Food and Agriculture Organization (FAO) [43] estimated that 22·4% of the Canadian population was susceptible to invasive listeriosis. However, age demographic data used in that study are over a decade old, and the proportion of Canadians aged ⩾65 years was 12·5%, compared to 15·7% used in this model. In addition, the prevalence of other risk factors across all age groups was 31·4% herein, compared to 4·6% used (pregnancy and immunodeficiency) in the WHO/FAO study [43].

Several risk factors, including but not limited to alcoholism, antacid use, corticosteroid therapy, and laxative use were identified, but dose-response data were unavailable and therefore they were not considered in the model. Inclusion of these risk factors could have a profound impact on the model results. For example, when prevalence data for alcoholism are included in the model (estimated as the prevalence of chronic drinking in Canada as defined by Health Canada [48], or 14·4% of the population aged ⩾15 years), and arbitrarily adopting the dose-response R value for liver disease, the projected number of cases in a simulated contamination event increased by 21%. The model does not explicitly incorporate additive or synergistic risk in those with multiple risk factors, although the methods in the derivation of relative risk values may partially account for this. Goulet et al. [Reference Goulet30] classified patients with >1 underlying risk factor according to the most immunosuppressive condition. Further research on listeriosis risk factors, their overlap, and their resulting contributions to the risk of listeriosis would greatly benefit subpopulation risk characterization.

This model was designed to be user friendly; however, in doing so, some aspects of more complex or all-encompassing QMRA models were not included. While the focus herein has been on pre-packaged delicatessen meat, delicatessen meat sliced at retail should also be considered. Although it is unlikely these products would be distributed to hospitals or long-term-care homes, the risk associated with counter-sliced meats is considerably greater than prepackaged meats [Reference Endrikat49, Reference Quesenberry50], thus possibly representing an important outbreak source for the general population. Additionally, consumption habits in vulnerable subpopulations should be included when data are available, as they are likely to differ [Reference Heathcock51, Reference Jevšnik, Hoyer and Raspor52].

The QMRA model provides several novel aspects. It provides a simple user interface to allow for use by public health officials not necessarily well-versed in modelling, similar to other tools [Reference Evers and Chardon9]. However, unlike these generic tools, it incorporates uncertainty and variability in inputs, and is pre-populated for a specific pathogen and suite of food products. Therefore, it is an easily modifiable tool that can be adapted to reflect the particular circumstances of L. monocytogenes contamination events in Canada. Further, the integration of several regions, products, and vulnerable subpopulations allow for risk comparisons and the identification of high priority populations to direct public health interventions. Example simulations presented herein provide an indication of how the tool can be used to identify higher risk food-treatment combinations and distribution scenarios.

SUPPLEMENTARY MATERIAL

For supplementary material accompanying this paper visit http://dx.doi.org/10.1017/S0950268816000327.

ACKNOWLEDGEMENTS

This work was funded by the Public Health Agency of Canada under the Food and Consumer Safety Action Plan. We are grateful to Ainsley Otten, Andrew Fedoruk, Mary Stoesser, and Natasha Elms for their assistance in risk factor analysis. We thank Robert Williams for providing some risk factor prevalence data.

DECLARATION OF INTEREST

None.

References

REFERENCES

1. Thomas, MK, et al. Estimates of the burden of foodborne illness in Canada for 30 specified pathogens and unspecified agents, circa 2006. Foodborne Pathogens and Disease 2013; 10: 639648.CrossRefGoogle ScholarPubMed
2. Roberts, AJ, Wiedmann, M. Pathogen, host and environmental factors contributing to the pathogenesis of listeriosis. Cellular and Molecular Life Sciences 2003; 60: 904918.CrossRefGoogle Scholar
3. Safdar, A, Armstrong, D. Listeriosis in patients at a comprehensive cancer center, 1955–1997. Clinical Infectious Diseases 2003; 37: 359364.CrossRefGoogle Scholar
4. Scallan, E, et al. Foodborne illness acquired in the United States – major pathogens. Emerging Infectious Disease 2011; 17: 615.Google ScholarPubMed
5. Clark, CG, et al. Surveillance for Listeria monocytogenes and listeriosis, 1995–2004. Epidemiology and Infection 2010; 138: 559572.CrossRefGoogle ScholarPubMed
6. Weatherill, D. Report of the independent investigator into the 2008 listeriosis outbreak: Health Canada, 2009 (http://www.cpha.ca/uploads/history/achievements/09-lirs-rpt_e.pdf). Accessed 27 September 2015.Google Scholar
7. Health Canada. Policy on Listeria monocytogenes in ready-to-eat foods. Bureau of Microbial Hazards, Food Directorate, Health Products and Food Branch, 2011 (http://www.hc-sc.gc.ca/fn-an/alt_formats/pdf/legislation/pol/policy_listeria_monocytogenes_2011-eng.pdf). Accessed 27 September 2015.Google Scholar
8. Smith, BA, et al. Microbial food safety risk assessment. In: Doores, Y, Tortorello, ML, eds. Compendium of Methods for the Microbiological Examination of Foods, 5th edn. Washington, DC: American Public Health Association, 2013.Google Scholar
9. Evers, EG, Chardon, JE. A swift quantitative microbiological risk assessment (sQMRA) tool. Food Control 2010; 21: 319330.CrossRefGoogle Scholar
10. Pouillot, R, et al. Listeria monocytogenes dose response revisited – incorporating adjustments for variability in strain virulence and host susceptibility. Risk Analysis 2015; 35: 90108.CrossRefGoogle ScholarPubMed
11. Ross, T, et al. Quantitative risk assessment of Listeria monocytogenes in ready-to-eat meats in Australia. International Journal of Food Microbiology 2009; 131: 128137.CrossRefGoogle ScholarPubMed
12. Vose, D. Risk Analysis: A Quantitative Guide, 3rd edn. West Sussex, UK: John Wiley & Sons, 2008.Google Scholar
13. Dewis, G. Controlling the temperature in Canadian homes: Statistics Canada, 2008. (Environment accounts and statistics technical paper series; catalogue no. 16–001-M, no. 6.).Google Scholar
14. Huang, L. Computer-controlled microwave heating to in-package pasteurize beef frankfurters for elimination of Listeria monocytogenes . Journal of Food Process Engineering 2005; 28: 453477.CrossRefGoogle Scholar
15. EcoSure. 2007 United States cold temperature evaluation design and summary pages, 2008. (http://foodrisk.org/exclusives/EcoSure/). Accessed 27 September 2015.Google Scholar
16. Pouillot, R, et al. Estimating parametric distributions of storage time and temperature of ready-to-eat foods for U.S. households. Journal of Food Protection 2010; 73: 312321.CrossRefGoogle ScholarPubMed
17. Pradhan, AK, et al. Quantitative risk assessment for Listeria monocytogenes in selected categories of delicatessen meats: impact of lactate and diacetate on listeriosis cases and deaths. Journal of Food Protection 2009; 72: 978989.CrossRefGoogle Scholar
18. Health Canada. Notice of modification to the list of permitted preservatives to enable the addition of potassium lactate and sodium lactate to standardized meat products and poultry meat products, 2012 (http://www.hc-sc.gc.ca/fn-an/alt_formats/pdf/consult/nom-adm-0001/document-consultation-eng.pdf). Accessed 27 September 2015.Google Scholar
19. U.S. Food and Drug Administration, Center for Food Safety and Applied Nutrition, U.S. Department of Agriculture, Food Safety and Inspection Service, and Centers for Disease Control and Prevention. Quantitative assessment of the relative risk to public health from food-borne Listeria monocytogenes among selected categories of ready-to-eat foods, 2003 (http://www.fda.gov/downloads/Food/FoodScienceResearch/UCM197330.pdf) Accessed 27 September 2015.Google Scholar
20. Kusumaningrum, HD, et al. A quantitative analysis of cross-contamination of Salmonella and Campylobacter spp. via domestic kitchen surfaces. Journal of Food Protection 2004; 67: 18921903.CrossRefGoogle ScholarPubMed
21. Goh, SG, et al. Transmission of Listeria monocytogenes from raw chicken meat to cooked chicken meat through cutting boards. Food Control 2014; 37: 5155.CrossRefGoogle Scholar
22. Foerster, C, Figueroa, G, Evers, E. Risk assessment of Listeria monocytogenes in poultry and beef. British Food Journal 2015; 117: 779792.CrossRefGoogle Scholar
23. Farmland. Hickory smoked hot dogs – original (http://www.farmlandfoods.com/products/hickory-smoked-hot-dogs-original.html). Accessed 27 September 2015.Google Scholar
24. Cates, SC, et al. Consumer knowledge, storage, and handling practices regarding Listeria in frankfurters and delicatessen meats: results of a Web-based survey. Journal of Food Protection 2006; 69: 16301639.CrossRefGoogle Scholar
25. Hitchins, AD. Assessment of alimentary exposure to Listeria monocytogenes . International Journal of Food Microbiology 1996; 30: 7185.CrossRefGoogle ScholarPubMed
26. Lindqvist, R, Westöö, A. Quantitative risk assessment for Listeria monocytogenes in smoked or gravad salmon and rainbow trout in Sweden. International Journal of Food Microbiology 2000; 58: 181196.CrossRefGoogle ScholarPubMed
27. Broemeling, AM, Watson, DE, Prebtani, F. Population patterns of chronic health conditions, co-morbidity and healthcare use in Canada: implications for policy and practice. Healthcare Quarterly 2008; 11: 7076.CrossRefGoogle ScholarPubMed
28. Statistics Canada. Table 051–0001 – Estimates of population, by age group and sex for July 1, Canada, provinces and territories, annual (persons unless otherwise noted): CANSIM (database). Accessed 27 September 2015.Google Scholar
29. Buchanan, RL, et al. Use of epidemiologic and food survey data to estimate a purposefully conservative dose-response relationship for Listeria monocytogenes levels and incidence of listeriosis. Journal of Food Protection 1997; 60: 918922.CrossRefGoogle ScholarPubMed
30. Goulet, V, et al. Incidence of listeriosis and related mortality among groups at risk of acquiring listeriosis. Clinical Infectious Diseases 2012; 54: 652660.CrossRefGoogle ScholarPubMed
31. Health Canada. 2008 Listeriosis outbreak in Ontario, chronology of events, 2009 (http://www.health.gov.on.ca/en/public/publications/disease/docs/listeriosis_outbreak_chronology.pdf). Accessed 27 September 2015.Google Scholar
32. Marsden, JL, et al. Salmonella spp. and Listeria monocytogenes risk assessment for production and cooking of blade tenderized beef steaks, 2001 (http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/01-013N/BeefSteaks.pdf). Accessed 27 September 2015.Google Scholar
33. Mead, P, et al. Nationwide outbreak of listeriosis due to contaminated meat. Epidemiology and Infection 2006; 134: 744751.CrossRefGoogle ScholarPubMed
34. Ross, T, et al. Listeria monocytogenes in Australian processed meat products: risks and their management: Final report, 2009.Google Scholar
35. Zhang, L, et al. Growth of Listeria monocytogenes in different retail delicatessencatessen meats during simulated home storage. Journal of Food Protection 2012; 75: 896905.CrossRefGoogle ScholarPubMed
36. Abou-Zeid, KA, et al. Survival and growth of Listeria monocytogenes in broth as a function of temperature, pH, and potassium lactate and sodium diacetate concentrations. Journal of Food Protection 2007; 70: 26202625.CrossRefGoogle ScholarPubMed
37. Burnett, SL, et al. Growth or survival of Listeria monocytogenes in ready-to-eat meat products and combination delicatessen salads during refrigerated storage. Journal of Food Science 2005; 70: 301304.CrossRefGoogle Scholar
38. Shelef, LA, Yang, Q. Growth suppression of Listeria monocytogenes by lactates in broth, chicken, and beef. Journal of Food Protection 1991; 54: 283287.CrossRefGoogle ScholarPubMed
39. Myers, K, et al. The effect of high hydrostatic pressure, sodium nitrite and salt concentration on the growth of Listeria monocytogenes on RTE ham and turkey. Meat Science 2013; 93: 263268.CrossRefGoogle ScholarPubMed
40. Pouillot, R, et al. Quantitative risk assessment of Listeria monocytogenes in French cold-smoked salmon: I. Quantitative exposure assessment. Risk Analysis 2007; 27: 683700.CrossRefGoogle Scholar
41. Pouillot, R, et al. Listeria monocytogenes in retail delicatessens: an interagency risk assessment model and baseline results. Journal of Food Protection 2015; 78: 134145.CrossRefGoogle ScholarPubMed
42. Yang, H, et al. Consumer phase risk assessment for Listeria monocytogenes in delicatessen meats. Risk Analysis 2006; 26: 89103.CrossRefGoogle Scholar
43. World Health Organization. Risk assessment of Listeria monocytogenes in ready-to-eat foods. Technical report, Microbiological Risk Assessment Series, 5th edn: Food & Agriculture Org., 2004.Google Scholar
44. Schuchat, A, et al. Role of foods in sporadic listeriosis: I. Case-control study of dietary risk factors. Journal of the American Medical Association 1992; 267: 20412045.CrossRefGoogle ScholarPubMed
45. Gerner-Smidt, P, et al. Invasive listeriosis in Denmark 1994–2003: a review of 299 cases with special emphasis on risk factors for mortality. Clinical Microbiology and Infection 2005; 11: 618624.CrossRefGoogle ScholarPubMed
46. Popovic, I, Heron, B, Covacin, C. Listeria: an Australian perspective (2001–2010). Foodborne Pathogens and Disease 2014; 11: 425432.CrossRefGoogle ScholarPubMed
47. Rocourt, J. Risk factors for listeriosis. Food Control 1996; 7: 195202.CrossRefGoogle Scholar
48. Health Canada. Canadian Alcohol and Drug Use Monitoring Survey, 2011 (http://www.hc-sc.gc.ca/hc-ps/drugs-drogues/stat/_2011/summary-sommaire-eng.php). Accessed 27 September 2015.Google Scholar
49. Endrikat, S, et al. A comparative risk assessment for Listeria monocytogenes in prepackaged versus retail-sliced delicatessen meat. Journal of Food Protection 2010; 73: 612619.CrossRefGoogle Scholar
50. Quesenberry, H, et al. FSIS comparative risk assessment for Listeria monocytogenes in ready to eat meat and poultry delicatessen meats. Food Safety and Inspection Service, U.S. Department of Agriculture, 2012 (http://www.fsis.usda.gov/shared/PDF/Comparative_RA_Lm_Report_May2010.pdf). Accessed 27 September 2015.Google Scholar
51. Heathcock, R, et al. Survey of food safety awareness among HIV-positive individuals. AIDS Care 1998; 10: 237241.CrossRefGoogle ScholarPubMed
52. Jevšnik, M, Hoyer, S, Raspor, P. Food safety knowledge and practices among pregnant and non-pregnant women in Slovenia. Food Control 2008; 19: 526534.CrossRefGoogle Scholar
Figure 0

Fig. 1. Conceptual risk assessment model for L. monocytogenes in ready-to-eat delicatessen meats and hot dogs.

Figure 1

Table 1. Input parameters for the exposure assessment component

Figure 2

Table 2. Subpopulation descriptions and their corresponding relative risk values and listeriosis dose-response model R parameters

Figure 3

Fig. 2. Probability distributions of total estimated listeriosis cases for consumption of contaminated beef, ham, and turkey delicatessen meats, and hot dogs, with and without growth inhibitors in Canada. Distributions result from Monte Carlo simulations using Latin Hypercube Sampling with 100 000 iterations. Distributions are shown in a log scale, and central tendencies are not constant across log transformations. (See Table 3 for median estimates of listeriosis cases.)

Figure 4

Table 3. Median listeriosis cases and the 5th and 95th percentile estimates for each of the products following a simulated L. monocytogenes contamination event in Canada, as determined by Monte Carlo simulations using Latin Hypercube Sampling with 100 000 iterations. Relative differences as compared to the product with the greatest public health risk are provided to demonstrate the utility of the model for comparing various product and product treatment scenarios

Figure 5

Fig. 3. Listeria monocytogenes median contamination levels throughout the retail-to-consumption pathway for beef, ham, and turkey delicatessen meats, and hot dogs with and without growth inhibitors. Contamination levels are based on weighted average medians across all portion types taken at the beginning and end of each step in the pathway, as determined using Monte Carlo simulations using Latin Hypercube Sampling with 100 000 iterations. C/C, Cross-contamination. Initial concentrations were equivalent across products, and final median concentrations are indicated. Each vertical line indicates the beginning of one stage and the end of the previous stage. Contamination levels at entry and exit of each stage are linked with straight lines for ease of visualization only, and do not indicate linear increases within stages.

Figure 6

Fig. 4. Relative percent of listeriosis cases in eleven subpopulations given consumption of contaminated turkey delicatessen meat with growth inhibitors in Canada, as determined using Monte Carlo simulation using Latin Hypercube Sampling with 100 000 iterations. Relative size of each subpopulation in Canada is also shown. Relative population size is equivalent to the relative number of contaminated portions consumed by each subpopulation. (See Table 4 for median estimates of listeriosis cases and portions.)

Figure 7

Table 4. Median portions consumed, probability of illness, listeriosis cases and respective 5th and 95th percentile estimates for the consumption of contaminated turkey delicatessen meats with growth inhibitors in each subpopulation included in the model, assuming a hypothetical contamination event. Probability of illness was determined as a weighted average across all portion types. Results were determined by Monte Carlo simulations using Latin Hypercube Sampling with 100 000 iterations

Supplementary material: File

Falk supplementary material

Falk supplementary material 1

Download Falk supplementary material(File)
File 86.5 KB