Hostname: page-component-7c8c6479df-p566r Total loading time: 0 Render date: 2024-03-27T12:25:37.973Z Has data issue: false hasContentIssue false

Systematic reviews in nutrition: standardized methodology

Published online by Cambridge University Press:  17 May 2012

Carmina Wanden-Berghe*
Affiliation:
Department of Physiology, Pharmacology and Toxicology, University Cardenal Herrera - CEU, Elche, Spain Home Care Unit, University General Hospital, Alicante, Spain
Javier Sanz-Valero
Affiliation:
Department of Public Health, History of Science and Gynecology, Miguel Hernández University, Alicante, Spain Department of Community Nursing, Preventive Medicine, and Public Health and History of Science, University of Alicante, Alicante, Spain
*
*Corresponding author: Carmina Wanden-Berghe, fax +34677415560, email carminaw@telefonica.net
Rights & Permissions [Opens in a new window]

Abstract

The objective of this study is to establish a methodological proposal in order to carry out qualitative systematic reviews and apply these findings to a review of Omega-3 Fatty Acids with respect to health and illness. Based on a methodological proposal, a general protocol was developed to provide a sound basis for the preparation of the reviews in this journal supplement. A systematic technique was proposed in order to revise the existing scientific literature on Omega-3 Fatty Acids, with particular emphasis on aspects relating to health and illness. The aim of qualitative systematic reviews is to collate and summarise the results of the primary studies reviewed which will be carried out through a descriptive synthesis. It can be concluded that systematic reviews provide a summary of the existing primary documents on a specific scientific question. The detailed and explicit methods used lead to the identification, critical evaluation and synthesis of the scientific literature. Furthermore, both bias and random effects are reduced, resulting in more reliable data from which to draw conclusions and make recommendations to support decision-making.

Type
Full Papers
Copyright
Copyright © The Authors 2012

The objective of a systematic review is to integrate the existing knowledge in a particular subject or, more specifically, concerning a scientific question. A systematic review is carried out under the following conditions:

  1. (1) A review is conducted by a methodology which is previously set out and fully detailed in a protocol that is thorough, critical and designed to limit bias.

  2. (2) There is an analysis and integration of the findings from the primary research studies regarding the health problem posed.

In the case where the study design included in the systematic review enables the combination and statistical treatment of the results, a meta-analysis or quantitative systematic review can be carried out.

The first reference to a systematic review was made by Sir George Biddell Airy (1801–1892)(Reference Airy1), the British mathematician and astronomer. His manual describes the method used to carry out a quantitative synthesis of the results of different studies, which Glass(Reference Glass2 )subsequently named “Meta-Analysis”. In the biomedical field, one of the first examples of meta-analysis was published in 1904 by Karl Pearson(Reference Pearson3), who concluded that the vaccine for Intestinal Fever was not effective enough to be recommended as a vaccination. Statistical methods were later developed for meta-analysis, and from the 1970 s onwards they were applied with greater frequency, with a large number of publications specifically by Glass(Reference Glass2, Reference Glass, McGaw and Smith4) and Stjernswärd(Reference Stjernswärd5, Reference Stjernswärd, Muenz and von Essen6).

The benefits of systematic reviews are derived from the pursuit of their specific objectives, namely the description of a specific event; the analysis of the impact of certain factors on an illness; the appraisal of diagnostic tests; the assessment of the treatment; the identification of special population groups or an evaluation of the characteristics of individual research(Reference Delgado Rodriguez7).

Reviews can be proposed with different starting points: for example, with the purpose of examining existing knowledge surrounding particular questions posed. In these cases, if an association is found, a hypothesis can be put forward to test the possible relationship between exposure and effect, its meaning and magnitude, and if the same effect is experienced by the different population subgroups. Furthermore, methodological problems or other flaws in the study of the association to be analysed may be detected.

For any initial analysis in a systematic review, it is very important to establish the protocol to be followed, as this will determine the criteria for the selection and inclusion of articles (study design, language, quality, etc), the strategies to avoid any possible bias as well as the format for presenting the results.

This study thus provides a methodological proposal for qualitative systematic reviews.

Systematic review methodology

Although there exist different approaches for scientific literature reviews, systematic reviews are considered to offer the most reliable method. The systematic review protocol guarantees that the review is carried out with the rigours demanded of all research and should contain the following sections:

  • – Formulation of the problem to be studied.

  • – Identification of the studies through a bibliographic search.

  • – Selection of the studies and a critical quality assessment.

  • – Data extraction and synthesis of results.

  • – Conclusions and Recommendations.

Defining the research question

This is the preliminary, fundamental step of all scientific literature systematic reviews, since adequate formulation of the research question will determine the correctness of the process(Reference Rothman, Greenland and Lash8), which will in turn generate the protocol.

The research question must be based on the following elements: specific population to be researched; evaluation of the intervention; a comparison using other techniques or options, if appropriate, and the results which will give a measure of the intervention studied(9, 10). These elements comprise the acronym PICO (Population, Intervention, Comparison and Outcomes).

Bibliographic searches and data sources

The identification and location of studies must be carried out via a strategic and systematic bibliographic search which identifies the appropriate studies. It must also be easily replicated and applicable to the majority of existing health science databases.

This search must take into account the elements of the review research question, as well as the most appropriate study designs(11). Consequently, an effective review search formula is developed, determining the suitable Descriptors and Qualifiers(12, 13)for each of the PICO elements, combining them with the appropriate Boolean operators. It is important to establish the criteria for inclusion in the review beforehand, in order to determine which articles should be extracted.

However, it is important to consider that the exhaustiveness of the search is more important than its precision (where more irrelevant articles are identified), although the ideal is to strike a balance between the degree of exhaustiveness (highly sensitive searches) and precision (specific searches).When deciding the scope of the bibliographic search formula, it is always preferable to use noise-generating words, which may result in retrieving irrelevant articles that can subsequently be discarded, than risk losing relevant articles that have not been retrieved by the search. In the event of the latter case, where highly defined searches are used, it is advisable to redefine the search formula.

Once the search is completed on the computerised databases (a systematic review should strive to be as broad as possible in the use of bibliographic sources), other information sources need to be consulted in order to minimise any possible publication bias. Furthermore, the bibliography included in the selected articles should be checked with a view to locating additional studies; “grey literature” should be retrieved, such as doctoral theses, conference proceedings, research projects, reports from institutions or from the pharmaceutical industry, and information should be requested from experts.

It is important for the search strategy to be clearly described in the protocol and subsequently in the publication of the systematic review, including the Descriptors, their combination, the period covered by the search, the types of articles being searched, and any other information that helps to accurately reproduce the search carried out.

Concerning documentary typology, it is important to highlight that the Cochrane Collaboration focuses on systematic reviews of randomised controlled trials as they tend to provide more reliable information than other sources of evidence on the differential effects of the various health care alternatives(14).

Selection and critical appraisal of studies

Strict compliance with this section will limit bias, facilitate the interpretation and discussion of the results as well as ensure the replication of the process. This section must establish the most suitable research design for tackling the question under review.

Studies will be selected according to the inclusion or exclusion criteria established by two independent reviewers. Whenever possible, the independent reviewers will not be aware of the authors, journals or results of the articles to be selected. It is useful to measure the correlation between the data obtained by the independent reviewers, and establish beforehand the minimum level of agreement that is required.

Experts in a specific area often have previously formed opinions that can bias their evaluations, both in terms of the relevance and the validity of the articles to be included. Therefore, although it is important that at least one of the reviewers is well versed on the issue of the review, it may also be beneficial to have a second reviewer who is not an expert in the field(14).

The fact that a scientific article has undergone a peer review does not necessarily guarantee the quality of the research(15). The authors' scientific competence and the prestige of the biomedical journal are important criteria, but not sufficient in themselves to guarantee the credibility of the research(Reference Gérvas and Pérez Fernández16).

Problems relating to replicability and bias can affect two stages of the review process: the final decision regarding study inclusion criteria and quality appraisal(15). A systematic review will minimise these problems and provide appropriate conclusions only if a thorough appraisal is carried out on the primary studies which form the basis of the review. Thus, before deciding to include an article, an appraisal of its methodological rigour must be carried out. A critical appraisal is aimed at discovering if the methods and, as a consequence, the results of a study are valid. This quality assessment should be carried out confidentially and by two independent reviewers. Qualitative and quantitative scales are useful for quality assessments as they will have a positive impact on the reliability of the conclusions. However, these methods introduce a certain inter and intra observer variability, therefore it is necessary to establish the minimum degree of agreement that needs to be reached between the reviewers. The following sources are cited to assist in quality evaluation:

  • – American Medical Association: JAMAevidence(17).

  • – CRD's guidance for undertaking reviews in health care(11).

  • – U.S. Preventive Services Task Force (USPSTF)(18).

  • – Scottish Intercollegiate Guidelines Network (SIGN)(19).

  • – EBM-Tools, Center for Evidence-Based Medicine (CEBM)(20).

  • – Consolidated Standards of Reporting Trials (CONSORT)(21).

  • – Jadad scoring or the Oxford quality scoring system(Reference Jadad, Moore and Carroll22).

  • – Evidence Base, National Institute for Health and Clinical Excellence (NICE)(23).

  • – Canadian Agency for Drugs and Technologies in Health (CADTH)(24).

  • – Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)(Reference Moher, Liberati and Tetzlaff25).

Data extraction and synthesis of results

The objective of this stage of the review is to compile a narrative summary of the individual studies as this will facilitate the general appraisal. Meta-analysis techniques will be used to combine the quantitative results of the individual studies.

Data extraction is the process through which information is obtained from the primary studies which have been selected from the bibliographic search(26). Data will be collected using an appropriate form which will have been previously included in the protocol. Subsequently, the data retrieved will be transferred to the evidence or review tables. Specifically, the information relating to the acronym PICO (Population, Intervention, Comparison and Outcomes) will be included, as well as the time variable.

On occasions, the lack of relevant information may make it necessary to contact the corresponding author of the retrieved articles, and if necessary request further information.

The descriptive presentation of the results should begin with a statement on the number of accepted articles, indicating those that were included or excluded and the rationale behind the decision. A flow chart of the entire process may assist in explaining this section with greater clarity.

The non quantitative synthesis of the results is set out in the review table to help readers to compare the evidence gathered on the subject. The accuracy and the synopsis of the information in the tables is one of the main steps involved in a systematic review.

Furthermore, together with the PICO variables, the variables related to design and methodological quality are also important in assessing the consistency of the articles included in the review.

Conclusions and recommendations

A systematic review is a synthesis of the best available evidence on a specific scientific question obtained using a methodology which minimises the margin of error. It could be said that the results speak for themselves.

Nevertheless, the final product is not a mere summary of what is known about an intervention, it also highlights the areas that require further research(10). In the discussion section, all reviews should deal with any important methodological limitations of the studies included and of the methods used in the review that may affect any decisions that need to be taken, or future research.

The conclusions are only justifiable when the process of data collection, analysis and integration is done thoroughly and systematically. The efforts involved in conducting a systematic review can be beneficial in identifying knowledge gaps in the subject and suggesting recommendations for future initiatives(15) based exclusively on the knowledge reviewed. A common error is to reach conclusions which go beyond the evidence analysed.

It is however appropriate to present the primary results of the review and its implications for practice, as well as to consider and formulate new questions that may arise from the revised research as regards application to different population groups and new interventions, for example.

Systematic reviews provide a good evaluation of the available evidence, but do not offer any guarantees. The need for prior knowledge, common sense and clinical judgement will always be necessary to interpret the results of this review.

Applying the methodology to a review of Omega-3 Fatty Acids in health and illness

For the production of this journal supplement, a systematic technique was proposed as the methodology for the review of scientific literature on Omega-3 Fatty Acids related to health or illness.

Data sources:

The data used in the different studies of this journal supplement were obtained by consulting and accessing various online health science databases. Consequently, the following sources can be recommended:

  • – Medlars Online International Literature (MEDLINE), via PubMed

  • – EMBASE

  • – Web of Knowledge, Institute for Scientific Information (ISI)

  • – The Cochrane Library Plus

  • – International Pharmaceutical Abstracts (IPA)

  • – Psychology Information (PsycINFO)

  • – Cumulative Index to Nursing and Allied Health Literature (CINAHL)

  • – Biological Abstract

  • – Food Science and Technology Abstracts (FSTA)

  • – Latin American and Caribbean Heath Sciences Literature (LILACS)

  • – Pan American Health Organization Library (PAHO)

  • – World Health Organization Library Information System (WHOLIS)

Data processing

The articles studied were those published in any country or by any institution or individual researcher, and also those published since the beginning of the indexing of each source used.

The differences regarding the selection of primary articles, due to the language of publication, the study design, and the methodological study are noted in each of the reviews conducted.

For document retrieval, the Medical Subject Headings (MeSH) developed by the U.S National Library of Medicine (NLM) were used wherever possible.

Likewise, subheadings were also used, whenever the formulated search strategy deemed it appropriate.

By studying the NLM Thesaurus, the appropriate MeSH (or Major) were obtained for each of the reviews. Similarly, boundaries, such as type of articles, species, and ages, were used.

Tags were included in the search strategy when there was no suitable MeSH available, or when they enabled a more exhaustive search formula.

Search formulae were initially developed for use in the MEDLINE database, via PubMed, by using Boolean connectors and subsequently adapting them to the aforementioned databases. In any case, these formulae can be systematically reproduced at any time in the respective databases.

Data collection

The format for data collection from the primary studies was agreed upon in the protocol and was designed to facilitate the selection of the necessary information. These tasks can be carried out by specific software programmes that are used in systematic reviews as they permit the organisation of the articles and the data that needs to be handled. One of these tools is the RevMan (Review Manager)(27) software used to prepare and maintain Cochrane reviews. It enables the following: preparation of the manuscript text; construction of the tables that show the characteristics of the studies; performance of a meta-analysis if appropriate and the graphical presentation of the results. Together with RevMan, Archie (the Cochrane Collaboration's central server for managing documents and contact details) forms the Cochrane Information Management System (IMS), which is designed to assist contributors to the Cochrane Collaboration.

Another computer application designed for collaborative online work is the Systematic Reviews Collaborative Tool (SysCollab)(28), specifically developed for this journal supplement, although soon a publicly available version will allow the simultaneous participation of the researchers involved in all stages of the review. It will enable the management and selection of the information from the articles, the establishment of the protocol, the generation of the forms to analyse the quality of the documents and real time and deferred communication between participants.

Furthermore, the text of the review can be drawn up as a collaborative effort, along with the generation of the tables and graphics.

Although consistent with the structure proposed by Cochrane, this software also incorporates the evidence based criteria of other medicine organisations. Regardless of the format or software used, the synthesis of the data in the review tables should include, in addition to the PICO acronym, all other variables that enable the evidence gathered from the primary studies to be compared.

Selection of articles

The final selection of the articles was made according to the inclusion or exclusion criteria defined in each of the reviews, although in all cases, the following issues were taken into consideration:

  • Typology. The type of chosen document is indicated in each of the reviews.

  • Study design. Selection depended on the characteristics of the question posed by each of the reviews, and those which included some degree of recommendation based on the levels of scientific evidence set out by the US Agency for Health Research and Quality(18), were included.

  • Quality. Articles were excluded if they did not meet the minimum requirements as regards quality of design, in their Spanish version, in accordance with to the Critical Appraisal Skills Programme, (CASPe)(29).

  • Control of bias. Studies which could not guarantee the control of bias (for example, co-morbidities that could distort the outcome of the study).

  • When ad hoc criteria were used for certain reviews, they were included in the methodology and the possible limitations of ad hoc criteria were also mentioned in the discussion.

Furthermore, as a secondary search to guarantee exhaustiveness and reduce possible publication biases, the bibliographic list of the articles selected in the primary search was examined. The aim was to identify studies that were not detected in the electronic review. Likewise, the available “grey literature” was identified in the studies where possible.

A flow chart was recommended to clearly explain the rationale for the inclusion or exclusion of the articles.

Acknowledgements

Both authors have contributed equally to the development of this work, in accordance with the Vancouver Norms. We certify that there is no conflict of interest. This article received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

References

1 Airy, GB (1861) On the algebraic and numerical theory of errors of observations and the combination of observations [monograph on the Internet]. Cambridge: MacMillan and co. Available at: http://www.jameslindlibrary.org/illustrating/records/on-the-algebraical-and-numerical-theory-of-errors-of-observation/title_pages (accessed 15 February 2011).Google Scholar
2 Glass, GV (1976) Primary, secondary and meta-analysis of research. Educational Researcher 10, 38.CrossRefGoogle Scholar
3 Pearson, K (1904) Report on certain enteric fever inoculation statistics. BMJ 3, 12431246.Google Scholar
4 Glass, GV, McGaw, B & Smith, ML (1981) Meta-analysis in social research [monograph on the Internet]. Newbury Park: Sage Publications. Available at http://www.jameslindlibrary.org/illustrating/records/meta-analysis-in-social-research/title_pages (accessed 15 February 2011).Google Scholar
5 Stjernswärd, J (1974) Decreased survival related to irradiation postoperatively in early operable breast cancer. Lancet 304, 12851286.CrossRefGoogle Scholar
6 Stjernswärd, J, Muenz, LR & von Essen, CF (1976) Postoperative radiotherapy and breast cancer. Lancet 307, 749.CrossRefGoogle Scholar
7 Delgado Rodriguez, M (2010) Revisión sistemática de estudios: Metanálisis. Barcelona: Signo Editores.Google Scholar
8 Rothman, KJ, Greenland, S & Lash, TL (2008) Modern Epidemiology, 3th ed. Philadelphia, USA: Lippincott Williams & Wilkins.Google Scholar
9 The Cochrane Collaboration (2011) The Cochrane Policy Manual [homepage]. Oxford, UK: The Cochrane Collaboration. Available at http://www.cochrane.org/policy-manual/welcome (accessed 15 February 2011).Google Scholar
10 The Joanna Briggs Institute. (2001) Introduction to Systematic Reviews. Changing Practice [serial on the Internet]. Available at http://www.joannabriggs.edu.au/pubs/best_practice.php (accessed 15 February 2011).Google Scholar
11 Centre for Reviews and Dissemination (2009) Systematic Reviews: CRD's guidance for undertaking reviews in health care [monograph on the Internet]. York, UK: National Institute for Health Research, University of York. Available at http://www.york.ac.uk/inst/crd/index_guidance.htm (accessed 16 February 2011).Google Scholar
12 Medical Subject Headings (MeSH) [Database on the Internet] (2011) Bethesda: National Center for Biotechnology Information NCBI, U.S. National Library of Medicine. Available at: http://www.ncbi.nlm.nih.gov/sites/entrez?db = mesh (accessed 16 February 2011).Google Scholar
13 Health Sciences Descriptors (DeCS). [Database on the Internet] (2010) Sao Paulo: Virtual Health Library. Available at http://decs.bvs.br/I/homepagei.htm (accessed 16 February 2011).Google Scholar
14 Iberoamerican Cochrane Centre Module (2011) Handbook (Spanish version) [homepage]. Barcelona, Spain: Centro Cochrane Iberoamericano. Available at: http://www.cochrane.es/?q = es/handbook (accessed 23 February 2011).Google Scholar
15 González de Dios J, Balaguer Santamaría A. (2007) Revisiones sistemáticas y metanálisis (I): conceptos básicos. Evid Pediatr [serial on the Internet]. Available at: http://www.evidenciasenpediatria.es/DetalleArticulo/_LLP3k9qgzIh7aNQBiadwmV2uNpzhRtGoySyIrAFidII1cH_sSbFkbmyd7FSp9aUz2s8mhuZE3MdxEj-U4cn17g#articulo-completo (accessed 23 February 2011).Google Scholar
16 Gérvas, J & Pérez Fernández, M (2001) La revisión por pares en las revistas científicas. Aten Primaria 27, 432439.CrossRefGoogle Scholar
17 American Medical Association (2011) JAMAevidence: Core topics in evidence-based medicine [homepage]. Available at http://jamaevidence.com/index (accessed 15 February 2011) Chicago, USA: American Medical Association & The McGraw-Hill Companies.Google Scholar
18 U.S. Agency for Healthcare Research and Quality (2011) U.S. Preventive Services Task Force (USPSTF) [homepage]. Rockville, USA: Agency for Healthcare Research and Quality, USPSTF. Available at http://www.ahrq.gov/clinic/uspstfix.htm (accessed 19 February 2011).Google Scholar
19 NHS Quality Improvement Scotland (NHS QIS) (2011) Scottish Intercollegiate Guidelines Network (SIGN) [homepage]. Edinburgh, UK: NHS QIS. Available at http://www.sign.ac.uk/index.html (accessed 22 February 2011).Google Scholar
20 Center for Evidence-Based Medicine (CEBM) (2011) Tools for each step of the EBM process [homepage]. Oxford, UK: CEBM, University of Oxford. Available at http://www.cebm.net/index.aspx?o = 1023 (accessed 22 February 2011).Google Scholar
21 Consolidated Standards of Reporting Trials (CONSORT). (2010) The CONSORT Statement [homepage]. The CONSORT Group. Available at http://www.consort-statement.org/consort-statement/ (accessed 22 February 2011).Google Scholar
22 Jadad, AR, Moore, RA, Carroll, D, et al. (1996) Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 17, 112.CrossRefGoogle ScholarPubMed
23 National Institute for Health and Clinical Excellence (NICE). (2010) Evidence Base [homepage]. Health Development Agency, NICE. Available at http://www.nice.org.uk/aboutnice/whoweare/aboutthehda/evidencebase/evidence_base.jsp (accessed 22 February 2011).Google Scholar
24 Canadian Agency for Drugs and Technologies in Health (CADTH) (2003) Guidelines for Authors of CADTH Health Technology Assessment Reports [monograph on the Internet]. Ottawa, Canada: CADTH. Available at http://www.cadth.ca/index.php/en/related-links/authors-guide (accessed 22 February 2011).Google Scholar
25 Moher, D and The PRISMA Group. Liberati, A & Tetzlaff, J, et al. (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [homepage] the PRISMA Statement. Available at: http://www.prisma-statement.org/index.htm (accessed 21 August 2011).CrossRefGoogle Scholar
26 Atienza Merino G, Maceira Rozas MC, Paz Valiñas L. (2008) Las revisiones sistemáticas, guías clínicas [monograph on the Internet]. A Coruña, Spain: Fisterra, atención primaria en la Red, Available at http://www.fisterra.com/guias2/fmc/rrss.asp (accessed 15 February 2011).Google Scholar
27 Review Manager (RevMan) [Computer program]. (2008), version 5·0. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration.Google Scholar
28 Systematic Reviews Collaborative Tool (SysCollab) [Computer program] Granada, Spain: University of Granada.Google Scholar
29 CASPe (2011) Critical Appraisal Skills Programme Español (CASPe) [homepage]. Alicante, Spain: CASPe. Available at http://www.redcaspe.org/ (accessed 23 February 2011).Google Scholar