Systematic reviews and meta-analyses are considered to be the 'gold standard' of scientific evidence, in other words they are at the top of this hierarchy. Being a 'gold standard' means these types of studies present the most reliable evidence.
What is a Systematic Review?
A systematic review asks a research question and answers it by summarising the existing evidence that meets a set of specified criteria.
What is a Meta-analysis?
Some systematic reviews will present their results using meta-analysis. This is a statistical method that combines the results of the includes research in the systematic review to generate an average results (or effect). Meta-analyses can add value as they produce more precise estimates of the effect of a treatment/intervention than considering each study individually.
What are the stages of a systematic review and meta-analysis?
To think this is a simple process would be a silly move. The review starts with a research question and a protocol/research plan. Studies need to be searched for using a highly sensitive search strategy. The retrieved studies need to be screened for eligibility (done by at east 2 people working independently). Relevant data needs to then be extracted and the quality of the included studies needs to be assessed. Then finally after all that, data needs to be synthesised (meta-analysis) and results presented.
The systematic review follows a standard process which I have outlined below:
1. Identifying a research question: you need to know what you are going to be investigating. What patient group, what treatment/intervention, what outcomes etc. The more specific, the better. This also has to be something that has not already been researched in a review before, so researching databases to ensure that similar reviews have not already been published is crucial. My review is titled 'The effects of exercise alone or combined exercise and dietary interventions on markers of insulin resistance and metabolic health in postmenopausal women with breast cancer: A systematic review and meta-analysis of randomised trials'. Very specific, and covers pretty much all the points - treatment, outcomes, population, study type etc.
2. Define an inclusion and exclusion criteria: why is having an eligibility criteria important? Having pre-specified eligibility criteria helps to produce more accurate, objective, and meaningful results as having it ensures that the literature considered in the review is relevant to the study question. Establishing a 'PICO' can help big time with this. PICO = Population, Intervention, Comparison, Outcomes. Using these categories can help you easily define your inclusion/exclusion criteria, ensuring you don't miss anything important out.
3. Search for studies: a search of the large citation databases, such as MEDLINE, the Cochrane Library (CENTRAL), Scopus, Web of Science etc. will be sufficient. However, for a more comprehensive search, it is worth including a search of the Grey Literature. When you have your chosen databases, you will need to devise a search strategy using search terms relating to you study question. This is where your chosen PICO can also come in handy.
4. Select studies (title/abstract and full-text screening): this to me is the boring part as it involves A LOT of reading, especially if your search finds a lot of results. Mine had about 3500! First, you must read through every title and abstract of the studies you've found as a result of your searches, and check these against your pre-specified eligibility criteria. Those deemed eligible are moved on to the next step, and those deemed in-eligible, or irrelevant, are excluded. The next step is to read through the entire full-texts of the studies which passed the title/abstract screenings, and deem which of ones will be eligible and those to be excluded. Those you decided to be eligible will be included in the review! YAY!
5. Extract data: data extraction is the process of extracting pieces of information from the studies you have assessed for eligibility in your review and organising the information in a way that will help you synthesise the studies and draw conclusions. From each study, the following data may need to be extracted, depending on the review's purpose: title, author, year, journal, research question and specific aims, conceptual framework, hypothesis, research methods or study type, outcome results, and concluding points. Special attention should be paid to the methodology, which is important when it comes to quality assessment. If a meta-analysis is being conducted, then raw and refined data from each result in each study will need to be collected.
6. Assess quality: assessing the quality of evidence contained within a systematic review is as important as analysing the data within. Results from a poorly conducted study can be skewed by biases from the research methodology and should be interpreted with caution. Not every scientific study is conducted perfectly. Quality assessment therefore helps in minimising the risk of bias and increases confidence in review findings.
7. Synthesise and present results: synthesis is the stage where extracted data are combined and evaluated, and will determine the outcomes of the review. The way data is synthesised and presented depends on the type of data being handled, e.g., quantitative or qualitative. What ever tools you use, the general purpose is to show the outcomes and effects of various studies and identify issue with methodology and quality. This means that your synthesis might reveal a number of elements, including:
- Overall level of evidence,
- The degree of consistency in the findings,
- What the positive effects of an intervention or treatment are,
- How many studies found a relationship or association between two things.
8. Quantitative synthesis (meta-analysis): data in a quantitative systematic review is presented statistically. Meta-analysis combines and evaluates data from multiple studies to draw conclusions about outcomes, effects, shortcomings of studies and/or applicability of findings. Data is commonly represented in the form of a forest plot - a way of combining results of multiple studies in order to show point estimates arising from different studies of the same condition or treatment.
So, as you can see from what you have just read, it is not that simple. And this is where I talk about my review process so far. It's not been that easy for me as I have faced a couple of set backs which have consequently delayed the progress and completion of my review.
Okay, so where do I begin. It was all going quite smoothly for me. I had completed and registered my protocol, created my search strategy, completed my searches and my abstract and full-text screenings. However, I found I was left with NO eligible studies. You read right, no eligible studies to include in my review. I'm hoping that this happens more often than I think. However, with such a specific study population of interest I guess it is to be expected. I did what most PhD students would in this situation, run in panic to their supervisor. All was calm, we decided to alter one or two things in my eligibility criteria to make it less strict and then decided we would conduct a secondary analysis on those things we had removed. This is the point where I would love to tell you that that was enough, and I have now completed and written up my review paper. But this is not that story. Even with a changed eligibility criteria, I still had no eligible studies to include, cry :(.
So this is where I panicked, for real. I pretty much felt stuck, like there was absolutely nothing I could do. I didn't know what to do. Again, I went back to my supervisor to give him the bad news. Following this, we decided to re-do all of the searches again with an edited search strategy - we removed any words relating to 'diet' as this wasn't our main intervention of interest in hopes to receive more studies to re-screen. As a result, we have received around 3500 results, compared to around 350 in my previous search attempts. I have now completed the title/abstract screening along with 3 other researchers and I am now ready to start the full-text screenings.
My take away: My main takeaway from this, is to expect the research process not to go 100% to plan, and to allow it. I had originally planned to have finished my review write up by now, but obviously to what I've described above that goal has not been achieved. I've learned to be more patient with the research process and that there is always a solution when things don't go my way!
No comments
Post a Comment