We continue our New Year, New Review series by tackling one of the primary steps in the evidence synthesis process: conducting full-text screening. MetaReviewer v1.2 now has the functionality to help review teams complete this part of the process. We created a Coding Form Template specifically for use with the full-text screening process that can be found at this link.
Prior Synthesis Work that Needs Completing
MetaReviewer v1.2’s current capabilities start with the conduct of full-text screening. This means that, before a review team interacts with MetaReviewer, several aspects of the review process need to be completed.
- Developing a review protocol and inclusion/exclusion criteria
- Conducting the systematic literature search
- De-duplicating citations
- Screening titles and abstracts
Once the review team completes abstract screening, MetaReviewer can be used for the remaining aspects of the synthesis process. The project manager should create a citation file for uploading that identifies each found citation, whether it was eligible based on abstract screening, and any other pertinent citation information. Users should create the citation file using the philosophies outlined in the User Guide, specifically around creating Citation IDs and Study IDs.
Before Using Coding Form Template for Screening
Prior to interacting with the coding form template, review teams should identify the criteria for determining eligible studies. The criteria is often similar to the abstract screening criteria, but it should be specific and exacting. The end result of screening full-text studies should be a list of studies that will be included within the evidence synthesis. Review teams should strive to operationalize any and all criteria that determines which studies are deemed eligible and which studies are deemed ineligible.
Modifying the Screening Template
With the full-text screening criteria defined, review teams should access, copy, and move the coding form template for screening into their team’s Google Drive. The first page of the coding form describes this process.
Once the coding form is within the Drive folder, it can be modified to meet the review team’s needs. We created the template with a basic format that will likely fit most evidence synthesis team needs, but it is purposefully vague and lacking context. Project managers should modify each question to fit the criteria of the team’s review.
For example, the second question asks “Does this report include an eligible sample?”. Two response options are listed, “Yes” and “No”. Although this is a perfectly reasonable question, it is not specific enough to a singular review so as to be useful because – what is an eligible sample changes based on the context. A review team conducting a synthesis on English Language Learners, for instance, will make much different determinations about sample eligibility compared to review teams screening for general population studies. The “description” row allows project managers to provide a short description of the eligible sample directly within the MetaReviewer user interface. The screenshot below illustrates this idea.
Uploading and Piloting the Form within MetaReviewer
Modifying the coding form to meet the review team’s needs is one important step, but it isn’t the last one a project manager will take prior to implementing the full-text screening process.
One crucial step to conducting a lasting and high-quality full-text screening process is piloting the form within MetaReviewer. We like to think of piloting the form in two phases.
The first phase involves only the project management person or small team. In this phase, the project manager uploads the form into MetaReviewer using the “Add new form” button within the “Manage coding forms and view responses” portal.
Once properly uploaded, users should click on the blue text link to open a test version of the form.
Users can now view the form as other members of the review team will see it for each individual study. In our experience, simply seeing the form within the MetaReviewer environment will be enough to catch errors or things to change, both big and small. Users should make those changes within the Google Document.
When ready to upload a new version of the coding form into MetaReviewer, users must first delete the outdated form in MetaReviewer first. An error message appears – like the one below – when a user attempts to upload the previous form with the same title. (Archiving a form also results in an error message.)
Should project managers not wish to delete the old form, and instead track the forms progress, one workaround is to create different numerical IDs associated with each update. Simply changing the 1 to a 2 will results the ability to upload a new version.
In phase two of the process, the leadership team includes the full review team in the piloting process. Each member of the review team can screen a single study and MetaReviewer will record each response. Project managers can download all of these responses and compare each team member’s answer.
Reviewing Progress within the Studies Portal
After piloting is completed and the full-text screening process has begun, project managers need to know about the team’s progress. Within the Studies portal, users can sort or filter on the Screening Status column to easily see the studies in process. (Once full-text screening is complete, users can also filter on the Eligibility Status column to see only Eligible studies, too.)
Another approach is to download the Studies portal data, using the “Download” button on the page. Doing so will result in an excel file that can be easily manipulated and analyzed to track progress along the way.
We look forward to see what you create in MetaReviewer!