Training montages are one of my favorite film sequences. They’re great. They condense what would be a boring amount of repetitive content into a snappy set of scenes, they briefly fill you with unrealistic optimism that you, too, could learn an impressive skill, and the soundtracks are always fire. Most syntheses will need their own training phase, most likely to train coders to extract information from studies. MetaReviewer can’t condense that training into montage form, but it can help you organize your training process. In this post, I want to share a few ways to manage coder training in MetaReviewer. Let’s get down to business.
Let’s say you’ve got 4 coders that you need to train for your project. You want them to all code the same study (or studies) to find out where there might be confusion about the coding protocol. However, at present, MetaReviewer only allows you to assign two coders to a study. The reconciliation function can only handle two responses as well. There are a couple of ways to get around these problems.
First, you could just ignore the assignment function and tell all your coders to code the same study. Even though only two can be officially assigned, you can enter as many coding form responses on a study as you’d like. Just have each person click on your coding form on your training study page and fill out the form.
Selecting the “Click to view responses” button, to the right of the coding form title, will show you a list of all responses, completed or in progress. When you’re ready to compare their answers, you can click the Download button on this page. This will create a csv file with each coder’s response as its own labeled row. This way, you can see if coders agreed with each other (and you) and identify where coders might have gone astray.
A second option is to create duplicate study entries for your training study and utilize the assignment feature. So, if you had four coders to train, create two study entries for training and assign two coders to each. You can link the same citation to multiple studies, so you’ll only have to enter information for your training article once.
An advantage of this approach is that it allows you use the reconciliation feature, enabling you (or your coders) to easily compare their results and practice the reconciliation functionality.
Regardless of your approach, your training process will likely produce more coding form responses than you’ll need. When you're done with training, you can simply delete the responses (or studies) that you no longer need. This will declutter your project space while keeping some of the coding for your eventual synthesis. Conversely, if you want to keep your training responses for future reference but not have to delete those rows out of your final data, you could set up a project just for training purposes.
These are just a few ways to organize coder training in MetaReviewer. There are likely other paths that could work well for your project. If you ever want to work through ideas or ask about MetaReviewer’s capabilities, don’t hesitate to hit up the help desk. Until then, happy training!
