The joy I feel when I finally start data extraction on an evidence synthesis project can only be matched by the dread I feel when I try to make sense of how my team is progressing through coding. Every week, I want to know how many studies we've completed, which studies are ready for reconciliation or validation, and who is ready for new assignments. This information is essential to keep things moving forward week-to-week, to help me figure out how to motivate team members through a coding slump, and to re-adjust timelines as needed.

Previously in MetaReviewer, I had to rely on team members manually updating their user status and the study status to get this essential information. Because we're all human--and busy ones, at that--this is an error-prone system. People sometimes forget to update their statuses or enter incorrect statuses. As a result, I'm not always confident about how much progress we're making. This is not only headache-inducing as I try to make assignments or report on coding progress, but it can hinder overall progress on the project if studies are held up in the pipeline because they aren't marked correctly.

That all changes with one of MetaReviewer v.1.3.0's new features. Now, a user's workflow status automatically updates as they take actions, like opening a form, submitting a form, or flagging a study for discussion. Similarly, a study's status automatically updates as users work through their assignments. This makes it easier for project managers to create progress reports and track individual team members' assignments. It also helps individual team members stay on top of their workload by providing real-time updates of the studies to which they're assigned. Keep reading to learn about how to set your project up to support automated status updates and how this all works for common assignment configurations.

Setting forms to workflows

To get started, you simply need to set forms to their corresponding user workflows and select a default form for each workflow. Click through the images below to see how to set forms to workflows:

Once you've selected a form as the default form for a workflow, you will see it listed above that workflow on every individual study page in your project:

workflows_with_forms

In this example, MetaReviewer now know that it should update the study's coding status based on what coders assigned to that study do with their responses to the "INSERT NAME" coding form.

Ok, so how do automated status updates work?

We designed the automated status updates with one of the most common assignment configurations in mind: two coders and a reconciler. In general, there are two ways that a project administrator might make these assignments: all at once or sequentially.

All-at-once assignments

A project administrator could decide to assign coding to three team members in one fell swoop. Initially, all three users' statuses and the study coding status will be "Assigned - not started." When coders open the default coding form, their statuses and the study's coding status will automatically update to "Assigned - in progress." All it takes is one coder to get started for a study's coding status to shift into action! When a coder submits their response for the form set to the coding workflow, their individual user status for that assignment will update to "Submitted." Only when both coders have submitted their responses will the study's coding status update to "Ready for reconciliation/validation."

At this point, the assigned reconciler will see in their Assignments tab that the study is ready for their attention.

assignments_tab

Once the reconciler opens their reconciliation form, the study coding status will update to "Reconciliation/validation in progress." At this point, coders' forms will be locked so they can't make edits to their responses while reconciliation is in progress. If the reconciler determines that coders need to revisit their forms, the reconciler will need to manually update the study coding status to "Revisions needed." In this case, both coders' forms will be unlocked for editing, and both the user and study coding statuses will revert to "Assigned - in progress." The study's coding status will only update to "Complete" once the reconciler has submitted their reconciliation form.

Sequential assignments

A project administrator might, instead, decide to make assignments sequentially. For instance, they might assign Coder 1 to a study but leave the other roles unassigned because they aren't sure yet who they want to assign (or if they even want to double code...more on that soon). In this case, when Coder 1 submits their coding form, the study's coding status will update to "Ready for reconciliation/validation." If the project administrator sees in the Studies Portal that there is a study that is "Ready for reconciliation/validation" but it doesn't have a Coder 2 or Reconciler/Validator assignment, that's a sign that they need to make assignments for that study.

They could assign a second coder and leave the reconciler/validator assignment blank, which would revert the study status back to "Assigned - in progress." But once Coder 2 submits their form, the project administrator will see the study is, again, "Ready for reconciliation/validation" and will need to make an assignment to complete the coding workflow.

Alternatively, the project administrator might decided against assigning a second coder or a reconciler/validator. In this case, they would need to select "No assignment needed" for the both roles for the study coding status to automatically update to "Complete."

Finally, they might decide that they want to assign a validator but don't need a second coder. In this case, they would need to select "No assignment needed" for Coder 2 and make an assignment for the reconciler/validator. The study coding status will remain "Ready for reconciliation/validation."

Ready to play?

confused

If you're feeling overwhelmed, don't worry; we're not going to share any more "rules of automation" today. Get in MetaReviewer, make a sandbox project, and play around with the assignments so you can see how it works for yourself. As always, email the Help Desk with any questions you have and keep an eye on the Learn page for more information about automated status updates!