Effect Size EasY in PYthon – or eseypy (Pronounced: easy-pie) for short – is one of our newest updates to MetaReviewer. Let’s learn about the next iteration of estimating effect sizes in MetaReviewer and why eseypy is important for MetaReviewer’s future.
What is eseypy and why is it necessary?
We launched MetaReviewer as a full access application on December 1, 2023. For the past 14 months, we’ve been seeing – and hearing from users via the Help Desk – that MetaReviewer’s original effect size calculation functionality didn’t live up to our standards and promises. In short, when users added more than a few effect size rows to a coding form, the application couldn’t keep up with the calculations. Therefore, it appeared to some users like MetaReviewer was not updating the effect size calculations in real time. This was frustrating to us as users and especially as MetaReviewer maintainers.
We know how vital the estimation of effect sizes within MetaReviewer is to users – and to evidence synthesis – so we vowed to make it right.
The result is eseypy. Essentially, instead of storing the effect size calculations within the coding forms, we store the effect size calculations within a Python library called eseypy. When users click the ‘Calculate effect size’ button, MetaReviewer sends the information to the library, eseypy estimates the effect size, and that estimated effect size is returned to MetaReviewer. (Note: this all happens on the backend of the program; all you have to do now is push the ‘Calculate’ button!)
What can eseypy calculate currently?
We want to ensure that the current functionality is perfected before moving to additional effect size types. We have therefore loaded eseypy with the same 13 effect size types that it currently handles – all standardized mean-difference effect size types. Our latest version of the MetaReviewer User Guide provides more detail on the types and their exact calculations.
What will eseypy calculate in the future?
We know that – while important – those 13 effect size types represent only the tip of the effect-size-type-iceberg. Fortunately, this new architecture will allow us to roll out new effect size types more frequently.
To start, the next update will incorporate cluster-level effect sizes articulated within the What Works Clearinghouse Version 5.0 Handbook (WWC, 2022). We’ve already begun the process of making these updates, so we hope to roll these updates out by the summer 2025 – hopefully sooner! In the longer-term, we anticipate incorporating calculations that move beyond the standardized mean-difference. We also will incorporate ways to convert between effect size types – for example converting an odds ratio to a standardized mean-difference, or vice versa.
What do I need to do to use eseypy within MetaReviewer?
The short answer is – not much!
If you are new(ish) to MetaReviewer, and using coding form templates generated this month (January 2025), then you don’t need to do anything. Simply upload the coding form, enter your effect size data, press the new “Calculate” button – and watch the magic unfold before your eyes.
If you are an experienced MetaReviewer user, with a project created before 2025 that included effect size calculations, then you will need to do a few small things to set up your project for success.
First, navigate to the project homepage as you normally would. If you are a project admin, you will see a banner at the top of the screen confirming that you would like to update the coding form to correspond with eseypy. Press confirm and the coding forms within MetaReviewer will update.
Second, navigate to the “Manage coding forms and view responses” page. From there, click on the “View responses” button for the coding form with effect sizes. On the Responses page, you will see a new button that says, “Calculate effect sizes”.
Clicking it will display a pop-up screen where you will press “Calculate”. This will run eseypy calculations across all the studies with effect size responses. It will also show you where missing data will prevent the calculation of an effect size.
Calculate effect sizes with peace of mind
We think this new effect size interface will have a great impact on current and future synthesis teams. Let us know what is working well and what you’d like to see improved.
