Sysrev matches and exceeds Covidence's review capabilities.

As Part 1 of this blog showed, Sysrev is a cheaper alternative to Covidence, that facilitates both rigorous and more generalized document reviews, and has unique research support capabilities.

Part 2 will focus on specific features of Covidence as a review platform and the Sysrev analogs.  As you will see, Sysrev is also a strong alternative for features Covidence supports like import, screening and extraction and analytics & exporting.

Importing

Covidence allows references to be imported using either EndNote or Pubmed XML format, as well as text documents in RAS or CSR formatting.  

Sysrev also permits references to be imported via XML file.  Sysrev users can also use an in-application Pubmed Search to query for articles, import specific articles by PMIDs, or upload a .zip of PDFs.  The ability to batch upload PDFs greatly increases the versatility of Sysrev.  In addition to publications, projects can focus on technical sheets such as MSDS, reporting documents, patents, or any number of possibilities.

Update Nov: 11th - Sysrev now supports import using the RIS file type, which is supported by many reference and citation managers.  Learn more at https://blog.sysrev.com/ris-import-support/

For projects that involve a continuous review of new documents, Sysrev can integrate custom data sources (e.g. clinicaltrials.gov).  Contact us (tom@sysrev.com) to learn more.

"Import Articles Dashboard" - Project Admins can upload documents via an in-application PubMed Search, PMIDS, XML, or a batch PDF upload

Screening and Extraction

Covidence assumes a two part screening process – a first pass of just Title & Abstracts and then a second review of the full article – before a final Extraction step.  Each "task" within the review can be completed by 1 or 2 reviewers.  Tasks are assigned randomly.

For Title and Abstract screening, users are able to select yes, no, or maybe, while Full Article screening requires a final include or exclude designation.  Users are able to add inclusion/exclusion criteria, as well as highlights, tags, and notes which help the reviewers in their decision marking process.  

The Extraction phase is where Covidence is both the most developed and the most limited.  As a part of the extraction phase, each Covidence review has a pre-built "Quality Assessment" dashboard which asks the reviewers to look for red flags associated with bias, as well as incomplete or selective data, that may be associated with individual studies.  

The actual Data Extraction dashboard includes a number of data categories: Summary, Identification, Methods, Population, Interventions, and Outcomes.  Within each category, the user can create any number of data points for the reviewers to extract within the pre-built scenarios.  

Alternatively, Sysrev users are able to screen and perform data extraction within the same step.  As opposed to Covidence where inclusion/exclusion criteria are simply listed as reminders at the top of the review dashboard, Sysrev allows for inclusion criteria to be "hard-coded" into labels.  That is, the Project Admins can create labels which are required for a document to be included, decreasing the chance of mis-labeling.  

For the actual data extraction, Sysrev users can create as many boolean, string, or category labels as they choose – providing true flexibility in screening and extraction tasks.  

"Manage - Label Definitions" Dashboard - Project Admins can choose any number of labels for each review

Analytics & Export

Sysrev gives unparalleled insight into the progress of each Project.  For each article, Sysrev tracks every label applied by every reviewer, including conflict resolutions. Unlike Covidence which does not allow voting histories to be exported, Sysrev has a number of Export functionalities including exporting individual answers.  This is especially important as Sysrev permits the scenario where every reviewer reviews every document.   Along with tracking pure progress, Sysrev also calculates a concordance metric to give insight into each reviewer's performance.  

Overview Dashboard for https://sysrev.com/p/3144 depicting total project status, individual reviewer progress, and the inclusion/exlcusion AI predictions

Artificial Intelligence - Predicting Inclusion

Within each project, Sysrev attempts to learn from the reviewers which of the uploaded documents meeting inclusion criteria.  In fact, Sysrev chooses the order in which documents are reviewed so that it can learn the inclusion/exclusion criteria as quickly as possible.  At their own discretion, Sysrev users can filter articles based on the inclusion/exclusion predictions.

In conclusion, Covidence is a mature platform which expects reviews to be conducted in a very pre-defined way.  Conversely, Sysrev was built with flexibility in mind – any extraction task on any document.  It is by that flexibility that Sysrev is not only a viable alternative, but a powerful (and much cheaper) replacement for Covidence.