Evidence Prime Webinar: GRADEpro from A to Z: in-depth information and practical examples

‍

The third Evidence Prime webinar has been another huge success.
Thank you all for your active participation!

We would like to thank our speakers, Justyna Lityńska and Artur Nowak.

If you want to read more about our speakers and the event, click here: Webinar info.

If you want to learn more about GRADEpro, open an account, and test it for free, you can do it here: https://www.gradepro.org/. 

The theme "GRADEpro from A to Z" has sparked a great deal of interest. As a result, we welcomed a very active audience of almost 200 participants from all over the world, representing various fields of expertise. In addition, nearly 500 registrants received the video recording after the event. 

‍If you have missed this opportunity and would still like to receive the recording of the sessions, don't hesitate to get in touch with us at webinars@evidenceprime.com.

During our previous webinar, "Why GRADE?" we talked about why to use GRADE methodology for your guideline development. Then, we gave you a quick run-through of GRADEpro tool and showed you how it could help you create guidelines according to the best practices of GRADE.

We have received a lot of requests from the participants to provide more detailed information and practical examples of how you use the tool. In the response, we have created this webinar, where we presented introductory information on:

1. What GRADEpro is: We have demonstrated how to set up Your GRADEpro profile
2. We have talked about data management in GRADEpro with a practical example of how to use the tool and guide the process.
3. We described panel interactions and talked more about PanelVoice and other features, engaging a panel of experts.
4. Finally, we showed how to disseminate the recommendations and gave you a sneak peek into production to demonstrate the new features coming out soon, such as the publication platform and text editor. 
   

To read more about our tool GRADEpro and test it for free, check https://www.gradepro.org/. 
To learn more about Panel Voice and other features, visit our constantly expanding Knowledge base, https://kb.gradepro.org/.
To use our solutions in your organization, contact us at gradepro@evidenceprime.com or sales@evidenceprime.com.  

We received a lot of fantastic feedback during the webinar and plenty of questions, not all of which we could answer during the sessions due to the time limit. Therefore, we have compiled a short summary of our Q&A session where we answer all your inquiries and solve all your doubts. 

Q&A Session

General questions

Is the project public if I develop it in GRADEpro?

No, all projects are private. They can only be accessed by users that you invite to join. Thank you for the question!

Do you scientifically review the outputs before publishing them on your site? 
The question was answered LIVE

Of course, we have to remember that at GRADEpro, we are doing our best to understand the GRADE domain and accommodate our software with it, but we are not GRADE domain specialists. Justyna is one of the members of our team who are reviewing the outputs published on our pages. Our team does not have an expert in every domain in which scientists create recommendations. We check whether the crucial aspects are filled in: if the authors' names are provided, which is essential because the users can contact them regarding the data. Justyna verifies whether the quality assessment has been done because publishing an empty table is not something anyone would do. Mostly it means that a mistake was made. 

Those administrative elements are checked. Whether the data is scientifically correct is up to you and up to the scientific community, who can then contact you and point out, for example, some room for improvement. That's how it's done.

‍

Criteria and outcomes

What criteria should we apply while conducting an analysis of the risk of bias?

It depends on the type of the studies. For randomized controlled trials, you can use the Cochrane Risk of Bias tool that Justyna presents during the webinar.

What criteria do you use to select an inclusive and unbiased panel?

It isn't easy to recruit a good panel. However, we believe some research provides guidance on how to do this.

From the GRADEpro perspective, we provide a module to collect the panel members' Conflict of Interest (COI) information. We also have a workflow that enables you to select panel members based on the information they provide in their COI forms.

What are the criteria for choosing the outcomes in GRADE? For example, should it only be consistent with the meta-analysis outcome? What about the adverse events?

Selecting the outcomes is a complex process that is sometimes a subject of long debates and discussions. It has been described in one of the GRADE articles: GRADE guidelines: 2. Framing the question and deciding on important outcomes. GRADEpro provides a dedicated feature allowing for selecting the most important outcome together with a panel of experts. It has been covered in this part of the webinar.

How do you decide on the importance of outcomes and EtDs?

One of the GRADEpro modules allows you to ask the panel to rank the questions and outcomes according to their importance.

If you ask how to assess the importance of the EtD criteria, this is often done in a panel meeting, but we don't have a structured process to facilitate this.

Is it possible to assess several outcomes simultaneously, or do you have to assess each outcome separately?
The question was answered LIVE

Thank you for this question. Every outcome has to be assessed separately. There is no option to assess them together because every outcome is a separate entry to the table. In addition, these outcomes may come from different studies, so the assessment would be different. However, it doesn't take a lot of time, as demonstrated during the presentation - it's just a few clicks. If you are confident that the assessment should be repeated for the options below, it takes just a few seconds to click on it.

What if a specific outcome was assessed by different measures? Do we treat it as one outcome, or do we treat each outcome measure separately?
The question was answered LIVE

Most of the time, you would like to convert these outcomes into one. In general, you need to have a single estimate for one outcome for decision-making. Having multiple outcomes that are different measurements of the same outcome does not help make sense of the data, so you need some process to merge them. Of course, this probably would require a more in-depth answer from our colleagues at GRADE Working Group, who usually deal with these aspects. 

‍

Summary of Findings (SoF) and Evidence to Decision (EtD) frameworks

What's the difference between using GRADEpro to create SoF tables (for SRs) and EtD tables (for CPGs) from the start and updating existing SoF/ EtD previously developed by other organizations?

For updating existing SoFs and EtDs, there is the Adolopment module that is based on the GRADE framework described in "GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and development of trustworthy recommendations: GRADE-ADOLOPMENT" . It allows reusing the existing data in SoFs and adapting the judgments in the EtD.

How should we consider patient values and preferences to make recommendations?

Regarding patient values, we can refer you to the tooltips and detailed judgment tools in the EtD in GRADEpro. For more information, you can check the GRADE Working Group - they provide GRADE workshops which are very helpful from the domain point of view. We can also refer you to our GRADEpro Handbook and the GRADE articles in the Journal of Clinical Epidemiology.

‍

Types of studies and analyses

I have conducted a systematic review to recommend rating scales to be used in studies/trials. Would GRADEpro be a suitable tool for this purpose?

If you conducted a systematic review on the tools/scales to be used in surveys collected as part of trials, you could certainly use GRADEpro to create an evidence table to synthesize the results of this research.

Is there a GRADEpro platform for incorporating evidence from qualitative studies? 
The question was answered LIVE

Thank you for this question. In general, GRADEpro can be used to assess any type of data you wish. However, suppose no option that you see is useful for you. If no option available answers your question, you can always use the "narrative outcome" option to describe your outcome in words. It is quite often used by people who do only qualitative work. That's the most common option for it. If anything else is needed, you can always contact us, and either we will find an existing feature for you or add it to the list of future features to be created because GRADEpro is a work in progress.

Evidence-based medicine is evolving, and so is GRADEpro.

Can GRADEpro be used for systematic reviews of articles?
The question was answered LIVE

Yes, of course. Looking at the numbers, at our nearly 100 000 users, probably most of them use GRADEpro for systematic reviews. It also reflects the pyramid of evidence, where we generally have more systematic reviews than guidelines. GRADEpro can be used for systematic reviews, mainly for creating evidence tables and interactive presentations of it (iSoF).

With the new publication module that we presented during the webinar, you can also create a document or manuscript for the systematic review itself and either export it to Word for future processing or even publish it directly with our platform.

This functionality enables you to create living systematic reviews. By publishing these reviews on a web platform, you can update them at any time, provide timestamps, and provide the full history of all the versions. 

Is GRADE assessment necessary for all systematic reviews or only for guidelines?

Whether or not you decide to take this step is up to you. However, if you want to follow the GRADE domain rules fully, the body of evidence should be assessed for the certainty of evidence. 

Do all Systematic Reviews require GRADEpro? 
The question was answered LIVE

Thank you for this question. No one can force you to use GRADEpro for all systematic reviews. However, this is very helpful, mainly because, for example, some people are using tables created in word documents or other external tools. GRADEpro creates those tables for you - they are ready as you begin. You only need to populate them with data, making the job faster and easier for you and saving time. And as it saves time, it also saves money. 

Do we use GRADEpro only to rate the quality of evidence?
The question was answered LIVE

No, not only. It can also help you organize your work because it provides more elements that we couldn't show during our webinar. For example, checklists of the guideline creation process can guide you through the process after the data is assessed, through the creation of recommendations together with the panel, and then during the publication process. So, after the meta-analysis, you can perform all the steps with GRADEpro. 

Do we use/need GRADEpro in scoping review?
The question was answered LIVE

In general, the methodology for conducting the literature review should not influence the possibility of using GRADEpro. Instead, it depends on the type of outcomes that you extract and measure. In GRADEpro, you can still create narrative outcomes, so even if your scoping review does not produce any quantitative evidence that can be presented as dichotomous or continuous outcomes, you can still do it in GRADEpro. 

How useful is GRADEpro for performing Meta-analysis?
The question was answered LIVE

The meta-analysis is not in the scope of GRADEpro, as we are not providing any statistical methods. Most of the time, researchers use tools like RevMan to do meta-analysis and import the results of these statistical models to GRADEpro.

How to fill in the information on single-arm analysis? In my case, it is an analysis of RCT and NRSI on a specific treatment (survival).
The question was answered LIVE

For single-arm analysis, we have a couple of different study designs within GRADEpro. If you have a single-arm analysis, especially for survival, the survival curve compares this with the standard treatment or something like this. So, this would probably need to be discussed in detail,  either over email or maybe even with our scientific colleagues at GRADE Working Group. Still, in general, people are using GRADEpro for a similar purpose. Please reach out to support@gradepro.org, and we can always try to help you, if not directly, then at least by forwarding your inquiry to our colleagues.

Compared to Delphi panel study applications, how does GRADEpro excel?

GRADEpro's Panel Voice and other panel-engaging features are designed specifically for the purpose of guideline development. The users don't need to create surveys from scratch - the templates and forms are ready for use and only need to be populated with data from the user's research. For example, the Conflict of Interest tool provides different templates utilized by organizations such as WHO. The results of these forms are automatically incorporated in further panel communication in GRADEpro (panel members conflicted for particular questions are automatically excluded from the recipients' list when forms for these questions are being sent). Additionally, results from the forms sent via GRADEpro are automatically summarized and presented in GRADEpro with no need for manual input from the manager.

‍

Other platforms and tools

What are the advantages or disadvantages of using RevMan, then importing findings into GRADEpro?

Most of the time, you need to conduct a meta-analysis before creating an evidence table. Statistical modeling is not in the scope of GRADEpro but can be done in RevMan. You can then import the meta-analysis results either from RevMan 5 or RevMan Web.

Are you planning to integrate GRADEpro with other platforms? Covidence, RevMan and Rob2?

The RevMan Web integration is available. It can be enabled per review by the managing editor. With this feature, you can create GRADEpro tables from within RevMan Web. The statistical data is automatically transferred to GRADEpro, where users only need to assess the quality of evidence. Then the content is also automatically transferred back to RevMan Web, and the table is inserted into the review. The RoB2 tool is also available in GRADEpro, and it has been presented here.

What is the difference between the Cochrane Support Summaries and recommendation panelists?

Judging by the information we found on Cochrane Support Summaries, it is more comparable to the Evidence to Decision framework. It is a defined template that can be used to present evidence in a concise manner, so it is easier to comprehend.

The recommendation panelists or panel members, on the other hand, are a group of experts selected to provide their insight when creating a recommendation based on the evidence collected, analyzed, and assessed in the course of the systematic review. Those people could be specialists in the medical field in which the recommendation would be based, and also nurses, patients, policymakers, and all other parties that would, later on, be affected by the effects of introducing the recommendation.

So while a Cochrane Support Summary is a synthesis of the results from a systematic review tailored for policymakers in low-income countries, the panel members are a group of people who discuss a similar document (EtD) to come up with a recommendation.

‍

GRADEpro app

What is the most current version of GradePro for Windows 11?

GRADEpro for Windows is no longer supported. What Justyna presents during the webinar is a new version that works through a browser.

What is the URL to access GRADEpro on the web?

You can access GRADEpro, create an account and test it for free at https://gdt.gradepro.org/app/.

Who should we contact to determine the price of using GRADEpro for our project?

You can start by visiting our pricing page  to determine which of the features you'd be most interested in, and then you can contact us at sales@evidenceprime.com for a detailed quote.

‍

Video recordings

Will this webinar be recorded and available publicly?

Yes, the webinar is recorded, and the video recording is available and sent to all the participants.

Is it possible to watch the previous webinar "Why GRADE?" referred to during the presentations?

Yes, the previous webinar "Why GRADE? 15 reasons to start using GRADE for your guideline development" is available publicly online. 

You can watch it here.

Is the recording of a quick run-through of GRADEpro available?

Yes, the recording: "How GRADEpro supports you to create guidelines according to the best practices of GRADE?" is available online. 

You can watch it here.

Other comments

How can we collaborate with both speakers to organize a webinar on GRADE evidence for our project?

Thank you for your interest in collaborating with us. Please get in touch with us regarding your project and ideas at webinars@evidenceprime.com.

We invite and encourage everyone interested in presenting your projects during our webinars to do so! 

‍

‍

Quotes and feedback

Excellent presentation! Very useful.
Thank you so much for the presentation and the answer!!
Thank you for such great talks and introduction!

Thank you, everyone, for your active participation!

Useful links and further readings:

For more information about upcoming webinars and events, Subscribe to our Newsletter:

Read more about our speakers: 

* GRADEpro website: https://www.gradepro.org/ 
* GRADEpro app: https://gdt.gradepro.org/app/ 
* GRADEpro Knowledge base: https://kb.gradepro.org/ 
*GRADEpro Panel Voice: https://www.gradepro.org/panelvoice 

Watch our previous webinar: Why GRADE? 15 reasons to start using GRADE in your guideline recommendations.
Watch a quick run-through of GRADEpro: How GRADEpro support you in creating guidelines according to GRADE's best practices?

Follow us on our social media for more updates:

Evidence Prime:

Laser AI:

‍

‍

GRADEpro:

‍

‍

‍

‍

Recording

Available upon request. Please get in touch with us at webinars@evidenceprime.com.

If you have any more questions, comments, or suggestions for the following topics that would be of your interest, please do not hesitate to contact us at webinars@evidenceprime.com.

Evidence Prime team

‍