GRADEpro’s inside scoop at the GRADE Working Group Meeting

Our Product Manager, Justyna, on 24-25 May 2023, attended the GRADE Working Group meeting at the Medical Faculty in Split, Croatia. The event was co-organised by Tina Poklepović-Peričić, the co-director of Cochrane Croatia, and her team.

It was a great pleasure to meet everyone who could attend in person and receive face-to-face updates on the latest developments, achievements, and plans of the GRADE Working Group.

As always, the event consisted of two days of discussions on the future GRADE ideas, often in the lecture hall with all attendees present or also in smaller rooms later on, when a few ideas were discussed in parallel.

Thomas Piggott presented a paper on the GRADE for Essential Medicines, which showed guidance on applying GRADE quality and certainty assurance in the process of preparing International and National Lists of Essential Medicines. An example of such a list is the WHO Model Lists of Essential Medicines. Evidence Prime helped create an electronic version of this list. We’re glad to hear that the GRADE Working Group is taking steps to improve the process of managing such lists in the future.

The following presentation, by Beth Radke, focused on the Risk of Bias assessment across studies of exposure, which is a part of the GRADE Environmental Health project group.

Miloslav Klugar presented a suggested version of the GRADE Guidance paper focused on the Adolopment process of contextualising evidence. Miloslav was presenting on behalf of the GRADE Adaptation project group, led by himself and Tamara Lotfi. We are very interested in further progress in that field, as GRADEpro offers an adolopment module which has been used extensively, e.g. during the contextualisation part of the eCovid19 project, allowing the RecMap users (including the Czech team led by Miloslav) to adapt or adopt the recommendations available on the portal.

The following agenda item was a discussion on GRADE structured vocabulary based on a presentation delivered by Paul Whaley. The idea of unifying terminology used in the GRADE domain is a compelling yet essential step in developing this field, and we are excited to see the results of this project.

Wojtek Wiercioch and Gian Paolo Morgano presented their GRADE Guidance paper on the Decision and judgment thresholds. Their mathematical model, derived from extensive research done in the scientific community, will help future guideline panels assess the magnitude of the effects within the Evidence to Decision framework (EtD). We are working with Wojtek and Gian Paolo on applying the thresholds in GRADEpro.

The final presentation of the first day was delivered by Jessica Beltran on the EtD (evidence to decision) for Multiple-Intervention Comparisons. We are proud to have already collaborated with this project and created GRADEpro tools supporting it. You can learn more about it through one of our monthly webinars, or read more here.

If you are actively doing guidelines in GRADEpro and would benefit from the multiple intervention EtD framework, please get in touch with us at support@gradepro.org.

After this presentation, the attendees participated in separate presentations in smaller groups, which regarded topics such as Publication bias in quantitative research, GRADE for rare diseases, GRADE for Planetary Health and GRADE Reporting. As our representative, Justyna attended the GRADE Economic Evaluation project group presentation and discussion guidance for rating inconsistency in Economic Evaluation led by Maria Ximenas Rojas. The discussion was a part of the project on bringing certainty assessment to the economy-related criteria of the EtD.

The second day was just as riveting. We have been following various meetings and presentations.

In the morning, Farid Foroutan presented ideas on GRADE Prognosis Concept on Prognostic Model Discrimination, with solutions ranging from clinicians’ intuition to machine learning. The following presentation, delivered by Emily Senerth, focused on the GRADE Guidance on Evidence to Decision Framework for Environmental and Occupational Health. The EtD framework described in the paper is currently available for all GRADEpro users.

Qiukui Hao presented another paper on the GRADE concept for construction for indirect comparison, which is a crucial issue concerning the vital matter of the Network Meta-Analysis. Next, Samer Karam and Holger Schünemann led a discussion on the important topics of Certainty of Evidence for outcome values and the values’ variability. The double presentation was met with an enlightening discussion and many ideas from members of the group.

Antonio Bognanni gave a presentation that was an introduction to an emerging project to explore the idea of intermediate levels of certainty rating. With the intermediate certainty levels being a vital issue to address, we will follow this project with great interest.

The final presentations of the second day were delivered by Linan Zeng on GRADE for rating certainty when the target threshold is null, or the point estimate is close to null, and Hassan Murad presented remotely on rating dose response gradients.

Some of the following subgroup discussions referred to presentations from the previous day, such as Paul Whaley’s workshop on the structured GRADE vocabulary or a debate led by Miloslav and Tamara regarding GRADE adolopment. Other presentations included GRADE for tests and diagnosis (when there is no or an inappropriate reference standard) by Holger Schünemann and Reem Mustafa and GRADE training and credentialing delivered by Philipp Dahm. Miranda Langendam and Elena Parmelli led the workshop that Justyna attended on the Quality Indicators (a tool for measuring the effectiveness of guideline implementation), which Evidence Prime has been working on together with the European Commission Initiative on Colorectal Cancer team.

‍

All in all, the meeting brought a lot of new concepts and substantial development in the existing projects of the GRADE domain. The Adriatic Sea and Croatian sun proved to be an excellent environment for a strong flow of ideas and heated discussions. We are proud to be a part of such a productive community and will continue supporting the efforts of the GRADE Working Group with our tools and features. We cannot wait to see the fruits of the following meetings, which will happen in September in conjunction with the London Cochrane Colloquium and the Guidelines International Network Conference in Glasgow, where we will be hosting a workshop related to the theme of Automation and Technology to support guidelines development. We will dive more in-depth into the topic - RecChat: Testing an AI-Based Conversational Search Engine for the eCOVID-19 RecMap.

We look forward to seeing you there again!