Summary

DSEN MAGIC has been incredibly productive with 61 queries that are completed/ ongoing and 79 associated methods projects completed that were not funded by DSEN. DSEN MAGIC has the capacity and expertise to conduct multiple types of knowledge synthesis, from rapid reviews to living reviews for DSEN KUs.

DSEN MAGIC has several concerns regarding the recent DSEN Evaluation, such as the comparator selected, calculation of timing, inclusion of feasibility assessments, and dual role of one of the DSEN organizations.

DSEN MAGIC is available and interested to collaborate with the development of DSEN 2.0 and agree that the addition of a recognized scientific leader is needed for DSEN, a different coordination process is required, and that researchers and KUs need to make decisions jointly regarding the future of DSEN.

DSEN MAGIC’s Progress

• DSEN MAGIC has been working closely with DSEN knowledge users
  (KUs) since 2011 using an integrated knowledge translation approach on 61 (48%) of all DSEN queries.
• DSEN MAGIC received 21 queries in the past year alone.
• DSEN MAGIC successfully completed 8 rapid reviews in the 2019-2020 year
  with 2 to 3 week timelines for DSEN KUs.
• DSEN MAGIC completed 79 methods projects to advance the science of knowledge synthesis, none of which were funded by the CIHR DSEN grant.
• DSEN MAGIC has capacity and expertise to complete rapid reviews or living reviews for DSEN KUs needs, addressing concerns raised in the evaluation.

DSEN MAGIC’s Concerns with DSEN Evaluation

• Since the DSEN researchers were not provided an opportunity to respond to the evaluation, we felt it was important to share the DSEN MAGIC’s perspective.
• The comparator used in the evaluation for the MAGIC team was the ODPRN. A more suitable comparator is NICE or AHRQ. ODPRN reviews were noted as being much quicker than DSEN, however, these were led by MAGIC PIs.
• It is unclear how the timing for project completion was calculated for the evaluation. An alternative way would be to calculate the time from PMAP approval to when preliminary results were provided. Some projects take several months for the PMAP to be fully approved by the KUs.
• Time spent on feasibility assessments and/or scoping the query were not included in the evaluation and this should also be accounted for, especially for queries that did not proceed.
• The dual role of CADTH was not contextualized in the evaluation. CADTH received funding from DSEN in the past, CADTH sits on the DSEN Advisory Board, and 2 queries that came through DSEN were completed by CADTH in the 2019-2020 year. This raises concerns over potential conflicts of interest.

DSEN 2.0 Model

• DSEN MAGIC agrees that a new, visible and collaborative scientific lead is urgently needed for DSEN.
• DSEN MAGIC agrees that a different coordination process is required for DSEN and recommends the Strategic Patient-Oriented Research Evidence Alliance approach to query coordination (i.e., queries go through a central office and a meeting is arranged with KUs within 24 hours to discuss the query scope and timelines).
• DSEN MAGIC agrees that DSEN researchers and KUs need to make decisions jointly about the future of DSEN.

Summary

DSEN MAGIC has been incredibly productive with 61 queries that are completed/ongoing and 79 associated methods projects completed that were not funded by DSEN. DSEN MAGIC has the capacity, nimbleness, and internationally-acknowledged methodological expertise to conduct multiple types of knowledge synthesis, from rapid reviews to living reviews for DSEN KUs.

DSEN MAGIC attended the DSEN Evaluation Consultation meeting on February 10th and February 11th 2021. DSEN MAGIC is well- positioned and enthusiastic to collaborate with the development of DSEN 2.0. This 1-pager outlines DSEN MAGIC’s achievements, along with our collective feedback to build stronger organizational relationships.

DSEN MAGIC’s Progress

• DSEN MAGIC has been working closely with DSEN knowledge users (KUs) since 2011 using an integrated knowledge translation approach on 61 (48%) of all DSEN queries.
• DSEN MAGIC successfully completed 8 rapid reviews in the 2019-2020 year with 2 to 3 week timelines for DSEN KUs.
• DSEN MAGIC completed 79 methods projects to advance the science of knowledge synthesis, none of which were funded by the CIHR DSEN grant. The dissemination of these methods results ensure that leading edge and innovative methods were made available to the DSEN KUs.
• DSEN MAGIC has the capacity, nimbleness, and internationally- acknowledged methodological expertise to complete rapid reviews or living reviews for DSEN KUs needs, addressing concerns raised in the evaluation.

Feedback on the DSEN 2.0 Model

• Host organization: The host organization needs to be free of COI from industry, including accepting funding from industry. The host organization must have a strong understanding of complex methods required to conduct all types of KS, including NMA. The host organization must be at arms length; they must not be responding to queries themselves or implementing the queries, as this would be a COI.
• Scientific leader: The scientific leader must be free from COI from industry. The scientific leader must have a researcher background with experience in regulatory leadership. The scientific leader must understand the science of KS. The scientific leader should not be a pre- existing DSEN PI.
• Funding mechanism: Must have a stable funding mechanism to ensure the long-term sustainability and retention of high-calibre scientists and staff. This will also allow the teams to divert staff (for a short intensive period) to address the need for queries that are urgent in nature, as was the case for the MAGIC team during COVID. If more overhead is being charged for contracts, there may be decreased amount of funds going to queries. There is also a substantial administrative burden on institutes for executing contracts, which may increase timelines. Institutes do not value contracts in the same way as grants, as grants add to the “prestige” of the institute. As well, institutes place less value on evaluating researchers’ career progression when they are attracting contracts versus grants.