Metrics

Topic: Metrics

Time to shake up site monitoring metrics? Join the MCC Centralized & Site Monitoring Process Metrics Group Sept 16th

Friday, September 10, 2021

Are you fed up with the standard metrics for site monitoring – are Monitoring Visit Reports completed on time? Are follow-up letters issued on time? The MCC Centralized & Site Monitoring Process Metric Development Work Group has developed process maps that depict the new monitoring processes that bring together centralized, remote and onsite monitoring processes. How well do traditional monitoring metrics align with the new processes? We’ll be thinking afresh on this process – what are the Key Performance Questions we want our metrics to help us answer? Can we design new metrics that will provide additional actionable insights?

More information about this metric development work group can be found by visiting the MCC Centralized & Site Monitoring Metrics Work Group page. MCC members can register for the meeting by logging in to the member portal.

NEW!! MCC Site Activation Milestone Metrics Toolkit v 1.0 Now Available in the Member Portal

Friday, August 20, 2021

The MCC Site Selection & Start-Up Process Metrics Work Group – comprised of more than 250 individuals representing over 60 organizations – worked collaboratively from October 2019 to May 2021 to develop the assets contained in the Site Activation Milestone Metrics Toolkit v1.0. MCC had received feedback that the existing MCC-defined metrics in the Clinical Trial Performance Metrics v1.3 needed review and update. Processes have continued to develop since the existing metrics were defined. In addition, MCC revised the approach to defining metrics – providing much more context for the implementation and use of each metric, and defining metrics at different levels, as well as using the MCC Metrics Development Framework to help ensure metrics have a clear, defined purpose. The MCC Site Selection & Start-Up Process Metrics Work Group was formed to review the existing metrics and define new ones, as needed. The Work Group aimed to develop standardized metrics to:

  • Facilitate comparison / benchmarking across industry
  • Monitor processes and identify process improvement opportunities
  • Provide objective evidence of impact of process improvement efforts
  • Ensure the use of a balanced set of metrics (i.e. measure more than just “time”)

Additional metrics toolkits will be developed and published in 2022 as the work group completes its review of the site selection and start-up process.

More about this metric toolkit and the metric development work group can be found by visiting the MCC Site Selection & Start-Up Metrics Work Group page. MCC members can access the toolkit by logging in to the member portal.

Not a member? Contact Terry Holland, MCC Membership Director, to discuss membership options.

MCC Metrics Available in a Metric Master File

Friday, May 21, 2021

Is your organization interested in viewing all of the MCC metrics in a single file?  The MCC Metric Master file is now available for members to view in the MCC member portal. This file contains the detailed definitions of more than 300 portfolio/program, study, country and site level performance metrics. The metric definitions include data fields that allow you to filter the metric listing in various ways, including:

  • reporting level (portfolio, study, etc.)
  • metric level (basic, advanced, aspirational)
  • metric type (time, cost, quality)
  • study process phase (start-up, conduct, closeout)

MCC encourages members to review metrics in their functional/process metric toolkits so you don’t miss the opportunity to view the toolkit overview materials and take advantage of the metric selection tools.

Members can download the file by logging into the member portal and visiting the Existing Metric Set library.

Not a member? Contact Terry Holland, MCC Membership Director, to discuss membership options.

CTO Podcast: Optimizing Clinical Trial Monitoring with Keith Dorricott and Linda Sullivan

Tuesday, April 27, 2021

Clinical Trial Optimization Podcast

EPISODE 19

Optimizing Clinical Trial Monitoring with Keith Dorricott and Linda Sullivan

MCC launched its groundbreaking monitoring process metrics working group last year with the purpose of developing new metrics that take into account changes in the clinical monitoring process, including risk-based assessment and centralized monitoring integration. That’s the important topic discussed in Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). The collaborative working group started with the challenging process of developing a process map that depicts the new monitoring approach that integrates centralized monitoring with remote and onsite monitoring, Dorricott notes. The group has been utilizing the MCC metric development framework described in a previous podcast to identify critical success factors, key performance questions, and ultimately metrics that organizations can utilize to oversee and management new monitoring models. The group anticipates the first installment of metrics to be published by the end of May. Sullivan notes that MCC has knowledge portals, including those on monitoring, that members can utilize to optimize their clinical trials. For more on this topic, check out the Centralized & Site Monitoring Process Metrics page.

Subscribe today!

Apple Podcasts

Spotify

Stitcher

Android

CTO Podcast: How Organizations Can Take Advantage of MCC’s Risk Assessment and Mitigation Management Tool To Optimize Clinical Trials with Keith Dorricott and Linda Sullivan

Wednesday, April 21, 2021

Clinical Trial Optimization Podcast

EPISODE 18

How Organizations Can Take Advantage of MCC’s Risk Assessment and Mitigation Management Tool To Optimize Clinical Trials with Keith Dorricott and Linda Sullivan

MCC has developed a valuable risk assessment and mitigation management tool that organizations can use as the basis for implementing the risk-based quality management process outlined in ICH E6 (R2), section 5.0. That’s the important topic discussed in Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Dorricott facilitates MCC’s QbD-RBQM-Quality Improvement Community of Practice Group. Dorricott describes how MCC members have worked collaboratively over the last five years to develop a Risk Assessment and Mitigation Management Tool (RAMMT) that moves beyond the RACT tool that was developed by another industry group. The initial drive was from members wanting MCC metrics to be aligned with individual risks to assist with risk scoring. Sullivan points out that when ICH E6 (R2) was released, the group conducted a gap analysis to determine whether the tool fully supported the approach outlined in the guidance. The tool was updated to support a new approach that includes identifying critical processes and data and utilizing MCC process maps and key risk indicator metrics already developed by MCC metric development groups. Sullivan and Dorricott note that it’s important for risk indicators to detect risk and emerging issues as early as possible to reduce, and maybe eliminate, the negative impacts. However, Sullivan and Dorricott add, when you’re scoring your risks, you should score the ‘as-is’ situation and document any additional mitigations such as improved detection, or additional training and guidance. Otherwise, the mitigations may not actually be implemented and the risk will be higher than anticipated. Indeed, Sullivan says, documentation is key to risk assessments now being required by regulators. This topic is covered in the white paper, MCC Risk Assessment & Mitigation Management Tool 2.0: Risk-Based Quality Management Aligned with ICH E6 (R2).

Subscribe today!

Apple Podcasts

Spotify

Stitcher

Android

eCOA Process Metrics Work Group Expanding Coverage of the Metric Toolkit

Wednesday, April 14, 2021

The utilization of eCOA/ePRO services has accelerated in recent years, fueled in part by the COVID pandemic. In late 2020, the MCC eCOA Process Metrics Work Group conducted a review of the eCOA metric set v2.0 (released in October 2018) and identified several areas that needed exploration to address gaps. The first area of focus considered was the translations process. They started by developing a high-level process map, which required multiple iterations before finalization. Work Group members found the process map very useful for establishing the Key Performance Questions and metrics needed to address the gap.

At the April 2021 meeting, they will move on to discuss another key part of the process – training. Currently, the metric toolkit includes a metric measuring staff training completion. However, the group plans to cover a full range of training, including site staff and study participants (for ePRO). Additionally, they will explore how to measure whether the training is effective. In subsequent meetings, they will discuss whether additional study conduct metrics related to data quality should be added, too.

Additional information about this MCC member group can be found by visiting the eCOA Process Metrics work group page. MCC Members can log onto the MCC member portal to register for the session.

Not a member? Contact Terry Holland, MCC Membership Director, to discuss membership options.

MCC’s Metric Development Framework Defines Metrics That Answer Key Performance Questions

Monday, April 12, 2021

Successful performance metrics focus on what drives success across the enterprise. They help answer important questions. They have a purpose. Don’t measure things simply because you can, or that aren’t a concern for your organization. That may sound obvious, but many organizations measure things that simply don’t matter. Just because something is a problem for a competitor, or you heard it mentioned at a conference, doesn’t mean that the associated metric is meaningful to your trial or for your processes. You face challenges unique to the types of trials you’re running and the processes you have adopted. You cannot afford to waste resources on collecting, reporting and analyzing metrics that don’t answer your questions.

Learn more – podcast and download the paper.

New MCC Vendor Oversight Community of Practice Group Launching on 07 April 2021

Thursday, March 18, 2021

MCC’s Vendor Oversight Metric Development work group will transition to a Community of Practice group in April 2021. MCC vendor oversight metric sets have been developed and members are ready to form a Community of Practice group. Community of Practice groups are a virtual community of MCC members that share a common interest or concern who meet on a regular basis to share ideas and practical approaches to address challenges. Meeting topics are determined by members of the Vendor Oversight Community of Practice.

The initial meeting will feature Linda Sullivan, Co-Founder and Executive Director, MCC and Keith Dorricott, MCC Ambassador leading a discussion on the use of metrics in vendor oversight. Topics to be discussed include:

  • The impact of metrics on behavior
  • Vendor oversight vs the process perspective
  • What metrics do and do not tell you
  • Advantages and challenges of standardizing metrics

The MCC Vendor Oversight Metric Development work group – comprised of more than 200 individuals from 56 companies – developed four metric toolkits between April 2017 – January 2020 which include detailed metric definitions, implementation guides, metric selection tools and demo videos in addition to an overview video.

  • MCC Vendor Oversight Finance Metric Toolkit 1.0 – a metric set focused on the financial oversight of CROs. Includes rate of incurring costs, accuracy of budget, cycle times for contracts and change orders, and investigator site payment timeliness.
  • MCC Vendor Oversight Quality Metric Toolkit 1.0 – a metric set focused on the quality oversight of CROs / vendors. Includes staff turnover, training, issue escalation, action completion, and includes metrics covering sub-contractor oversight.
  • MCC Vendor Oversight Timeliness Metric Toolkit 1.0 – a metric set focused on the timeliness of meeting milestones and SUSAR reporting across studies.
  • MCC Vendor Oversight Relationship Assessment Metric Toolkit 1.0 – a metric set for use when analyzing data from a relationship survey between sponsor and CRO or CRO and vendor. The toolkit includes detailed guidance on setting up, implementing and analyzing data from a survey.

The toolkits are available for members to download in the MCC member portal.

The session is open to all MCC Members – contact MCC for registration details.

For more information about the Community of Practice Group, please visit MCC website.

Not a member? Contact Terry Holland, MCC Membership Director, to discuss membership options.

Updated Cardiopulmonary Process Metrics Toolkit Now Available to Members

Thursday, May 21, 2020

The MCC Cardiopulmonary Process Metrics Toolkit v2.2 (ABPM, ECG, and Spirometry metrics) is now available for MCC member organizations to download from the portal.

Thank you to all who provided input and feedback as we carried out this update. The toolkit includes “core” metrics that are applicable to the set-up, conduct and oversight of ambulatory blood pressure monitoring (ABPM), electrocardiogram (ECG) and spirometry services in clinical trials. Additionally, the toolkit included ABPM-, ECG- and spirometry-specific metrics. The update includes the following modifications:

  • Consolidation of all metrics into one metric workbook – comprising the core metrics, and ABPM-, ECG- and Spirometry-specific metrics.
  • Metric “Core 3” has been split into two metrics – one measuring the proportion of sites where equipment was shipped on time and another measuring the proportion of sites where equipment is reported as having an error. In the original metric, the measurement was for the proportion of sites where equipment was received on time and members agreed that this was very challenging data to collect.
  • Performance targets updates for the ECG metrics 1a, 1b and 1c, and the example formula corrected for 1b and 1c.
  • The glossary has been added to the metric workbook – similar to other MCC metric sets.
  • Metric selection tools and implementation support tools have been developed

The changes were reviewed and accepted by MCC members in March 2020. Please visit the MCC Cardiopulmonary Process Metrics Work Group page for additional information.

Not a member? Contact Terry Holland, MCC Membership Director, to discuss membership options.

MCC Imaging Process Metrics Toolkit v3.0 Now Available

Thursday, May 14, 2020

The MCC Imaging Process Metrics Toolkit v3.0 is now available for MCC member organizations to download from the portal. Thank you to all who provided input and feedback as we carried out this extensive update. The updated metric toolkit includes the following:

  • Updated Metric Definitions
    • Portfolio / Study / Site level metrics
    • Basic / Advanced metric designations
    • Potential Key Risk Indicators
  • New toolkit overview, implementation guide and introductory webinar (13 mins).
  • Updated process map
  • Additional components to assist with your implementation and use of the metrics:
    • Four metric selection tools which help you select the right metrics for your situation and to list the associated data elements needed
    • A metrics reporting planning worksheet to help plan the reporting of the metrics you select

Please visit the MCC Imaging Process Metrics Work Group page for additional information.

Not a member? Contact Terry Holland, MCC Membership Director, to discuss membership options.