Metrics

Topic: Metrics

MCC Metrics Available in a Metric Master File

Friday, May 21, 2021

Is your organization interested in viewing all of the MCC metrics in a single file?  The MCC Metric Master file is now available for members to view in the MCC member portal. This file contains the detailed definitions of more than 300 portfolio/program, study, country and site level performance metrics. The metric definitions include data fields that allow you to filter the metric listing in various ways, including:

  • reporting level (portfolio, study, etc.)
  • metric level (basic, advanced, aspirational)
  • metric type (time, cost, quality)
  • study process phase (start-up, conduct, closeout)

MCC encourages members to review metrics in their functional/process metric toolkits so you don’t miss the opportunity to view the toolkit overview materials and take advantage of the metric selection tools.

Members can download the file by logging into the member portal and visiting the Existing Metric Set library.

Not a member? Contact Terry Holland, MCC Membership Director, to discuss membership options.

CTO Podcast: Optimizing Clinical Trial Monitoring with Keith Dorricott and Linda Sullivan

Tuesday, April 27, 2021

Clinical Trial Optimization Podcast

EPISODE 19

Optimizing Clinical Trial Monitoring with Keith Dorricott and Linda Sullivan

MCC launched its groundbreaking monitoring process metrics working group last year with the purpose of developing new metrics that take into account changes in the clinical monitoring process, including risk-based assessment and centralized monitoring integration. That’s the important topic discussed in Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). The collaborative working group started with the challenging process of developing a process map that depicts the new monitoring approach that integrates centralized monitoring with remote and onsite monitoring, Dorricott notes. The group has been utilizing the MCC metric development framework described in a previous podcast to identify critical success factors, key performance questions, and ultimately metrics that organizations can utilize to oversee and management new monitoring models. The group anticipates the first installment of metrics to be published by the end of May. Sullivan notes that MCC has knowledge portals, including those on monitoring, that members can utilize to optimize their clinical trials. For more on this topic, check out the Centralized & Site Monitoring Process Metrics page.

Subscribe today!

Apple Podcasts

Spotify

Stitcher

Android

CTO Podcast: How Organizations Can Take Advantage of MCC’s Risk Assessment and Mitigation Management Tool To Optimize Clinical Trials with Keith Dorricott and Linda Sullivan

Wednesday, April 21, 2021

Clinical Trial Optimization Podcast

EPISODE 18

How Organizations Can Take Advantage of MCC’s Risk Assessment and Mitigation Management Tool To Optimize Clinical Trials with Keith Dorricott and Linda Sullivan

MCC has developed a valuable risk assessment and mitigation management tool that organizations can use as the basis for implementing the risk-based quality management process outlined in ICH E6 (R2), section 5.0. That’s the important topic discussed in Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Dorricott facilitates MCC’s QbD-RBQM-Quality Improvement Community of Practice Group. Dorricott describes how MCC members have worked collaboratively over the last five years to develop a Risk Assessment and Mitigation Management Tool (RAMMT) that moves beyond the RACT tool that was developed by another industry group. The initial drive was from members wanting MCC metrics to be aligned with individual risks to assist with risk scoring. Sullivan points out that when ICH E6 (R2) was released, the group conducted a gap analysis to determine whether the tool fully supported the approach outlined in the guidance. The tool was updated to support a new approach that includes identifying critical processes and data and utilizing MCC process maps and key risk indicator metrics already developed by MCC metric development groups. Sullivan and Dorricott note that it’s important for risk indicators to detect risk and emerging issues as early as possible to reduce, and maybe eliminate, the negative impacts. However, Sullivan and Dorricott add, when you’re scoring your risks, you should score the ‘as-is’ situation and document any additional mitigations such as improved detection, or additional training and guidance. Otherwise, the mitigations may not actually be implemented and the risk will be higher than anticipated. Indeed, Sullivan says, documentation is key to risk assessments now being required by regulators. This topic is covered in the white paper, MCC Risk Assessment & Mitigation Management Tool 2.0: Risk-Based Quality Management Aligned with ICH E6 (R2).

Subscribe today!

Apple Podcasts

Spotify

Stitcher

Android

eCOA Process Metrics Work Group Expanding Coverage of the Metric Toolkit

Wednesday, April 14, 2021

The utilization of eCOA/ePRO services has accelerated in recent years, fueled in part by the COVID pandemic. In late 2020, the MCC eCOA Process Metrics Work Group conducted a review of the eCOA metric set v2.0 (released in October 2018) and identified several areas that needed exploration to address gaps. The first area of focus considered was the translations process. They started by developing a high-level process map, which required multiple iterations before finalization. Work Group members found the process map very useful for establishing the Key Performance Questions and metrics needed to address the gap.

At the April 2021 meeting, they will move on to discuss another key part of the process – training. Currently, the metric toolkit includes a metric measuring staff training completion. However, the group plans to cover a full range of training, including site staff and study participants (for ePRO). Additionally, they will explore how to measure whether the training is effective. In subsequent meetings, they will discuss whether additional study conduct metrics related to data quality should be added, too.

Additional information about this MCC member group can be found by visiting the eCOA Process Metrics work group page. MCC Members can log onto the MCC member portal to register for the session.

Not a member? Contact Terry Holland, MCC Membership Director, to discuss membership options.

MCC’s Metric Development Framework Defines Metrics That Answer Key Performance Questions

Monday, April 12, 2021

Successful performance metrics focus on what drives success across the enterprise. They help answer important questions. They have a purpose. Don’t measure things simply because you can, or that aren’t a concern for your organization. That may sound obvious, but many organizations measure things that simply don’t matter. Just because something is a problem for a competitor, or you heard it mentioned at a conference, doesn’t mean that the associated metric is meaningful to your trial or for your processes. You face challenges unique to the types of trials you’re running and the processes you have adopted. You cannot afford to waste resources on collecting, reporting and analyzing metrics that don’t answer your questions.

Learn more – podcast and download the paper.

New MCC Vendor Oversight Community of Practice Group Launching on 07 April 2021

Thursday, March 18, 2021

MCC’s Vendor Oversight Metric Development work group will transition to a Community of Practice group in April 2021. MCC vendor oversight metric sets have been developed and members are ready to form a Community of Practice group. Community of Practice groups are a virtual community of MCC members that share a common interest or concern who meet on a regular basis to share ideas and practical approaches to address challenges. Meeting topics are determined by members of the Vendor Oversight Community of Practice.

The initial meeting will feature Linda Sullivan, Co-Founder and Executive Director, MCC and Keith Dorricott, MCC Ambassador leading a discussion on the use of metrics in vendor oversight. Topics to be discussed include:

  • The impact of metrics on behavior
  • Vendor oversight vs the process perspective
  • What metrics do and do not tell you
  • Advantages and challenges of standardizing metrics

The MCC Vendor Oversight Metric Development work group – comprised of more than 200 individuals from 56 companies – developed four metric toolkits between April 2017 – January 2020 which include detailed metric definitions, implementation guides, metric selection tools and demo videos in addition to an overview video.

  • MCC Vendor Oversight Finance Metric Toolkit 1.0 – a metric set focused on the financial oversight of CROs. Includes rate of incurring costs, accuracy of budget, cycle times for contracts and change orders, and investigator site payment timeliness.
  • MCC Vendor Oversight Quality Metric Toolkit 1.0 – a metric set focused on the quality oversight of CROs / vendors. Includes staff turnover, training, issue escalation, action completion, and includes metrics covering sub-contractor oversight.
  • MCC Vendor Oversight Timeliness Metric Toolkit 1.0 – a metric set focused on the timeliness of meeting milestones and SUSAR reporting across studies.
  • MCC Vendor Oversight Relationship Assessment Metric Toolkit 1.0 – a metric set for use when analyzing data from a relationship survey between sponsor and CRO or CRO and vendor. The toolkit includes detailed guidance on setting up, implementing and analyzing data from a survey.

The toolkits are available for members to download in the MCC member portal.

The session is open to all MCC Members – contact MCC for registration details.

For more information about the Community of Practice Group, please visit MCC website.

Not a member? Contact Terry Holland, MCC Membership Director, to discuss membership options.

Updated Cardiopulmonary Process Metrics Toolkit Now Available to Members

Thursday, May 21, 2020

The MCC Cardiopulmonary Process Metrics Toolkit v2.2 (ABPM, ECG, and Spirometry metrics) is now available for MCC member organizations to download from the portal.

Thank you to all who provided input and feedback as we carried out this update. The toolkit includes “core” metrics that are applicable to the set-up, conduct and oversight of ambulatory blood pressure monitoring (ABPM), electrocardiogram (ECG) and spirometry services in clinical trials. Additionally, the toolkit included ABPM-, ECG- and spirometry-specific metrics. The update includes the following modifications:

  • Consolidation of all metrics into one metric workbook – comprising the core metrics, and ABPM-, ECG- and Spirometry-specific metrics.
  • Metric “Core 3” has been split into two metrics – one measuring the proportion of sites where equipment was shipped on time and another measuring the proportion of sites where equipment is reported as having an error. In the original metric, the measurement was for the proportion of sites where equipment was received on time and members agreed that this was very challenging data to collect.
  • Performance targets updates for the ECG metrics 1a, 1b and 1c, and the example formula corrected for 1b and 1c.
  • The glossary has been added to the metric workbook – similar to other MCC metric sets.
  • Metric selection tools and implementation support tools have been developed

The changes were reviewed and accepted by MCC members in March 2020. Please visit the MCC Cardiopulmonary Process Metrics Work Group page for additional information.

Not a member? Contact Terry Holland, MCC Membership Director, to discuss membership options.

MCC Imaging Process Metrics Toolkit v3.0 Now Available

Thursday, May 14, 2020

The MCC Imaging Process Metrics Toolkit v3.0 is now available for MCC member organizations to download from the portal. Thank you to all who provided input and feedback as we carried out this extensive update. The updated metric toolkit includes the following:

  • Updated Metric Definitions
    • Portfolio / Study / Site level metrics
    • Basic / Advanced metric designations
    • Potential Key Risk Indicators
  • New toolkit overview, implementation guide and introductory webinar (13 mins).
  • Updated process map
  • Additional components to assist with your implementation and use of the metrics:
    • Four metric selection tools which help you select the right metrics for your situation and to list the associated data elements needed
    • A metrics reporting planning worksheet to help plan the reporting of the metrics you select

Please visit the MCC Imaging Process Metrics Work Group page for additional information.

Not a member? Contact Terry Holland, MCC Membership Director, to discuss membership options.

MCC Releases the Vendor Oversight Finance Metric Toolkit 1.0

Friday, February 14, 2020

The Vendor Oversight Metric Development Work Group has launched its Finance Metrics Toolkit 1.0 that includes robust metric definitions for sixteen metrics and additional components to assist with metric selection, implementation and use.

The finance metrics are defined at four different levels; Portfolio, Study, Country and Investigative Site, and have clarity on purpose, definition, target and use.

The components of the toolkit are:

  • Metric Toolkit Overview and Implementation Guide
  • Metric Toolkit Overview (recorded webinar)
  • Metric Workbook

Metric Selection & Implementation Support Tools include:

  • Key Performance Questions to metrics
  • Issues to metrics
  • Basic/Advanced metrics to data elements
  • Data elements to metrics
  • Metrics reporting planning worksheet

MCC members can download the Finance Metrics Toolkit 1.0 by logging into the member portal and visiting the MCC Metric Sets and Tools page.

This release concludes the work of the Vendor Oversight Metric Development Work Group. The many accomplishments of the work group include:

  • Four metric toolkits
  • Detailed definitions of 59 metrics – developed using the MCC Metrics Development Framework
  • 28 glossary terms defined
  • Detailed guidance on relationship surveys published
  • Agreement on different approaches to aggregation of data across studies
  • 225 individuals registered to meetings representing 56 companies

The work group now moves into the metric implementation phase and no further metrics will be developed during this phase. Beginning in June, there will be quarterly Implementation Q&A meetings for Vendor oversight.

The first Implementation Q&A will be June 17, 2020 at 11AM EDT. Participants will review the metrics across all four sets for vendor oversight – timeliness, quality, finance, and relationship and have an opportunity to raise questions, discuss challenges, make suggestions on implementation. MCC members can register for this meeting by logging into the member portal, clicking the Members & Work Groups tab and scrolling down to Vendor Oversight under the Metrics Development Tab.

For additional information about the MCC Vendor Oversight Metric Toolkit v1.0 download the white paper.

Learn more about the MCC Vendor Oversight Work Group.

Not a member? Contact Terry Holland, MCC Membership Director, to discuss membership options.

Site Selection and Start-Up Work Group Launches with Public Meeting

Wednesday, October 30, 2019

Development of site start-up metrics and results from a public survey were the main topics during a public meeting that relaunched the MCC Site Selection and Start-Up Work Group. The well-attended meeting had 97 participants representing 35 organizations.

Citing data from the Tufts Center for the Study of Drug Development, Linda Sullivan, MCC Executive Director and Co-Founder, discussed the need for a focus on site start-up metrics. Attendees then reviewed the results from a public survey that sought input on high level processes, current process issues, and current metrics utilized to oversee and manage the processes.

Highlights from the survey included:

  • Suggestions for changes to the high level process map which have been added to the “working version” of the high-level process map
  • Significant issues in the process that were most selected are related to the time taken for site contracting and site selection
  • Most organizations measure high level cycle times; from site selection to activation and from draft contract sent to contract final
  • Most organizations measure the proportion of sites either not enrolling or enrolling only one patient

After plotting the metrics on the high level process map, the discussion moved to how the interactions between regulatory, ethics and site contracting vary by country and site and should provide context to metric values. The MCC Metrics Development Model was described with reference to the Site Contracting Metrics 1.1, which are available for those in member organizations to download.

The next Site Selection and Start-Up Work Group meeting will be on November 5, 2019 and all future meetings will take place on the first Tuesday of the month. These meetings are only open to MCC member organizations. The work group will be using the MCC Metrics Development Framework to develop a comprehensive metric toolkit that includes basic and advanced standardized metrics that answer key performance questions related to this process.

The MCC encourages members to join the work group! The meetings are a pre-collaborative space for you to meet your industry peers, share experience and develop metrics to help our industry.

Members can register for the November 5 meeting by visiting the Site Selection and Start-Up Metrics Work Group page on the MCC member website or contact Lara Knitter, MCC Customer Service & Product Support, at lknitter@metricschampion.org for assistance.