With more organizations focusing on metrics, the MCC has received an increase of questions ranging from how to use metrics, why some metrics are better than others, which type of metrics is best to use, as well as questions about specific MCC Metrics. This column provides a forum for us to share these questions and answers with you.
MCC Members at the Gold or Platinum benefit level, can access the “MCC Ask the Experts – Questions and Answers Archive” by logging into the MCC member website. Interested in becoming a member or have questions about your membership? Contact membership director, Terry Holland.
A: The MCC Centralized and Site Monitoring Process Metrics Work Group has been discussing how to use audit and inspection data to provide insights into whether issues have been detected and appropriately remediated. The challenge you’re raising is that trending the number of issues over time can be misleading because the opportunity to detect those issues varies over time – and the pandemic has brought this into sharp focus. In addition to reviewing the trending of audit findings, you should add a metric that normalizes the issue counts to take into account the opportunity of detection. For example, if there are 10 site audits with findings related to Principal Investigator oversight, and only 10 site audits were conducted, that’s indicating a systemic issue – the issue was found on 100% of the opportunities. Whereas, if there were 100 site audits, for which 10 had findings related to Principal Investigator oversight, this indicates the issue is not so prevalent – i.e. the issue was found on 10% of the opportunities. So, a measure of prevalence of findings, trended over time, might provide a better guide. In addition to looking at overall prevalence, you might consider looking at prevalence of issues by issue type. This requires clearly defined categories for issue types – a MCC group is developing a categorization framework for organizations to utilize in the future.