Ask the Experts- MCC Answers Your Metric Questions

With more organizations focusing on metrics, the MCC has received an increase of questions ranging from how to use metrics, why some metrics are better than others, which type of metrics is best to use, as well as questions about specific MCC Metrics. This column provides a forum for us to share these questions and answers with you.

MCC Members at the Gold or Platinum benefit level, can access the “MCC Ask the Experts – Questions and Answers Archive” by logging into the MCC member website. Interested in becoming a member or have questions about your membership? Contact membership director, Terry Holland.

March 2019

A: MCC metric definitions include four types of metrics:  Quality, Timeliness, Cycle Time, Cost/Efficiency. (See Ask the Experts: "Cycle Time vs. Timeliness metrics – which should I use?") Query rate is defined as a quality metric.

We also have different levels of metrics. For query rate, we have:

  • Study level query rate [Ref DM-04aQ-S]. The Key Performance Question (KPQ) that the metric is designed to help answer is, "Is high quality data being entered into the database?"
  • Site level key risk indicator [Ref RBM006]. The KPQ for this metric is, "Is the site delivering high quality, evaluable data?" It compares query rate among sites in the same study to determine if particular sites are outliers, either high or low relative to average performance.

Each metric is designed to answer a different question; one at study level and one at a site level by comparing sites within a study. At study level, high query rate is most likely due to a systemic issue such as eCRF design issues, high protocol complexity, IT programming issues or poor instructions for eCRF completion. Low query rates may also be a concern – perhaps automated and/or manual queries are not being generated when they should be. It is important to recognize that systemic-level issues can’t be addressed by individual sites. But how can you determine whether you have a systemic-level issue or site-level performance issues? Review the site level key risk indicator to determine whether a subset of sites have high query rates or the rates are high among most sites. If you have a subgroup of sites with high query rates, you need to investigate further. You may find that the sites with high rates are early enrollers that have encountered a systemic level issue before the other sites. Alternatively, you may discover site-specific issues such as staff turnover with poor knowledge transfer or training, or other resource issues. MCC metric definitions include suggestions for additional analysis and investigation you could carry out if the metric value is not to target.

The data from these metrics can highlight concerns. But, as with all metrics, further investigation is needed to understand why the data is as it is, whether it is truly a concern, and whether actions are needed to improve the performance.

So, in direct answer to the question, the query rate can be both an indicator of site quality and a reflection of a systemic-level quality issue such as eCRF design. Understanding whether there is an issue at one, some or all sites may point you in a particular direction in terms of possible cause.

Meet the Experts

Keith Dorricott, MCC Ambassador and Director,
Dorricott Metrics and Process Improvement, LTD

Linda B. Sullivan, Co-Founder & President,
Metrics Champion Consortium

Click here to submit a question