We’re implementing analytics-enabled quality management to perform automated evaluation of customer interactions. Do we still need to do calibration sessions?
Analytics-enabled quality management (AQM), which leverages capabilities that include speech, text, and desktop analytics to automatically evaluate up to 100% of customer interactions, enhances, but does not completely eliminate, traditional quality management (QM) processes, including calibration. Although AQM handles the “heavy lifting” in a QM program, human involvement is still part of the process. For instance, interactions that are flagged for review by AQM, per user-defined parameters, are still sent to a QM analyst or supervisor for further evaluation. In addition to AQM evaluations, it is a best practice for supervisors to conduct some live monitoring of interactions to facilitate in-the-moment agent coaching and/or recognition.
Calibration is a critical component of a successful AQM program, as it is in a traditional QM process. In an AQM environment, a set of interactions (voice- and/or text-based) are tagged for calibration and sent to all reviewers (QM analysts and supervisors). Each reviewer manually evaluates and scores the transactions. The QM manager generates a report that reflects scoring variances between the reviewers for each transaction at the question level. The scoring variances are also calculated against the automated AQM evaluation. A calibration meeting is held to review scoring results and to discuss any inconsistencies. DMG recommends that agents be invited to participate in calibration sessions so they can gain an appreciation of the effort and rigor applied to the QM process, better understand the evaluation criteria, as well as to provide valuable insight from the front line. Once all opinions are presented, the group must reach a consensus on the best way to score each question. The QM criteria guidelines, automated AQM evaluation forms, procedures and training materials should be modified or updated based on the consensus outcome.
When conducted properly, calibration sessions help agents provide a consistently outstanding customer experience, foster collaboration, and establish agreement on how quality monitoring criteria should be applied to each question on an evaluation form, whether scored via AQM or manually. These sessions are also a great way to uncover opportunities for additional training and best practices for handling interactions.