In Part 1 of this series, we examined the need for Key Performance Indicators (KPIs) in Pharmacovigilance (PV) and listed some of the most important metrics to be first developed (Part 1, Nov 27th, 2018). In this second article, we share some considerations that will be helpful when first establishing KPIs:
· Define measurable metrics. Often we start with a long list of activities we wish to monitor including timeline and quality parameters. In addition to prioritizing based on a risk-based approach (refer to Part 1), the availability of the data will also be a determining factor in selecting our first set of metrics. The first metrics to be developed are those than can be measurable from a reliable source that will ensure accuracy and objectivity, keeping in mind that they will be scrutinized by internal and potentially external stakeholders. If data is not being collected, and that is often the case for quality parameters, you will first need to develop the process and checklists to collect these data.
· Setting thresholds of compliance. Consider common PV standards to establish what compliance level is acceptable. An example: for serious ICSR reporting compliance to Health Authorities, your target should not be below 95%, ideally 98-100%. As reference, 15-day compliance results for Marketing Authorisation Holders and Sponsors in the EU are shown in the 2017 Annual Report on EudraVigilance. In 2017, the compliance rate of 15-day ICSR reporting to EudraVigilance was 94.9%.
· Start small. Producing metrics for the first time can seem daunting so every effort should be made to be pragmatic. Select a time period that is manageable. As a starting point, it is recommended you look at the current month or quarter. As you progress you will accumulate annual data. As a second stage, it may be appropriate to conduct a retrospective review to establish historical compliance and show progress. You are likely to be asked for current and previous year data during audits and inspections.
· Dive In. The ideal process or system does not exist, it will need to be refined on an ongoing basis. Waiting for all the pieces to come together (e.g. new database, new technology, perfectly formatted data) will likely result in long delays. Starting with a narrow scope is the first step to creating a culture of self-directed compliance management. Don’t hesitate and start the process despite the many challenges and limited resources. Implement a pilot phase if needed with interim results until your data is in good shape.
· Engage other functions. PV compliance is a company-wide responsibility and many activities critical for PV compliance are not conducted by the PV department. The development of metrics to capture such activities are an effective way to demonstrate oversight and provide prompt support to other departments. To achieve this, ensure you engage other functions early in the planning phase and include them in appropriate workstreams to leverage their expertise and capabilities (e.g. Regulatory Affairs for Labeling Updates and Aggregate Report Submission, Quality Assurance for CAPAs and Deviations).
· Partners and service providers: KPIs to monitor the performance of delegated or contractual obligations is a must. Ensure you include in PV agreements the need to receive metrics from partners and vendors that capture compliance of delegated activities. Also, to ensure ongoing collaboration in addressing any issues that may arise from the review of compliance data.
In summary, selecting measurable metrics, assigning thresholds of compliance, diving in with a narrow scope and pragmatic approach and working closely with all internal and external stakeholders will ensure you achieve your first significant milestone - a foundation set of metrics to analyze, communicate and build on that will instill a practice and culture of compliance.
Laura Larrú
References:
EMA/7552/2018 - 2017 Annual Report on EudraVigilance for the European Parliament, the Council and the Commission. https://www.ema.europa.eu/documents/report/2017-annual-report-eudravigilance-european-parliament-council-commission-reporting-period-1-january_en.pdf, Accessed Jan 2019.