All Pharmacovigilance (PV) departments at some point have faced or are facing the challenge of developing Key Performance Indicators (KPIs) to monitor and improve their pharmacovigilance systems.
In this article we review some of the important considerations for developing such KPIs.
In the EU, the use of performance indicators is defined in the Modules of Good Pharmacovigilance Practices (GVP) to continuously monitor the quality of key processes in PV. Module I – PV Systems and Their Quality Systems and Module II – Pharmacovigilance System Master File (PSMF) outline the framework for the KPIs. Guidance on compliance monitoring for each PV process is provided in each Module of GVP.
Beyond meeting regulatory requirements, establishing a KPI system enables PV organizations to assess the health of their PV systems, verify compliance with PV requirements, keep an ongoing oversight of delegated responsibilities from vendors and partners and be prepared for audits and inspections.
Larger organizations may have consolidated systems to control processes and extract metrics. However, as if often the case with rapidly growing small to mid-sized organizations, the PV department is faced with the challenge of developing KPIs with limited resources. In such situations, the following considerations can be helpful to start and successfully consolidate the KPIs:
Create a Standard Operating Procedure (SOP) for KPI Define the process that your company will follow to design and produce metrics, monitor the compliance and improve the system. This is a cyclical process where different departments may contribute.
Create a List of Metrics or KPIsAssign priority KPIs according to a risk-based approach, taking into consideration the impact on patient safety and regulatory compliance. Have an implementation plan for developing the KPIs with realistic timelines. That means considering the available resources and specially focusing on the highest priority metrics to start their development first without forgetting other metrics that will be developed at a later stage. Metrics have an ability to become company-wide focal points of attention, so they should reflect key activities. Some of the most important metrics will include:
ICSRs Reporting to Health Authorities: 7-day and 15-day reports submission timeliness and quality of ICSRs. May include 90-day reporting in the EU.
Periodic Report Submission to Health Authorities: timeliness and quality of reports.
CAPAs from Audit and Inspection findings, Deviations: ensure CAPAs and Deviations are monitored until closure with a system to raise signals to prevent non-compliance with agreed plans.
Signal Detection systems: ensure signal detection activities are being performed regularly, consistent with the requirements defined by company SOPs.
Risk Management Plans: compliance with submission due dates, adherence to commitments and quality of documents.
Safety Labeling Updates: ensure new safety information is implemented in a diligent way from signal detection or competent authority assessment (PRAC or PSUR Assessments, etc.), through to CCSI/CCDS updates and finally Local Product Information changes.
Develop systems or trackers that will be used to extract the metrics
This may be one of the most difficult parts of the KPI project. For ICSR, look for opportunities to extract metrics directly from the safety database in an efficient and accurate manner. Where it is not possible to automate the production of metrics using IT systems (e.g. Periodic Reports submissions, Risk Management Plans, Signal Detection, Labeling Updates, etc.), an excel log or tracker may be a good start to document decision dates, rationales, submission due dates and actual submission dates. If you use an excel log, make sure the tracker is saved in a controlled environment with access restrictions and ideally with the ability to track record of changes. The process and responsible function to update the tracker should be defined in a company SOP or Work Instruction.
Overall, a robust KPI system together with an audit and quality system will enable the PV professionals to understand the strengths and weaknesses of the PV system with the aim to fix the issues and improve the performance of their processes. By making these visible at a broader level within the organization, will ensure a consolidated focus on compliance, as well as optimal support from the Company's leaders.
Laura Larrú
LauraLarru@JPCCAssociates.com
References:
EMA/541760/2011 Guideline on good pharmacovigilance practices (GVP) Module I – Pharmacovigilance systems and their quality systems. 22 Jun 2012. https://www.ema.europa.eu/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-module-i-pharmacovigilance-systems-their-quality-systems_en.pdf Accessed Oct, 2018.
EMA/816573/2011 Rev 2. Guideline on good pharmacovigilance practices (GVP) Module II – Pharmacovigilance system master file (Rev 2). 28 March 2017 https://www.ema.europa.eu/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-module-ii-pharmacovigilance-system-master-file-rev-2_en.pdf Accessed Oct, 2018.