Vendor Audit: What It Is and What to Expect From It

The pharmaceutical industry is one of the most highly regulated industries, and for good reason. These organizations are tasked with protecting the rights and well-being of subjects, along with the integrity of data and analysis. As a result, audits are a crucial part of our business.

The word "audit," however, can be intimidating, no matter how well prepared you think you are, as we know from our work on FDA audits. On top of making sure your own organization is ready for an audit, you also need to prepare a plan for auditing your vendors. A major part of a client's quality assurance program involves a periodic audit of contracted organizations, partners, and those who provide services that support various aspects of clinical trials.

EG Life Sciences has abundant experience in this field, and we're going to break down the purpose of audits, how these audits work, how you can prepare, and how we can help.

Audit Purpose and Processes Overview

The Purpose of an Audit

In general, sponsor biotech and pharmaceutical companies outsource a certain set of services to vendors. Such outsourcing decisions are based on a lack of availability of resources within the sponsor organization.

As an example, for a small to mid-size pharmaceutical company, it is not possible to have the in-house skill sets and resources required to execute all functions associated with the data life cycle. This includes technical infrastructure and processes, as well as human resources to perform various activities related to data acquisition, database management, data management, biostatistics, statistical reporting, and medical writing.

Since these are essential functions, small to mid-size sponsor companies often require bundling these services and outsourcing them to external vendors, often referred to as Clinical Research Organizations (CROs). In the case of larger pharma companies, the same outsourcing for data cycles may come from meeting the requirements triggered by rapid fluctuations in the demand for technical and human resources.

While sponsor companies are required to outsource services, it is critical for them to ensure that they are associating with a "right vendor." Beyond the legal and financial compliance requirements, the sponsors need to confirm that the vendors meet the necessary regulatory and reporting requirements when it comes to the qualification of a vendor. Secondly, sponsors need to ensure that they can accurately understand the extent of the risk they are taking on by associating with certain vendors. Such qualification and risk assessments are done by the sponsor through a formal audit process. Given the strict regulations imposed by pharmaceutical industry regulatory bodies around the world, the audit process in the pharmaceutical industry is much more rigorous than in other industries. Figure 2 describes why a vendor gets audited.

The Audit Process Overview

In general, sponsor companies select their vendors through their due diligence process. The due diligence is undertaken through requests for information (RFI) and requests for proposal (RFP), followed by bid defense. Every sponsor has a different set of rules, expectations, and requirements for the selection of a vendor. Prior to initiating the association and starting the business operations, sponsors take an audit of vendor processes, systems, infrastructure, and personnel. Such an audit is called a "pre-qualification" audit.

Upon the satisfactory outcome of the "pre-qualification" audit, the vendor gets selected by the sponsor for the functional operations that the sponsor wishes to conduct. After the association, the sponsor needs to take a routine audit to ensure that the vendor processes promised initially are followed by the vendor upon continued operations. Secondly, such routine audits also ensure that the vendor has resolved or addressed the observations from any prior audits. Most sponsors have a rule of auditing every service provider at least once in a span of two to three years subject to continued association with the vendor. Vendors who are deemed higher potential risk are audited more frequently.

The second type of audit is the "vendor qualification audit." This audit is usually taken for vendors where the sponsor has an existing contract. This audit can be a routine audit as a quality assurance requirement, or it can be triggered by a qualified quality event through business operations. Depending on the cause of such an audit, this audit can have a different scope and stakeholders involved. This audit can happen frequently or with a predefined frequency based on a mutual agreement between the sponsor and vendor.

The third type of audit is triggered through a submission process from a regulatory body. In many circumstances, the US Food and Drug Administration, the European Medicines Agency (EMA), or the Pharmaceuticals and Medical Devices Agency (PMDA) may request additional process-specific information pertaining to vendor processes utilized in the submission of clinical data to a regulatory body. This audit can be much more comprehensive in the case of clinical research organizations.

Now that we've discussed the process, the three types of audits can be summed up as follows:

• Pre-qualification audit: As stated above, these audits are undertaken prior to business being awarded to a potential new service provider, and they evaluate the vendor’s ability to meet the sponsor's expectations and industry standards. Vendor pre-qualification audits focus on new vendor capabilities, systems, and processes. Vendor qualification audits are guided by 21 CFR 312.52 and ICH 5.5.2, which regulate the transfer of obligations between a contract research organization and sponsors.
• Re-qualification and in-process audits: These audits are also known as Quality Assurance (QA) audits. They refer to a periodic formal review by the sponsor’s quality assurance departments to examine the vendor’s organizational structure, review procedures followed, and review selected documentation-related services performed by the vendor for the sponsoring organization. The purpose of a Vendor QA audit is to assess potential risk and make sure the vendor continues to meet the sponsor’s expectations.
• Extension of regulatory audit from sponsor to vendor: In general, the sponsor's pharmaceutical companies are audited by regulatory bodies for compliance with processes. When sponsors use vendors for key processes that impact the drug development process, the regulatory audit process is extended to the vendor. Such audits are an extension of an audit from regulatory bodies to sponsors.

Next, we'll provide an overview of these vendor quality assurance audit procedures, including expectations of a sponsor’s auditors, audit findings, and how to appropriately respond to audit findings. Before we get started, we'll share a humorous analogy of the role these types of audits play in sponsor-vendor relationships.

Audit Types, Conduct, and Response

Pre-Qualification Audit

After vendor due diligence is complete, sponsors will conduct a vendor pre-qualification audit. This audit includes formal validation and assessment of vendor capabilities and focuses on following the aspects of the business. This audit is conducted by quality assurance personnel representing the sponsor. The goal of quality assurance personnel in this case is to evaluate all processes and procedures of the vendor before awarding business. Various processes generally assessed during a pre-qualification audit are discussed below:

Business Processes

A formal review of the vendor’s organizational setup along with project management processes is conducted. An auditor reviews the vendor’s work allocation processes, cost estimation, and study milestone management processes. A review of the project plan is done for consistency with the sponsor’s expectations. The vendor's resourcing capabilities, along with any previous regulatory remarks, are reviewed. An auditor is also interested in operational quality and performance metrics assessment processes, and the auditor will also assess the vendor’s disaster recovery and business continuity plan. Overall Key Performance Indicators (KPIs) are evaluated, including repeat business, the vendor’s finances, staff retention rates, and more.

Quality Management System (QMS) Processes

For the partnership between the sponsor and vendor to be successful, a robust quality management system is essential. A pre-qualification audit involves an evaluation of QMS-related processes and related documentation like Standard Operating Procedures (SOPs), standard templates, work instruction and guidance documents, training processes and records, delivery quality management processes, personal CVs, and job descriptions, along with hiring and employee evaluation processes. An auditor will also check vendor policies and procedures related to data privacy and protection. Computer System Validation required as per 21 CFR part 11 compliance is also reviewed in detail, along with due diligence done by the vendor before deploying any new software for project execution. Finally, an auditor will also evaluate if the QMS processes of the potential vendor align with the sponsor and are in accordance with industry standards.

Personnel Qualification and Interviews

A review of vendor staff qualifications and interviews of key personnel are important to ascertain if qualified resources are available for project execution. The qualification of vendor staff is compared against job descriptions, and their training records are evaluated. Some of the key staff will be interviewed by the sponsor to assess their qualifications and experience.

Infrastructure Review

The auditor will also evaluate building and facility infrastructure, including access to the workplace, along with server room and data center security. Plus, the auditor will review IT infrastructure related to remote work by vendor personnel, like VPN, remote staff access, and monitoring processes.

Re-qualification and In-process Audits

Vendor re-qualification and in-process audits, also known as QA compliance audits, refer to a periodic formal review by the sponsor’s quality assurance departments. This audit examines the vendor’s organizational structure, reviews procedures followed, and selects documentation related to the service performed by the vendor for the sponsoring organization. The purpose of a Vendor QA audit is to assess potential risk and make sure the vendor continues to meet the sponsor’s expectations.

Scope of Audit

A QA compliance audit assesses compliance with relevant Standard Operating Procedures (SOPs), applicable guidelines/regulations, contracts, and work orders, and it will include an assessment of the protection of the rights, safety, and well-being of patients and consumers. In general, the following areas will be reviewed during a typical QA audit.

• The service provider’s organizational structure, personnel qualification, employee oversight, trainings and turnover, and project management processes relevant to the execution of the sponsor’s clinical trials
• QMS-related documents like SOPs, templates, etc.
• Review of documentation related to the relevant contract, including work orders, budget and change orders, etc.
• Documentation of selected sponsor studies for which the vendor provided services
• Operational quality and performance metrics (quality, error rate, re-work rate, and productivity, etc.) and previous quality deviation management (frequency, RCA, CAPA, etc.)
• Computer systems, data and information security, and privacy safeguard processes
• Periodic software and computer system validation documentation
• Review of previous negative findings in regulatory inspections, if applicable
• Previous audit findings and their related corrective and preventive action plan (CAPA)

Method of Audit Conduct

Most vendor audits are currently conducted remotely utilizing Zoom or MS Teams with auditors requesting documents to be uploaded on cloud platforms prior to the audit. An audit is an agenda-driven meeting that generally takes place for about two business days.

Gap Assessment and Comparative Assessment With Previous Audit Findings

Auditors will review a gap assessment provided by sponsor auditors during a previous pre-qualification audit or other type of audit, and they will review an agreed-upon vendor's action plan along with its effectiveness and timely implementation. Any delay in the vendor’s CAPA plan implementation or less than optimal action plan implementation will be notified as an audit observation. A vendor is also expected to provide documentary evidence of gap assessment CAPA implementation.

Review of QMS

The QMS refers to a comprehensive set of policies and procedures in place to meet sponsor and regulatory requirements, which includes protecting the rights and well-being of subjects and maintaining the integrity of clinical study data and analysis. GCP requires that all data transformations need to be validated and documented. To meet these stringent guidelines, it is essential that the SOPs and procedures that a vendor implements meet the sponsor’s expectations. An auditor reviews current policies, procedures, and SOPs associated with QMS processes, along with the expected documentary evidence of its appropriate implementation. An auditor will review various templates used by the vendor, e.g., the study validation template to make sure it meets the sponsor’s standards and captures required information.

Gap Assessment of Current Processes

The gap assessment of current processes is done by an auditor who reviews the record of activities performed by the vendor from selected studies to assess compliance with relevant SOPs, proper documentation, and applicable guidelines related to the protection of the rights, safety, and well-being of patients and consumers. During an audit of selected studies, it is expected that key personnel responsible for the execution of relevant clinical study activities be present and answer questions from auditors. An auditor will review CVs and the training curriculum of vendor personnel who worked on selected studies and activities to ascertain if these activities were carried out by qualified personnel. Auditors will assess if proper procedures were followed for maintaining data integrity and study result validity. A formal review of every study milestone documentation is done, including study startup activities like a list of table creation and updates, documentation related to the data monitoring committee (DMC), a study test run, unblinding, and a final run of study deliverables, etc. Auditors will expect evidence of study activities being executed correctly as per industry/sponsor standards in formal study documentation (Trial master file), and any missing, incomplete, or inappropriate documentation will have to be explained by accountable vendor personnel. If the auditor discovered any deviation in quality during the execution of study activities, the sponsor will look for documented evidence of root cause analysis and relevant CAPA.

Findings and Resolutions

An audit report is a formal communication of observations from an audit provided by the sponsor’s auditor team to the service provider. Audit observations are classified into three types based on their potential impact on study subjects’ safety and well-being and the quality or integrity of data produced and reported in a clinical trial.

• Minor Observations: These observations include ones that do not impact patient safety/well-being or the quality of data/clinical trial integrity. However, multiple minor observations potentially can become a major observation if they may potentially impact patient safety or data integrity. A good example of a minor observation would be an incomplete employee training record.
• Major Observations: Major audit findings result from deviations that might potentially impact the safety and well-being of subjects and/or the quality and integrity of clinical trial data. An example of a major audit finding is inaccurate QC documentation. A major observation can lead to a failed audit.
• Critical Observations: Critical audit findings include those deviations that will adversely impact the rights, safety, or well-being of the subjects and/or the quality and integrity of data. Examples of critical audit findings include data with incomplete source documentation or code with multiple hardcoding instances without appropriate notes on file. A critical finding in an audit is not acceptable, and a service provider who receives a critical finding or several critical findings is considered to have failed the audit. In fact, receiving a critical observation amounts to an audit failure.

A CAPA needs to be implemented by the vendor upon receipt of the audit report. A CAPA is a formal document containing vendor audit observations and a remediation plan, along with an implementation timeline. It is to be noted that a sponsor would expect documentary evidence of appropriate and timely execution of a CAPA.

Regulatory Audit

While supporting regulatory submissions and even reporting of clinical trial results and progress, the sponsor companies are frequently audited by regulatory bodies. For the US FDA, these are managed through the forms FDA 482 (notice of inspection) and FDA 483 (observation of inspection). The audits from regulatory bodies are to ensure that the sponsor is conducting the clinical trial processes with integrity and as per the regulatory guidance and requirements to ensure patient safety. In clinical data operations, these audits mean ensuring that the processes utilized in clinical data processing, management, and analytics are validated thoroughly. Since in many cases such processes are owned by CROs and vendors, the regulatory audit gets extended to evaluate the vendor processes. These audits include the following aspects:

Instance-Driven Process Documentation Request

The clinical data submission process goes through various milestones which require interaction with and approval of regulatory bodies. This includes but is not limited to the FDA advisory committee meeting, Real-Time Oncology Reporting (RTOR), and periodic data and status reporting requirements from the FDA. For EMA, this includes periodic reporting, as well as reporting requirements specific to the General Data Protection Regulation (GDPR). There are similar reporting events from other regulatory bodies like PMDA, MHRA, and Health Canada. These reporting events may trigger routine questions and potentially an audit from regulatory bodies. In many circumstances, such an audit could be a remote audit requesting the gathering of necessary and sufficient information related to the conduct of a clinical trial by the sponsor. In some instances, such an audit can get into a more detailed assessment of processes.

Personnel Interviews

Personnel qualifications and employee records are important components of regulatory audits. This includes the resume, training files, and job descriptions of each individual responsible for conducting clinical trial operations processes. Conducting interviews gives necessary confidence to the regulatory personnel about the qualifications and experience of the staff who perform clinical operations.

Necessary and Sufficient Documentation Development and Fulfillment

In many circumstances, the clinical trial does not go as planned. If so, the sponsor needs to thoroughly document the discrepancies and anomalies in the data and processes to ensure that the trial is conducted with integrity and to ensure patient safety. Such documentation is evidenced through documents like "note to file," and the content of such a document can initiate a requirement for a regulatory audit. As a result of these audits, regulatory bodies may request additional documentation and related evidence from the sponsor and vendors.

Conclusion

At some point, you will be faced with a vendor audit. To pass the audit, it is essential for both sponsors and vendors to understand regulatory requirements and how they impact programming processes. Programming teams need to be prepared for audits by ensuring timely and appropriate documentation of programming activities. Our documentation should be in accordance with the current sponsor standards and follow QA guidelines. Even though the audit process can seem lengthy and complex, these audits exist to ensure that you are not only following proper practices but that everything you do keeps patient safety and well-being first.

If you feel you need support in this area, EGLS has the expertise to help you with a vendor audit, along with other audits and overall audit processes. See how EG Life Sciences can help!

Other Sources

• "Glossary." Good Clinical Practice Network. Accessed 17 June 2022. https://ichgcp.net/1-glossary
• "The Global Guideline for GCP Audit." Good Clinical Practice Network. Accessed 17 June 2022. https://ichgcp.net/the-global-guideline-for-gcp-audit
• C. Marchand, A.Tinazzi " What Auditors Want." Paper presented at PHUSE, 2016.