GxP professionals understand the need for quality and quality system and we discuss quality with one another on a daily basis. But how do we measure it? How do we quantify our results? Once again we turn to Len Grunbaum and Emma Barsky, regular contributors to GxP Perspectives, for their insight on how to quantify quality for the development manufacture, and distribution of health products such as drugs, medical devices, and biologics.
In its simplest form, the definition of “quality” is “how good something is.” But what exactly does this mean for the life science industry, whose frame of reference is defined by regulations which are often vague and which provide little or no guidance regarding how they should be implemented?
In light of this, we would like to offer some ideas regarding how to measure – quantify – how good your “quality” is in tangible and practical terms. We contend that such metrics are useful in order for company management to make sound decisions regarding whether and/or where the quality system (i.e., the operational infrastructure that promotes and facilitates “quality”) requires improvement. The following key indicators are not all-inclusive (nor are the items mutually exclusive), but they provide meaningful ways to assess your “quality”:
• Number of successful external and internal audits as a percentage of the total number of external and internal audits: the higher the percentage of successful external audits (e.g., by existing/potential clients, regulators), especially when you have a large number of them, the better your “quality.” Passing one audit with flying colors is great but passing multiple audits with few minor or no observations is way better. It not only sets a trend regarding “legitimate” quality but it also validates the company’s degree of quality from different perspectives. This scenario allows any company to claim that its quality system has withstood scrutiny from a variety of companies and/or regulatory agencies over a long period of time.
While “looking good” to the outsiders is great, “feeling good” about what is under the covers is even better. Therefore, if thorough internal audits do not find any issues that either directly (critical observation) or indirectly (major observation) impact subject/consumer safety and/or data/product integrity, then “quality” is inherent to the operations.You may wonder how a subjective term like “success” is defined in this context. Fair question. A result of “no audit findings” (e.g., no FDA 483s, no audit observations) is the clearest measure of success. A relative handful of “minor/cosmetic” issues is not perfect but is certainly acceptable in this context. To the extent that the number of observations may be “critical” or “major,” as defined above, the audit will certainly be viewed as less successful or even unsuccessful. One should also remember that it is a common thing in the industry to consider a large number of minor observations as a major issue because this scenario gives an impression of a negative trend, the latter of which is not conducive to having quality operations.
• Number of “directed” (i.e., “for cause”) audits as a percentage of total audits: because directed audits are performed to follow up on actual or perceived regulatory compliance problems, the higher the number of “directed” audits, the more questions will be raised about your “quality.” “Directed” audits could be external (i.e., performed by existing clients or regulatory authorities) or internal (i.e., performed by internal quality staff). The higher the number of problems confirmed, the weaker the quality system. Even if these types of audits indicate in general that there are no actual problems, or a minimal number of problems, a large number of such audits should prompt questions regarding why the perception exists that the degree of “quality” is such that an investigation is required.• Number of investigations/CAPAs: an investigation is a formal and documented process performed to gather information (e.g., root cause, impact) regarding a specific problem encountered (e.g., a customer complaint, a missing controlled document) and which, depending on the outcome of the investigation, may lead to corrective and preventive actions. An “excessive” number (the definition of which is admittedly subjective in nature) of investigations, even if satisfactorily completed and closed, gives an impression that the underlying cause has never been properly identified and/or corrected.
• Number of repeating issues as a percentage of the number of audits performed: repeating issues are symptomatic of a quality system that does not correct or otherwise effectively address problems. While isolated incidents are not necessarily a reflection on the company’s overall quality, incidents that span multiple project teams and/or departments and/or are observed more than once may be indicative of quality-related problems. It is very difficult to convince anyone of the quality of operations when problems that are systemic in nature become evident.
The higher the percentage of audits that contain repeating issues, the more likely that this may be viewed as 1) management indifference, 2) lack of management involvement, 3) inappropriateness of personnel qualifications and/or 4) inability/unwillingness to invest in “quality.”• Number of business opportunities lost due to unsuccessful external audits as a percentage of the number of external audits: audits are sometimes performed as a basis for determining whether a business relationship should be consummated or continued (e.g., you will be chosen as a vendor/supplier, an existing relationship will be sustained) or expanded (e.g., a company will be awarded additional projects). Some life science companies (e.g. pharma, biotech) have to get clearance from the FDA prior to being able to market their product. Support companies (e.g., CROs, contract manufacturers) may have to undergo due diligence inspections to establish/maintain/enhance a business relationship. The higher the percentage of such opportunities lost (e.g., loss of a potential or existing client, project cancelled/not awarded, FDA did not grant an approval) because of poor audit results, as a percentage of external audits performed, the stronger the indication that your “quality” is dangerously weak. This, in turn, has a financial “bottom line” impact on the company: loss of business opportunities can also be translated into wasted R&D cost and/or lost anticipated revenue, both of which become a major risk to the company’s financial health.
In addition to the items listed above, there is another important quantifiable component to “quality,” which is too often being overlooked or not being considered at all. This component is what we define as “the monetary expenditure associated with ‘quality.’” Namely, we are talking about an operationally quantifiable parameter – cost of establishing and maintaining “quality” operations. Most will argue that “quality” is very expensive no matter what. We firmly believe that it does not have to be that way if the underlying causes, which directly and unnecessarily contribute to the extra cost of doing business, are either eliminated or minimized. Here are a few examples to give you a flavor of what can contribute to increased costs when it comes to meeting the regulatory responsibility of instituting and sustaining “quality”:
• Regulatory compliance decisions that are not defined in writing and/or are not defensible.
• Cumbersome and inflexible procedures that require more resources than necessary to execute them without “procedural deviations.”
• Inefficient procedures that require the same activity to be done more than once in order to be in compliance.
• Ineffective procedures that do not reach the desired objective of being in compliance after the first execution.
• Unclear procedures that result in too many on-going corrections in order to inject “quality” into operations.
• Too many procedures that company staff must follow without any value added.
• Contradictory procedures that lead to generating Notes-To-File, CAPAs, deviations, investigations, etc. because compliance to one procedure results in non-compliance with one or more other procedures.The above-listed activities not only translate into the need to spend more time and money in an attempt to have operational quality, but a number of these items translate into further quality-related costs to the company. Examples of the latter include, but are not necessarily limited to, taking the time to respond to observations or even worse yet, an FDA-483 or a Warning Letter. We think the point we are trying to make is clear…
Our bottom line is that you can make both your QA and CFO happy by quantifying “quality” in terms that will be understood and appreciated by both. This means that sound decisions can be made regarding whether and/or where to apply precious company time and resources help ensure that your “quality” is as good as it can be without putting the business out of business.
Emma Barsky and Len Grunbaum
Partners of The Practical Solutions Group, LLC
In News from FDA: Yet another weight loss danger in Japanese “rapid weight loss” pills. Read the story: foodconsumer.org
Join the GxP Perspectives Linkedin Group Here
Or get an email subscription (on the right sidebar)