Friday, 18 May 2018

Common Mistakes in Method Validation and How to Avoid Them - Part 3: Accuracy


The validation of analytical methods is undoubtedly a difficult and complex task. Unfortunately this means that mistakes are all too common. As a trainer and consultant in this area I thought it might be useful to take a look at some common mistakes and how to avoid them. In this series of articles I will pick out some examples for discussion related to the method performance characteristics as listed in the current ICH guidance, ICH Q2(R1), namely: Specificity; Robustness; Accuracy; Precision; Linearity; Range; Quantitation limit; and Detection limit.
In previous articles I wrote about some common mistakes associated with ‘Specificity’ and 'Robustness'. This time I’ll take a look at ‘Accuracy’. The common mistakes that I have selected for discussion are:
1.       Not evaluating accuracy in the presence of the sample matrix components
2.       Performing replicate measurements instead of replicate preparations
3.       Setting inappropriate acceptance criteria
The definition of accuracy given in the ICH guideline is as follows: ‘The accuracy of an analytical procedure expresses the closeness of agreement between the value which is accepted either as a conventional true value or an accepted reference value and the value found.’ This closeness of agreement is determined in accuracy experiments and expressed as a difference, referred to as the bias of the method. The acceptance criterion for accuracy defines how big you are going to let the bias be and still consider the method suitable for its intended purpose.
The term accuracy has also been defined by ISO to be a combination of systematic errors (bias) and random errors (precision) and there is a note about this in the USP method validation chapter, <1225>: ‘A note on terminology: The definition of accuracy in 1225 and ICH Q2 corresponds to unbiasedness only. In the International vocabulary of Metrology (VIM) and documents of the International Organization for Standardization (ISO), accuracy has a different meaning. In ISO, accuracy combines the concepts of unbiasedness (termed “trueness”) and precision.’
From the point of view of performing validation, the difference in the definitions doesn’t make a lot of difference, we usually calculate both bias and precision from the experimental data generated in accuracy experiments. Personally I prefer the ISO definition of accuracy.
Mistake 1: Not evaluating accuracy in the presence of the sample matrix components
Since the purpose of the accuracy experiments is to evaluate the bias of the method, the experiments that are performed need to include all the potential sources of that bias. This means that the samples which are prepared should be as close as possible to the real thing. If the sample matrix prepared for the accuracy experiments is not representative of the real sample matrix then a source of bias can easily be missed or underestimated.
TIP: The samples created for accuracy experiments should be made to be as close as possible to the samples which will be tested by the method. Ideally these ‘pseudo-samples’ will be identical to real samples except that the amount of the component of interest (the true value) is known. This can be very difficult for some types of sample matrix, particularly solids where the component of interest is present at low amounts (e.g., impurities determination).
For impurities analysis, it may be necessary to prepare the accuracy samples by using spiking solutions to introduce known amounts of material into the sample matrix. Although this carries the risk of ignoring the potential bias resulting from the extraction of the impurity present as a solid into a solution, there isn’t really a workable alternative.
Mistake 2: Performing replicate measurements instead of replicate preparations
Performing replicate preparations of accuracy ‘pseudo-samples’ allows a better evaluation of what differences in the data are due to the bias and what are due to variability of the method, the precision. A minimum of 9 replicates are advised by the ICH guidance and these should be separate preparations. For solids, this could be 9 separate weighings into 9 separate volumetric flasks, as per the method.
However, the preparation does depend on the nature of the sample matrix and the practicality of controlling the known value for the component of interest. As discussed above, sometimes in the case of impurities methods, solutions may be required for practical reasons even though the sample matrix exists as a solid. In this case 9 separate weighings does not offer more representative ‘pseudo-samples’ and thus a single stock solution for the impurity would probably be a better choice.
TIP: Assess the sample matrix and try to prepare separate replicates when possible so that the data produced is as representative as possible and includes typical sources of variability.
Mistake 3: Setting inappropriate acceptance criteria
As mentioned previously, the acceptance criterion for accuracy is based on how much bias you will allow in the results from the method. It is obviously better not to have any bias in a method but there is always a certain amount of potential bias associated with the combination of the sample matrix, the level of the components of interest in the sample, and the instrumentation used for the measurement. For the method to be capable the bias needs to be less than the specification for the result. For example, if a drug substance specification requires that there must be between 99 to 101 %w/w of the drug present, then a method which has a bias of 2% is not going to be acceptable.
TIP: Make sure that the acceptance criteria set for accuracy in method validation are compatible with the requirements for the method, and in particular, the specification for the test.
References
1.       ICH Q2 (R1): Validation of Analytical Procedures: Text and Methodology, 2005, www.ich.org
2.       USP <1225> Validation of Compendial Methods, www.usp.org
In the next instalment, I will write about common validation mistakes for the method performance characteristic of precision. If you would like to receive the article direct to your inbox, then sign up for our eNewsletter. You will receive lots of helpful information and you can unsubscribe at any time. We never pass your information on to any third parties.
If you would like to learn more about method validation, and method transfer, then you may be interested in the 3 day course on the topic from Mourne Training Services Ltd. The course has two versions, one applied to small, traditional pharmaceutical molecules and one for large, biological/biotechnology derived molecules. Visit the MTS website for more information.

   

Tuesday, 15 May 2018

Tuesday, 8 May 2018

Tuesday, 1 May 2018

Wednesday, 25 April 2018

Common Mistakes in Method Validation and How to Avoid Them - Part 2: Robustness

A rather unfortunate mistake!
The validation of analytical methods is undoubtedly a difficult and complex task. Unfortunately this means that mistakes are all too common. As a trainer and consultant in this area I thought it might be useful to take a look at some common mistakes and how to avoid them. In this series of articles I will pick out some examples for discussion related to the method performance characteristics as listed in the current ICH guidance, ICH Q2(R1), namely: Specificity; Robustness; Accuracy; Precision; Linearity; Range; Quantitation limit; and Detection limit.

In the previous instalment I wrote about some common mistakes associated with ‘Specificity’. This time I’ll take a look at ‘Robustness’. The common mistakes that I have selected for discussion are:

1. Investigating robustness during method validation

2. Not investigating the right robustness factors

3. Not doing anything with the robustness results

The purpose of a robustness study is to find out as much as possible about potential issues with a new analytical method and thus how it will perform in routine use. Usually, we deliberately make changes in the method parameters to see if the method can still generate valid data. If it can, it implies that in routine use small variations will not cause problems. This definition is provided in the ICH guideline: “The robustness of an analytical procedure is a measure of its capacity to remain unaffected by small, but deliberate variations in method parameters and provides an indication of its reliability during normal usage.

There is another aspect to robustness that doesn’t neatly fit under this definition which applies to the performance of consumable items in the method, such as chromatography columns. The performance of the column when different batches of the same column packing are used may vary. Although column manufacturers aim for batch to batch reproducibility, most practitioners of HPLC will have come across at least one example of this problem. Another issue is the aging of the column, the column performance generally decreases with age and at some stage the column will have to be discarded. Strictly speaking, these column challenges would actually come under the heading of intermediate precision, following the ICH guideline, but it makes much more sense to investigate them during method development as part of robustness.

The method validation guidelines from both ICH and FDA mention the importance of robustness in method development and how it is a method development activity but they do not define whether it needs to be performed under a protocol with predefined acceptance criteria. Since the use of a protocol is a typical approach in most pharma companies it brings me to my first common mistake associated with robustness.

Mistake 1: Investigating robustness during method validation


What I mean by this is that the robustness investigation is performed during the method validation, i.e. the outcome of the investigation is not known. I do not mean the approach where the robustness has already been fully investigated and then it is included as a section in the validation protocol for the sole purpose of generating evidence which can be included in the validation report.

If robustness is investigated during validation for the first time, the risk is that the method may not be robust. Any modifications to improve robustness may invalidate other validation experiments since they are no longer representative of the final method. It will of course depend on what modifications have to be made. As FDA suggests… “During early stages of method development, the robustness of methods should be evaluated because this characteristic can help you decide which method you will submit for approval.”

TIP: If for some reason robustness hasn’t been thoroughly evaluated in method development then investigate it prior to execution of the validation protocol using a specific robustness protocol. If any robustness issues are identified, these can be resolved prior to the validation. The nature of the robustness problems will determine whether the resolution is just a more careful use of words in the written method or if method parameters need to be updated. 

Mistake 2: Not investigating the right robustness factors


If you choose the wrong factors you may conclude that the method is robust when it isn’t. Typically what happens then is that there are a lot of unexpected problems when the method is transferred to another laboratory, and since transfer is a very common occurrence in pharma, this can be very expensive to resolve.

When choosing robustness factors it is tempting to read through the method and select all the numerical parameters associated with instrumentation. For example, when assessing HPLC methods there is a tendency to only look at the parameters of the instrument without consideration of the other parts of the method, such as the sample preparation. Unfortunately, sample preparation is an area where robustness problems often occur. Detailed knowledge of how the method works is required to identify the most probable robustness factors.

TIP: The most important factors for robustness are often those which were adjusted in method development. Review all the steps in the method to choose robustness factors and use a subject matter expert to help if necessary. 

Mistake 3: Not doing anything with the robustness results


The reason for investigating robustness is to gain knowledge about the method and to ensure that it can be kept under control during routine use. Very often robustness data is presented without any comments in the validation report and is not shared with the analysts using the method. This tick-box approach may be in compliance with regulatory guidance but it is not making the most of the scientific data available. The discussion of the method robustness in the validation report should be a very useful resource when the method needs to be transferred to another laboratory and will assist in the risk assessment for the transfer.

TIP: Review the robustness data thoroughly when it is available and ensure that there is a meaningful discussion of its significance in the validation report.

References:

1.       ICH Q2 (R1): Validation of Analytical Procedures: Text and Methodology, 2005, www.ich.org
2.       FDA Guidance for Industry: Analytical Procedures and Methods Validation for Drugs and Biologics, 2015, www.fda.gov


In the next instalment, I will write about common validation mistakes for the method performance characteristic of accuracy. If you would like to receive the article direct to your inbox, then sign up for our eNewsletter. You will receive lots of helpful information and you can unsubscribe at any time. We never pass your information on to any third parties.

If you would like to learn more about method validation, and method transfer, then you may be interested in the 3 day course on the topic from Mourne Training Services Ltd. The course has two versions, one applied to small, traditional pharmaceutical molecules and one for large, biological/biotechnology derived molecules. Visit the MTS website for more information.


   

Tuesday, 3 April 2018

MTS Recommends... What's New In MHRA's Revised Data Integrity Guidance — A Detailed Analysis

What's New In MHRA's Revised Data Integrity Guidance — A Detailed Analysis 

A very detailed summary of all the changes in this guidance from the original GMP guidance issued in 2015, and the draft GxP version released in 2016.

By Barbara Unger
Pharmaceutical Online, March 19, 2018