Paula Brown, PhD, director for natural health and food products at the British Columbia Institute of Technology, gave a talk on method selection at the recent Dietary Supplement Analytical Forum put on in Salt Lake City by the United Natural Products Alliance. The main goal of the forum was to look at DNA testing methodologies in light of the pressure put on the industry by New York Attorney General Eric Schneiderman and others and to see how these testing methods can be integrated into existing quality control systems. Brown cautioned that despite Schneiderman’s having placed DNA barcode testing on some sort of gold standard pedestal, the method must still pass through the same eye of the needle as any other test. Does it help answer the pertinent questions about a product’s identity, purity and potency?
“We’ve really got two different things going on here,” Brown said. “First there is the question of what do we chose to analyze, what reference materials are we using? Then there is the issue of product quality.”
Fit for purpose?
The key question, Brown said, is to first identify what the questions are that need to be answered, and then match the tests and methods used to those end points. In that sense, in Schneiderman’s first attack on the industry there was a clear disconnect in this process, as the test chosen and way it was done could not have answered the question of whether the products his office tested matched their specifications and label claims.
There are dozens of tests from which to choose for analyzing dietary supplement ingredients, Brown said. The old saw about documentation when dealing with FDA is that if it isn’t written down, it didn’t happen. Brown said there’s another when evaluating methods: Just because it’s written down, doesn’t mean it’s good.
“You can’t assume that literature methods have been optimized or validated,” she said.
You also can’t assume that even a validated method will fit your purpose, she said.
“People often reach for a USP method but those are often developed for a specific matrix,” she said. “Pharmacopoeial methods that are developed for one endpoint may not be suitable for another. If you were just looking at the anthocyanins, you wouldn’t be able to tell an apple from a tart cherry.”
Brown recommended setting up a matrix of sorts to start the process of what methods might be appropriate for a particular product. This starts with examining the specification. Is a particular plant part called for? If so, can the methods used tell one plant part from another? (DNA testing, for example, cannot). Can the method identify and quantify known adulterants for the material, whether they are related species or marker chemicals that have been added in? DNA testing might be of some use here for finding related species, assuming the tester knows what to look for. Finally, can the tests used differentiate between a natural and a synthetic material? In that case, a method that could quantify the ratios of sterioisomers might be called for.
(As an aside, the American Botanical Council is developing a series of Laboratory Guidance Documents that look at the plethora of methods available for the testing of various ingredients and offer information quality control personnel can use to choose among the various methods.)
Brown said a cast study of the use of DNA testing in honey shows that method validation for this technology is just as important as for any other. The results the researchers got varied from testing kit to testing kit. A 100% identification response is unrealistic, Brown said. Such a result would in fact lead a responsible tester to assume that they’d done something wrong, she said. A 90% to 95% Probability of Identification (POI) is a statistically more supportable finding, she said.
“No measurement is perfect. There is error in every measurement,” she said.
Validation is the key
There’s no magic bullet in method validation, Brown said. It’s still a matter of looking at the selectivity, linearity, accuracy, precision (repeatability/reproducibility), limits of detection and quantification and analyte stability associated with the tests you want to use.
Brown said that many experts with long experience in the industry have been put in a reflective mood by Schneiderman’s actions and the resultant public relations fallout that have come on the heels of the 20th anniversary of DSHEA.
“We were just talking this morning about how we are 20 years down the road and nothing has changed. I don’t think that’s true. But choosing the right methods and validating them is still a key,” she said.