FDA cGMP WARNING LETTERS
This article was originally published in The Gold Sheet
Executive Summary
...issued to foreign drug manufacturers in FY 1997 dropped by two-thirds from the previous year. The inspection performance declined for domestic firms, which received 50% more agency warnings than in FY 1996. Warning letters to oral solid manufacturers correspond with recall actions during FY 1997. Failure investigations, stability programs, and process validation remain key FDA compliance concerns. HPLC testing and labeling/ packaging procedures are also on the agency’s radar screen. [The drug cGMP warning letters issued in FY 1997 are listed on pp. 8-16. The listing includes the recipient's name, letter date, plant location and a description of the problem areas cited.
|
FDA cGMP WARNING LETTERS issued to foreign drug manufacturers in FY 1997 dropped by two-thirds from the previous year. The inspection performance declined for domestic firms, which received 50% more agency warnings than in FY 1996. Warning letters to oral solid manufacturers correspond with recall actions during FY 1997. Failure investigations, stability programs, and process validation remain key FDA compliance concerns. HPLC testing and labeling/ packaging procedures are also on the agency's radar screen. [The drug cGMP warning letters issued in FY 1997 are listed on pp. 8-16. The listing includes the recipient's name, letter date, plant location and a description of the problem areas cited. BULK SUPPLIERS DRAW FEWER FDA WARNINGS The number of warning letters issued by FDA to foreign drug manufacturers for good manufacturing practice deficiencies declined significantly during the agency's fiscal year 1997 compared to FY 1996. ä Only 13 GMP warning letters were sent to foreign firms between October 1996 and September 1997, less than half of the 31 letters issued abroad in FY 1996. There were 35 foreign warning letters in FY 1995, the first year the agency began using the warning letter format for cGMP compliance enforcement overseas. The drop for FY 1997 occurred in spite of the fact that FDA performed nearly as many inspections of foreign drug firms during the year as in FY 1996. Although the count has not been finalized for the past year, FDA estimates that the total number of inspec-tions will be just under the 286 performed in FY 1996. The decline reflected a sharp drop in the number of warning letters issued to foreign bulk manufacturers, which received only seven warning letters during FY 1997 compared to 24 the previous year. Foreign dosage form producers received six in FY 1997 and seven in FY 1996. Bulk firms receiving warning letters included Guangdong (China), Nippon Rikagkuyakuhin (Japan), Bidachem (Italy), Huls (Germany), Gist-Brocades (Netherlands), Irotec (Ireland), and Gedeon (Hungary). Finished dosage firms in the foreign warning letter pool included manufacturers of injectables - Nippon Kayaku (Japan) and Connaught (Canada); oral solids - Novopharm (Canada); ophthalmics - Dr. Mann Pharma (Germany); sterile implants - Zeneca (England); and veterinary products - Wyeth-Ayerst (Canada). ä The improved compliance record among foreign bulk firms parallels the effort internationally to improve the written GMP standards for active ingredient manufacturing. A series of guidelines have emerged recently from various regulatory agencies and associations addressing bulk drug GMPs ("The Gold Sheet" December 1996). In their wake, the International Conference on Harmonization steering committee agreed at a meeting in early February to develop a GMP guideline for APIs which will incorporate aspects of these recent international documents. FDA is preparing to publish for public comment a second draft of its domestic API GMP guideline in March. Instead of finalizing the document, the agency is putting it out in draft form, in part to allow more flexibility in pursuing the ICH effort. An international guideline being developed through the Pharmaceutical Inspection Convention (PIC) was released for comment in late 1997, and the work on that guideline will also factor into the ICH API project. Countering the downturn in foreign warning letters, there was a 50% increase in the number of FDA warning letters sent to domestic manufacturers in FY 1997. ä In that period, 88 domestic facilities received FDA warnings, up from 59 in FY 1996. The domestic increase kept the number of total warning letters issued by FDA hovering around 100 for the fourth straight year ("The Gold Sheet February 1997). As in FY 1996, only a small percentage of the domestic manufacturers who received a warning letter last year were bulk firms (4 of 88). Topical manufacturers fared slightly better in FY 1997 than in the previous year, receiving 15 warning letters last year vs. 20 in FY 1996. The types of problems discovered by FDA at these firms were similar for the two years, with stability programs, standard operating procedures (SOPs), and master/ batch records remaining the most frequently cited problems by the agency ("The Gold Sheet" February 1997). As in FY 1996, there was little correlation between the topical manufacturers receiving warning letters and those recalling products. Only two topical firms receiving the cGMP warnings in FY 1997, Speciality Chemicals and G&W (topicals/supposi-tories), conducted recalls reported in FDA's 1997 weekly enforcement reports ("The Gold Sheet" January 1998). G&W also recalled a topical cream the previous year ("The Gold Sheet" January 1997). Warning Letters Correlate With Oral Solid Recalls Oral solid manufacturers received five times as many warning letters in FY 1997 than the previous year - 20 vs. 4. ä Unlike topical manufacturers, there was a strong correlation between FDA findings of significant cGMP problems and product recalls among the oral solid group. Almost two-thirds (12 of 20) of the oral solid warning letter recipients recalled one or more oral solid products in 1997. Among these firms, Novopharm, Par, and Searle also had recalls reported in the 1996 enforcement listings. For ten of these firms, the cGMP problems were spotted at the same plant at which the recalled product was manufactured. About half of the recalls preceded the inspection triggering the letter, and the other half followed the adverse plant inspection. ä In at least two cases, there was an explicit correlation between the FDA findings described in the warning letters and the product recall. For example, ICN initiated a recall of trioxsalen tablets during an FDA inspection of its Bryan, Ohio facility, which included a review of the trioxsalen manufacturing and control process. On February 25, 1997, six days after the month-long inspection began, the firm notified the agency that it would withdraw all unexpired product on the market because of dissolution failures. A warning letter was issued to ICN in May which highlighted the inspection findings that the firm had failed to follow either the USP dissolution method or to validate that the dissolution method in use was equivalent or superior to the compendial method. Content Uniformity Issues Raised At Whitehall Robins At American Home Products' subsidiary Whitehall Robins, there was a direct link between the recall of a lot of ketoprofen caplets announced in September 1997 and an FDA inspection that concluded a month earlier. According to the warning letter, Whitehall did not reject a lot of the ketoprofen which failed content uniformity specifications. FDA noted that an out-of-specification result was obtained for one of ten tablets tested in April 1996. The results from this test were invalidated by Whitehall because of a technical error attributed to the analytical equipment. An additional three tablets failed content uniformity specifications the following day. Again, the results were invalidated for the same reason. Whitehall released the lot after passing results were obtained by using a different high pressure liquid chromatograph. Following FDA's inspection, however, Whitehall obtained failing content uniformity results for two tablets after conducting additional tests, prompting the recall of ketoprofen. In the warning letter, FDA acknowledged receipt of the firm's response to the inspection and noted that parts of the proposed corrective action plan were adequate. FDA requested that the firm submit the blend uniformity and content uniformity validation data to the Center for Drug Evaluation and Research (CDER) for review before resuming production of keptoprofen tablets. The agency also stressed that the firm needed to ensure that out-of-specification results are traced and fully investigated. 30% Of Letters Cite Out-Of-Spec Investigations The industry as a whole continues to struggle with the interpretation of FDA's requirements for out-of-specification (OOS) result investigations. The frequency of warning letters citing cGMP deficiencies in this area reflects the uncertainty that exists. About 30% (30 of 101) of the letters sent to firms in FY 1997 reference deficient failure investigations. Out-of-specification investigation practices have been a focal point of FDA inspections for the past several years. The agency's expectations in this area were affirmed and better defined in the 1993 U.S. district court ruling on the U.S. vs. Barr case ("The Gold Sheet" February & October 1993). A guidance to help firms understand the ramifications of the Barr decision regarding failure investigations has been under development at FDA since the case concluded. However, a draft has not yet been made public for comment. The out-of-specifica-tion investigation guidance will address agency expec-tations regarding documentation, retesting, resampling and outlier tests. ä The warning letters show that about two-thirds of firms (19 of 30) cited for OOS investigation-related problems failed to conduct such reviews in certain situations that warranted them. For example, FDA alleged in a February warning letter that Medco did not investigate failing analytical results for validation media runs for an eye drop product. Similarly, an April warning letter to Milex highlighted the firm's failure to investigate OOS stability results for vaginal suppositories. In other cases, FDA alleged that G&W did not explore the causes of failing analytical results for multiple analytical tests, including: dissolution, stability, impurity, and content uniformity. Novopharm was cited for averaging out-of-spec dissolution results and failing to conduct an investigation. Other problems found with failure investigation practices in FY 1997 included: n n undocumented and/ or unsubstantiated investigation conclusions, n n failure to retain and/or document all OOS results, and n n lack of timely investigations. ä A January 1997 warning letter to Mikart detailed multiple concerns with the firm's OOS investigation program. In the letter, FDA maintained that the firm failed to properly document reasons and supporting data for the invalidation of out-of-spec results. FDA investigators were prompted to audit Mikart's failure investigation practices due to a high number of investigations reported by the firm in the five months prior to the agency inspection. FDA reported that the firm conducted 158 investigations in this time period, 82 of which focused on test results from the stability laboratory and 76 from the quality control lab. Of these OOS investigations, 70% or 111 of 158 were related to product potency and/or quality. The FDA investigators were particularly con-cerned that 114 erroneous finished product test results
were dismissed by Mikart due to analytical error or other lab mistake. ä Several factors led the FDA team to question the reliability of Mikart's conclusions. These included numerous test results which showed that product samples had either high or low assay, dissolution, content uniformity, or pH. Further, there were repeat failures for the same analytical test on the same lot, and same-test failures on consecutive lots of the same product. Other concerns raised by FDA included finished products which failed multiple tests and the firm's practice of discarding samples before analytical test results were obtained. FDA Wants Support For "Analyst Error" Conclusions According to their "establishment investigation report" (EIR) on the inspection, the FDA investigators selected 16 of the Mikart failure investigations for more careful review and found that five were in violation of the cGMPs. The EIR maintains that, in one case, the attribution of a failing test result to "apparent" analyst error was not supported by the analyst's notebook. Furthermore, the original dissolution sample was not analyzed to "confirm the firm's hypotheses of a sample preparation error." The investigators also reported that the use of the word "apparent" in describing the cause of the aberrant test result indicated that "the analyst was speculating about the cause of the initial result." A second failure investigation was judged deficient because Mikart did not retest a sample which failed content uniformity tests. The OOS result was attributed to a standard preparation error. However, the FDA report notes that this was never confirmed by retesting the sample. Again, the report points out that the firm used the word "appears" in describing the cause of the aberrant result. FDA found that Mikart also speculated on the possible cause of a failing stability test result. According to FDA, Mikart concluded following an investigation that there was a "possible incomplete release of analytes from the tablet powder." As in the other cases examined, the firm failed to retain the sample to test its investigation hypothesis. The high number of failing results attributed to laboratory error also led FDA to question the capabilities of Mikart's analysts. FDA noted that a procedure should have been established to track errors in order to identify which analysts had the most problems. Another issue raised by the FDA investigators was that Mikart often began its failure investigations more than 30 days after receiving aberrant results. The investigators noted that the agency expected such investigations to be completed within 30 days. ä The corrective plan outlined in Mikart's response to the inspectional findings was found to be acceptable by FDA. In the response, the firm committed to conducting OOS investiga-tions within 30 days of the discrepant analytical result. Indicative of the problems firms face in interpreting current FDA expectations, Mikart explained that the unusually high number of failure investigations within the five-month period was the result of the firm's practice of reviewing analytical results at the upper and lower limits of the specifications. To avoid implementing "parallel systems for handling laboratory situations," Mikart documented these reviews as failure investigations. The company affirmed in its response that it will rely on other systems to monitor such results in order to better document the number of actual OOS investigations. Mikart also maintained that the number of analyst errors needed to be viewed in terms of the large workload performed in its analytical laboratories. The firm contended that there was only a "3.5% error rate as compared to the number of samples received." Sample Retention Issue Raised At Mikart Although the firm initially challenged FDA's assertion that analytical samples needed to be retained until test results were obtained, Mikart agreed to do so in order to substantiate future OOS investigative conclusions. ä According to the EIR, Mikart originally argued, based on discussions with other firms and industry lawyers, that the retention of laboratory samples is not a standard industry practice. In discussing the matter with the agency investi-gators, Mikart also maintained that the labor-atory workload and equipment limitations prevented analysts from retaining samples. When asked if sample retention was an established FDA policy, the investigators informed the firm that it is not a specific GMP requirement. However, the FDAers maintained that retesting of samples would have helped Mikart confirm its stance that analyst error caused out-of-specification results. Mikart stated in its response that the laboratory had instituted a policy for the retention of "an unfiltered portion of the sample removed from the dissolution vessels" and "standard stock preparations" until results are calculated. The new laboratory procedure required Mikart to purchase additional refrigerators and glassware. The firm recognized in the response that the retesting does not supplant the original data, but rather is used strictly to support the failure investigation. ä Mikart also noted that it would upgrade its quality control and stability investigation logs from a manual to a computerized system in order to trend the performance of analysts, tests, and equipment. A follow-up inspection in May 1997 confirmed that the company had taken adequate corrective action. During the summer, two Mikart abbreviated new drug applications were approved by CDER. Contract Lab Reporting Of OOS Results Cited Out-of-specification result handling was also a problem for two analytical laboratories in 1997. Metuchen Analytical and Consumer Product Testing both were cited for not notifying customers of failing analytical results. According to the FDA warning letter, Metuchen modified analytical methods after obtaining OOS results and reported the resulting in-specification values to their customers. For example, for propylene glycol, the firm obtained percent water content results of greater than the .2% specification with the unmodified analytical procedure. After modifying the test method, the results fell within the specification. However, FDA noted that "the laboratory report supplied to the customer showed that none" of the OOS results were conveyed. ä Consumer Product Testing was cited for neglecting to report failing assay values for verapamil HCl to its client. FDA asserted that the firm did not conduct an investigation into the OOS results. For prednisone tablets, the firm reported to its client the average of an OOS assay value and an in-specification value. The warning letter also pointed to the firm's practice of averaging failing and good loss-on-drying values for pectin without investigating the cause of the OOS result. FDA noted in the warning letter to Metuchen that the firm's corrective action plans were adequate. In the letter to Consumer Product Testing, on the other hand, FDA stated that the company's July 22 response letter to "many" of the observations listed on the FD-483 were "unsatisfactory" and requested that additional follow-up action be taken. Stability Programs Faulted On 40% Of FY 1997 Letters The agency's continuing focus on the stability testing component of laboratory operations is exemplified by the number of warning letters which reference problems in the area. Deviations in stability programs appeared in more than a third, or 40 of the 101 warning letters issued during the fiscal year. Stability-related citations appeared slightly less frequently in the FY 1997 warning letters than in the previous year when the problem area was highlighted on 44 of 90 warning letters. However, the types of stability-related problems reported are consistent with those found in FY 1996 ("The Gold Sheet" February 1997). ä Stability programs have been a target during inspections of topical producers in particular. As in FY 1996, stability-related deficiencies were noted in over half of the warning letters issued to topical firms last year. Oral solid and oral liquid manu-facturers were also frequently cited for deficient stability testing practices. Overall, about a third of the manufacturers cited for stability-related problems (13 of 40) failed to conduct stability tests on some or all of their products. Included in this group were three repackaging firms, Diversified Healthcare Services, Columbus Chemical Industries, and Patient First Corporation. In other cases, FDA pointed to a firm's failure to conduct stability testing at the required time intervals. For example, Vision Pharmaceuticals was cited for not analyzing stability at the required intervals for the active ingredient for sterile eye drops. ICN Pharmaceuticals also failed to conduct stability tests at the appropriate intervals, FDA alleged. In a May 1997 letter to Vivus, FDA maintained that the firm was negligent in running stability tests at three month test points for several lots of its alprostadil suppositories. The firm's stability program also lacked a written protocol for products manufactured and marketed in September 1996. A stability protocol for February 1997 failed to specify which lots would be placed on stability, FDA reported. Mikart and Glenwood-Palisades were also cited for failing to conduct stability tests at the required test intervals. At Glenwood-Palisades, FDA noted in a May warning letter that "many stability stations have been missed since the firm moved" to its Piscataway location. FDA also maintained that Glenwood-Palisades' analytical stability tests were not shown to be stability-indicating. ä The failure to use stability-indicating methods or methods simulating storage conditions were noted on letters to Berlex Laboratories, Cosmetic Specialty Labs, and Tri-Med. Insufficient stability data to support expiration dates was a cGMP deviation at nine firms: Teva, Vintage, Roberts, SoloPak, Eight in One Pet Products, Eon Labs, Olan, Orion Life Systems, Propharma, and Wide River Chemical Company. Other stability-related problems cited in the FY 1997 warning letters include the failure to follow protocols or SOPs (Sybron, G&W, and Advanced Pharmaceuticals), the testing of an inadequate number of batches (Medco), and lack of stability monitoring of reprocessed lots (Alra). HPLC Testing Is Lab Problem Area While inspecting stability and other analytical laboratory functions, FDA investigators have been paying close attention to the performance of high pressure liquid chromatography. Nine of the 102 warning letters issued in FY 1997 specifically mention HPLC-related problems. In four of these cases, the HPLC concerns are linked with related deficiencies in the firm's stability program. ä For example, the warning letter to Tri-Med maintained that the firm had not shown analytical methods used for stability testing to be stability indicating. "No attempt is made to identify or evaluate degradation products during routine stability testing," FDA said, adding that "the HPLC recorder is turned off as soon as the analyte peak elutes." The letter also cites Tri-Med for neglecting to document in method validation reports chromatogram assay peaks "with shoulders and unresolved peaks" on release and stability for Tri-Vitamin drops with fluoride and Tri-Care syrup. Integration of HPLC peaks was found to be inconsistent in that the firm used computer integration for one lot of Tri-Tex liquid and a manual technique for another lot. The firm was also deficient in completing validation for all its analytical test methods and products, FDA said. ä In the letter to G&W, FDA noted that the firm used incomplete or inadequate HPLC methods for assay, release, and/or stability testing of its topical products. Maintenance and calibration of HPLC equipment was another problem refer-enced at G&W's facility. Following FDA's inspection, G&W recalled 20 lots of indomethacin suppositories for failing stability assays. FDA noted in the warning letter that the firm failed to conduct stability tests on the product during 1994. Two other products, floucinolone acetonide cream and phenylephrine HCl suppositories, were recalled by the firm in 1997 because of failing stability results. HPLC Suitability Needs To Be Established A failure to document that the liquid chromato-graphic methods used for stability testing are stability indicating for each active ingredient was also alleged against Catherx. FDA further asserted in the warning letter that the firm failed to document: "the standard injection used to determine the validity of the standard curve in the assay of finished products by liquid chromatography;" and "the review of the original laboratory records by a second responsible person for liquid chromatographic assays and IR spectra." ä Evaluation and identification of HPLC peaks found during stability testing of the biobatch and validation lots for albuterol inhalant was one of the problem areas found at Medisol Labs. Other warning letters, including those to Abbott and Scientech, pointed to a failure to properly test the suitability of HPLC methods for the analytical tests being run. Installation qualification for HPLC equipment was a problem area for Allied Laboratories. Neil Labs allegedly utilized an HPLC method which was not validated for chemical testing of several oral solid products. Irotec Labs, a foreign API supplier, was cited for several deficient HPLC practices. FDA noted in a May 1997 warning letter that the firm failed to report and review all chromatographic data. FDA questioned the firm's practice of discarding data that significantly deviated from the average. The agency noted that "this selective retention and reporting of data can raise doubts about the overall quality and reliability of the data reported." The letter emphasized that although some data may be scientifically invalid, "all analytical data must be retained." FDA also questioned whether Irotec's HPLC work on its drug substances stressed products enough to identify potential degradants. The agency com-mented in the letter that "the potential degradants have not been identified by exposing the drug substances to extreme conditions." The warning letter requested submission of corrected SOPs for the reporting of thin layer chromatography and HPLC-related test results for release and stability lots. ä Current issues regarding HPLC validation were discussed by FDA St. Louis Lab officials William Furman and Thomas Layloff at a workshop held during the Fall 1997 PDA/FDA conference. Furman pointed to a lack of consensus regarding the point at which an "adjustment" to an HPLC procedure becomes a "modification" requiring a revali-dation of the system. He explained that he has been working on a guidance document which might be appropriate to publish in USP's Pharmacopeial Forum for comment that would help define allowable limits on HPLC adjustments. FDA will expand its guidance in the laboratory area with the release of an updated stability guideline. Clearance of the new stability guideline has been held up due to concern about infringing upon the guideline drafting process at ICH, which may include a rewrite of Q1A, "Stability Testing of New Drug Substances and Products" ("The Gold Sheet" September 1997). However, FDA is sensitive to the need for updated guidance in view of the number of problems it is finding with current programs. Process & Cleaning Validation Remain In Focus The fiscal year 1997 warning letters show that firms continue to struggle with validating manufac-turing processes. Out of the 101 warning letters issued in the period, 35 referenced problems with process validation. In most cases, firms were cited for failing to validate one or multiple manufacturing processes. ä Cleaning validation also continues to be a recurrent problem area - 16 firms were cited for such problems in FY 1997. Cleaning validation was a significant problem for bulk firms and, in particular, foreign bulk manu-facturers. Almost half (five of 11) of the companies cited for deficiencies in their cleaning validation programs were bulk manufacturers. Of the seven foreign letters sent to API producers, four failed to conduct cleaning validation. ä The proper handling of labeling and packaging has been another area of FDA inspection concern, and this focus is likely to continue in view of a resurgence of labeling-related recalls in 1997 ("The Gold Sheet" January 1998). The FY 1997 warning letters show that investigators are continuing to find quite a few violations with packaging and labeling handling procedures. Eleven of the 101 warning letters issued in the period contained citations regarding labeling controls. Of the firms receiving these letters, two were repackagers - Amerisource and Alphagen. Both firms also initiated drug product recalls last year. FDA cited Alphagen for multiple labeling control deficiencies. According to the warning letter, the firm could provide no documented proof that labeling had been examined upon receipt or at any point prior to release. There was no inventory of labels used on drug products, nor were there records available for the purpose of conducting labeling reconciliation. Alphagen recalled guaifenesin capsules during the FDA inspection due to mislabeling. The firm reported that some of the bottles may mistakenly have contained pancreatic enzyme capsules. Blister Packaging In Focus At AmeriSource AmeriSource (American Health Packaging) conducted a series of product recalls in 1997 involving blister packaging ("The Gold Sheet" January 1998). Altogether, the company initiated recalls for six products repackaged at its plant. In the warning letter to the firm, FDA contended that the controls present to prevent labeling mix-ups were not adequate. As examples, the agency noted that: n n there was no controlled access to labels and inserts; n n inserts were not physically separated during storage; n n approved inserts/outserts were not identified in master production records; and n n master production records included two different labels as the approved labels. For its blister packaging lines, the warning letter asserts, AmeriSource did not establish written master production and control records. FDA also found the firm's employee training program to be inadequate. Apothecary Products was also cited in a warning letter for various packaging/labeling-related deficien-cies. Prior to the June inspection that prompted the letter, the firm recalled acetaminophen and acetamino-phen/pseudoephendrine HCl tablets after finding the wrong product identifications on convenience packs. ä Matching a significant uptick in recalls for the past year, injectable firms also received more warning letters than in FY 1996; 14 injectable firms received letters last year compared to five in FY 1996. The recurrence of recalls over the past five years stemming from pseudomonas and other forms of microbial contamination has prompted CDER's compliance office to alert field investigators to focus on water systems during inspections ("The Gold Sheet" January 1998). In turn, water system citations have been turning up fairly frequently in the warning letters issued by the field staff. In FY 1997, for example, 12 of 101 warning letters reference practices regarding the validation, control, and testing of water systems. ä Half of these 12 letters referencing cite the failure firms' failure to validate their deionized or purified water systems (PrimaPharm, Nippon Rikagkuyakuhin, DermaRite, Nephron, R.J.S., and Rhone-Poulenc Rorer). The International Society for Pharmaceutical Engineering, in conjunction with FDA, has issued for comment a draft of a "Water and Steam Guide" - one of a series of 12 "Baseline Pharmaceutical Engineering Guides," which will be forthcoming. The ISPE guide includes a chapter on microbial control of water systems.
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||