A Rebuttal Explaining the PTAB Invalidation Rate Fallacy in the GAO Reports

This is a response to Robert Grantham’s recent post titled “Patents Subjected to IPRs are the Perfect Vehicle to Assess the USPTO’s Patent Quality Problem.” This post claims to challenge my recent article titled “Fallacy Dispelled: Invalidation Rates of Adjudicated Patents Convey Nothing About Quality of All Issued Patents.” However, the post does not challenge my article, but instead falsely knocks down a strawman the post invented by mischaracterizing my article.

My request from the Government Accountability Office (GAO) was to correct its Reports that erroneously assert that the Patent Trial and Appeal Board’s (PTAB’s) high patent invalidation rates (percentages) reflect the same high invalidity rates (other percentages) of all issued patents. The post acknowledges my request but asserts that “Katznelson contends there is no relationship between PTAB patent invalidation and overall examination quality.” That statement is a plain falsehood. By deliberately omitting the critical term “rates,” the post assertion above mischaracterizes my article as if it made a categorical statement rather than a specific contention limited to the numerical comparison of the “invalidation rates” at the PTAB and the invalidity rates of all issued patents. Contrary to the post’s assertion, my article never denied that “invalidation” at the PTAB is reflective of examination quality– it reflects the poor examination quality of about half the patents selected for adjudication at the PTAB. However, the post deliberately ignores my critical point that the GAO reports’ equating the numerically high invalidation rate at the PTAB with the numerical invalidity rate underlying all issued patents is the fallacy that my article explains. Contrary to the post’s allegation, my article does not call a fallacy the simple notion that examination quality relates to PTAB invalidation generally.

Nevertheless, the post further dissembles by stating: “Contrary to being a fallacy, examination quality is the leading contributor (93% of the time) for Final Written Decisions (FWDs) against the patent holder,” and further stating that “[e]xamination quality is indisputably a controlling factor, not a fallacy, in PTAB invalidations.” Nowhere in the post can one find any reference to the numerical discrepancy contention of the two categories of invalidity rates. The post contains neither any reference, nor a challenge to the applicability of the underlying Priest-Klein selection effect, nor any reference to the fact that about 50% plaintiff win rates is observed not only in patent cases but more broadly in all civil cases selected for adjudication, as shown in Table 1 of my request for GAO correction.

The post shows a profound misapprehension of the implications of selection of patents and prior art references for adjudication. Despite detailed explanations in my article, the post treats “facts” found in cases selected for adjudication as if they are the same “facts” for all issued patents. The post states: “It is apparently quite easy to dismiss the 84% PTAB kill rate as not having anything to do with the actual evidence, since that has been precisely the case ….” “A far more telling statistic,” the post continues, “is ‘…in fact, 93% of the grounds studied relied on at least one prior art reference that was not before the examiner during prosecution.’” The post thus propounds: “How is it possible that within a set of ‘specially selected’ valuable patents critical evidence is missed 93% of the time?”

First, my article does not dismiss the high “kill rate” at the PTAB — it explains it as applicable only for those close call cases selected for adjudication — not for all issued patents. Second, the fact that 93% of the grounds for PTAB invalidation relied on prior art references that were not before the examiner during prosecution, is because these references were selected after the fact by the patent challenger precisely because they were not considered by the examiner in prosecution. In most patent applications, the number of relevant prior art references is staggering (e.g. see U.S. Pat. No. 7,292,835 with 856 listed references), but most references are normally not cited. It is well-known that prior art references that were not considered by the examiner are specifically sought after and identified by patent challengers for adjudication because they increase the likelihood of IPR institution and reduce the burden of overcoming the presumption of patent validity. Transonic Sys. v. Non-Invasive Med. Techs. Corp., 75 Fed. Appx. 765, 783 (Fed. Cir. 2003) (“There is an increased burden for proving invalidity when an Examiner considered the asserted references during prosecution.”) Here, too, the post is blind to this selection effect. In many cases where such new references are identified, they are simply cumulative to those the examiner has cited in prosecution, and where the result would have been the same had the examiner considered the reference.

Alternative Illustration of Reason the High Invalidation Rates of Adjudicated Patents Convey No Information on the Underlying Validity of All Issued Patents

Grantham, the post’s author, clearly missed the reason for the fallacy on invalidity rates that my article exposed. At the risk of some repetition, I now provide an alternative presentation by combining two separate figures of my request for correction from the GAO into one figure provided here for easier comprehension.

Two assumed distinct scenarios of different underlying issued patent validity rates are depicted in part (a) of the figure. The first scenario assuming that 6% of issued patents are likely invalid is depicted by the function Q(x) shown in broken line. The second scenario assuming that about one in three (33%) of issued patents are likely invalid is depicted by the distribution of Q₁(x) in solid line in part (a) of the figure. Both plots are of the ordered distributions of the prior probability that patents in the respective population percentile x (horizontal axis) are valid. Prior uncertainty of the validity of some patents in the populations is shown by the gray zones labeled “close calls,” which are expanded horizontally for both plots in the view of part (b). However, note that although the “close calls” zones of Q(x) and Q₁(x) arise in different percentile regions of the patent population, Q(x) and Q₁(x) in part (b) rise and cross the 50% probability point in a similar way. Neither the fact that only 6% of the underlying patent population in the first scenario, nor 33% in the second scenario are likely invalid, changes the result that adjudicated patents are selected only from the “close calls” zones where Q(x) and Q₁(x) are each near their 50% point.

For each scenario, the holder of the patent and its challenger will each form their own opinion and respective perceived probability of their success. Under the Priest-Klein theory, the divergence of their respective expectations of the outcome would normally arise only in uncertain cases in the narrow “close calls” zones, which cases may be selected for litigation.

Part (b) of the figure schematically plots two additional probability curves, showing the probability of success in prospective litigation as perceived by the patent challenger and by the patent holder respectively. The perceived probabilities may slightly vary based on differing specific information available to the parties; however, the plots capture the overall general trend with respect to the varying magnitude of the prior probability of validity in Q(x) and Q₁(x). In cases below and above the uncertainty gray zone, there would be no appreciable divergence of expectations of the parties as to the outcome of litigation were it to proceed, and they will avoid either asserting or challenging the patent, or otherwise settle. Therefore, the validity of the patents in both such cases and their numbers will never be reflected in adjudications, which are relatively extremely rare. According to FY-2024 PTAB statistics, only 409 patents were adjudicated during that year; the rest were not instituted or were settled. This constitutes only 0.012% of all 3.5 million U.S. patents in force.

In conclusion, in both scenarios depicted by Q(x) and Q₁(x), the patent invalidation rate in adjudication will be similarly high (around the 50% point). However, that high rate essentially conveys no information about the relative fraction of all issued patent percentiles below the “close calls” gray zone that are likely invalid, compared to those above this zone that are likely valid. Simply stated, patents selected for adjudication are specifically from the transition zone between two distinct patent quality levels in populations of unknown relative size. The GAO Reports erroneously infer their relative size from adjudication outcomes for patents in the uncertainty transition zone. This is a fatal error that dooms these conclusions to the dustbin.

Contrary to the post’s final pronouncement, incoming USPTO Director John Squires misspoke when essentially echoing the GAO Reports’ fallacy. During his confirmation hearing he said that patents in “IPRs have a 68% defect rate; if the American patent system was a factory, 68% of the products we put out are found defective in a later proceeding.” Fortunately, Mr. Squires later in his written response to Questions for the Record corrected his earlier verbal statement, and explained that “[t]he statistics I mentioned are those published by the USPTO concerning claim cancellation upon challenge at the PTAB which are a small subset of all issued patents, not a measure of quality at the front end” (emphasis added).

Are Patents Adjudicated in IPRs the ‘Perfect Vehicle’ to Assess USPTO Examination Quality Problems?

The Post’s title declares that patents subjected to IPRs constitute the “perfect vehicle,” i.e., the best sample, to assess examination quality deficiencies. The post explains that “[t]he fact that patents subjected to an [IPR] are ‘specially selected’ is the very reason that this class of patents are a suitable vehicle for studying the cited evidence.” This latter pronouncement further manifests its author’s apparent misapprehension of the nature of this “special selection.”

As explained above, the reality is that this “special selection” of patents adjudicated in IPRs is of the “close calls” cases for which there was substantial prior uncertainty as to patentability. That is, substantial uncertainty as to whether the examiner failed in making the final determination — the parties could not agree on that issue. Moreover, that uncertainty as to whether the examiner erred is manifested throughout even after the PTAB decision, as the Federal Circuit reverses or remands about 22% of PTAB decisions in IPRs.

Therefore, while evaluating those “close calls” selected patents for adjudication can be informative, a significant fraction of that sample from the uncertainty zone contains patents correctly issued through no examination error. Therefore, it is not the most productive sample that fully and fairly represents actual examination failures. Instead, focusing the limited USPTO quality assurance resources on other patents for which there is no material uncertainty as to examination failures is preferable as more productive. Such are cases where the opposing parties have both implicitly agreed that the patents contain likely unpatentable issued claims. These are patents at the lower edge of the uncertainty zone, which include PTAB cases that were settled, or where the patent owner disclaimed an issued patent claim under 35 U.S.C. § 253(a), or amended an issued patent claim under 35 U.S.C. § 316(d) prior to the PTAB decision.

Points in the Post with Which I Agree

The post makes important points on examination quality with which I fully agree. Based on my experience, I agree with the post’s contention that examiners overly rely on keyword searches rather than structured methodical strategies to find the most relevant art, and do not sufficiently search the non-patent literature. It observes that the Office of Patent Quality Assurance (OPQA) does not “review or grade an examiner’s search strategy,” an observation consistent with the lack of evidence that such is included in OPQA’s procedures. Finally, as to OPQA’s name, it has been my contention at least since my 2010 detailed comments on examination quality and my 2019 statement before the Senate Subcommittee on IP, that the USPTO should stop using the term “patent quality,” neither in OPQA’s name, nor in its mission. This is because erroneously rejected patent applications are not patents and a patent must have been issued for evaluating its quality. This term is strictly a measure of allowance errors. The term that should be used instead is “examination quality” because it is unbiased between allowance and rejection errors and because it correctly identifies the problem: examination — not patents. Accordingly, to send the message removing this bias, the USPTO should change OPQA’s name to “Examination Quality Assurance Office” (EQAO), a proposal I made publicly to then-USPTO Director Michelle Lee, who sent the wrong message by ignoring it.

Image Source: Deposit Photos
Image ID:158530450
Copyright:iqoncept