Finding the Best Patents – Forward Citation Analysis Still Wins

best-stars-businessman“We would like you to find the best patents in this pile of 50,000 candidates. Oh, and we need it done for $30K.” We hear requests like this so often we’ve built processes and tools to help us address them.  Our team has over 60 years of experience developing, evaluating, monetizing, litigating, and licensing patents; we’d like to share some of our experience and methodology with you.

Let’s return to that pile of 50,000 patents – how can we find the highest quality patents reliably and efficiently?

We’ve identified five primary factors for consideration in patent ranking (in order of weighting):

  1. Forward citations (45%)
  2. Age of patent from priority date (19%)
  3. Independent claim count (adjusted by number of means claims) (14%)
  4. Claim 1 word count (12%)
  5. Family size and international filings (10%)

We were surprised to discover that forward citations dominate the analysis. We evaluated millions of patents – and consistently forward citations were the biggest predictor of a higher value patent. More on this below.

What Ranking Tools Can Do

When we start projects sorting through thousands of patents, we first meet with the client to define success. In other words, what does it mean for the client to be successful in a project like this? Finding the best patents? Eliminating the worst patents?

Ultimately, we want to find patents that meet the clients’ needs. This means quickly eliminating 95% of the patents that are less likely to fit those needs. For this, we built a tool and ranking system that helps us identify the patents that are both most and least likely to be useful. With this smaller pool, we can start human review looking for the patents the client seeks.

ROL Group’s 2016 Patent Ranking System

Where do you look for benchmarks of better patents? We believe that patents that are bought and sold better reflect patents more likely to meet business needs. We started with our database tracking over $7B of patents that companies are trying to sell, or have sold.  We also looked at the characteristics of patents that had been litigated.

Our requirements for a ranking system:

  • Ranking system must be fully transparent – all aspects and formulas available to both ourselves and our clients for review, discussion, and per-project adjustments. Most existing commercial systems hide their ranking systems and are therefore precluded
  • Factors based in data and intuitively explained – each factor we use should be based in data but also intuitively explainable to clients

In developing the new ranking system, we began with our previous heuristics-based system and tested the existing factors and others against our patent deals database as well as data sets of litigated patents. The new ranking factors were determined based on simulations comparing different potential weights.

Forward Citation

We found that forward citations (later patents that cite the subject patent) were the most significant factor in identifying patents that were likely to be purchased. In fact, the patents that were sold—or even highlighted by brokers, e.g. the representative patent—in a brokered patent package exhibited an even more extreme number of forward citations than litigated patents.

Why forward citations? Why not claim length or any number of other factors? We believe that forward citations are a proxy for industry-wide R&D investment in a technology area. With more investment, there are generally more products. With more products, there is a higher chance of infringement. Infringement drives value and most likely meets a client’s needs. Specifically, a purchase either eliminates the client’s own infringement or provides a tool to use against someone else).

Our analysis focused on looking at forward citation counts for four primary sets of patents: (i) a set of all issued patents from 2005-2014, (ii) a set of litigated patents from the same period, (iii) a set of patents from the brokered market that were sold from 2009-2014, and (iv) the representative patents from brokered patent packages. The results were striking; the sold (set iii) and representative (set iv; e.g. the patent highlighted by brokers in packages) patents had exponentially more forward citations than the broad set of issued patents (set i).

Because there was evidence of significantly higher forward citations in the set of litigated patents (set ii) compared to the broad set of issued patents (set i), we decided to use the forward citations counts deltas between litigated patents and issued patents to set our ranking metric.

Turning to the chart, the light green line shows the forward citation count by years from publication date for litigated patents (set ii). The dark green line shows the forward citation count by years from publication date for the broader set of issued US patents (set i). As is evident in the first three years, there is minimal difference between the two data sets, but then a clear gap emerges.


For patents more than three years from the publication date, we identified four regions for ranking adjustments:

  • Region A: The patent being ranked massively exceeds the number of expected citations for a litigated patent (rank = 1)
  • Region B: The patent being ranked has more citations than expected for a litigated patent, it is defined to be the same size as region C
  • Region C: The patent being ranked has more citations than expected for a typical patent, but not more than a litigated patent
  • Region D: The patent being ranked has fewer citations than expected for a typical patent

Age of Patent from Priority Date

We know that our clients are generally looking to purchase patents that are actively adopted and in use in industry, but also are looking for sufficient remaining life to get the benefit of their purchase. For example, if a client is buying for a potential dispute that has not yet materialized, at least five years of remaining life is generally desirable.

From our time at Rambus, we know that patents in the range of 8-12 years from priority had the highest probability of being valuable in licensing. There is, additionally, a wealth of academic research on the timing of litigation vs. remaining life of patents. See, e.g. Brian Love, “An Empirical Study of Patent Litigation Timing” Univ. of Penn. Law Review, Vol 161, p 1309 (2013). As well as work by Mark Lemley together with John Allison and David Schwartz, “Understanding the Realities of Modern Patent Litigation”, 92 Texas L. Rev. 1769 (2014). Additionally, as seen in our prior article on Intellectual Venture’s (IV’s) patent portfolio and our forthcoming article (IAM Magazine Issue 77), IV’s purchase windows overlap heavily with the ranges we model as well.

We used the information from those papers as well as our experience to model this factor:



TO BE CONTINUED: Up next I will discuss the remaining factors in some detail, and explain why they are somewhat less helpful in determining ranking priority.


Warning & Disclaimer: The pages, articles and comments on do not constitute legal advice, nor do they create any attorney-client relationship. The articles published express the personal opinion and views of the author as of the time of publication and should not be attributed to the author’s employer, clients or the sponsors of Read more.

Join the Discussion

3 comments so far.

  • [Avatar for Erik Oliver]
    Erik Oliver
    March 30, 2016 03:10 pm

    @lou – Thanks for your thoughtful comments, evidence of use is ultimately the gold standard. The issue here is how do you find those in a big pile? You’ve highlighted a number of refinements that could all potentially improve the quality of the algorithm.

    @christopher – Thanks for the additional literature pointers. We are not claiming causation, we are looking for tools to quickly identify where to look in the stack more carefully. We are not using the citation count for valuation.

  • [Avatar for Louis Carbonneau]
    Louis Carbonneau
    March 28, 2016 01:17 pm

    As a broker who has sold over 2000 patents, I will tell anyone that by far the most important factor to predict whether a patent will sell at all, let alone for good value, is whether it is infringed or not. If no one is (actually or imminently) practicing the patent, then no one needs the rights associated therewith. A high citation count, while it certainly allows to quickly zoom on patents in a large portfolio that indicate the potential for a higher value, don’t tell the whole story and again, this only works if a given patent ends up being infringed.
    Second, you do have to exclude self citations as they often skew the picture. Third, most people ignore that there are usually more forward citations on the published application than on the ensuing patent and that you have to add both together to show the whole picture. Finally, the patent in any given family who has the most citations (almost always the first one) is rarely the most valuable one. You usually get your best claim language allowed on your second or third attempt with the USPTO and often take what the examiner is willing to give you the first time in order to get a first an allowance and then file a continuation to fight for the claims that you wanted to have in the first place.
    So in short, there is no silver bullet with forward citations. You have to look at those in a context.

  • [Avatar for Christopher Eusebi]
    Christopher Eusebi
    March 25, 2016 10:24 am

    There has been much analysis of bibliographic data on emerging technologies—in particular, many different methods have been used to analyze patent citations. A recent analysis of patent citations demonstrates that the relationship between patent value and citations is non-linear and not monotonic.
    Abrams, David S., Ufuk Akcigit and Jillian Popadak, “Understanding the Link between Patent Value and Citations: Creative Destruction or Defensive Disruption?,” University of Pennsylvania (2013), available online as of May 23, 2014 at:

    What does this mean? You cannot use patent citation for valuation – period.

    As practitioners – we all know that the Examiner’s all have their favorite references that are used not because of their very early date or innovativeness– but because of their encyclopedic nature. We also know attorneys who intentionally cite their client’s patents and applications to game this analysis

    Further, in non-academic literature see the Washington post article.

    While there are factors which can be used for valuation; the statistics clearly show citation analysis is not one.

    Let’s all remember correlation is not causation.

Varsity Sponsors

IPWatchdog Events

PTAB Masters™ 2024
January 29, 2024 @ 1:00 pm - January 31, 2024 @ 2:00 pm EST
Artificial Intelligence Masters™ 2024
March 4, 2024 @ 1:00 pm - March 6, 2024 @ 2:00 pm EST
Trade Secrets Masters™ 2024
April 8, 2024 @ 1:00 pm - April 10, 2024 @ 2:00 pm EDT
Patent Litigation Masters™ 2024
May 13, 2024 @ 1:00 pm - May 15, 2024 @ 2:00 pm EDT
Patent Prosecution & Portfolio Management Masters™ 2024
June 24, 2024 @ 1:00 pm - June 26, 2024 @ 2:00 pm EDT

From IPWatchdog