Finding the Best Patents – Comparative Patent Ranking Systems – Citations Still Matter

RankingOver the past 18 months, our clients have begun to show greater interest in international patents (e.g. non-US). Increasing client interest in international patents corresponds with the general rise in importance of international patents (continuing ascension of the Chinese market, potential for unitary patent for Europe), more anti-patent owner decisions in the US, and greater patent litigation outside the US. Client interest spurred us to review our existing patent evaluation tools and methodologies for potential improvements. We found that our current ranking system can be used for EP patents, especially with further calibration using EP data sets, and also that an integrated ranking approach that allows comparison and ranking of EP and US patents relative to one another is possible.

Identifying the most pertinent patents in a list of thousands can be tedious and difficult. An automated, first-order process for ranking patents lightens the load considerably. Patent ranking tools are designed to identify the patents both most and least likely to be useful. Our existing ranking system (Part1Part2) is uniquely focused on the brokered patent market. We developed a refined ranking system focused on US patents, based on ROL Group’s 60-plus years of experience developing, monetizing and evaluating patents, and analysis of characteristics of our private data set of over $7B of patents that companies are trying to sell, or have sold. Since open-sourcing our current ranking approach, we have seen additional adoption of our ranking system, e.g. by AQUA Licensing LLC, a patent broker, who is using the ranking to improve its evaluation of packages and the information shared with prospective purchasers.

To best advise our clients, we wanted to explore if, and how, patent ranking for EP patents has been conducted by others and how similar, or different, those ranking systems are. We conducted a literature survey to identify ranking systems—particularly EU-centric ranking systems—to better understand the insights and contrasting views on this topic. Because value is context-specific, it is hard for researchers to develop “one right model,” but we saw tremendous convergence of ranking factors across the literature. Thus, it turns out that ranking systems for EP patents are quite similar to ROL Group’s approach to ranking US patents and that, once again, citation counts in various forms dominate the analysis.

A quick note on terminology: value is the context-specific utility of a patent to that given company at a specific time, and ranks are relative estimations of likely value across a number of common contexts. The language of the reviewed papers tends to use these terms somewhat synonymously, and, for concordance with the language of the paper in describing the ranking factors, we refer to the value correlations that the systems identified. The remainder of this post documents the literature survey we conducted and highlights the overlapping nature of the ranking systems.

How the study was conducted

Identifying relevant literature was done by searching for papers on patent ranking and reaching out to experienced individuals in the academic sphere. Once we started reading a few papers, we were able to identify a number of relevant articles quickly. Cross-references between the different papers was common, so it was clear that there was a close-knit body of scholarship on this subject. A full list of reviewed literature references can be found at the end.

Additionally, there is a number of commercial methods for patent ranking. Many of these companies often have their own methods. For example, in the US, Innography is well known for providing patent ranking, and their ranking system, branded as PatentStrength™, is based on 12 factors outlined in one of the articles we survey in this paper (Allison et al., 2003). Intriguingly, around 2014, Innography also released a new ranking tool, branded as CustomStrength™, which allowed customers to define their own custom ranking in which factors and weights are selected by the customer themselves. This would allow customers to build a custom ranking formula based on almost any of the approaches from the surveyed literature.

In total, we selected ten research papers to study, with an emphasis on systems that attempt to rank EP-patents. Two papers were dropped from detailed consideration because their research scope did not align with this topic upon deeper investigation. That left eight papers that were relevant to studying EP-ranking systems in our comparison. Of the eight, three papers were EU-centric and two were US-centric. The remaining three papers were country-agnostic but relevant for EP patents. The remainder of this post will compare and contrast the eight studied systems as well as ROL Group’s system.


A total of 22 different ranking factors were identified. Even though each of these 22 factors measures something unique, many were partially overlapping. We grouped the factors into the following seven categories:

Grouping # Systems # Factors Group Description Factors
Citations 8 8 Factors relating to citation counts, e.g. forward/backwards, as well as more sophisticated measures Patent Scope, Originality Index, Generality Index, Backward Citations, Backward Citations to Non-Patent Literature, Forward Citations, Breakthrough Invention, Radicalness Index
Family 8 3 Factors relating to family size, e.g. number of countries and additional prosecution through divisionals, continuations and similar practices Patent Family Size, Patent Jurisdiction, Families of Applications and Patents
Timing 5 3 Factors relating to time in prosecution and/or age of the patent Patent age, Grant Lag, Prosecution Length
Claims 4 4 Factors relating to the claims of the patent Length of First Claim, Independent Claims Count, Length of the Specification, Claims Count
Post-issuance proceedings 3 1 Factors relating to post-issuance proceedings, e.g. opposition Whether a patent has been upheld by opposition or not
Maintenance 3 1 Factors relating to maintenance fee payments Patent renewals
Owner 1 2 Factors relating to the original owner Size of original assignee, Domestic ownership

The different systems used between five and twelve of the 22 distinct factors. Interestingly, although litigation is often viewed as correlating with greater value—and is often used to calibrate ranking systems –no system used litigation as an explicit factor in any of the studied systems. One system used oppositions to boost rankings for patents that had undergone.

The citations category is the most common to patent ranking systems and is used by all eight studied systems. Citations in general, and forward citations in particular (later patents that cite the subject patent), were found in the papers to be positively correlated to the likely value of the patent, meaning that a high number of citations is correlated with a higher patent value. Also included in the citations category are a range of factors that measure the originality, generality, or radicalness of the patent, as well as the breadth of the patent specification scope (i.e. the number of IPC-codes the patent has listed). The citations category includes the highest number of distinct ranking factors (eight) and is one of two categories used by all of the systems studied.

The family category is also used by all eight ranking systems and, considering how much emphasis all different ranking systems put on it, seems to be of significant importance. However, ROL Group’s analysis of family size versus patents sold on the brokered patent market, as well as litigated patents, failed to identify this as a strong value indicator. Multiple papers consider this factor to be supportive in its nature, meaning that it is not of great importance itself but rather gains importance when combined with other factors, such as citations. The specific ranking factor for the total size of the patent family was the most frequently used factor of the family ranking factors.

The timing category is used by five of the eight ranking systems and is most commonly represented by the patent age factor (which measures the age of the patent from its priority date). The papers that mentioned specific timeframes for which patents were deemed more likely to be valuable agreed with ROL Group’s ranking system that patents see their economic value peak during 8-12 years after their priority date. Prosecution length is also categorized under timing and measures the time a prosecuted patent spends in prosecution, with the underlying reasoning being that the longer a patent spend in prosecution, the more valuable it is. However, the opposite seems to be true according to other systems, since shorter grants are also viewed as more valuable based on an assumption that assignees are more keen to speed up the patent application process for more valuable patents. The ROL Group did not look for correlations in prosecution duration in developing its ranking system.

The claims category is considered by half of the compared systems and is in made up of four factors, where the only factor used by multiple systems is the claims count. The claims count factor assigns patents with more claims higher value. Some argue that the number of independent claims, as well as the length (word count) of the first claim or the entire patent specification, is another measure of value. Interestingly, none of the systems analyzed the claims directly, e.g. natural language analysis and/or comparison against literature.

The remaining three categories are grouped under other because they were much less frequently used than the main categories discussed above. Whether a patent has been upheld in an opposition (or potentially other similar proceedings) is a binary factor which we categorized as post-issuance proceedings. Three of the eight ranking systems deem this to be a value-indicating factor, where upheld patents would be considered to have a higher economic value. Interestingly, Allison et al. (2003) found that patents that had been litigated successfully more than twice started to have a higher rate of failure.

The maintenance category also contains only one factor, patent renewal. Three ranking systems measure the number of patent renewals. The more times a patent was renewed, the more value was attributed to the patent.

The last category is owner, utilized by one of the eight ranking systems. It consists of two factors: size of original assignee and domestic ownership. The paper that advocates for these two factors is US-centric and proposes that more valuable patents tend to be issued to individuals or small companies rather than large companies and that patents with a foreign owner tend to be less valuable than patents with a domestic owner.


We were pleased to find that the known ranking approaches for EP patents do not differ significantly from ranking systems for US patents. We do not believe from the available literature that the differences in the ranking systems derive from differences in how to best determine a rank based on the national origin of the patent. This insight will enable us to tailor our system further for EP patents (e.g. by further calibrating with EP data sets). More generally, the insight supports an integrated ranking approach that allows comparison and ranking of EP and US patents relative to one another.


Aistemos,. CIPHER Briefing Patent Strength. 2014. Web:. 8 July 2016.

Allison, J.R., Lemley, M.A., Moore, K.A. and Trunkey, R.D., 2003. Valuable patents. Geo. Lj, 92, p.435.

Fischer, T. and Leidinger, J., 2014. Testing patent value indicators on directly observed patent value—An empirical analysis of Ocean Tomo patent auctions. Research Policy, 43(3), pp.519-529.

Giuri, P., Mariani, M., Brusoni, S., Crespi, G., Francoz, D., Gambardella, A., Garcia-Fontes, W., Geuna, A., Gonzales, R., Harhoff, D. and Hoisl, K., 2007. Inventors and invention processes in Europe: Results from the PatVal-EU survey. Research policy, 36(8), pp.1107-1127.

Harhoff, D., Scherer, F.M. and Vopel, K., 2003. Citations, family size, opposition and the value of patent rights. Research policy, 32(8), pp.1343-1363.

Measuring The Strength Of Patents | Innography. N.p., 2016. 8 July 2016.

Oliver, E., Richardson, K., and Costa, M. “Finding The Best Patents – Forward Citation Analysis Still Wins – Ipwatchdog.Com | Patents & Patent Law” Part 1Part2. | Patents & Patent Law. N.p., 2016. Web. 15 July 2016.

Squicciarini, M., Dernis, H. and Criscuolo, C., 2013. Measuring Patent Quality.


Warning & Disclaimer: The pages, articles and comments on do not constitute legal advice, nor do they create any attorney-client relationship. The articles published express the personal opinion and views of the author as of the time of publication and should not be attributed to the author’s employer, clients or the sponsors of

Join the Discussion

No comments yet.