“It is an exciting time to be an IP lawyer, or any other type of lawyer that works in these areas. These technologies will likely touch every aspect of our lives and generate what may seem like an endless list of legal, social, and ethical challenges.”
Augmented Reality (“AR”), along with Virtual Reality (“VR”), is rapidly growing in prominence and will be transformative to the way we live, work, learn and play. Both AR and VR will undoubtedly bring a whole set of novel IP issues for individuals, companies, IP practitioners and the courts. Like any new technological area, such as cyber law for the nascent internet technology in the early 1990s, many legal issues need to be addressed and many more are yet to be discovered as this area evolves.
AR and VR are sometimes used in overlapping ways, but each presents unique IP law issues. AR keeps the real world as the backdrop and enhances it with digital details, layering new strata of perception and supplementing one’s reality or environment with additional material. The digital details could be anything that may be perceived by the five senses: touch, sound, taste, scent or vision. The most common and most developed AR is visual information. Unlike AR, VR is a way to generate realistic images, sounds and other sensations that put you smack in the middle of an imaginary world. VR is a new world built from the ground up. VR is the primary technology for the metaverse, which will be comprised of virtual worlds.
Relatively early and familiar examples of AR can be found in televised sports, where the first down marker in a football game is shown as a yellow virtual line across the field, the hockey puck is highlighted with a virtual blue trail, or a virtual box is drawn next to the batter during a baseball game to illustrate the strike zone. Another familiar example is the video game Pokemon Go, which created a public safety nightmare as gamers went searching high and low for different Pokemon characters while walking and crossing busy city streets with their heads buried in their smartphones. More recent applications for AR run across almost all industries, and potential applications are as unlimited as the human imagination. Examples include:
- Healthcare—AR can assist doctors in performing surgeries by showing them the exact positioning of an implant or incision in 3D, which is especially useful if the area is difficult to see or visualize. AR surgical simulators can be used to train doctors.
- Travel—AR can enhance the tourist experience. It could bring a museum or a historical landmark to life by allowing the user to see what the exhibit, landmark or historical site may have looked like hundreds or thousands of years ago. Imagine visiting the Roman Colosseum with the stands full of Roman citizens and a virtual representation of a gladiator fight on the Colosseum floor.
- Real Estate/Interior Design—Realtors and homebuilders could use AR to show in 3D what a furnished house would look like as a buyer walks through the empty or partially built structure. Interior designers could use AR to show you exactly what your restyled room or bathroom remodel will look like when completed, again in 3D as the redesign is overlaid on the existing room.
- Clothing/Fashion—Apparel, jewelry and other accessories could be tried on virtually with product information and matching outfit proposals.
- Education and Sports—AR could be used to train workers, teach students, and coach athletes. Workers could also use AR to ensure they are doing their job safely and efficiently.
- Entertainment—Entertainment giant Disney, for example, has fully embraced AR. Its parks use AR on many of its rides and attractions. The new “Secret Life of Pets” ride allows riders to see themselves as pets on video screens. The pets on the screens mimic the hand gestures and facial expressions of the riders. Disney has also developed AR technology that could let visitors wear AR costumes of their favorite characters for souvenir snap shots.
- Gaming—The gaming industry is the leader in the number of experiences and AR related patents likely to issue.
With rapid innovation and commercial use of AR technology, there seems to be an almost endless supply of potential IP issues. Some of the IP issues that have already presented themselves or will likely present themselves in the near future are discussed below.
Filing patents for AR is similar to other technologies, but there are important differences regarding user interfaces, namely the AR hardware and motion-tracking technologies required to go from 2D to 3D. Tens of thousands of U.S. patent applications have been filed and published relating to AR and VR headsets alone. The major filers include familiar names: Microsoft, Intel, Meta, Samsung, Google, LG, and Sony. There are also smaller, lesser-known players who are major patent filers, such as Magic Leap, a company specializing in bringing AR and VR to the world. In addition to the increased filings of patent applications for the hardware components for AR, one of the major trends is the rise of method claims for the use of AR for practical applications, such as some of those listed above.
Patent litigation has already commenced for AR hardware and certain applications or uses of AR. These cases are being filed by practicing entities and non-practicing entities (“NPEs”) alike. On the hardware side, HTC’s Vive headset has been a subject of patent infringement by an NPE. ESP, Inc. v. HTC Corp., Case No. 3:17-cv-05806 (N.D. Cal.). ESP asserted three patents against HTC pertaining to the measuring of 3-D pose and orientation using on-board photodetectors and stationary light sources for virtual reality applications. HTC tried to dismiss the case on grounds that the patents were directed to ineligible subject matter under Section 101. HTC argued that the claims were directed to an abstract process of using objects as tools to determine one’s position, as sailors have done by looking at stars for centuries. The court said it was a close call, but found that ESP’s patents passed the Alice test, which will likely be a central issue for litigation involving AR patents. The claims passed Step 1 of the Alice test because the photodetectors were not directed to the abstract idea, but were tied to their placement on a manipulated object, such as eyeglasses, which was integral to improving the system. Step 2 of Alice was also overcome according to the district court because the claims contain an inventive concept. Ultimately, however, HTC filed an IPR and prevailed. IPR No. 2018-01032. Another case to watch on the hardware side is D3D Enterprises, LLC v. Microsoft Corp., Case No. 6-20-cv-01699 (M.D. Fla.).
For patents relating to the application or use of AR, Lennon Image Technologies LLC (“Lennon”), a NPE, engaged in a patent litigation campaign against the who’s who of retailers, such as Macy’s, and Bloomingdale’s. The retailers had websites that featured technology to virtually try on apparel and accessories. The retailers’ AR technology used a computer’s webcam to capture a person’s image to then allow the user to superimpose apparel, jewelry or accessories on the image. Despite claiming differences between Lennon’s patent and the retail use, all of the defendants removed the “try on” feature from their website or settled.
Considering the ever-increasing number of patent applications filed in the AR space, AR developers are concerned about a wave of patent litigation that may stifle innovation, particularly from NPEs. The Lennon campaign is a perfect example where the assertion of an AR patent by a NPE squelched the development and use of promising AR “try on” applications in the retail clothing and fashion industry.
A wave of AR patent filings and potential litigation raises another open question: How much detail or description for the claimed uses of AR will be required in the patent specification to satisfy the written description and enablement requirements of Section 112? There has been an increasing trend of patents being invalidated by courts for failure to satisfy either written description or enablement. As the industry and speculators rush to capture IP space in the AR world, one could easily imagine relatively skinny patents with method claims directed to a wide variety of potential applications or uses of AR. But the AR method may be very rudimentary and not commercially viable with the technology available at the time of filing the patent application, yet it may become very valuable as AR technology improves. In a rapidly evolving area, that timespan for the technological improvement could very well be within the 20-year lifespan of the patent. Unlike other burgeoning technological areas such as antibody patents, where the U.S. Court of Appeals for the Federal Circuit and the U.S. Patent and Trademark Office (USPTO) have provided guidelines for satisfying 112 requirements, there are no such guidelines for AR or VR from the USPTO or other patent offices. Nor are there any district court or Federal Circuit decisions addressing this issue.
Trademarks and Copyrights
Most of the legal issues in AR will likely involve trademark and copyrights. There is always the possibility that third parties may engage in trademark infringement in AR or VR. AR, however, is unique in that virtual trademarks can be made to appear as if they are anywhere in physical space, literally providing advertisers a whole new planet Earth to plaster their trademark logos on buildings, billboards, people’s clothes, Mount Rushmore, the Washington monument, trees, the sky… basically anywhere. This, of course, would create a plethora of legal issues. For example, there may be “false connections” between the advertiser and the owner of the object bearing the advertiser’s trademark. This may lead to claims of likelihood of confusion as to whether the physical business, property owner or the government–in the case of the Washington Monument–sponsors the virtual advertisement.
Businesses may also advertise their trademarks near their competitor. For example, a Jimmy John’s advertisement may appear next to any Subway restaurant, or Pepsi may display its trademark near any Coke trademark in the real world. Imagine seeing people who are actually drinking a can of Coke in the real world appear as if they are drinking a can of Pepsi in the AR world! We could envision litigation relating to such practices that may likely be similar to the search engine keyword cases in the last decade.
There has been some uncertainty whether trademarks that cover goods in the real world could be extended to protect virtual representations. The now dated Marvel v. NCSoft case illustrates this uncertainty. See 2005 WL 878090 (C.D. Cal. March 9, 2005). Marvel sued NCSoft, a video game company, for providing tools to players to design superhero costumes for their avatars that allegedly infringed the trademarks and copyrights of Marvel’s superheroes. The court dismissed Marvel’s trademark claims, stating the players did not use the brands in commerce and the use of Marvel Superhero names by players was not of an infringing nature. With changing times and technology, the AR marketplace is now very different. As a result, many companies and brands, such as Nike, Roblox, Gucci, Prada and others are seeking out registrations for their trademarks in classes specifically relating to virtual goods for use in virtual or augmented reality environments.
In copyright law, AR will likely add new dimensions to the concepts of fair use and derivative works. For example, when AR artists use the physical world as their canvas, there will be copyrighted works everywhere in the form of artwork, statues, architecture, etc. Would morphing a copyrighted work into another similar object infringe the right to create a derivative work of the copyrighted work? Or is it fair use? What if the software needs to first copy the copyrighted work and then alter it? Is that copyright infringement?
AR could also enhance other senses besides sight such as hearing. Would amplifying music from a distant concert that would normally be difficult to hear, constitute copyright infringement? As you can see, the IP questions abound.
The Law Meets Sci-Fi
It is an exciting time to be an IP lawyer, or any other type of lawyer that works in these areas. These technologies will likely touch every aspect of our lives and generate what may seem like an endless list of legal, social, and ethical challenges. Expect to see substantial transactions and litigation, as companies stake out and secure IP rights for AR applications. Along with the legal issues, these technologies may bring many of the concepts we’ve seen in sci-fi movies into reality (sort of!).
Image Source: Deposit Photos
Image ID: 126089794
Join the Discussion
No comments yet.