A Proposal for Addressing the AI Memory Supply Shortage

In economic theory, corporate value creation can be attributed to many factors and conflicting preferences. Prioritizing different factors produces a set of different outcomes and the outcome that is seemingly most efficient becomes the accepted norm. Sometimes however, corporate actors rely on accepted business practices which, while taken in the pursuit of the most optimal outcome, facilitate market distortions. Hit by a compute-driven demand shock and faced with resource scarcity, Samsung, SK Hynix, and Micron have opted to allocate production capacity in a way that has exacerbated an industry- wide memory card supply shortage. But to fully understand why the price of hardware components and consumer electronic devices has been steadily surging, we must first start with Nvidia.

The Events Leading Up to the Memory Supply Shortage

Once upon a time, Nvidia was a small and promising hardware company making graphics processing units (GPUs) for a diverse customer base. Rather than shipping products with built-in obsolescence, it invested in their durability and performance earning loyal consumers and dominating more than 90% of the graphics card market over its competitors AMD and Intel. And rather than entrenching itself in that singular market, it used software as a way of amplifying the usability of its hardware, turning its GPUs into general-purpose devices that could run compute-intensive applications such as machine learning.

To further improve performance, Nvidia next optimized its GPUs based on compute-intensiveness and product use, creating separate GPU series to accommodate its diverse client base of gamers, programmers, video editors, and especially after late 2022, AI developers. The AI market in particular caused a positive demand shock for compute which was initially absorbed by cloud service providers, who then turned to Nvidia to expand compute capacity, driving the price of Nvidia’s stock ten times higher in the process. Much of this, Nvidia had already anticipated, and it responded by allocating significant production resources to match demand. But one important component of its GPU architecture was memory and it was a crucial bottleneck for increasing Nvidia’s output.

As a result, Nvidia started reconfiguring its high-end GPU design to optimize performance even further, while seeking a way to address the memory bottleneck. One of its ideas was to acquire Groq, a competing company that developed the Language Processing Unit. Its distinct architecture could speed up the process for large language models to complete inference-related tasks and mitigate the memory supply shortage because it didn’t rely on the specific type of memory that was in high demand. This was desirable both for Nvidia and Nvidia’s customers. But the issue with acquiring Groq was that it was Nvidia’s direct competitor.

Nvidia wanted to avoid a similar scenario to what had transpired a few years ago, when its attempted acquisition of Arm was challenged by the Federal Trade Commission, forcing it to abandon the effort entirely. So, it hired a fancy law firm whose lawyers proposed structuring the acquisition in a way that they believed was safe from antitrust scrutiny. A kind of modern day poison pill, except, instead of greedy activists, it would protect Nvidia from pesky trustbusters by circumventing merger review. Nvidia likely understood that the structuring itself could attract more attention and possibly greater scrutiny of the deal, but it took a calculated risk thinking that the antitrust agencies couldn’t unwind it, and unlike their predecessors, they wouldn’t even try. As it turned out however, the Department of Justice did launch an investigation and some of us have shown that the harm from these types of acquisitions can be remedied after all.

In the meantime, much like the cloud service providers, Nvidia had to turn to memory manufacturers to expand memory capacity. The three largest manufacturers of memory cards are Samsung, SK Hynix, and Micron, who collectively own more than 90% percent of the memory card market. Like Nvidia, all three are hardware companies, and like Nvidia, all three have distinct product lines depending on the end-user. Conventional memory cards (DRAM) are sold to retailers and other electronics companies, and high-end memory stacks (HBM) to Nvidia and other AI labs. But unlike Nvidia, the memory trio was much less prepared and responded by reallocating production capacity to meet demand much more aggressively. This had a major impact on the availability of conventional memory cards causing significant spikes in retail prices.

As a consequence, Samsung, SK Hynix, and Micron were hit with a class action lawsuit alleging that they artificially inflated consumer prices by cutting production for conventional memory in a concerted effort that runs afoul of the antitrust laws. According to the complaint:

“Commodity DRAM serves the mass market—every consumer, every business, every device. HBM serves a narrow ultra-high-end segment of AI customers. HBM commands prices several times higher per gigabyte and requires more than twice the silicon wafer area to manufacture. The conspiracy alleged herein involves Defendants’ coordinated decision to starve the commodity DRAM market of supply while redirecting resources to HBM.”

The Legal Standard for Conscious Parallelism, Conspiracy, and Plausibility

Section 1 of the Sherman Act makes it illegal to conspire in restraint of trade. To infer a conspiracy, a plaintiff must prove the existence of an agreement. Specifically, the evidence must be enough to suggest that it is more likely than not that there is an agreement and not a kind of independent behavior which is rational and economically sound irrespective of the actions of other competitors. According to Theatre Enters. v. Paramount Film Distrib. Corp., 346 U.S. 537, 540–41 (1954):

“The crucial question is whether respondents’ conduct toward petitioner stemmed from independent decision or from an agreement, tacit or express. To be sure, business behavior is admissible circumstantial evidence from which the fact finder may infer agreement. But this Court has never held that proof of parallel business behavior conclusively establishes agreement or, phrased differently, that such behavior itself constitutes a Sherman Act offense.”

Circumstantial evidence that shows a conscious commitment to a common scheme, beyond mere conscious parallelism, is known as the plus factors. And determining the probative value of each of those plus factors in oligopolistic markets, has been a particularly challenging exercise in Section 1 cases.

The Class Action Against Samsung, SK Hynix, and Micron

In their complaint, the plaintiffs allege the following plus factors. First, there was parallel business conduct among the memory manufacturers to (i) make significant cuts to DRAM production, (ii) shift manufacturing capacity to HBM, prioritizing Nvidia’s orders for high-end memory and OpenAI’s needs for its data center build-out, (iii) shut down older product lines for conventional memory cards (DDR4/DDR3) and (iv) screen customer requests to ensure priority for Nvidia and the AI labs.

Second, these actions were against self-interest. Micron shut down Crucial, one of its business units that sold DRAM directly to consumers at a time when product scarcity had contributed to strong demand, high profits, and significant price increases for conventional memory cards. Moreover, refusing to balance production capacity for conventional memory in these circumstances, even while continuing to prioritize long-term HBM contracts, was detrimental to Micron’s short-term profits.

Third, the market characteristics are conducive to coordination as the memory market is an oligopoly with high barriers to entry and inelastic product demand. Furthermore, the manufacturers have a record of participating in a criminal conspiracy to fix prices. Last, they rely on public signaling through earnings calls, and enjoy an opportunity to collude through participation in industry events.

In their motion to dismiss, the manufacturers respond with three simple points. First, the decision to prioritize high-end memory output was reasonable business judgment. Second, conscious parallelism in an oligopoly is not a conspiracy. And third, since the alleged plus factors merely describe conscious parallelism, such claims, similar to those of an earlier class action, do not meet the “Twiqbal” standard to plausibly infer an agreement.

The Difference from Past Class Actions Against Samsung, SK Hynix, and Micron

In 2018, a class action made similar claims against the same three companies and was dismissed by a district court. The Ninth Circuit later upheld that dismissal, on the grounds that the trio’s actions were independent business conduct that had occurred in an oligopoly. But there are at least three factors that distinguish the claims in the new class action from those of 2018.

To begin with, numbers matter. According to the complaint, contract prices for DRAM are currently up approximately 697% and retail prices up 100% to 500%. In the earlier class action, the average retail price for DRAM rose by around 40% between 2016 and 2017, and contract pricing by 60% over 2017. Those increases pale in comparison to the recent price hikes.

Next, Micron’s shutdown of its consumer unit Crucial at a time when demand was strong, Micron’s profits high, and DRAM prices high, is a good example of what Bill Kovacic has referred to as “a super plus factor.” As he puts it:

“Restrictions in supply by subsets of firms when demand is strong, profits are high, and prices are relatively high, however, leads to the strong inference of collusion—namely, it is a super plus factor—for two reasons. First, there are substantial foregone profits from restricting supply when demand is strong. Second, buyers will take measures to resist price increases in such an environment, and it is quite unlikely that unilateral conduct by sellers would not take advantage of the opportunity to sell incremental units at high prices.”

The probative value of this factor is even more significant because Micron engaged in dominant-firm conduct even though it wasn’t large enough on its own to act as the market leader (according to the complaint SK Hynix has a 33.2% market share, followed by Samsung’s 32.6% market share, and Micron’s 25.7% market share). And yet, for some reason, Micron was still undeterred by the risk of opportunistic behavior by its rivals.

Then again, shifting capacity to prioritize demand from a more profitable and sophisticated customer like Nvidia and establish long-term contractual relationships with AI labs, is a reasonable business decision. But if, despite the aggressive production cuts, the memory manufacturers were unable to match the demand shock in the near-to-medium term – something that is likely to be focused on during discovery – and yet they still aggressively suppressed output for retail memory, then there is a question of ulterior motive. And indeed, one plausible explanation is that they acted under the guise of market demand for HBM to purposely throttle supply for DRAM to extract higher producer surplus at the expense of their consumers.

Last, the memory manufacturers rightly point to the earlier class action where the courts found most of the plus factors alleged to be evidence of conscious parallelism rather than conscious commitment to a common scheme. However, the Ninth Circuit decision also distinguished the memory market as one that is conducive to a conspiracy and noted the firms’ prior conduct as a plus factor that supports this claim.

How the Agencies and State Attorneys General Can Respond

The memory supply shortage is hurting small businesses in the component and computer assembly space the most, but not exclusively. Major manufacturers of laptops, smartphones, and gaming consoles are affected as well.

And yet, similar to the memory manufacturers, the antitrust agencies have finite resources. Mindful of this predicament, there is often hesitation in launching an investigation without having a clear expectation of achieving concrete results. Given the constraints in the range of tools available, and to minimize the risk of false-positive error costs from overzealous enforcement, some may err on the side of caution unless there is a cost-effective approach to induce market behavior. Fortunately, such an approach is available here.

The agencies can simply file a joint statement of interest urging the court to give due consideration to  (i) the unprecedented price hikes, (ii) the Kovacic super plus factor, and (iii) the conditions of a highly concentrated market that are conducive to a conspiracy, in deciding the motion to dismiss. After helping the plaintiffs clear Twombly, the agencies can sit back and let the process of discovery proceed. If the plaintiffs discover a smoking gun, the class action will settle for a sizable amount, giving the agencies and state attorneys general, including those who actively defend competition, a good indication to launch a formal investigation. If not, the antitrust agencies lose nothing. In other words, the agencies can use the class action as a proxy to test the waters prior to taking matters into their own hands.

Time to Finish the Job

The three memory manufacturers were among those who were once held accountable for orchestrating a criminal conspiracy to fix prices under the Bush administration. If the same oligopolists are shown to be responsible for another conspiracy, one that is more harmful in effect and more lasting in duration, then it’s up to the Trump administration to finish the job. In the meantime, others can scout ahead for it.

Image Source: Deposit Photos
Author: ilixe48
Image ID: 883698562