Despite the AI boom and skyrocketing graphics processing unit (GPU) prices, the slump in the GPU rental market continues. The Nvidia H100, priced at nearly $30,000, is available for rent at just $1 per hour. And Nvidia’s RTX 4090 can be found for as little as $0.20 per hour.
Training or running AI in the cloud has never been so cheap. The executives of some marketplaces are looking for customers on X.
“On-demand H100 for $0.99/hr, 4090 for $0.20/hr,” Yuchen Jin, co-founder and CTO at Hyperbolic, posted on X.
“Likely the cheapest GPUs around.”
These prices are some of the lowest yet for specialized hardware capable of hundreds of trillions of simultaneous calculations per second. In 2023, due to high demand and low availability, H100s, powerful GPUs used for AI training, were rented for over $8 per hour.
Some rental prices now seem detached from retail prices.
For comparison, a single H100 costs around $28,000 at Newegg. It would need to run non-stop over three years, not including electricity and system prices, to recoup the cost.
The situation is better for the Nvidia RTX 4090, which until recently was the most powerful “gaming” GPU, retailing for around $2,000. It was recently replaced by the RTX 5090. However, neither could be found in stock.
on-demand H100 for $0.99/hr, 4090 for $0.20/hr at Hyperbolic
undefined Yuchen Jin (@Yuchenj_UW) February 21, 2025
likely the cheapest GPUs around
tell me what you're building, and I'll spot you free credits for an 8xH100 node for at least a few hours to start. pic.twitter.com/SIwpzrZ0gL
In one year, the RTX 4090 running for $0.2/h would bring $1,750, excluding electricity, system, and other associated costs.
Jin believes these prices will stay “long term.”
“We are a GPU AirBnB, so the suppliers specify the price”, the post reads.
Many other marketplaces also offer H100s for prices below $2 per hour, even when choosing a system with a single GPU.
What’s going on?
According to a detailed analysis by Eugene Cheah, CEO at featherless.ai, AI startups have shifted their focus from training large foundational models to fine-tuning already existing and capable open-source AI models. This drives the demand for AI inference (deployment), which is less computationally intensive.
Many investors have previously signed long-term rental contracts, and the resale of unused capacities also contributes to the oversupply.
Nvidia’s dominance is now also challenged by AMD, Intel, and other competitors offering alternatives, while Nvidia itself has released newer, even more powerful, and expensive GPU architectures.
“Don’t buy H100s. The market has flipped from shortage ($8/hr) to oversupplied ($2/hr) because of reserved compute resales, open model finetuning, and decline in new foundation model co’s,” Cheah summarized.
🆕 $2 H100s: How the GPU Rental Bubble Bursthttps://t.co/6pUzyGfucI
undefined Latent.Space (@latentspacepod) October 11, 2024
A rare guest post, from returning guest @picocreator!
H100s used to be $8/hr (if you could get them).
Now there's 7 different places sometimes selling them under $2. What happened?@nvidia used to have… pic.twitter.com/XrsDmrft8Y
For many GPU owners, renting has become unprofitable, and investments in new infrastructure are riskier. Cheah has demonstrated that once the rental cast for H100 falls below $1.65 per hour, revenues no longer recoup the investment, and the price needs to be above $2.85 to beat the internal rate of return provided by the stock market.
Cheap computing power benefits many developers who couldn’t otherwise afford expensive equipment. That includes Chinese companies that can’t buy high-end equipment due to US export controls. The Wall Street Journal (WSJ) has previously reported that Chinese companies use GPU rental as a loophole to access banned Nvidia chips.
Your email address will not be published. Required fields are markedmarked