A Reddit user’s staggering bill at first looked like a scam or a joke. “Netlify just sent me a $104K bill for a simple static site,” they wrote after a mysterious traffic spike. In a quirky response, the company’s customer support team reduced the bill to $5,225. And when the story started trending online, the CEO decided that the user wouldn’t be charged at all.
Distributed denial of service (DDoS) attacks are often seen as a nuisance, rarely causing any lasting damage. However, a Reddit user calling themselves ‘liubanghoudai24’ received a devastating bill from their cloud provider after what they believe was a DDoS attack.
“So I received an email from Netlify last weekend saying that I have a $104,500.00 bill overdue,” their post reads.
The Netlify dashboard revealed extremely high bandwidth usage for four days, which peaked at 60.7TB on February 16th. They immediately suspected that the website had been attacked.
“It's not impossible, but why attack a simple static site like mine?” the user wondered. “This site has been on Netlify for four years and is always okay with the free tier. The monthly bandwidth never exceeded even 10GB and has only ~200 daily visitors.”
Their website, jyutping.org, stores a romanization scheme for the Cantonese language, some lessons, and tutorials.
After contacting customer support, it turned out the website contained one mp3 file, which was downloaded millions of times to reach the bandwidth of 164.1TB.
Offered a discount
“After looking into this, it seems you have a hit song on your site,” the email from Netlify customer support reads. “Maan Bou Jan Sang Lou by Teresa Tang. I was not aware of her, but she seems to be a popular Taiwanese singer. This song is 99% of your bandwidth usage over the past 30 days.”
The letter further explained that a lot of bandwidth was generated from user agents that “are quite ancient using Google Cloud addresses”.
“This would include devices such as circa 2010 iPads, Windows 98 & Windows 6 computers. So either you have a fanbase with a passion for older technology, or this was likely a DDoS attack. To me, this seems to be the latter,” the email continued and suggested hosting such files on third-party platforms, such as YouTube or SoundCloud.
After explaining the standard practice of reducing the bill to 20% after such attacks, which would be $20,900 in this case, the Netlify support team offered a better deal.
“I've currently reduced it to about 5%, which is $5,225. I know this is still a lot of money, and I apologize for the inconvenience. If you like, I can raise this internally to see what else can be done.”
The user wasn’t happy with that and decided not to pay but post their story on Reddit and Hacker News instead.
“This feels more like a scam to me. Why do serverless platforms like Netlify and Vercel not have DDoS protection or at least a spending limit? They should have alerted me if the spending skyrocketed. I checked my inbox and spam folder and found nothing. The only email is ‘Extra usage package purchased for bandwidth.’ It feels like they deliberately do not support these features so that they can cash grab in situations like this,” they said.
CEO responded with a post: “They're not getting charged for this”
One user on Hacker News with the alias ‘bobfunk’ introduced himself as the Netlify CEO and assured users that the bill would be forgiven. Cybernews was unable to verify the CEO’s identity independently. However, many previous posts from the same user and his bio support the claim of him being Matt Biilmann, the founder of Netlify.
“Our support team has reached out to the user from the thread to let them know they're not getting charged for this. It's currently our policy not to shut down free sites during traffic spikes that don't match attack patterns but instead forgive any bills from legitimate mistakes after the fact. Apologies that this didn't come through in the initial support reply,” the post reads.
The user also assured those doubting whether Netlify would forgive the bill if the story didn’t go viral.
“We've forgiven lots and lots of bills over the last 9 years, and they haven't gone viral,” ‘bobfunk’ said. “While I've always favored erring towards keeping people's sites up, we are currently working on changing the default behavior to never let free sites incur overages.”
Cybernews reached out to Netlify for additional comments and will update the story with a response.
Update: The situation is not as clear anymore
In another twist, the DDoS attack version of the story is being ruled out
“Since the user opened a ticket with us this past Sunday, we’ve been actively researching this situation. Initially, we thought it might have resulted from a DDoS attack, which we stated in our first response. After some investigating, it looks as though the spike in traffic was not caused by a DDoS after all,” Dorian Kendal, CMO at Netlify, told Cybernews.
Instead, now they believe that this was a sustained download event of an mp3 file over a stretch of multiple days.
“We’re working directly with the user to better understand what’s happening on their end, so we can uncover what caused the dramatic increase in downloads,” Kendal said.
“We’ve confirmed that the user was notified multiple times about the additional bandwidth that was being consumed on their site, but given their lack of response to these notifications, we believe that we should revisit and improve the messaging and urgency that’s being communicated.”
The company said that they’re continuing to work with the user and have completely waived any financial liability.
“We’re also taking steps to update our systems such that any future billing abnormalities like this are surfaced and resolved prior to involving any customer,” Kendal assured.
“Irrespective of how extremely rare this situation is, no user of Netlify should ever be concerned that this could happen to them. We’re taking meaningful steps to ensure that this is the case – both systematically and by reviewing our company policies.”
Netlify said that it takes the security of its platform and users' sites very seriously, and it prioritizes keeping sites online and removing bad actors as it detects them.
“We use a blend of automated and manual mitigation techniques to identify and squash abusive traffic. We are continually learning new traffic indicators and building these into the system to help protect sites.”
Updated on February 28th [08:00 a.m. GMT] with a statement from Netlify
Your email address will not be published. Required fields are markedmarked