A few weeks ago I received an email from Netlify telling me that I had exceeded 50% of my allocated bandwidth.
I thought this was strange as my site is highly optimised and receives very little traffic. I logged into Netlify’s Analytics dashboard and sure enough I’d used 128GB of data.
This started to ramp up around the February 9th and got progressively worse until I implemented a fix on February 21st.
Unfortunately, unless you pay for Netlify’s Analytics package ($9/m) you would have no way to identify the problem.
It transpires that bots were requesting /default.php (755,000 times) and I was—correctly—returning a 404 response and serving my 404 page.
Unfortunately Netlify does not allow us to filter or throttle requests (although you could do this with NGINX Reverse Proxy) so the only option is to accept them and handle them appropriately.
Thankfully Netlify offers a way of redirecting traffic so the solution I came up with was to redirect them away from my Netlify site, rather than serving them a 404.
I created a file named _redirect in the /static directory of my Gatsby project.
Each line in the file accepts an incoming address, a returned address and an HTTP response.
# /static/_redirect
https://samuelcotterall.com/news/index.php https://devnull.samuelcotterall.com/ 301!
https://samuelcotterall.com/default.php https://devnull.samuelcotterall.com/ 301!
https://samuelcotterall.com/default.php/* https://devnull.samuelcotterall.com/ 301!As you can see from the bar chart at the top of this page the bandwidth dropped back down to zero once I implemented these redirects (Feb 21st).