How I stopped thousands of spam hits on my Alpha server

[Steve Wood]

alphatogo.com and iadn.com have been out there for a while and so my server gets thousands of hits daily from bad bots, hackers and spiders from search engines I don't want. I have a properly configured robots.txt that does a partial job keeping out the bots that obey those settings, but many of them do not obey those settings. Alpha Anywhere has a feature to block by IP address but that is of minimal use against this overwhelming tide.

So I finally installed a "IP Blocker Firewall" on my server. The firewall allows me to block all known bad IP's from industry maintained blacklists, and/or by whole country. The "known bad IP" lists are automatically refreshed on a schedule (because the lists change daily). Right now I am blocking by blacklist, several whole countries and a few additional IP ranges based on my observation.

Blocked hits never get to the Alpha web server and so consume NO Alpha web server resources.

This still leaves my server open to all of search engines I want, and most of those who I expect to be valid customers. Because the blocking is fairly aggressive right now, I am sure this will occasionally block a legitimate hit -- so the Blocker is keeping a list of all IPs that hit the server (both allowed and blocked) so in a month or so I can change my tactic and try to block just the rogue IP ranges that are actually hitting my server rather than by whole country.

Doing this has reduced the hits on my server by those thousands per day.

Note, here are my robots.txt and sitemap.xml files. You should always have these files in your website root for basic SEO. My sitemap.xml is auto-generated every month from a scheduled xbasic script.

[Clipper87]

Re: How I stopped thousands of spam hits on my Alpha server

very interesting

[imjoken]

Re: How I stopped thousands of spam hits on my Alpha server

Steve,

This is very helpful info. Do you know how the IP Blocker handles having multiple IP's (alpha instances) on the same physical server?

[Steve Wood]

Re: How I stopped thousands of spam hits on my Alpha server

Jay, the whole process takes place in front of the web server, so it has no knowledge of instances or Alpha. It deals with hits to the Microsoft server across ALL ports, not just port 80/443.

EDIT: That means it takes in to account ALL of the IP's hitting your server regardless of what Instance they go to. If you look at the image in my first post you see the several IPs it is hitting on my server. All of my IPs start with 50.23., so I don't recognize a couple of them, which is a bit confusing still.

[Clipper87]

Re: How I stopped thousands of spam hits on my Alpha server

Steve,

One question I'd like to ask; have you been comparing products or is this THE "reference" product according to your findings ?
Thx
Frank

[Steve Wood]

Re: How I stopped thousands of spam hits on my Alpha server

Frank, Its the only product I could find.

[Clipper87]

Re: How I stopped thousands of spam hits on my Alpha server

OK Thx! I'm going to buy that.

[Tim Sutherland]

Re: How I stopped thousands of spam hits on my Alpha server

Is this similar to RDP Guard?

I believe this is the app that Zebra Hosting recommends.

[Steve Wood]

Re: How I stopped thousands of spam hits on my Alpha server

RDP Guard, guards RDP (primarily).

(And, I was the one who recommended RDP Guard to Zebrahost.)

[mikeallenbrown]

Re: How I stopped thousands of spam hits on my Alpha server

Thanks for this information, Steve. What about those of us who use a load balancer?

[imjoken]

Re: How I stopped thousands of spam hits on my Alpha server

The spammers are pinging the server/WAS instance IP addresses directly which bypasses the LB.

[GaryRockley]

Re: How I stopped thousands of spam hits on my Alpha server

Hi Steve,

I'm curious about this.

Since any server needs to have Port 80 open, how is this a firewall?

As an IP blocker, most IP blockers just drag down server efficiency, especially the native Server ones. So, consequently, legit users get penalized for attempted hacks & robots, because the server now has to essentially look up IP addresses. This begs a question, have you tested server response times after installing this?

Assuming minimal adverse impact on server efficiency, and noting your comment about blocking 'a whole country', can one block a range of IPs without having to specify each one? For example, 210.211.64.1 to 210.211.64.66? Thus blacklisting a spefic known range of Persona non Grata?

As for utilty, if the Server is secure, and again thinking about server management of resources, why care about robots and attempted hacks?

Server side processing of illegit robots is of minimal overhead to the server (and quite frankly, anyone can create a robot to ignore robots rules). Probaly better to invoke invoke a timer block of repeated requests from the same IP inside of say, 20ms.

Hacking attempts are of a bigger concern, but since most hackers run over proxy servers (wherby they already opened a port on a usually unsuspecting innocent conduit), and bounce around hundreds at a time, blocking an IP on an insecure server seems to me as little more than a band aid on a severed artery.

otoh, if the server is secure, blocking IPs also seems pointless and does little more than degrade server efficiency.

Or am I missing something?


Not trying to be critical of your reasoning, and I think that anything that can improve server security is mostly valid, but I just don't see IP blockers as the solution.


Regards,

Gary.

[CharlesParker]

Re: How I stopped thousands of spam hits on my Alpha server

Steve,
I rent a server from Zebra Host, do I just download and install this software on my server and it works as per my config or does this need to be installed by Zebrahost?
Sorry to ask such a noob question, but having zero knowledge I can ask no other...

[Steve Wood]

Re: How I stopped thousands of spam hits on my Alpha server

Charles - depends on your comfort level on the server. Its a pretty ordinary installation, but choosing what to block takes some time reviewing your logs and then deciding what to do. It takes a lot of observation over a period of time. And of course I tried my own IP like an idiot and temporarily locked myself out. (I knew I could get back in from another server).

Gary - I agree that it is not absolutely necessary to physically block hits to the server, hacking attempts or otherwise as long as the server is secure. The hacking attempts are all for PHP, completely bogus folder or file names or other non-Alpha resources they will never find. But the shear number of them was getting to me, like I said thousands upon thousands. My Alpha logs were full of 404 page errors, and search engine spider hits from various countries where I do not do business, all at 10 pages per second. And when that activity was high, id DID cause Alpha server strain. In fact, many times users here on the forum complained about server crashes or poor performance, and I helped investigate, it turned out to be from excessive unnecessary indexing of their site. One that comes to mind was getting their large PDF documents indexed and it always crashed the WAS.

I am currently blocking 274000 individual and ranges of IPs, and I do not see any performance degradation. But as I said I am recording the good and bad hits and plan to scale that down in a month to just block the ones that are actually hitting the server plus the industry-managed blacklists. The software gives me some aggregate data to help with that. BTW, on my server it is mostly China, Hong Kong, Pakistan, Russia, Ukraine and Israel. I also blocked IP ranges for several USA based search engines like Yandex.

Another thing I will say is that I learned a lot about my server by doing this. I had no idea so much was going on, and on ports other than 80/443. It is amazing to watch (I should do a recording). I need to know as much as I can about this both for my own use and for my customers. It IS important to pay attention to what is hitting your server. Case in point -- I have a customer that has highly personal information on their server (records of deaths) and a competitor fired up a SQL Injection attack on our server. It ran thousands of hacking attempts (with no success, Yeah Alpha). In that app I capture every single hit to a database table. I think you can see there it is important to monitor and also important to keep the number of useless spider hits to a minimum at least for sake of my database table.

Before I tried the above solution, I had written my own IP Blocker for hits to port 80. It used an xbasic script to capture all 404 Errors to non-Alpha related pages (PHP, empty folders, etc) The xbasic used a command line to write a Firewall Rule to block that IP. The problem was that after a couple days I had hundreds of new Rules in my Firewall, too many to manage.

Gary -- I don't understand your paragraph that includes "but since most hackers run over proxy servers ", so I don't know how to reply. The IP Blocker tool covers all ports, not just 80/443.

Lastly I would say selectively blocking by IP is not much different than you or I blacklisting our inbound email traffic. We do it because our time and peace of mind are precious. I also have to deploy anti-hacking measures over on the IADN Forum. If I drop the measures it instantly is flooded with bogus signups running ads or scraping for email addresses.

[Clipper87]

Re: How I stopped thousands of spam hits on my Alpha server

Steve,

I think Gary means hackers are using proxy servers to hide their IP so they can act as if legit. I also took a closer look at the website of ipblocker, especially the contactpage does not really impress me; who is this guy, what is this software doing (is it thrustworthy?) I usually like some more background but you sure have triggered my interest; I see a lot of traffic on my server too that for sure is not from people I'll ever do business with & various indexing engines that don't intrest me. I also notice that IIS seems to offer solutions for this provided that you subscribe to some list (I bet you also have to subscribe to some list rather than just buying one time a list ?) https://www.countryipblocks.net/coun...and-web.config