IP Banning - Socoder

-=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- (c) WidthPadding Industries 1987 0\|638\|0 -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=-
Socoder -> Site & Server -> IP Banning
Mon, 15 Oct 2018, 09:36
Jayenkai	IP Banning OK, after a week studying the stats, and making plans for the IP Banner, I finally doubled up security around the script, and got it all working (hopefully) correctly. If you ever see a strange looking "Jay definitely made this, because it looks utterly shit and completely fake" Login box, DO NOT try to login, or refresh that page, because repeated loads of that page will trigger the IP ban. If you DO get to see it, via normal usage, let me know what you did that might've triggered it, and I can whittle down the false-flags. eeeeee... I've banned myself about 20 times, today!!! That's not been fun! Note : IP Bans are temporary and last about 5 minutes. Not enough to completely ban you, but definitely enough to stop a LOT of the spam/hacky bots from causing massive server over-usage. -=-=- ''Load, Next List!''
Mon, 15 Oct 2018, 09:44
rockford	Yay! I'm not banned. Yet!
Mon, 15 Oct 2018, 12:36
jm2bits	You could just install Fail2Ban on the server: https://www.linode.com/docs/security/using-fail2ban-for-security/ I have used it in the past to great effect. (Here is a link to someone configuring it for HTTP traffic: https://blog.layershift.com/fail2ban-block-malicious-traffic/ )
Mon, 15 Oct 2018, 13:34
Jayenkai	Fail2Ban is working for the background stuff, and was one of the initial setup things in the many guides I followed to get the Linode Server up and running. But bloody hell, it's not easy to work with, is it!? Or rather, it doesn't work in a way which is compatible with my brain's operating system. I'm looking at stats where a bot will repeatedly try to access wp-admin.php, wp-setup.php, wp-init.php, login.php, wiki.php, uploader.php, log.php and more. Files which legitimately may be part of a site. Not "crawling" via pagelinks, but "guess-crawling" via hole poking. I figured I could probably do a quicker job with a bit of htaccess and a very simple script. If dodgy url, send to "login" page. If sent to login page too much, ban ip temporarily. To the best of my ability, I've hopefully managed to do this without accidental false flags. .. And if I did it by desperately trying to figure out Fail2Ban, I'm fairly certain I wouldn't have managed to have done it without failure!! This is less of a "Could do..", and more of a "Don't trust myself to.." I've done it in a way in which I'm fairly confident that I've done it correctly, and now that it's in place, I can easily add extra functionality to target some really specific types of bots... But, to repeat, if you do see the "dodgy looking log-in page" under normal usage, please report why and how. -=-=- ''Load, Next List!''
Sat, 27 Oct 2018, 15:37
Jayenkai	I've significantly enhanced the IPBanning stuff, tonight, and will again be watching it over the next few days. It might help.. .. Or it might not.. Who knows! Currently trying to block a chinese botnet that doesn't appear to have any particular traits other than "lots of requests all at once". Hopefully tonight's tweaks will solve the issue, but as always, if you see any weird shit happening, be sure to let me know. -=-=- ''Load, Next List!''
Sun, 28 Oct 2018, 03:51
Jayenkai	Hmm.. BingBot managed to get itself banned about 3 or 4 times, last night.. Over 800 requests in under 5 minutes?! Bing isn't usually THAT bad, but all signs point to it definitely being Bing.. Hmmm... I've added a crawl-delay setting to the robot.txt, and will see how that changes things. -=-=- ''Load, Next List!''
Sun, 28 Oct 2018, 05:08
Jayenkai	You can see Bing being a silly sausage, right at the start of this graph. It seemingly learned how to behave after that, but you can still see the SoCoder (Blue) stats are heavily clumped together, until Bing decides that it's crawled enough, at which point the server has time to breath properly again. God damnit, Bing!!! (Oh, and those later spikes are me waking up and triggering the daily SQL backups and things!) -=-=- ''Load, Next List!''
Sun, 28 Oct 2018, 16:34
cyangames	Aye I've had similar issues with bing bot over on sites I run, it's a pretty stupid bot -=-=- Web / Game Dev, occasionally finishes off coding games also!
Fri, 16 Nov 2018, 16:34
Jayenkai	FFS.. Today's Top-10 user stats.. 86289 - "Mozilla/5.0 (compatible; Seekport Crawler; http://seekport.com/)" 6691 - Windows User 731 - Bing 703 - Google 538 - "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 524 - Android User 255 - MacOS User 250 - "CCBot/2.0 (https://commoncrawl.org/faq/)" 228 - iPhone User 203 - iPad User --v Apologies if the site's been slow and clunky at any point, today. -=-=- ''Load, Next List!''
Sat, 17 Nov 2018, 04:49
rockford	It does appear to be a bit slower this morning.
Sat, 17 Nov 2018, 06:05
Jayenkai	Like swatting flies.. You splat one against the window, and they all seem to want a go!! -=-=- ''Load, Next List!''