|
Socoder -> Site & Server -> Spiderbots Descend |
|
| Fri, 11 Oct 2024, 05:42 | |
cyangames
|
Just a word of warning, bytespider and claudebot are mass crawling again, if you run a website, block them via .htaccess
-=-=- Web / Game Dev, occasionally finishes off coding games also! |
| Fri, 11 Oct 2024, 05:42 | |
Jayenkai
|
Logs for the past hour suggest... GPTBot : 7,193 Misc (Actual users, maybe?) : 2165 BingBog : 705 Meta : 219 Bytedance (TikTok) : 166 Google : 143 Claude : 40 -=-=- ''Load, Next List!'' 
|
| Mon, 14 Oct 2024, 05:26 | |
HoboBen
|
Dunno what your opinion is on ChatGPT is ethically. Figure, if we post here, it's public, and ChatGPT has as much right as any body to lurk the forum. But it's gonna package up what we say and regurgitate it without attribution and destroy the environment at the same time. And it's slurping up more data than anybody else. Do you block it? -=-=- blog | work | code | more code |
| Mon, 14 Oct 2024, 06:09 | |
|
Jayenkai
|
I don't block it. But discuss whether I should. To be fair, all the various AI's would have scraped the site half-a-dozen times each, by now, even long before it was a "thing" to avoid. Bing's ALWAYS scraping stuff, and they're chucking all that into CoPilot, and Google Google the Google Google, so the only "real" option is to pretty much block EVERYTHING. -=-=- ''Load, Next List!''
|
| Mon, 14 Oct 2024, 09:50 | |
|
Jayenkai
|
Stats for the past hour GPTBot : 8,741 Bing : 688 Probably actual users : 269 Google : 294 Bytedance : 167 CCBot : 58 Meta : 34 DotBot : 9 Claude : 5 -=-=- ''Load, Next List!''
|
| Fri, 08 Nov 2024, 10:25 | |
|
Jayenkai
|
Stats for the past hour Google : 3,124 dataforseo : 521 Bing : 453 Probably actual users : 184 Bytedance : 97 Meta : 59 AmazonBot : 26 Claude : 8 DotBot : 4 GPTBot : 1 .. Google.. And I double checked the IPs, and they're definitely Google. Half of them are listed as GoogleBot, and the other half "Google Other"...!?! So, I presume their AI crap. I mean.. What am I supposed to do about that!? Block Google!?!?! -=-=- ''Load, Next List!''
|
| Fri, 08 Nov 2024, 15:08 | |
|
cyangames
|
I think you can change the crawl rate within webmaster tools...well search console nowadays
-=-=- Web / Game Dev, occasionally finishes off coding games also! |
| Fri, 08 Nov 2024, 16:25 | |
|
Jayenkai
|
Oh yeah, I forgot about "you have to fart about online". Quite why it can't pay attention to the settings in robots.txt, I've no idea. -=-=- ''Load, Next List!''
|
|
|
| |||
|
| ||||