Rising Sun Member Forums  

Go Back   Rising Sun Member Forums > For Sale + Help > Rising Sun Forum Help Corner

Reply
 
Thread Tools
  #1  
Old 02-21-2012, 11:13 AM
corsair23's Avatar
corsair23 corsair23 is offline
Rising Sun Member
 
Join Date: Jul 2006
Location: Littleton
Posts: 8,698
Default Baidu Spiders?

Not sure why I noticed it, but I noticed that there were a LOT of "guests" with the IP address of 180.76.5.XXX

So, out of curiosity I did a Google search on 180.76.5 and one of the hits was:

https://www.vbulletin.com/forum/show...-Baidu-spiders


Not sure if they really affect our web site's performance but just now for instance, out of 75 "people" online, 36 of them were Baidu spiders - So almost 1/2. I know there are Google and Yahoo and other bots out there but typically you only see a couple instances of each at any one time.

Anyway, I'm not web guru but thought I'd bring it up...
__________________
Jeff Z. (the "not quite as skinny" one)
TLCA #17037
'97 LX450 - aka "The Whale"
'97 FZJ80 Antique Sage AE #267, stock
12/74 FJ40, 2F, SM420, 4" Lift, ARBs, 33" MTRs

:

"...anything else i can do for you guys, how about i wash your car or mow your lawn while you figure out your firewall system? I am now boarderline insane/unibomber." Kipper

"That assumes I'm even capable of pulling and stabbing..." Jacket

"I really like having a detachable unit." Beater
Reply With Quote
  #2  
Old 02-21-2012, 11:23 AM
Hulk's Avatar
Hulk Hulk is offline
Cruise Moab Committee
 
Join Date: Aug 2005
Location: South Side!
Posts: 11,423
Send a message via AIM to Hulk
Default

Looks like we just need to create a robot.txt file. I'll work on this.
__________________
Matt Farr, Centennial, Colorado | Webmaster: TLCA.org
1996 FZJ80 · TLCA #4189 · WØRDY
www.rustybrain.com/cruisers · my Rising Sun bio · Facebook · Twitter · Need satellite Internet? Check out: Exede Internet

If you think you can or think you can't, you're right.
Reply With Quote
  #3  
Old 02-21-2012, 11:32 AM
corsair23's Avatar
corsair23 corsair23 is offline
Rising Sun Member
 
Join Date: Jul 2006
Location: Littleton
Posts: 8,698
Default



Looks like 180.76.6.xxx might be another one?

http://www.forumpostersunion.com/showthread.php?p=86441


The two together added up to 43 out of the 75 users online just a minute ago when I counted. Seems like they are the bain of many web folk
__________________
Jeff Z. (the "not quite as skinny" one)
TLCA #17037
'97 LX450 - aka "The Whale"
'97 FZJ80 Antique Sage AE #267, stock
12/74 FJ40, 2F, SM420, 4" Lift, ARBs, 33" MTRs

:

"...anything else i can do for you guys, how about i wash your car or mow your lawn while you figure out your firewall system? I am now boarderline insane/unibomber." Kipper

"That assumes I'm even capable of pulling and stabbing..." Jacket

"I really like having a detachable unit." Beater
Reply With Quote
  #4  
Old 02-21-2012, 01:20 PM
60wag's Avatar
60wag 60wag is offline
Rising Sun Member
 
Join Date: Aug 2005
Posts: 1,796
Default

I know nothing about bots but have noticed the high number of users lately. How does the .txt file keep the bots out?
__________________
Bruce
'96FZJ80
'03 KTM 450EXC
"If you know what you're doing, then you aren't learning anything." - A. Lincoln
Reply With Quote
  #5  
Old 02-21-2012, 01:56 PM
corsair23's Avatar
corsair23 corsair23 is offline
Rising Sun Member
 
Join Date: Jul 2006
Location: Littleton
Posts: 8,698
Default

Quote:
Originally Posted by 60wag View Post
I know nothing about bots but have noticed the high number of users lately. How does the .txt file keep the bots out?
Matt can probably explain it but this was a link within the first link I posted:

http://www.robotstxt.org/robotstxt.html
__________________
Jeff Z. (the "not quite as skinny" one)
TLCA #17037
'97 LX450 - aka "The Whale"
'97 FZJ80 Antique Sage AE #267, stock
12/74 FJ40, 2F, SM420, 4" Lift, ARBs, 33" MTRs

:

"...anything else i can do for you guys, how about i wash your car or mow your lawn while you figure out your firewall system? I am now boarderline insane/unibomber." Kipper

"That assumes I'm even capable of pulling and stabbing..." Jacket

"I really like having a detachable unit." Beater
Reply With Quote
  #6  
Old 02-22-2012, 09:02 PM
simps80's Avatar
simps80 simps80 is offline
Rising Sun Member
 
Join Date: Jan 2009
Location: Arvada, CO
Posts: 936
Default

dang 8:56pm
Currently Active Users: 104 (12 members and 92 guests)
google spider's goin' nuts!!
__________________
Mike | KØSMP |
1997 fzj80 | 1976 FJ40 | TLCA # 20038
Reply With Quote
  #7  
Old 02-23-2012, 12:27 AM
corsair23's Avatar
corsair23 corsair23 is offline
Rising Sun Member
 
Join Date: Jul 2006
Location: Littleton
Posts: 8,698
Default

Quote:
Originally Posted by simps80 View Post
dang 8:56pm
Currently Active Users: 104 (12 members and 92 guests)
google spider's goin' nuts!!
no doubt...

12:26am
103 users (3 members and 100 guests)

I counted 22 Google spiders and at least 50 Baidu spiders...Oh, and 1 MSN bot
__________________
Jeff Z. (the "not quite as skinny" one)
TLCA #17037
'97 LX450 - aka "The Whale"
'97 FZJ80 Antique Sage AE #267, stock
12/74 FJ40, 2F, SM420, 4" Lift, ARBs, 33" MTRs

:

"...anything else i can do for you guys, how about i wash your car or mow your lawn while you figure out your firewall system? I am now boarderline insane/unibomber." Kipper

"That assumes I'm even capable of pulling and stabbing..." Jacket

"I really like having a detachable unit." Beater
Reply With Quote
  #8  
Old 04-26-2012, 02:10 PM
corsair23's Avatar
corsair23 corsair23 is offline
Rising Sun Member
 
Join Date: Jul 2006
Location: Littleton
Posts: 8,698
Default

Seems like the spiders are getting worse...

Just checked and out of the 61 users online a few minutes ago, 52 were spiders with IP addresses starting with 180.76.5 & 180.76.6
__________________
Jeff Z. (the "not quite as skinny" one)
TLCA #17037
'97 LX450 - aka "The Whale"
'97 FZJ80 Antique Sage AE #267, stock
12/74 FJ40, 2F, SM420, 4" Lift, ARBs, 33" MTRs

:

"...anything else i can do for you guys, how about i wash your car or mow your lawn while you figure out your firewall system? I am now boarderline insane/unibomber." Kipper

"That assumes I'm even capable of pulling and stabbing..." Jacket

"I really like having a detachable unit." Beater
Reply With Quote
  #9  
Old 04-26-2012, 03:42 PM
wesintl's Avatar
wesintl wesintl is offline
Hard Core 4+
 
Join Date: Aug 2005
Location: in da house
Posts: 7,579
Send a message via AIM to wesintl
Default

so... they may suck but they are really only sucking bandwith which isn't an issue for us right now. :MEH: Not sure why it's got you so worried

Some don't respect the robot.txt file so you have to block them at htacces or other creative ways
__________________
See you on the trail
WØREK
Couple o' FJ40's, BJ70, UZJ100, TDI 66FJ40,82FJ60,97FZJ80
TLCA#4180
Reply With Quote
  #10  
Old 04-26-2012, 05:06 PM
corsair23's Avatar
corsair23 corsair23 is offline
Rising Sun Member
 
Join Date: Jul 2006
Location: Littleton
Posts: 8,698
Default

Spiders scares me
__________________
Jeff Z. (the "not quite as skinny" one)
TLCA #17037
'97 LX450 - aka "The Whale"
'97 FZJ80 Antique Sage AE #267, stock
12/74 FJ40, 2F, SM420, 4" Lift, ARBs, 33" MTRs

:

"...anything else i can do for you guys, how about i wash your car or mow your lawn while you figure out your firewall system? I am now boarderline insane/unibomber." Kipper

"That assumes I'm even capable of pulling and stabbing..." Jacket

"I really like having a detachable unit." Beater
Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -6. The time now is 12:05 AM.


Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.