|
|||||||
 |
|
|
LinkBack | Thread Tools | Search this Thread | Rate Thread | Display Modes |
03-14-2003, 07:07 AM
|
#1 |
|
Member (6 bit)
Join Date: Nov 2001
Location: North East UK
Posts: 33
  |
Stop Spiders
Hi, I have some MS web server space without terminal services access. Within it I have a number of access databases.
How can I stop spidering of my URLS and unauthorised downloading of the databases or asp files? I know of the robots.txt file, is this sufficient? Any help gratefully accepted 
Last edited by Simfal; 03-14-2003 at 10:59 AM. |
|
|
|
03-14-2003, 09:39 PM
|
#2 |
|
Member (6 bit)
Join Date: Feb 2003
Location: New Hampshire
Posts: 44
|
See the question mark after (eg.) /isbnInquiry.asp? Most search engine spiders get to the "?" in the query string and stop indexing because of the probability of getting caught in a spider trap.
http://www.chami.com/tips/internet/010198I.html |
|
|
|
|
| Bookmarks |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
Linear Mode
