Select the search type
  • Site
  • Web
Search
You are here:  Support/Forums
Support

Bring2mind Forums

Robots.TXT and DMX folder
Last Post 04/14/2009 11:18 PM by Peter Donker. 1 Replies.
Sort:
PrevPrev NextNext
You are not authorized to post a reply.
Author Messages
mangiov
New Member
New Member
Posts:6


--
04/14/2009 3:46 PM

Hi,

assuming that I keep the storage folder in the default path (/Portals/0/DMX in my case), would it be enough to put an entry in the ropots.txt file (i.e. Disallow: /portals/0/DMX) to stop search engines from indexing directly the repository file or it is better to move the repository out of the portal root? I have protected documents that obviously have to be accessed only through DNN/DMX security.

Thanks,

Vincenzo

Peter Donker
Veteran Member
Veteran Member
Posts:4536


--
04/14/2009 11:18 PM
Hi Vicenzo,

It depends on what the search engines do. External search engines like Google only see content through the DMX. They can't access the files that are stored on your server because the extension (.resources) is disallowed by IIS. These files are also not linked anywhere, so the crawler has no clue.

An internal search engine like Indesing Service might see all content on the hard disk. But then it must know what to do with the file. Again this has been seriously hampered by the extension renaming. A text file could be read, but a native .doc file becomes a bunch of unreadable code for the indexer.

In conclusion I don't think you need to change anything. You must be aware though that files that are visible for 'All Users' can be crawled.

Peter
You are not authorized to post a reply.