New Foswiki release 2.1.6 is available with important security fixes.
Sourceforge foswiki email lists being discontinued. Subscribe to the new Foswiki announce and discuss lists at MailingLists
This question about Not sure...: Asked

Prevent Crawling Certain Webs

I have the recommended robots.txt file at the top of my wiki so that robots can only access view links.

Is there a best practice for preventing indexing/crawling of one or more webs?

Is that as simple as, e.g., adding

to the top level robots.txt file, or is there something else I need to do?

If it makes a difference, I've got short URLs enabled via Apache.

-- VickiBrown - 27 Feb 2014


QuestionForm edit

Subject Not sure...
Version Foswiki 1.1.9
Status Asked
Related Topics
Topic revision: r1 - 27 Feb 2014, VickiBrown - This page was cached on 21 Aug 2018 - 21:55.

The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy