Issue Feature Rejected

#14798Robots.txt and indexed search

Jody Cleveland 06/06/05, 18:45 updated 10/07/16, 19:11

Description

I’ve got a handful of those pdf files that I don’t want
indexed. So, to satisfy google, and other search engines, I use a
robots.txt file so they aren’t indexed.

My requested feature is, getting typo to honor robots.txt files, and
skip indexing files listed in there.

(issue imported from #M1170)

Reviews 0

No change resolves or names this issue in its commit message, and no comment links one.