public obeyNoFollowTags($mode)
$mode | bool | If set to TRUE, the crawler will obey "nofollow"-tags |
No information |
If set to TRUE, the crawler will not follow links that a marked with rel="nofollow"
(like <a href="page.html" rel="nofollow">) nor links from pages containing the meta-tag
<meta name="robots" content="nofollow">.
By default, the crawler will NOT obey nofollow-tags.