Method: PHPCrawler::obeyNoFollowTags()



Decides whether the crawler should obey "nofollow"-tags
Signature:

public obeyNoFollowTags($mode)

Parameters:

$mode bool If set to TRUE, the crawler will obey "nofollow"-tags

Returns:

No information

Description:

If set to TRUE, the crawler will not follow links that a marked with rel="nofollow"
(like <a href="page.html" rel="nofollow">) nor links from pages containing the meta-tag
<meta name="robots" content="nofollow">.

By default, the crawler will NOT obey nofollow-tags.