public addURLFilterRule($regex)
$regex | string | Regular-expression defining the rule |
bool | TRUE if the regex is valid and the rule was added to the list, otherwise FALSE. |
If the crawler finds an URL and this URL matches with one of the given regular-expressions, the crawler
will ignore this URL and won't follow it.
Example:$crawler->addURLFilterRule("#(jpg|jpeg|gif|png|bmp)$# i");
$crawler->addURLFilterRule("#(css|js)$# i");
These rules let the crawler ignore URLs that end with "jpg", "jpeg", "gif", ..., "css" and "js".