|$regex||string||Regular-expression defining the rule|
|bool||TRUE if the rule was successfully added|
By default the crawler ONLY searches for links in documents of type "text/html".
Use this method to add one or more other content-types the crawler should check for links.
These rules let the crawler search for links in HTML-, CSS- ans XML-documents.
Please note: It is NOT recommended to let the crawler checkfor links in EVERY document-
type! This could slow down the crawling-process dramatically (e.g. if the crawler receives large
binary-files like images and tries to find links in them).