public setContentSizeLimit($bytes)
$bytes | int | The limit in bytes. |
bool |
If the crawler is receiving the content of a page or file and the contentsize-limit is reached, the crawler stops receiving content
from this page or file.
Please note that the crawler can only find links in the received portion of a document.
The default-value is 0 (no limit).