PHPCrawl webcrawler library/framework

Requirements


At least the following requirements are necessary to run phpcrawl (v 0.8) in basic single-process-mode:
  • PHP 5.2.1 or later version
  • PHP with OpenSSL-support for SSL-connections (https).
    Not necessary for http-connects.
In order to run phpcrawl in multi-process-mode, some additional requirements are needed:
  • The multi-process mode only works on unix-based systems (linux)
  • Scripts using the crawler in multi-process-mode have to be run from the commandline (PHP cli)
  • The PCNTL-extension for php (process control) has to be installed and activated.
  • The SEMAPHORE-extension for php has to be installed and activated.
  • The POSIX-extension for php has to be installed and activated.
  • The PDO-extension together with the SQLite-driver (PDO_SQLITE) has to be installed and activated.