Can you help me build a web crawler in PHP?

Jerry Yurow December 1, 2011

I am a rank beginner in PHP scripting. I want to build a web crawler that will find each internal link on a given website and place its information into an XML file. I want to capture each link’s name and its URL reference.

  1. Jeff Fabish
    December 1, 2011 at 10:34 pm

    Hi Jerry,

    No. I will not write the code for you, however James Bruce wrote an excellent article on building a PHP web crawler (part 2). I've included a list of PHP learning resources below. If I were you, I would start a little simpler. There are plenty of open source web crawlers of which you may 'borrow' the source from.

    PHP 101: PHP For the Absolute Beginner
    W3Schools - PHP Tutorial
    Tizag - PHP Tutorial
    PHPBuddy - PHP Tutorial
    LearnPHP - PHP Tutorial
    Quackit - PHP Tutorial
    KillerPHP (non-free) - PHP Tutorials (non-free) - PHP Programming: The Basics Tutorials (non-free) - Real World Programming With PHP 5 Tutorials (non-free) - PHP Tutorial (non-free)

    - Jeff