Readers like you help support MUO. When you make a purchase using links on our site, we may earn an affiliate commission. Read More.
I'm creating an app similar to Stumbleupon with a content discovery engine. So what would be the most simple and efficient web crawling software to use to index mostly web pages, photos, and videos?
2013-07-24 06:20:38
google webmaster is best.. as far as i used...
2013-04-07 18:48:02
Hmmm...some good ideas here.
2013-04-06 13:14:24
you can try teleport pro but it costs 49.95 $http://www.tenmax.com/teleport/pro/home.htm
2013-04-07 14:33:04
i agree with you mate, that one is good
2013-04-06 10:44:40
Here is a list: http://web-crawler.software.informer.com/although it must be said that I have no knowledge at all of this type of software but worth a look, it may fill your needs.
2013-04-06 05:57:37
Best website crawler is google webmaster. You can do everything in google webmaster. its not a software but more then a software.