Web crawler in Java :

Definition :

A Web crawler is a computer program that browses the World Wide Web in a methodical, automated manner or in an orderly fashion.

For more information visit the wiki page for Web_crawler

Sample Java Program :

package com.test.main;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;

public class WebCrawler {

/**
* @param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
try {
URL url = new URL(“http://blog.iguddy.com/”);
BufferedReader br = new BufferedReader(new InputStreamReader(url.openStream()));
String temp = “”;
while(null != (temp = br.readLine())){
System.out.println(temp);
}
} catch (Exception ex) {
ex.printStackTrace();
}

}

}