Retrieval of Data from Hidden Web using Web Agent

Abstract

Abstract
There is an increase in the number of data sources that can be queried across the World Wide Web (WWW). Such sources typically support HTML forms-based interfaces and search engines query collections of suitably indexed data. The data are displayed via a browser. As the Web is growing in recent years, huge amount of data are available under dynamic forms of publication, which are accessed by an HTML Form, known as legacy databases (known as hidden Web). To handle such types of situation fast generation of agents is required that can automatically fetch pages for further processing. In this paper, a method is explained for automatically generating agents to collect hidden Web pages (deep web pages). In this method a preexisting data repository is used for identifying the contents of these web pages. Number of experiments has been carried out with sites from different domains to show the accuracy of method.

Keywords: Wrapper, hidden web, web agent, WWW, repository