Web Crawlers and Web Crawling Algorithms - A Review
Keywords:
Web Crawlers, Web Crawling, Depth First Search (DFS), Breadth First Search (BFS), Query ProcessingAbstract
As there is profound web development, there has been expanded enthusiasm for methods that help productively find profound web interfaces. Because of accessibility of inexhaustible information on web, seeking has a noteworthy effect. On-going examines place accentuation on the pertinence and strength of the information found, as the found examples closeness is a long way from the investigated. Notwithstanding their importance pages for any inquiry subject, the outcomes are colossal to be investigated. One issue of pursuit on the Web is that internet searchers return huge hit records with low accuracy. Clients need to filter applicable reports from insignificant ones by physically bringing and skimming pages. Another debilitating viewpoint is that URLs or entire pages are returned as list items. It is likely that the response to a client question is just part of the page. Recovering the entire page really leaves the errand of inquiry inside a page to Web clients. With these two viewpoints staying unaltered, Web clients won't be liberated from the substantial weight of perusing pages and finding required data, and data got from one pursuit will be characteristically constrained.
References
- SmartCrawler: A Two-stage Crawler for Efficiently Harvesting Deep-Web, Feng Zhao, Jingyu Zhou, Chang Nie, Heqing Huang & Hai Jin, Interfaces .IEEE Transactions on Services Computing Volume: PP Year: 2015.
- Survey of Web Crawling Algorithms - Rahul kumar, Anurag Jain and Chetan Agrawal. Advances in Vision Computing: An International Journal (AVC) Vol.1, No.2/3, September 2014.
- Algorithms and Programming Problems and Solutions, Shen and Alexander, Springer Undergraduate Texts in Mathematics and Technology 2010
- Ben Coppin “Artificial Intelligence illuminated” Jones and Barlett Publishers, 2004, Pg 77.
- Dempster-Shafer theory for a query-biased combination of evidence on the Web- Vassilis Plachouras, Iadh Ounis. Springer-Verlag Berlin Heidelberg 2014.
- Effective Pre-retrieval Query Performance Prediction Using Similarity and Variability Evidence, Ying Zhao, Falk Scholer, and Yohannes Tsegay, C.Macdonald et al. (Eds.): ECIR 2008, LNCS 4956, pp. 52–64, 2008._c Springer-Verlag Berlin Heidelberg 2008.
- Inferring Query Performance Using Pre-retrieval Predictors, Ben He and Iadh Ounis, Department of Computing Science University of Glasgow fben,ounisg@dcs.gla.ac.uk.
- A Unified Framework for Post-Retrieval Query-Performance Prediction, Oren Kurland, Anna Shtok, David Carmel, and Shay Hummel, ICTIR 2011, LNCS 6931, pp. 15–26, 2011. c_Springer-Verlag Berlin Heidelberg 2011.
- Varying Approaches to Topical Web Query Classification , Steven M. Beitzel, Eric C. Jensen, Abdur Chowdhury,& Ophir Frieder, SIGIR’07, July 23–27, 2007, Amsterdam, The Netherlands, ACM ..
- Survey on – Self Adaptive Focused Crawler, Ms. Pallavi Wadibhasme, & Prof. Nitin Shivale, Pallavi Wadibhasme et al, / (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 6 (1) , 2015, 218-220 .
- Efficient Query Evaluation using a Two-Level Retrieval Process- Andrei Z. Broder, David Carmel, Michael Herscovici, Aya Soffer & Jason Zien.
- Evaluating Topic DrivenWeb Crawlers, Filippo Menczer, Gautam Pant,& Padmini Srinivasan
Downloads
Published
Issue
Section
License
Copyright (c) IJSRSET
This work is licensed under a Creative Commons Attribution 4.0 International License.