IJSRSET calls volunteers interested to contribute towards the scientific development in the field of Science, Engineering and Technology

Home > IJSRSET1621137                                                     

Web Crawlers and Web Crawling Algorithms - A Review


Vishakha Shukla, Dharmendra Roy
  • Abstract
  • Authors
  • Keywords
  • References
  • Details
As there is profound web development, there has been expanded enthusiasm for methods that help productively find profound web interfaces. Because of accessibility of inexhaustible information on web, seeking has a noteworthy effect. On-going examines place accentuation on the pertinence and strength of the information found, as the found examples closeness is a long way from the investigated. Notwithstanding their importance pages for any inquiry subject, the outcomes are colossal to be investigated. One issue of pursuit on the Web is that internet searchers return huge hit records with low accuracy. Clients need to filter applicable reports from insignificant ones by physically bringing and skimming pages. Another debilitating viewpoint is that URLs or entire pages are returned as list items. It is likely that the response to a client question is just part of the page. Recovering the entire page really leaves the errand of inquiry inside a page to Web clients. With these two viewpoints staying unaltered, Web clients won't be liberated from the substantial weight of perusing pages and finding required data, and data got from one pursuit will be characteristically constrained.

Vishakha Shukla, Dharmendra Roy

Web Crawlers, Web Crawling, Depth First Search (DFS), Breadth First Search (BFS), Query Processing

  1. SmartCrawler: A Two-stage Crawler for Efficiently Harvesting Deep-Web, Feng Zhao, Jingyu Zhou, Chang Nie, Heqing Huang & Hai Jin, Interfaces .IEEE Transactions on Services Computing Volume: PP Year: 2015.
  2. Survey of Web Crawling Algorithms - Rahul kumar, Anurag Jain and Chetan Agrawal. Advances in Vision Computing: An International Journal (AVC) Vol.1, No.2/3, September 2014.
  3. Algorithms and Programming Problems and Solutions, Shen and Alexander, Springer Undergraduate Texts in Mathematics and Technology 2010
  4. Ben Coppin “Artificial Intelligence illuminated” Jones and Barlett Publishers, 2004, Pg 77.
  5. Dempster-Shafer theory for a query-biased combination of evidence on the Web- Vassilis Plachouras, Iadh Ounis. Springer-Verlag Berlin Heidelberg 2014.
  6. Effective Pre-retrieval Query Performance Prediction Using Similarity and Variability Evidence, Ying Zhao, Falk Scholer, and Yohannes Tsegay, C.Macdonald et al. (Eds.): ECIR 2008, LNCS 4956, pp. 52–64, 2008._c Springer-Verlag Berlin Heidelberg 2008.
  7. Inferring Query Performance Using Pre-retrieval Predictors, Ben He and Iadh Ounis, Department of Computing Science University of Glasgow fben,ounisg@dcs.gla.ac.uk.
  8. A Unified Framework for Post-Retrieval Query-Performance Prediction, Oren Kurland, Anna Shtok, David Carmel, and Shay Hummel, ICTIR 2011, LNCS 6931, pp.  15–26, 2011. c_Springer-Verlag Berlin Heidelberg 2011.
  9. Varying Approaches to Topical Web Query Classification , Steven M. Beitzel, Eric C. Jensen, Abdur Chowdhury,& Ophir Frieder, SIGIR’07, July 23–27, 2007, Amsterdam, The Netherlands, ACM ..
  10. Survey on – Self Adaptive Focused Crawler,  Ms. Pallavi Wadibhasme, & Prof. Nitin Shivale, Pallavi Wadibhasme et al, / (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 6 (1) , 2015, 218-220 .
  11. Efficient Query Evaluation using a Two-Level Retrieval Process- Andrei Z. Broder, David Carmel, Michael Herscovici, Aya Soffer & Jason Zien.
  12. Evaluating Topic DrivenWeb Crawlers, Filippo Menczer, Gautam Pant,& Padmini Srinivasan

Publication Details

Published in : Volume 2 | Issue 2 | March-April - 2016
Date of Publication Print ISSN Online ISSN
2016-03-30 2395-1990 2394-4099
Page(s) Manuscript Number   Publisher
258-260 IJSRSET1621137   Technoscience Academy

Cite This Article

Vishakha Shukla, Dharmendra Roy, "Web Crawlers and Web Crawling Algorithms - A Review", International Journal of Scientific Research in Science, Engineering and Technology(IJSRSET), Print ISSN : 2395-1990, Online ISSN : 2394-4099, Volume 2, Issue 2, pp.258-260, March-April-2016.
URL : http://ijsrset.com/IJSRSET1621137.php