Skip to main content

What is Web crawler?

Image result for What is Web crawler?
A web crawler (also known as a web spider or web robot) is a program or automated script which browses the World Wide Web in a methodical, automated manner.
This process is called Web crawling or spidering.
  Many legitimate sites, in particular search engines, use spidering as a means of providing up-to-date data.
  Web crawlers are mainly used to create a copy of all the visited pages for later processing by a search engine, that will index the downloaded pages to provide fast searches.
  Crawlers can also be used for automating maintenance tasks on a Web site, such as checking links or validating HTML code.
  Also, crawlers can be used to gather specific types of information from Web pages, such as harvesting e-mail addresses (usually for spam).
_________________________
Labels:

Comments

Post a Comment

Popular posts from this blog

HOW TO BYPASS OR UNBLOCK BLOCKED SITES:-

Today everyone of us is a part of any college,office or any organization. Everywhere we find free open wifi’s , but the major problem with these are there are many websites which are blocked by the servers of college,ofiice or any organization. There are different type of securities that are implemented in these server to block some sites like Facebook ,Twitter etc. So In this post i’ll tell you easy and permanent method to access or bypass blocked sites Steps To Bypass Blocked Sites-   1. Connect your pc or laptop with wifi or your broadband connection in which you want to access blocked websites.   2. Open command prompt by clicking start and the typing “cmd” and then pressing enter.   4. 3. Now a command prompt window will appear.   5. Type ping Websitename (for exp: ping http://facebook.com) and press enter.   6. Now open your browser and enter the site which you had recently ping.   7.You will see the site will get...
Post a Comment
Read more

Black-box testing

Black-box testing is a method of software testing that examines the functionality of an application without peering into its internal structures or workings. This method of test can be applied virtually to every level of software testing: unit, integration, system and acceptance. It typically comprises most if not all higher level testing, but can also embody unit testing. Specific knowledge of the application's code/internal structure and programming knowledge in general is not required. The tester is aware of what the software is supposed to do but is not aware of how it does it. For instance, the tester is aware that a particular input returns a certain, invariable output but is not aware of how the software produces the output in the first place. In penetration testing, black-box testing refers to a methodology where an ethical hacker has no knowledge of the system being attacked. The goal of a black-box penetration test is to simulate an external hacking or cyber...
Post a Comment
Read more