Skip to main content

What is Web crawler?

Image result for What is Web crawler?
A web crawler (also known as a web spider or web robot) is a program or automated script which browses the World Wide Web in a methodical, automated manner.
This process is called Web crawling or spidering.
  Many legitimate sites, in particular search engines, use spidering as a means of providing up-to-date data.
  Web crawlers are mainly used to create a copy of all the visited pages for later processing by a search engine, that will index the downloaded pages to provide fast searches.
  Crawlers can also be used for automating maintenance tasks on a Web site, such as checking links or validating HTML code.
  Also, crawlers can be used to gather specific types of information from Web pages, such as harvesting e-mail addresses (usually for spam).
_________________________
Labels:

Comments

Post a Comment

Popular posts from this blog

Black-box testing

Black-box testing is a method of software testing that examines the functionality of an application without peering into its internal structures or workings. This method of test can be applied virtually to every level of software testing: unit, integration, system and acceptance. It typically comprises most if not all higher level testing, but can also embody unit testing. Specific knowledge of the application's code/internal structure and programming knowledge in general is not required. The tester is aware of what the software is supposed to do but is not aware of how it does it. For instance, the tester is aware that a particular input returns a certain, invariable output but is not aware of how the software produces the output in the first place. In penetration testing, black-box testing refers to a methodology where an ethical hacker has no knowledge of the system being attacked. The goal of a black-box penetration test is to simulate an external hacking or cyber...
Post a Comment
Read more

How to Use Your Phone as a Webcam?

A quick search on Google Play throws up several webcam applications. We tried 5 of the top 10 results, but most didn’t work properly. From the ones that didn’t crash repeatedly, SmartCam failed to show the video, and DroidCam needs you to install an app on Windows to work. IP Webcam is easily the best of the lot because it works through your browser, which gives it the cross-platform compatibility. To set up a security camera on Android, follow these steps. 1. Connect your computer and the phone to the same Wi-Fi network. 2. Install the IP Webcam app on your smartphone. 3. Close all other camera apps. Force close them from the app switcher before you proceed. 4. Launch the IP Webcam app. Scroll to the bottom and tap Start server. 5. The app will now fire up your phone’s camera and display a URL. Our URL was http://172.32.15.110:8080. 6. Enter this URL in any browser on your computer and hit Enter. 7. In the browser, you’ll see a drop down menu n...
Post a Comment
Read more