Skip to main content

What is Web crawler?

Image result for What is Web crawler?
A web crawler (also known as a web spider or web robot) is a program or automated script which browses the World Wide Web in a methodical, automated manner.
This process is called Web crawling or spidering.
  Many legitimate sites, in particular search engines, use spidering as a means of providing up-to-date data.
  Web crawlers are mainly used to create a copy of all the visited pages for later processing by a search engine, that will index the downloaded pages to provide fast searches.
  Crawlers can also be used for automating maintenance tasks on a Web site, such as checking links or validating HTML code.
  Also, crawlers can be used to gather specific types of information from Web pages, such as harvesting e-mail addresses (usually for spam).
_________________________
Labels:

Comments

Post a Comment

Popular posts from this blog

HOW TO BYPASS OR UNBLOCK BLOCKED SITES:-

Today everyone of us is a part of any college,office or any organization. Everywhere we find free open wifi’s , but the major problem with these are there are many websites which are blocked by the servers of college,ofiice or any organization. There are different type of securities that are implemented in these server to block some sites like Facebook ,Twitter etc. So In this post i’ll tell you easy and permanent method to access or bypass blocked sites Steps To Bypass Blocked Sites-   1. Connect your pc or laptop with wifi or your broadband connection in which you want to access blocked websites.   2. Open command prompt by clicking start and the typing “cmd” and then pressing enter.   4. 3. Now a command prompt window will appear.   5. Type ping Websitename (for exp: ping http://facebook.com) and press enter.   6. Now open your browser and enter the site which you had recently ping.   7.You will see the site will get...
Post a Comment
Read more

How Hackers Look Like and What They Think?

Self-educated hackers are more respected in hacker’s community because that person is really interested in hacking and he passed a lot of things that are not teached in schools and college. The most common things hackers do is computers electrical engineering, physics, mathematics, linguistics and philosophy. However, like every developer is not the hacker, hackers do not always have the skills of programming, but someone of them have. :) Clothing styles: Hackers dress simply, casually: jeans, T-shirt and shoes. T-shirts are usually with some humorous slogans. today there are T-shirts with pictures of Penguin (trademark Linux) or daemon (BSD). A small number of hackers prefers hiking boots. This prefer wearing of black clothes. Hackers care more about comfort, practicality and ease of maintenance of clothes. They hate business suits. OTHER INTERESTS: Hobbies that hackers do are widespread. Mostly they like science fiction, music, medievalism, chess, war games and intel...
Post a Comment
Read more

What is Pendrive Hacking ?

Windows allows the storage of the passwords, as do modern browsers. While this feature is convenient for users, it has imposed itself as a big security risk among organizations. We know that browsers store most passwords on daily basis, like MSN messenger, Yahoo, Facebook passwords, etc. Most people lack time and ask their browsers to save their passwords. As we know, there are many tools available to recover saved passwords, so in this article I will explain to you how to make a USB password stealer and steal saved passwords. Just to explain the concept, we are going to collect some password stealing tools, tools that are freely available on the internet and capable of stealing the passwords stored in the browsers or other windows files. Then, we create a batch program that will execute these combined programs and store the stolen usernames and passwords in a text file. To further spice up the penetration testing demonstration, we will also make this batch file execute as an...
Post a Comment
Read more