By Bill Morton
Did you know that when you Google “Red Wolves,” Google will only search pages that it has indexed, which is only about four percent of the total World Wide Web (Why Businesses Needs to Understand the Deep Web, 2013). The other ninety-six percent is known as the Deep Web, or Invisible Web. This unknown realm is known for housing terroristic communication, gun and drug trading, assignation bids, and even child pornography. But the Deep Web also holds private files for sharing and storing for business, academic, and personal use. Like typical websites, these pages are still encrypted and decrypted, but typical search engines will not have the websites private key needed to decrypt these pages therefor cannot be indexed. The Darknet is the specific part of the Deep Web that houses the illegal activity. This is mainly because it is ran by private networks link from peer-to-peer. It is suggested that when exploring this part of the web a user will need use a browser that will keep his/her location anonymous. The Onion Router, or Tor, is an example of such browser. (Tor: Overview, n.d.) How Tor achieves this by bouncing a connected computers IP address, which shows the location of where a user is connecting from, i.e. home, to volunteer nodes where it hides the IP address under many layers, like an onion. Why? Well this is a criminal underworld full of hackers and other cyber criminals.
The Deep Web, as defined in Learning to Crawl the Deep Web, is the portion of the World Wide Web that is not a part of the surface web; in other words, web pages that is not indexed by search engines. Search engines usually use “crawler” to index web pages, following links, and repeat. This will continue until it ...