The Internet (Deep and Dark Web)
KeywordsInternet of things Search engines the Web Social networks World-Wide-Web
The Internet is a global computer network that interconnects over 23 billion digital devices (Statista 2018).
Originally intended to interconnect just a few hundred mainframe computers in universities and corporate and government labs, the Internet now enables desktops, laptops, cell phones, and just about anything that contains a processor to access information or connect with other devices anywhere in the world. Often categorized as the Internet of Things (IoT), in recent years billions of devices such as consumer appliances, sensors, and even medical devices have been connecting to the Internet. These devices soon will outnumber computers (Tung 2017a). Estimates show that by 2020 there will be four devices connected to the Internet for every person on Earth (Tung 2017b).
The Internet is, however, much more than a physical global network for interconnecting devices. The Internet also provides the essential services that allow software applications and devices to function together and exchange information reliably. Internet communication protocols have allowed developers to produce the Internet applications upon which we now rely: the web, search engines, social networks, and many other web applications. Services include an addressing and naming system so devices can locate each other, mechanisms for reliable transmission of information over what is essentially an unreliable, best effort delivery network, and even mechanisms for ensuring that information stored in different formats on different devices can be interchanged. Without such services, applications such as the World Wide Web would not exist.
Internet History and Development
Much of the underlying Internet technology was developed over 30 years ago in several US government-funded projects (Leiner 1997). In the 1960s and 1970s, the Defense Advanced Research Projects Agency funded the initial prototype network known as the ARPANET, an effort to connect multiple heterogeneous networks through a system of gateway computers. In 1983, the ARPANET made the transmission control protocol/Internet protocol (TCP/IP) the standard communication protocols used in the network. Thus hosts and networks that made used TCP/IP standards could be added to the ARPANET.
In the mid-1980s, the National Science Foundation played a key role with its funding of NSFNET whose purpose was to interconnect supercomputing facilities at government laboratories and universities (National Science Foundation 2018). This project led to the development of a backbone network for the Internet and demonstrated its utility. The program paved the way for the transition to current model where private telecommunication companies provide most of the Internet network infrastructure and the service fees they charge finance infrastructure maintenance and development.
Today the Internet is a complex system of interconnected networks mostly owned by private sector telecommunication companies known as Internet Service Providers (ISPs). In some countries the networks are government owned and controlled. There are local and regional service providers as well as core service providers who interconnect regional providers and offer a type of backbone for the Internet. Moreover, in recent years, major content providers such as Google have developed their own global networks separate from the public Internet to interconnect their data centers. These content provider networks connect to ISPs to access the public Internet and attempt to avoid using the Internet core where possible. Typically, they rely on the public Internet to reach end users (Google 2018).
The Internet designers introduced a number of network technologies and made a number of technical decisions that have allowed the Internet to scale from a few hundred machines to billions of devices (Leiner 1997). One key technology was the packet-switched network. In such networks, a large file to be transmitted, for example, is broken into multiple packets which are then transmitted over the network and reassembled at the destination. Packets may not even follow the same path from source to destination. A second crucial idea was to keep the complexity and computation at the end points. Most processing involved in moving traffic is done at the sending and receiving end points, with minimal processing on gateway devices that move traffic through the network. This was a departure from the way telecommunications worked at the time where if a sender and receiver wanted to communicate, intermediate switches had to be configured to make the connection. A third key component was the Internet addressing and routing systems that allowed networks to be added to the Internet with minimal impact on the existing system.
Of course, the Internet has benefited from the incredible increase in computational powered realized during the past 30 years. The switches and routers, key building blocks that move traffic, have increased in speed by at least three orders of magnitude during this period. Nonetheless the decision to keep processing on intermediate switching/routing devices to a minimum is a key factor in the scalability of the network. In addition, separating the physical network layer from other network layers allowed the massive investment in software applications at the network, and transport and application layers of the network to remain in place as underlying network hardware technologies evolved and improved.
From the beginning the technical specifications have largely been in the hands of committees of stakeholders who work together to devise the open or publicly available communication protocols and specifications needed to make the Internet work. With open protocols talented programmers anywhere in the world could develop network services and applications and improve on existing software.
Internet Management and Governance
Governance of a valuable shared resource such as the Internet, a critical information infrastructure for commercial activity and many competing interests, requires technical standards as well as policies needed to address issues such as privacy, security, and access (Global Commission on Internet Governance n.d.). As a global system, the Internet has no central manager or authority. Each nation oversees the activities that take place on the Internet in that country – some are very open, while others are highly regulated. In addition, the private sector plays a key role through user terms of service agreements both for Internet access and use of various web applications. From the beginning, technical standards have followed the multi-stakeholder approach, a distributed form of governance that includes developers, users, and others with an interest.
The Internet Society (Internet Society 2018) is the umbrella organization that provides a home for and financial support for the Internet standards process. Two organizations manage the actual development of technical standards, the Internet Engineering Task Force (IETF), which oversees the development of detailed technical protocol standards, and the Internet Architecture Board (IAB), which considers the longer-term development of the Internet. The IETF publishes a set of technical protocol standards called Requests for Comments (RFPs) which define the standards for communication protocols and many Internet applications (IETF 2018). Another key organization, the Internet Corporation for Assigned Names and Numbers (ICANN), manages the Internet/Domain Name System, which provides names and addresses for Internet computers and services (ICANN 2018).
Two contentious Internet governance issues concern access and privacy. The net neutrality debate centers on the ability of an ISP to prioritize Internet traffic and provide higher levels of service for customers willing to pay for it. The fear is that customers who cannot afford a premium service will receive a lower level of service and be denied the full utility of the Internet. The European Union supports net neutrality (European Commission 2018a), while the Federal Communications Commission in the USA recently started allowing ISPs to charge more for preferred services. The issue of ISPs making use of customer data is being addressed by a number of states in the USA with some states passing legislation to restrict ISPs from accessing customer data without their explicit permission (National Conference of State Legislatures 2018). The European Union has a strict privacy regime which regulates the way ISPs and Internet search and social network companies handle user data (European Commission May 23, 2018b).
Deep and Dark Web
The deep web and the dark web characterize portions of the World Wide Web. Many conflate the two terms, but there is a distinct difference. Many consider the dark web to be a subset of the deep web. Both terms refer to web sites not generally accessible to search engines. Access to dark web sites requires special software which protects the anonymity of the both the web site and the browser computer. Web sites readily accessible to search engines and indexed by them are often referred to as being part of the Surface Web, which most estimates put at under 5% of the entire World Wide Web.
The deep web includes web pages that cannot readily be accessed by search engine web crawlers. Thus such pages are not indexed and won’t appear in a list of search results. Examples of such pages include those protected by captchas, pages that require a user login, pages returned when one fills out a form, or paywall-protected digital media content.
Access to dark web sites typically requires the use of the Tor browser (TOR 2018). This browser makes it very difficult for someone monitoring a network to identify the address of either the browser or the web site being accessed. TOR often is used to browse the Surface and deep web as well when anonymity is desired. Dark web users often take the extra precaution of using a virtual private network for additional assurance of anonymity.
Dark web sites don’t use extensions such as .org or .com but .onion, which is not part of the Internet/Domain Name System. Sites available on the Surface Web provide lists of sites in the dark web. The anonymity afforded by the dark web makes it widely used for nefarious activities including black markets, botnet operations, selling hacking services and malware, and a range of criminal activities. In addition, dark net sites also host whistleblowing forums and political discussion groups that might be suppressed in certain countries.
The Internet and the applications it supports are critical to economic and social activities throughout the world. As an indication of how much the world has grown to rely on the Internet as an information source, Google currently processes over 3.5 billion searches each day up from about 3.5 million per day when it was founded in 1998 (Internet Live Stats 2018). In less than a generation, Internet-based applications such as social networks, search engines, online shopping and advertising, e-mail, and other applications have created entirely new industries and changed the way organizations and people interact and get information. A key challenge now is the governance of this shared resource so it continues to spawn innovation and development and at the same time protects the security of its users and makes the Internet work for everyone (Internet Society 2017).
- European Commission. (2018a). Open Internet. Retrieved from https://ec.europa.eu/digital-single-market/en/open-internet-net-neutrality
- European Commission. (2018b, May 23). Data Protection in the EU. Retrieved from European Commission: https://ec.europa.eu/info/law/law-topic/data-protection/data-protection-eu_en
- Global Commission on Internet Governance. (n.d.). Who runs the Internet? The global multi-stakeholder model of Internet governance. Ottawa: Centre for International Governance Innovation and the Royal Institute of International Affairs.Google Scholar
- Google. (2018). Our Infrastructure. Retrieved from Google: https://peerig.google.com/#/infrastructure
- ICANN. (2018). Get Started. Retrieved from ICANN: https://www.ican.org/get-started
- IETF. (2018). Internet Standards. Retrieved from https://www.ietf.org/standards/
- Internet Live Stats. (2018). Google Search Statistics. Retrieved from Internet Live Stats: http://www.internetlivestats.com/google-search-statistics/
- Internet Society. (2017). 2017 Global Internet Report: Paths to Our Digital Future. Retrieved from Internetsocity.org: https://www.internetsociety.org/globalinternetreport/?gclid=EAIaIQobChMIzJbOwqiL3QIVix-GCh0c5AGHEAAYASAAEgI9X_D_BwE
- Internet Society. (2018). About the Internet Society. Retrieved from Internet Society: https://www.internetsociety.org/about-internet-society
- Leiner, B. C. (1997). Brief History of the Internet. Internet Society. Retrieved from https://www.internetsociety.org/internet/history-internet/brief-history-internet/
- National Conference of State Legislatures. (2018). Privacy Regulation Related to Internet Service Providers. Retrieved from http://www.ncsl.org/research/telecommunications-and-information-technology/privacy-legislation-related-to-internet-service-providers-2018.aspx
- National Science Foundation. (2018). A Brief History of NSF and the Internet. Retrieved from National Science Foundation: https://www.nsf.gov/news/special_reports/cyber/internet.jsp
- Statista. (2018). Internet of Things connected devices installed base world wide from 2015 to 2025. Retrieved from Statista: https://www.statista.com/statistics/471264/iot-number-of-connected-devices-worldwide/
- TOR. (2018). What is the Tor Browser. Retrieved from TOR: https://www.torproject.org/projects/torbrowser.html.en
- Tung, L. (2017a, Feb 7). IoT Devices will outnumber the world’s population for the first time this year. Retrieved from ZDNet: https://www.zdnet.com/article/iot-devices-will-outnumber-the-worlds-population-this-year-for-the-first-time/
- Tung, L. (2017b, Feb 7). ZDNet. Retrieved from IoT Devices Wil Outnumber the World’s Population for the First Time This Year: https://www.zdnet.com/article/iot-devices-will-outnumber-the-worlds-population-this-year-for-the-first-time/
- Finklea, K. (2017, Mar 10). Dark Web. Congressional Research Service. Retrieved from Congressional Research Service: https://fas.org/sgp/crs/misc/R44101.pdf