Few things have an air of mystery around them like the functioning of the internet. We rarely see how they operate or change their configuration. We are rarely fully aware of what’s going on so that we can make the most performant and secure decisions. In what I hope will be a few part series, I’d like to take aim at the mystery and replace it with practical knowledge.
While we are going to go deep to on various components, it will help to have familiarity with some concepts or know where to research them. Somebody early in a university program or deep in a bootcamp should feel at home. As an introduction, we’ll start by tracing a web request to lay out the components & systems. We’ll dive into how home networks generally work & how machines hop on a network. Finally, we’ll finish talking about corporate networks with a particular focus on how they serve your web traffic.
Let’s start with a common web request. You have a question, and you know Google can help find an answer. So you fire up a web browser and type www.google.com into the URL bar and hit enter. What actually happens? How does your computer find google? And how does the data actually move?
The first thing that happens is an address lookup via the Domain Name System. This system translates the human-readable names (www.google.com) into the IP addresses machines understand (something like 172.217.13.228). With that information in hand, the browser can make the actual request.
But Google is not physically connected to your computer. So a path must be found through the internet(which encompasses your home network, your service provider, Google itself, and maybe a few others) from your computer to a Google server. And the IP address is the key.
Your home router understands its role as intermediary between your home network and your service provider’s network. And your home computer has a rule to say roughly “when I want to reach an IP address that isn’t on my home network, ask the router to forward it on.” That rule is known as the “default route” and your router, in this case, is the default gateway.
Once the message is on the service provider’s network, the message will traverse a series of routers with more complicated (and constantly changing) rules about how to pass the message on. This game of telephone continues until, eventually, the message arrives at routers owned by Google, who also dutifully pass the message along. For now, we’ll assume that the IP address is enough to get to the message to a machine that can respond with the data representing Google’s search page. And our message is delivered and we get a web page. But as we’ll see later, that’s just scratching the surface.
Until now, we haven’t thought much about the message being sent to Google. The truth is that one request generates a flurry of messages. And those messages are intermixed with all of the other messages flying across the network at once. So how your machine separate the conversation going on with Google from the one going on with Netflix?
To accomplish the communication, your web browser hands the message off to a system under the covers. This system subdivides the message, takes responsibility for some aspect of it’s delivery, and hands it to another system under its covers. Each system makes the message more generic to the point that the base system has no idea what the message is — it is just exchanging electrical pulses between physical computers. Everything about the message is reconstructed in a path opposite from the deconstruction. These layers of responsibility are called the OSI Model.
If it all seems a bit abstract, it might be useful to think of a network as being a lot like a post office. It doesn’t matter if you’re mailing a wedding invitation or a postcard. The post office just sees an address/postage and (most of the time), your mail gets delivered. The post office is very unaware that a wedding invitation requires a mailed response. And definitely doesn’t care about your attendee’s preference for the chicken or fish. Those details are the responsibility for a layer above what the post office promises to do.
To bring this back to our web request example, the top layer, known as the application layer, roughly says “Please get me the search page.” It is responsible for knowing what you were asking for and how to interpret the response. A few layers down (Layer 4, the transport layer), a system called TCP is responsible for ordering, should the message get broken up. Say “Please get me the search page” was too large to fit over the network in a single chunk (different networks have different max transmission units, or MTUs). The TCP system is allowed to break that up into something like:
Message Part 0: “Please get me”
Message Part 1: “the search page”
The application layer has no idea that happened. TCP manages the splitting and recombination (in the correct order). TCP, in turn, delegates the actual delivery to lower layers. Layer 3 deals with the IP addresses how to make sure messages get to the correct destination. Layers 1 and 2 get into the details of how to turn a message into electrical impulses over a wire (or through wifi, over microwave, etc).
At this point, you have the basics. A machine on a network uses DNS to translate human-readable names into machine-readable names. The browser constructs a message, which is sent through several layers of software to be broken up and managed so that it can ultimately be sent as pulses through a wire, frequencies through the air, or flashes of light through fiber optics. But several questions remain. How does a machine get onto a network so that it has an IP address and know where to send DNS requests? How do routers know where to forward traffic?
And most interestingly, how can one server possibly handle so much traffic? Spoiler: it doesn’t & the illusion that such a server exists is the magic where web development gets really interesting. Stay tuned…