Network Triangles, Part 2

How does the switch know where to send your data?

In part 1, I mentioned the Ethernet header. This is what it looks like.

Destination MAC 6 bytes
Source MAC 6 bytes
Length 2 byte
Data 46-1500 Bytes
CRC 4 Bytes

The statement was made that a switch only sends a packet out on a port where the destination device is connected. Did you catch that, the switch knows something key here.

Never trust anything that can think for itself if you can’t see where it keeps its brain.  ~J.K. Rowling, “Dobby’s Reward,” Harry Potter and the Chamber of Secrets, 1999, spoken by the character Arthur Weasley

The switch knows were to send and where not to send the data based on what device is on a port. This is called port based routing.

We can think of the switch keeping a big table of which device is on what port.

Port 1 Port 2 Port 3 Port 4
d0:bf:9c:25:ab:98 d0:bf:9c:02:02:01 d0:bf:9c:02:02:01 d0:bf:9c:02:02:04
00:ab:cd:00:03:01 00:bc:de:a1:a2:34

When the switch receives a packet it looks in the table for the destination. That tells it which port to use when forwarding the packet.

Switches learn where things are on the network by looking at the packets they receive. This is not something we have to configure. If it was, you would need a team of dedicated data entry people recording network changes 24/7. It won’t scale.

So, what should a switch do if it does not know? If the destination is not in the table?

Easy, Act like a hub and send it everyplace.

Why Triangles are bad

Not evil, but bad. A loop in a simple Ethernet network is evil. A triangle merely bad.

I explained how our devices worked to my boss (who knew obviously) and the FAE.  The phone sends to the gateway, the gateway sends to the QoS box, and the QoS box sends to the phone.

The switch looks at the Ethernet header every time it receives a packet. It takes the senders address and puts that MAC into the table under the port the packet was received from.

So for a switch to know where to send something, that switch has to have already received a packet from the destination. If the switch has not received anything, the MAC address is not in the local table, and it acts like a hub.

I explained, “In the new network setup, phones send to the gateway. The gateway never sends directly to the phone. Well, only when the phone it turned on and first checks itself in to find a gateway.” They gave me a look, like when a dog turns it’s head wondering if you have a treat.

Real switches don’t use tables. That would be slow, there are hardware hash tables and such things. Plus, a switch can not keep the table forever. Eventually, it would run out of memory. If nothing has been received from a given MAC for a long time (about 10-20 minutes) the entry in the table is dropped. The table is not infinite. With some hacking we found a small Linksys switch had a 512 entry table. The Cisco switches were around 10K.

“The switch in the other building only has phones connecting to it. The gateways and QoS box are on this side if the street. Those switches have never seen a packet from a gateway. When the phone sends to the gateways, the switch has no entry in the table. It does not know what to do, so it broadcasts the packet, like a hub.” My boss caught on at this point and made the obvious statement, “oh, shit.” “The packets sent everywhere get bridged to the WiFi network, causing to much bandwidth to be used, and all sorts of ugly network problems.”

The FAE was not following. I went on, “A switch assumes two computers talk to each other. So, when one sends the other replies. The network in between sees all of the packets and then has a complete route between the two devices conversing on the network. We made a triangle, not two things talking to each other, but three in a round robin.” Understanding dawned in his eyes.

In five minutes the meeting was over. The FAE was saved from interrogation.

The fix

That took a while and is good for 3 or 4 more stories with good lessons about networks. Sorry, but not for today.

How did I get this insight? I had just read Internetworking by Radia Perlman.  I think it is one of the foundational books to understand how networks actually work. I loaned my copy to someone and never got it back, sigh.

The book goes much deeper into the way switches work. I had to keep it simple to limit the size of this blog post.

Remember this one thing. Networks have very simple rules. A switch works by the rule, “send packets out the same port a packet with the matching address can in”. the behavior becomes complex, just by applying one simple rule.

I think that is really cool, and fun to work with.

Leave a Reply

Your email address will not be published. Required fields are marked *