"Our 300M dedicated line, now download files only 300KB/s, the operator came twice and said the line is OK." On the morning of August 21, the executive director of a design company in Panyu was crying on the phone. The design company relied on large files for food, and a few GB of renderings, which was equal to the company shutdown.
This "carrier says the line is fine, users say the network speed is slow" dispute is probably on the intranet. We brought a line meter, laptop and a backup switch on-site.
Step 1: First prove that the external network is OK
Connect the laptop directly to the LAN port of the egress firewall. Dial test: speedtest downlink 295Mbps, ping the carrier gateway for 1ms, packet loss 0%This section from the external network to the firewall is completely normal, and the problem is in the internal network.This is an important step-to use the data to draw clear lines of responsibility and avoid wasting time in the wrong direction.
Step 2: Look at the switch indicator, the loop will "talk"
Their intranet structure is: firewall → one Huawei S5720 core switch → four TP-Link fool switches (TL-SG1024D, 24-port gigabit) distributed in each office area.
Go to the second office area and see the problem at a glance: the TL-SG 1024DAll port indicators are flashing in crazy syncUnder normal conditions, each port should flash randomly according to its own traffic. All ports flash at the same frequency, which is a typical feature of broadcast storms. Touch the fuselage again, hot.
Step 3: Grab the bag and hammer
Laptop received this switch free port, wireshark capture 10 seconds, more than 80,000 packets, of which 96% are broadcast frames (ARP request and destination MAC for FF: FF frame), the source MAC address in several office computers repeatedly cross jump between. This is the loop: a broadcast frame to come in, from all ports copied out, and was sent back another path, infinite loop amplification, a few seconds can put the gigabit link bandwidth and the CPU of the switch all ate up.
Looking along the network cable, the truth is simple:Last week, the new station was adjusted, and the intern had an extra network cable, which was inserted into port 12 and port 18 of the same switch.A fool switch has no Spanning Tree Protocol, and a loop can turn itself into a broadcast amplifier.
Step 4: Rectify, not just unplug the cable
Unplug the cable, the storm subsided within 10 seconds, and the entire network recovered to 295Mbps. But only unplug the cable without changing the structure, and someone will plug it back in next time. We did three things:
| rectification item | specific practice | cost |
|---|---|---|
| Core Switch Open Protection | Huawei S5720 enables RSTP globally, with stp edged-port enable + stp bpdu-protection on the downstream port. Upon receipt of BPDU, the shutdown port is shut down and an alarm is issued. | Equipment purchased, configuration free |
| Replace Access Switch | Three of the four stupid machines continue to be used (budget reasons), and the most critical second floor is replaced by H3C S5120V3- 28P (STP/loop detection is supported), about RMB 1800. | RMB 1800 |
| Port Identification and Training | All network ports are labeled (the uplink port is marked with red "No access to computers"), and a 15-minute training is given to the administration: No one is allowed to connect the network cable without permission. | free of charge |
Retest in the afternoon of the rectification day: the speed measurement of the four office areas was above 280Mbps at the same time, and the CPU occupation of the switch was reduced from 100% to 12%. After two weeks, everything was normal.
Why are stupid switches most afraid of loops?
Managed switches have STP (Spanning Tree Protocol), which can automatically discover loops and set one of the links to a blocking state, logically "cutting" the loop. Fool switches do not have this ability, nor do they have loop detection. When encountering loops, they are completely paralyzed. So our principle is:For office environments with more than 20 network ports, there must be at least one management switch at the access layer.H3C/Huawei/Ruijie entry model can be bought for RMB 1500–2500, this money cannot be saved.
Office slow speed screening order (sorted by hit rate)
- look at that switch lightFull port synchronous flash = loop/broadcast storm; single port flash and no large traffic = suspicious.
- boundary velocity measurement: In the export equipment LAN port direct connection speed measurement, normal problems in the internal network, abnormal to find the operator.
- Check Wi-Fi or cable.Only wireless slow, wired fast, mostly AP channel collision or excessive load, and switch has nothing to do with it.
- Proportion of watching radio by catching packets: wireshark Statistics → Capture File Properties, broadcast more than 30% of the frames to suspect loops or poisoning (ARP virus will also broadcast storms).
- Check network cable and negotiate speed: Gigabit port negotiation into 100M half-duplex, the speed will drop precipitously, usually crystal head oxidation or line sequence error, line measuring instrument a test will know.
Design company, e-commerce company, direct broadcast room and other "network is productivity" scenarios, it is recommended to do a complete intranet physical examination: topology sorting, loop risk points, Wi-Fi coverage heat map, switch firmware version, on-site diagnosis within Guangzhou, report, diagnosis fee RMB 500 (subsequent rectification contract can be deducted).