On July 19-21, a Panyu auto parts manufacturing enterprise moved its entire factory from the old factory to a new factory 38 kilometers away. Their self-built server room has 42 rack servers (ERP, MES, file service, virtualization cluster), 2 Synology NAS (drawings and quality inspection images), 1 firewall, 2 core switches, and 1 UPS. There is only one requirement:ERP and MES downtime cannot exceed 4 hoursThe production line starts Sunday night.
This type of project is most likely to be underestimated. Many enterprises think that relocation is "unplugging, loading and plugging". In fact, the relocation of 42 sets of equipment requires 5 times the workload of preliminary preparation.
Phase 1: Zhou, inventory and planning before relocation
Asset inventory (1st Zhou): 42 servers were registered one by one-brand model, serial number, U bit, IP, bearer service, RAID configuration, dependency relationship. A table was made, which was the only basis for the whole project. During the inventory, it was found that 6 of the customer's own accounts were missing (no one remembered the tester bought three years ago), and 3 machines had been offline but were still eating electricity in the rack-cleaning them conveniently.
Labelling system (1st Zhou): Label both ends of each wire (format: local equipment-port → opposite equipment-port), label three places for each machine (front, side and top, because only part of the surface can be seen after packing). Label printed with film, no stylus-handwriting will paste after handling friction.This step of cutting corners will take ten times as long to repay when the new server room is put on shelves.
Confirmation of preconditions for new equipment room (No.2 Zhou): The new factory room is not "there is room", we checked 8 conditions item by item:
| Item | ask for | Actual situation of new equipment room |
|---|---|---|
| power supply | Independent loop, capacity ≥ rack total power ×1.5, dual better | Single-channel 32A×3 loop, total capacity is enough, but there is no single channel (ATS is recommended to be installed later) |
| UPS | Capacity matching, battery new | Newly purchased 2 sets of 6kVA online + external battery rack (lasting 40 minutes) |
| air conditioner | Refrigeration capacity ≥ total heat generation ×1.3, 24 hours operation | 2 sets of 5 precision air conditioners N+1, up to standard |
| bearing | Rack full load about 800-1200kg/㎡ | Server room on the first floor, bearing standard |
| external network link | Must be opened and tested before moving | 2 500M special lines (main and standby), opened 2 weeks in advance and passed the test |
| Structured Cabling | Cable rack, cable tray, fiber optic to desktop | Completion of construction and testing 10 days ahead of schedule (including link test report) |
| Grounding and Lightning Protection | Ground resistance ≤4Ω | Measured 1.8Ω, up to standard |
| Fire and Access Control | Gas extinguishing, access control records | Heptafluoropropane + swipe card access control, up to standard |
Network and IP Planning (No. 2-3 Zhou): Is there any change in the network segment of the old and new plant? If so, the IP references of ERP/MES client configuration, printer, scanner gun and PLC should all be changed, which is extremely risky. Our suggestion isKeep the original segment unchanged(10.10.0.0/16 overall translation), only physical position changes, so that the application layer zero changes. This decision reduces downtime from a possible 2 days to less than 4 hours.
Backup (3rd Zhou, most critical): Make a full backup before relocation-ERP Databases, MES Databases, virtualization cluster configuration, firewall policy, switch configuration. Export all to mobile hard drive + cloud.Relocation is an operation with physical risks (transportation bumps, power-on shocks), and should not be started without full backup.
Stage 2: Sequential design for batch migration
You can't move it all at once--moving it all means stopping it. We divide it into three batches according to business dependency and downtime tolerance:
| batches | equipment | time | business impact |
|---|---|---|---|
| mother batch | 12 non-core: file servers, testers, backup NAS, print services | Saturday 08:00-14:00 | None (no work on Saturdays) |
| the second batch | 18 virtualized clusters (VMware, vMotion migrate VMs first and then host), 2 NAS | Saturday 15:00-Sunday 06:00 | Partial virtual machine suspension, non-production business |
| the third batch | ERP + MES database server 8 sets, firewall, core switch, UPS | Sunday 06:00-10:00 | 3 hours and 40 minutes downtime(4 hours required) |
Key Tips:Core equipment moved last, but network first. On Saturday, after the first batch is moved, configure the core switches and firewalls of the new server room and connect them to dedicated lines to verify that the internal and external networks are smooth and VPN is available. In this way, when ERP is moved on Sunday, the network environment of the new server room has already run for 16 hours, eliminating the uncertainty of links and configurations-otherwise, 4 hours will definitely not be enough to adjust the network and restore services on Sunday at the same time.
Stage III: Specifications for removal, packaging and transportation
- Normal shutdown, not unplugging: The server goes through the system shutdown process, RAID card cache is written back (when BBU has power), NAS goes through DSM shutdown, virtualization host enters maintenance mode. Direct unplugging has a probability of damaging RAID metadata and file system.
- hard drive sequential marking: The hard drives of all RAID servers are labeled by slot number (1-4), andnon-tray transport(Transport with reel + shock packaging). If the order is out of order, the array may not be recognized or even trigger reconstruction. If there is a problem during reconstruction, it is a data disaster. This is the most common rollover point for relocation.
- shockproof packing: Antistatic bag + pearl cotton + original carton for each server (customized aviation box without original box), wrapping film fixed tray for equipment in rack. Van with air cushion suspension for transportation vehicles, no convertible (heavy rain and exposure in Guangzhou in July are risks).
- Fiber optic individual protection: Fiber jumper cannot be coiled too tight (bending radius is not less than 10 times the cable diameter, usually ≥3cm), with a special reel storage, joint wear dust cap. Broken fiber cannot be seen by the naked eye, plugged in is high packet loss, check can be checked all day.
- UPS battery transport: Lead-acid batteries are dangerous goods, must be disconnected, terminals wrapped with insulating tape, placed upright (do not upside down and side), packed separately. Mixed in the equipment box once a short circuit will catch fire. Waste batteries to qualified recycling party disposal.
Phase IV: Power-on sequence of new equipment room (wrong equipment will burn)
Power-on is not "all plugged in and powered on". It must be in the order of dependence. After each step is verified, the next step:
- UPS and power distributionPower on UPS first, confirm that input voltage is normal, battery charging is normal, output is stable (measured with multimeter), and then turn on PDU.
- network equipment: Core switch → firewall → access switch, open one by one, wait for each to start (about 2-5 minutes) and confirm that the port light is normal. The network is connected first, and the remote management of the server behind can be used.
- storage deviceNAS, SAN, storage array first open, wait for all disks to be ready, array state is healthy (this step may take 10-20 minutes, anxious to open the server will find that the mount storage fails).
- Virtualization Hosting: Open the host, wait for vCenter/cluster recovery, and start the virtual machine one by one (do not fully open it with one key, it will generate a boot storm, and instantaneous IO and power peaks may destroy UPS).
- Database and Application Server: Finally open ERP/MES, start database services, verify data consistency (DBCC CHECKDB).
- business verification: It is not "the server can be powered on", but the business side verification-ERP can log in and bill, MES can collect data, scanning gun can connect, printer can output paper, client can access sharing, external network can connect VPN. We have made a 32-item acceptance list in advance and checked and signed one by one.
At 09:40 on Sunday, all 32 items passed the acceptance, ERP/MES resumed online, and the actual shutdown lasted 3 hours and 40 minutes. On Monday, the production line started normally.
Three pitfalls to avoid (we've seen cases of customers carrying their own rollover)
Pit 1: RAID hard drive order disorder.A customer moved 8 servers by himself, took out all the hard drives and packed them separately, and inserted them back into the new server room. As a result, 3 arrays reported "foreign configuration", one of which triggered reconstruction and rebuilt to 60%. The other old disk could not hold up-RAID5 two disks were offline, and finally spent RMB 6800 to recover data. Correct practices: no disk extraction, slot label, shockproof transportation.
Pit 2: The external network link has not been opened in advance.On the day of relocation, the special line of the new server room was not connected, and the engineers stared in the server room without network. The business stopped for one and a half days. The special line opening cycle is usually 5-15 working days (involving the operator's construction), and the order must be placed 3 weeks before the relocation.
Pit 3: The "invisible conditions" of the new server room were not checked.The cooling capacity of the air conditioner is not enough (42 sets of equipment generate about 25kW, two sets of 3 racks cannot be pressed at all, the server room 40℃ collectively reduces frequency), the load is not enough (the upstairs room is fully loaded with racks), and the grounding is poor (static electricity and network interference). These must be measured item by item before moving in, and cannot rely on "looks good".
What constitutes relocation costs?
| Item | Notes | Amount of this case |
|---|---|---|
| Preliminary survey and scheme design | New and old equipment room survey, asset inventory, network planning, batch scheme, acceptance list | RMB 6000 |
| Foundation construction of new server room | Structured cabling, rack installation, bridge, UPS installation (materials are calculated separately) | RMB 28000 |
| Equipment off-rack packaging and transportation | 42 sets of equipment + auxiliary material packaging, air cushion vehicle 38 km, loading and unloading | RMB 9500 |
| System Recovery | Installation and wiring, power-on commissioning, system recovery, business joint commissioning, 32 items of acceptance | RMB 18000 |
| duty guarantee | 72 hours on-site duty after relocation +3 months remote support | RMB 6000 |
| total | —— | RMB 67500 |
Comparison reference: The enterprise itself organizes the relocation to save this service fee, but the recovery cost of a data damage (RMB 3000–20000) plus the loss of a day of business shutdown (manufacturing enterprises usually RMB 50,000–200,000) far exceeds the service fee. The value of the relocation project is not in the transportation, but in "no accident".
ACCPC provides general contracting services for relocation and construction of server rooms: survey scheme, Structured Cabling, equipment migration, system recovery, acceptance report, on-site duty, which can be undertaken by Guangzhou and Pearl River Delta. We also help with IDC colocation - if the new site will not have its own server room, we can engage a licensed IDC provider on your behalf (two-way mains + UPS + diesel + 7×24 duty + BGP multi-line). Resource fees are billed and invoiced directly by the provider, while equipment relocation, racking, deployment and ongoing operations are handled by us; the total annual cost is usually lower than building your own server room. Telephone020-39029800.