Two years ago, when talking about domestic AI chips, the topic basically stayed at "Can it be used?". In 2026, this question has already had an answer, and the new question has become "What scene to use, how to calculate accounts, ecology cannot keep up".
When consulting local AI deployment for enterprises, domestic computing power has changed from "do not consider" to "must evaluate a round"_the reason is very practical: imported high-end cards cannot be bought (controlled) and cannot be bought (price), while domestic cards can already play the cost performance in reasoning scenarios.
Key data sets for 2026
| index | data |
|---|---|
| Domestic AI chip shipments | 1.65 million, breaking through 40% of the market for the first time |
| vacate goods and shares | 81.2 Ten thousand, with a share of 20.3%, ranking first |
| Cambricon 2026 revenue for the first half of the year | 5.996 billion RMB,+108% |
| Daily Token Calls in China | June 2026Breaking through 500 trillion |
| DeepSeek deployment plan | 160,000 Ascend 950 DT, Ulanqab 1GW Data Center,Reasoning without training.The delivery cycle exceeds 1 year. |
| GLM-5.3-Flash | 320B parameter/18B activation (MoE), 100,000 domestic chip clusters, performance benchmarking international top model,The price is about 1/40 of it. |
The specifications of the 950DT are worth looking at separately:144GB HiZQ 2.0 HBM, FP8 Computing Power 1 PFLOPS, Chip Interconnection Bandwidth 2TB/s. 144GB memory This specification is critical-it determines how large a single card can fit, and it also determines the service capabilities of inference scenarios.
Most notable signal: head player selects "inference only, no training"
DeepSeek's 160,000 clusters are clearOnly do reasoning, no training. The logic behind this choice has direct reference value for enterprises to do local deployment:
Training and reasoning are two completely different loads.
| dimensionality | drill | inference |
|---|---|---|
| Computing power characteristics | Hyperscale parallelism, cluster interconnection bandwidth is bottleneck, thousands of cards are required for synchronization | Single card or small cluster, memory capacity and throughput are bottlenecks |
| software stack dependency | Heavily dependent on CUDA ecology (distributed training framework, operator library, hybrid precision toolchain) | Relatively light, mainstream reasoning frameworks (domestic alternatives to vLLM, SGLang, TensorRT-LLM) have been adapted domestically |
| cost of failure | A training task runs for several weeks, with huge losses due to errors midway, and high stability requirements. | Single request failure can be retried, fault tolerance space is large |
| Domestic card maturity | Still catching up, ecology is the main obstacle | Available and cost-effective |
Conclusion:99% of local AI requirements are reasoning, not training.You don't need to train a large model. What you need is to deploy a ready-made open source model (or API model) locally to serve internal business-knowledge base Q & A, document processing, customer service assistance, code completion, image recognition. These are all inference loads, which are the areas that domestic cards can already play.
The case of GLM shows that ecology is in a positive cycle
GLM-5.3-Flash uses a cluster of 100,000 domestic chips to achieve a MoE architecture with 320B parameters/ 18B activation. Its performance is comparable to the top international models and its price is about 1/40. Two meanings:
- Model side actively adapts domestic hardware--It is no longer "software adaptation such as hardware", but the characteristics of domestic clusters are considered when designing the model (sparse activation of MoE just reduces the requirements for single card computing power and interconnection bandwidth).
- Price 1/40 is the decisive business variable. For enterprises, the cost of API calls is reduced by an order of magnitude, and many scenarios that were originally "think about it" become feasible-such as intelligent archiving of full documents and real-time quality inspection of each customer service call.
Feasibility Checklist for Enterprise On-premises Deployment
Don't start with "what card to buy", start with these four questions:
Question 1: Does your scenario really need to be deployed locally?
Hard conditions to trigger on-premises deployment (either is worth meeting):
- Data cannot be out of domain: Medical images, financial customer information, government-enterprise confidential documents, manufacturing process parameters — uploading such data to cloud API violates internal systems or regulations.
- Call volume is so large that API cost exceeds self-builtThe rough algorithm is that the monthly API cost exceeds RMB 15,000 and will last for more than one year. It is cost-effective to build a self-built server (RMB 150,000–400,000 for a reasoning server, depreciated for three years).
- Offline or low latency required: Factory floor, field equipment, intranet isolation environment, or real-time scenarios requiring response latency of less than 100 ms.
If all three are not met,Using cloud API directly is a more rational choice--No maintenance, no hardware depreciation, and model upgrades with the cloud. Many enterprises '"local AI" projects end up as a dusty server in the server room.
Question 2: How big is the model and how strong is the hardware?
| scene | Model size recommendations | hardware threshold | reference scheme |
|---|---|---|---|
| Individual/Group Knowledge Base Q & A | 7B-14B quantization (INT4) | 16-24GB Memory Single Card | RTX 4090/5090 workstation, RMB 20–40,000 |
| Department-level document processing, customer service assistance (10-50 people) | 14B-32B quantification, or MoE small activation model | Single card 48-96GB or dual card 24GB | Home-made reasoning card (Ascend 300I/Cambricon MLU370) or RTX 6000 Ada, 80 - 250 thousand RMB |
| Enterprise multi-department concurrency (50-500 people)+ RAG knowledge base | 32B-70B, or MoE (as GLM Series 320B/18B activated) | 4-8 Card Reasoning Server, Single Card 96-144GB | Atlas 800I A2 /Cambricon Siyuan Cluster, 50- 1.5 million RMB |
| Need fine tuning/LoRA training | Domain adaptation based on 7B-32B | Memory requirements are about 3-5 times that of inference | It is recommended to rent cloud computing power, and the cost performance of self-built fine-tuning clusters is extremely low. |
Key reminder: Memory capacity is more important than computing power (TOPS/FLOPS).In reasoning scenarios, models cannot run completely without memory, and insufficient computing power is only a bit slower. 144GB memory of the 950DT specifications, the value is that a single card can be installed 70B level model FP8 weight, do not need multi-card segmentation (multi-card segmentation will introduce communication overhead).
Question 3: Who will do the software stack and operation and maintenance?
This is the real bottleneck of domestic computing power landing, and it is also the number one reason for project failure:
- The migration costs of CUDA ecology.If your technical team has all the experience on CUDA, switching to CANN or Cambricon Neuware requires learning costs, operator coverage and maturity of the debugging tool chain (especially custom operators and cold model structures).
- Model adaptation is not "download and use".Mainstream open source models (Qwen, GLM, Llama, DeepSeek) have different degrees of adaptation on domestic cards. Quantification scheme, reasoning framework version and driver version should be matched, and trial and error cycle is counted in weeks.
- Operation and maintenance requires special personnel.Inference services need to manage concurrent scheduling, memory fragmentation, model hot updates, and monitoring alerts, which are not on the same level as operating and maintaining a Web server. Projects without full-time staff are basically in a state of "usable but no one dares to move" after half a year.
Practical advice:Preference is given to model + hardware combinations that have been certified by the manufacturerDon't go through the first pit yourself. Also, package the implementation and one-year operation to the integrator, which is cheaper than hiring yourself.
Question 4: To what extent is China-made compliance required?
If it is purchased by government enterprises, state-owned enterprises and public institutions, there are often China-made (information technology application innovation) catalogue requirements. At this time, the selection logic is completely different:
- It is necessary to check whether the product is in China-made catalogue and whether it has relevant qualification certification (such as commercial password, equal protection and localization rate requirements).
- Not only AI card, but also the whole machine (Kunpeng/Haiguang/Feiteng for CPU), operating system (Kirin/UOS), Databases (Dameng/Renmin Jincang/openGauss) and middleware should be considered in a complete set, otherwise the acceptance will not pass.
- Budget to keep enough: China-made package prices are usually 1.5-2.5 times that of commercial solutions, and the implementation cycle is longer.
Purchasing pitfalls: four common mistakes
- Just look at the calculation parameters to buy cards.FP16 computing power 1 PFLOPS but memory only 32GB, cannot run 70B model; memory 144GB but low interconnection bandwidth, multi-card parallel efficiency is poor. Parameters to see the overall match.
- Ignore lead times.DeepSeek's delivery cycle of 160,000 tablets exceeds one year-the order cycle of popular domestic cards is generally 3-12 months, and it will be longer in 2026 under the background of tight computing power. Project scheduling must include this, do not press "order two weeks to arrive" planning.
- Purchase in bulk without POC.Run a POC (2-4 Zhou) with your own real business data, real concurrent pressure, and real models to measure throughput, latency, stability, and O & M complexity. Projects that skip this step have a high rollover rate.
- Underestimating power and cooling.An 8-card inference server consumes 5-8kW, and the power supply and air conditioning of an ordinary office cannot be suppressed. Either put it in the room (need to confirm the power supply capacity and cooling), or host it to IDC.This point is directly related to the following liquid cooling topic — after the power of a single rack exceeds 20kW, air cooling will be powerless.
ACCPC provides enterprise AI infrastructure consulting and implementation: local deployment feasibility assessment (free), hardware selection and procurement (including domestic computing power and imported cards), power supply and cooling transformation of server room, IDC colocation coordination (we engage a licensed provider: two-way mains + UPS + diesel + 7×24 duty), system integration and one-year operation and maintenance packaging. We also do wholesale Guangzhou servers (Dell/HP/Lenovo/Ultramicro/Inspur/Huawei). We do not do "sell out and go" business--if the conclusion is "you do not need local deployment" in the evaluation stage, we will tell you directly, and then help you configure the cloud API solution. Phone 020-39029800.