AI accelerators for scientific computing promise high throughput but require algorithm restructuring. We analyze where the gap arises and how it is practically closed.
The first confrontation occurs at the architectural level. AI-oriented accelerators, such as NPUs, are optimized for dense tensor operations and low precision. Scientific computing requires the opposite: stable numerical accuracy, irregular memory access, and complex synchronization. This creates three systemic gaps: execution mismatch, precision gap, and data-movement gap. As a result, directly porting HPC code to NPUs leads either to accuracy degradation or to bottlenecks in memory and communication rather than in compute.
The solution in this work is built not around a single optimization but around the coordination of multiple levels. Heterogeneous execution (CPU + NPU), mixed precision computing, and explicit memory orchestration are employed. The trade-off here is evident: increased development complexity and dependence on architecture. However, this allows computationally dense parts to be offloaded to tensor blocks (Cube Units), while those sensitive to precision or control remain on the CPU. This approach does not eliminate the limitations of NPUs but circumvents them through task decomposition.
At the implementation level, memory management and computation partitioning play a key role. The Ascend architecture divides computations into Cube (high throughput) and Vector (flexibility), which requires explicit mapping of kernels. For example:
- dense operations (GEMM) scale on Cube and achieve high utilization
- matrix-vector operations remain memory-bound and limited by bandwidth
- complex cases (e.g., complex GEMM) require an additional unpacking stage and lose efficiency
To address the precision gap, two approaches are applied. The first is mixed precision with iterative refinement (as in HPL-MxP), where low precision is used for the main workload, and accuracy is restored on the CPU. The second is precision emulation, as in SGEMM-cube, where FP32 computation is decomposed into several FP16 operations with subsequent aggregation. This increases the number of operations but allows for the utilization of the high bandwidth of tensor blocks.
A separate class of problems is related to data movement. In tasks such as quantum modeling (PQSim) or Monte Carlo (SMC-X), the bottleneck becomes not compute but data movement. Techniques employed include:
- operation fusion to reduce the number of launches
- tile-based processing with data retention in on-chip memory
- transforming irregular algorithms into SIMD-friendly forms
SMC-X is particularly indicative: an originally irregular algorithm with branching is transformed into batch processing with masks, allowing for efficient use of vector units. This is a typical trade-off: a compromise on the “purity” of the algorithm for predictable execution on hardware.
Results show that performance depends not on peak FLOPS but on the consistency of three factors: accuracy, computation placement, and data movement. In compute-bound tasks, NPUs demonstrate high efficiency. In bandwidth-bound and irregular tasks, gains are achieved only after significant algorithm reworking. In some cases, competitive performance and scalability are observed, but there is no universality—much remains sensitive to specific architecture and stack.
The conclusion appears pragmatic. AI accelerators for scientific computing work, but not as a drop-in replacement for GPU/CPU. This is a different programming model where:
- accuracy becomes part of the algorithm
- memory is managed explicitly
- computations are distributed across devices
Industrially, this reflects a broader trend: the convergence of AI and HPC leads not to unification but to increased complexity in the stack. Systems where the algorithm, runtime, and architecture are aligned as a cohesive whole gain the most.
Information source
arXiv is the largest open preprint repository (since 1991, under the auspices of Cornell), where researchers quickly post working versions of papers; the materials are publicly accessible but do not undergo full peer review, so results should be considered preliminary and, where possible, checked against updated versions or peer‑reviewed journals. arxiv.org