What Can ABACUS Do Too? | A Unified Heterogeneous Implementation of Numerical Atomic Orbitals-Based Real-Time TDDFT within the ABACUS Package

Posted on In Word count in article: 1.4k Reading time ≈ 5 mins.

1. Introduction

Recently, a collaborative team from Peking University, University of Science and Technology of China, Institute of Physics, and other institutions has implemented a unified heterogeneous computing framework for real-time time-dependent density functional theory (RT-TDDFT) based on numerical atomic orbitals (NAO) within the domestically developed open-source density functional theory software ABACUS. By introducing a hardware-agnostic abstraction layer into the ABACUS code, the team successfully achieved efficient cross-platform acceleration from single-node multi-core CPUs to large-scale multi-GPU architectures while keeping the physics algorithm code clean and maintainable.

The framework can efficiently compute important optical properties such as optical absorption spectra and dielectric functions, and supports simulation of physical processes such as nonadiabatic Ehrenfest ion-electron coupled dynamics. In terms of accuracy, GPU results are numerically identical to CPU implementations and are in excellent agreement with existing mainstream software benchmarks. In terms of efficiency, a single GPU achieves a 3–4× speedup over a 56-core dual-socket CPU node, and multi-GPU acceleration has been realized while maintaining good parallel efficiency. Compared with similar software, this work not only achieves excellent cross-platform portability and multi-GPU acceleration for RT-TDDFT, but also solves the computational bottleneck of the velocity gauge under NAO basis sets through low-level heterogeneous acceleration algorithms. The related functionality has been officially released in ABACUS v3.9.0.26 and later versions; see the Chinese documentation tutorial: ABACUS real-time time-dependent density functional theory tutorial (applicable to LCAO basis sets, v3.9.0.26 and later).

The related research results were published under the title “A unified heterogeneous implementation of numerical atomic orbitals-based real-time TDDFT within the ABACUS package” in Computer Physics Communications, a classic journal in computational physics. Related link: (https://doi.org/10.1016/j.cpc.2026.110260).

2. Engineering Challenges of Heterogeneous Acceleration

When simulating ultrafast electron dynamics and light–matter interactions on femtosecond to attosecond timescales, RT-TDDFT is an extremely important first-principles computational method. With the increasing scale and duration of computations, porting such methods to heterogeneous accelerators such as GPUs has become an inevitable trend.

However, heterogeneous acceleration of RT-TDDFT with localized basis sets (such as numerical atomic orbitals, NAO) faces software engineering challenges. First, developing and maintaining low-level GPU code is extremely complex, involving cumbersome device memory management and architecture-specific kernel optimization. This not only easily leads to bugs such as memory leaks, but also seriously hinders the long-term sustainable development of scientific software. Second, current high-performance computing platforms are highly diverse (including NVIDIA GPUs, AMD GPUs, and domestic accelerators, etc.). If the code is deeply tied to a single hardware vendor’s programming model, it will lose the flexibility of cross-platform porting.

3. A Unified Heterogeneous Computing Framework: Hardware-Agnostic Design

To address the above software engineering bottlenecks, this work does not adopt the traditional approach of hard-coding low-level acceleration instructions in the physics logic code. Instead, it redesigns a three-layer synergistic hardware abstraction architecture for ABACUS:

  • Unified data container: This is the core of the entire heterogeneous framework. The team designed a Tensor class supporting multidimensional arrays. It not only encapsulates tensor shape and data type, but also fully takes over memory allocation and deallocation on the host side (CPU) and device side (GPU). Based on the RAII (Resource Acquisition Is Initialization) mechanism, developers do not need to manually manage memory or copy data between host and device, thereby fundamentally eliminating the risks of dangling pointers and memory leaks.
  • Unified linear algebra operator interface: Based on the Tensor container, the team encapsulated a set of polymorphic dense linear algebra operators. This interface automatically dispatches to the corresponding mathematical libraries according to the hardware environment (e.g., BLAS/LAPACK on the CPU side, cuBLAS/cuSOLVER on the GPU side, etc.). This allows developers of physics algorithms to focus on the physical formulation of wavefunction evolution (for example, solving the Crank-Nicolson propagator equation) without worrying about the underlying hardware environment.
  • Unified real-space grid integration interface: To address the integration bottleneck of real-space physical quantities (such as charge density and Hamiltonian construction), the team designed and implemented a unified heterogeneous grid integration interface. It is worth mentioning that for the position-dependent phase factor introduced under the velocity gauge, traditional methods suffer severe performance degradation in spherical grid integration. This framework implements efficient computation under the velocity gauge through batched atom-level GPU reduction algorithms.

Efficient heterogeneous computing framework for RT-TDDFT based on numerical atomic orbitals

4. Physical Validation and Computational Performance

In terms of physical validation, the heterogeneous framework demonstrates extremely high reliability. The researchers tested systems of various dimensions, from anthracene molecules to three-dimensional bulk silicon. In optical absorption spectra and nonadiabatic Ehrenfest molecular dynamics, the GPU results strictly coincide with CPU results in numerical accuracy and are in excellent agreement with benchmark data in the field.

Optical properties (dielectric function) of bulk silicon. (a) Time-dependent external electric field waveform. (b) Time-domain evolution of the macroscopic current density, demonstrating the strict consistency between CPU and GPU results. (c)–(d) Real and imaginary parts of the dielectric function under the hybrid gauge and velocity gauge. (e)–(f) Basis set convergence tests (DZP vs. TZDP). (g)–(h) Comparison of the dielectric function with SIESTA benchmark results [3]

In terms of performance, this engineering architecture achieves significant speedups:

  • Single-node performance: When performing wavefunction time-step evolution for a silicon supercell containing 1200 atoms, a single NVIDIA A800 GPU achieved about 3 to 4 times faster wall-clock time than a fully loaded 56-core dual-socket CPU node (Intel Xeon Gold 6348). Among them, the core wavefunction propagation and evolution module achieved up to 6–7× acceleration, and the spherical grid integration under the velocity gauge even achieved up to 12× acceleration.

Acceleration of the wavefunction evolution module. (a) Linear scale (b) Logarithmic scale (c) Time proportion

Acceleration of the spherical grid integration module under the velocity gauge. (a) Linear scale (b) Logarithmic scale (c) Time proportion

  • Multi-GPU scalability: With a distributed multi-GPU linear solver strategy, the framework exhibits excellent strong scalability on distributed supercomputing clusters across nodes. When simulating a complex system with 1728 atoms, scaling to 40 GPUs still maintains about 76% parallel efficiency.

5. Summary

Through systematic architectural design, this work abstracts the complexity of underlying heterogeneous hardware from physics algorithm developers and successfully achieves heterogeneous acceleration of the LCAO basis set RT-TDDFT module in ABACUS. This engineering approach of “code decoupling” and “unified interfaces” not only provides strong performance support for ultrafast simulations of complex electron dynamics, but also lays a highly scalable foundation for future scientific software to embrace more heterogeneous computing platforms and AI4S.

This research was supported by the National Key R&D Program of China (2025YFB3003603) and the Excellent Research Group Project of the National Natural Science Foundation of China (12588301). Multi-GPU strong scalability tests were completed on the SAI (Mr. Science) open-source supercomputing platform.

6. References

  • [1] Bao, T., Li, Y., Deng, Z., Zhao, H., Lu, D., Huang, Y., Lian, C., He, L., & Chen, M. (2026). A unified heterogeneous implementation of numerical atomic orbitals-based real-time TDDFT within the ABACUS package. Computer Physics Communications, 327, 110260.https://doi.org/10.1016/j.cpc.2026.110260
  • [2]Zhao, H., & He, L. (2025). Hybrid Gauge Approach for Accurate Real-Time TDDFT Simulations with Numerical Atomic Orbitals. Journal of Chemical Theory and Computation, 21,7, 3335–3341.https://doi.org/10.1021/acs.jctc.5c00111
  • [3] Pemmaraju, C. D., Vila, F. D., Kas, J. J., Sato, S. A., Rehr, J. J., Yabana, K., & Prendergast, D. (2018). Velocity-gauge real-time TDDFT within a numerical atomic orbital basis set. Computer Physics Communications, 226, 30-38.https://doi.org/10.1016/j.cpc.2018.01.013