From 0579176d51ab6cc2af577057140f75aaaac1d353 Mon Sep 17 00:00:00 2001 From: abacus_fixer Date: Mon, 14 Sep 2026 09:18:57 +0800 Subject: [PATCH 1/3] Remove unneeded README20260902 file --- README20260902 | 65 -------------------------------------------------- 1 file changed, 65 deletions(-) delete mode 100644 README20260902 diff --git a/README20260902 b/README20260902 deleted file mode 100644 index c7596ba9741..00000000000 --- a/README20260902 +++ /dev/null @@ -1,65 +0,0 @@ -# para* 重构计划（分支 2026-09-02-b） - -## 背景与起点 - -- **分支**：`2026-09-02-b`，起点 commit `2a7696f5d`（step-0 cleanup source_base parallel_*） -- **基线状态**：`source_base/` 下有 14 个 `parallel_*` 文件（2d/cell/comm/common/device/global/grid/reduce），**没有** `module_parallel/` 目录，**没有** ParallelPartition——从 0 开始 -- 之前分支上尝试过 ParallelPartition（8 个裸 MPI_Comm 成员）和 ParaTag enum 两套方案，都已推倒 - -## 核心设计（已定） - -**两个类，放在 `source/source_base/module_parallel/` 下：** - -### 1. `ParaWorld` —— 单个通信域 -- 内容：`tag`（字符串常量）+ `comm`（MPI_Comm，串行下不存在）+ `rank` + `size` -- 把原本散在 GlobalV 的并行参数（NPROC_IN_POOL、RANK_IN_POOL 等）收进对应域对象 -- 方法：`tag()` / `rank()` / `size()` / `comm()`（仅 __MPI）/ `valid()` / `static serial(tag)` 安全退化（size=1, rank=0） -- 串行编译：`comm()` 用 `#ifdef __MPI` 包住，`rank()`/`size()`/`tag()` 总可用 -- 域特有参数（如 npw_per_proc、2D 网格行列）**不放进类**，由函数按需另外传 - -### 2. `ParaCollection` —— 全域容器 -- 内容：`std::vector` -- 查找：`find(tag)` 按字符串 tag 线性查找，**找不到返回静态空域（安全退化，不抛异常）** -- tag 用常量（避免裸字符串拼写错误运行时才暴露） - -### 3. `ParaTag` —— 域标签常量 -- 8 大域：`pw` / `kmesh` / `bsame_kdiff` / `bdiff_ksame` / `rgrid` / `diag` / `matrix` / `atom` -- 对应原全局：POOL_WORLD / KP_WORLD / INT_BGROUP / BP_WORLD / GRID_WORLD / DIAG_WORLD / matrix / atom - -## 目标 - -- 函数通过**注入** `const ParaWorld&` 或 `const ParaCollection&` 获取通信域，不再读裸全局 POOL_WORLD/GlobalV -- wrapper（Parallel_Common::bcast_* / Parallel_Reduce::reduce_*）加 `ParaWorld` 重载，`#ifdef __MPI` 收进 wrapper 内部，调用点无 `#ifdef`、无 MPI_Comm，串行并行都能编译跑 -- 测试用 `ParaWorld::serial(tag)` 或一行工厂构造，**去掉 GlobalV/divide_pools/set_global_partition 样板** - -## 分步计划（每步一个 commit，确认合理再进下一步） - -| 步骤 | 内容 | commit 信息 | -|---|---|---| -| **step 1** | 建 `module_parallel/` 目录 + `ParaWorld` 类（tag 常量 + comm/rank/size + `serial()` + `valid()`），含单元测试 + CMake/Makefile.Objects 接线 | `feat(parallel): add ParaWorld comm-domain value type` | -| **step 2** | `ParaCollection`（`vector` + `find(tag)` 安全退化返回静态空域），含单元测试 | `feat(parallel): add ParaCollection domain container` | -| **step 3** | 用 `ParaWorld`/`ParaCollection` 表达 8 大域装配（替代旧 divide_pools 全局写法），接进 driver 初始化 | `feat(parallel): assemble domains into ParaCollection at driver` | -| **step 4** | `Parallel_Common::bcast_bool` 加 `ParaWorld` 重载（`#ifdef __MPI` 收进 wrapper，串行 no-op，旧签名保留） | `feat(parallel): bcast_bool overload taking ParaWorld` | -| **step 5** | rhog_io.cpp 打样：注入 `ParaWorld`，`bcast_bool(error, pw)` 一行无 `#ifdef`；read_rhog_test 改一行构造去 GlobalV | `refactor(io): inject ParaWorld into read_rhog` | - -## 命名与规范约束 - -- 文件名小写+下划线：`para_world.h/.cpp`、`para_collection.h/.cpp` -- C++11，4 空格缩进，大括号独占一行，不用 `using namespace std`，注释用英文 doxygen 格式 -- 不加默认参数，不用全局变量（ParaCollection 通过注入传递，不做全局单例） -- include guard 用短名，与同目录其它文件一致 -- 不用 goto，不用宏做域替换，struct 不裸露公有成员 -- 函数参数带校验（指针非空、int 范围合理） - -## 验证方式 - -- 编译目录：`/home/510Group/6_abacus_mc/abacus-mc/build_max_para_test`，命令 `make -j 30` -- 测试：`OMP_NUM_THREADS=1 ctest -V -R ` -- 注意：沙箱内 MPI 测试会因 `/dev/nvidiactl` 受限误报崩溃，需看 ctest 日志实际结果 -- 每步 commit 前确认编译 0 错误 + 相关测试通过 - -## 待确认细节（开工前） - -1. 文件路径 `source/source_base/module_parallel/para_world.h/.cpp` 是否 OK -2. `ParaWorld` 串行下 `comm()` 不存在（`#ifdef __MPI`），`rank()`/`size()` 返回 0/1，`tag()` 总可用——是否 OK -3. 从 step 1 开始，还是想先调整步骤划分 From e01c2156e3edf2a4eaf39a1032cb2e994bd26dae Mon Sep 17 00:00:00 2001 From: abacus_fixer Date: Mon, 14 Sep 2026 09:24:48 +0800 Subject: [PATCH 2/3] Fix build without LibRI: split BvK utils out of ri_util.h module_lr is built whenever ENABLE_LCAO is on, but lr_io_krlist.cpp unconditionally included module_ri/ri_util.h, which pulls in LibRI headers () and fails to compile when ENABLE_LIBRI is off (regression from #7849). Move the LibRI-free Born-von Karmen helpers (get_Born_vonKarmen_period, get_Born_von_Karmen_cells) into a new header ri_util_bvk.h; ri_util.h now includes it, and lr_io_krlist.cpp includes only the new header. Verified: target lr builds with ENABLE_LIBRI=OFF (build/), target ri builds with ENABLE_LIBRI=ON (build_std_para/). --- .../module_lr/utils/lr_io_krlist.cpp | 2 +- source/source_lcao/module_ri/ri_util.h | 8 +- source/source_lcao/module_ri/ri_util.hpp | 55 -------------- source/source_lcao/module_ri/ri_util_bvk.h | 73 +++++++++++++++++++ 4 files changed, 75 insertions(+), 63 deletions(-) create mode 100644 source/source_lcao/module_ri/ri_util_bvk.h diff --git a/source/source_lcao/module_lr/utils/lr_io_krlist.cpp b/source/source_lcao/module_lr/utils/lr_io_krlist.cpp index 84c05a2e14e..1d9b53d9fc2 100644 --- a/source/source_lcao/module_lr/utils/lr_io_krlist.cpp +++ b/source/source_lcao/module_lr/utils/lr_io_krlist.cpp @@ -1,6 +1,6 @@ #include "lr_io_krlist.h" #include "lr_io.h" -#include "source_lcao/module_ri/ri_util.h" +#include "source_lcao/module_ri/ri_util_bvk.h" #include "source_base/constants.h" #include #include diff --git a/source/source_lcao/module_ri/ri_util.h b/source/source_lcao/module_ri/ri_util.h index 67aef442ddc..f5659d87329 100644 --- a/source/source_lcao/module_ri/ri_util.h +++ b/source/source_lcao/module_ri/ri_util.h @@ -8,6 +8,7 @@ #include "source_cell/klist.h" #include "source_lcao/module_ri/conv_coulomb_pot_k.h" +#include "source_lcao/module_ri/ri_util_bvk.h" #include #include @@ -20,13 +21,6 @@ namespace RI_Util { - inline extern std::array - get_Born_vonKarmen_period(const K_Vectors &kv); - - template - extern std::vector> - get_Born_von_Karmen_cells( const std::array &Born_von_Karman_period ); - template inline std::array Vector3_to_array3(const ModuleBase::Vector3 &v) diff --git a/source/source_lcao/module_ri/ri_util.hpp b/source/source_lcao/module_ri/ri_util.hpp index 886c5f1b377..41ce8270019 100644 --- a/source/source_lcao/module_ri/ri_util.hpp +++ b/source/source_lcao/module_ri/ri_util.hpp @@ -12,61 +12,6 @@ namespace RI_Util { - inline std::array - get_Born_vonKarmen_period(const K_Vectors &kv) - { - return std::array{kv.nmp[0], kv.nmp[1], kv.nmp[2]}; - } - - template - std::vector> - get_Born_von_Karmen_cells( const std::array &Born_von_Karman_period ) - { - using namespace RI::Array_Operator; - std::vector> Born_von_Karman_cells; - for( int c=0; c{c} % Born_von_Karman_period ); - return Born_von_Karman_cells; - } - - template - std::vector> - get_Born_von_Karmen_cells( const std::array &Born_von_Karman_period ) - { - using namespace RI::Array_Operator; - - std::array sub_Born_von_Karman_period; - for(int i=0; i> Born_von_Karman_cells; - for( const std::array &sub_cell : get_Born_von_Karmen_cells(sub_Born_von_Karman_period) ) - for( Tcell c=0; c cell; - for(int i=0; i{c} % std::array{Born_von_Karman_period.back()})[0]; - Born_von_Karman_cells.emplace_back(std::move(cell)); - } - return Born_von_Karman_cells; - } - - /* example for Ndim=3: - template - std::vector> - get_Born_von_Karmen_cells( const std::array &Born_von_Karman_period ) - { - using namespace Array_Operator; - std::vector> Born_von_Karman_cells; - for( int ix=0; ix{ix,iy,iz} % Born_von_Karman_period ); - return Born_von_Karman_cells; - } - */ - inline std::map>> update_coulomb_param( const std::map>> &coulomb_param, diff --git a/source/source_lcao/module_ri/ri_util_bvk.h b/source/source_lcao/module_ri/ri_util_bvk.h new file mode 100644 index 00000000000..d8574258796 --- /dev/null +++ b/source/source_lcao/module_ri/ri_util_bvk.h @@ -0,0 +1,73 @@ +//======================= +// AUTHOR : Peize Lin +// DATE : 2022-08-17 +//======================= + +// Born-von Karmen supercell utilities. +// This header is free of LibRI dependencies so that modules built without +// LibRI (e.g. module_lr) can use it. + +#ifndef RI_UTIL_BVK_H +#define RI_UTIL_BVK_H + +#include "source_cell/klist.h" + +#include +#include + +namespace RI_Util +{ + inline std::array + get_Born_vonKarmen_period(const K_Vectors &kv) + { + return std::array{kv.nmp[0], kv.nmp[1], kv.nmp[2]}; + } + + // cell index c folded into [0, period) like (c % period + period) % period + template + std::vector> + get_Born_von_Karmen_cells( const std::array &Born_von_Karman_period ) + { + std::vector> Born_von_Karman_cells; + for( Tcell c=0; c{c} ); + return Born_von_Karman_cells; + } + + template + std::vector> + get_Born_von_Karmen_cells( const std::array &Born_von_Karman_period ) + { + std::array sub_Born_von_Karman_period; + for(size_t i=0; i> Born_von_Karman_cells; + for( const std::array &sub_cell : get_Born_von_Karmen_cells(sub_Born_von_Karman_period) ) + for( Tcell c=0; c cell; + for(size_t i=0; i + std::vector> + get_Born_von_Karmen_cells( const std::array &Born_von_Karman_period ) + { + std::vector> Born_von_Karman_cells; + for( int ix=0; ix{ix,iy,iz} ); + return Born_von_Karman_cells; + } + */ +} + +#endif From 5cba06b410cccaaa79a3a92b99f2f2369fe52f48 Mon Sep 17 00:00:00 2001 From: abacus_fixer Date: Mon, 14 Sep 2026 09:31:26 +0800 Subject: [PATCH 3/3] Fix timer_enable_nvtx: define __USE_NVTX on the targets that consume it __USE_NVTX was defined only on the final executable target, whose sole translation unit main.cpp contains no NVTX code. The two OBJECT libraries that actually guard NVTX calls with the macro -- base (source_base/timer.cpp) and driver (source_main/driver.cpp) -- never saw it, so every NVTX block was preprocessed away and timer_enable_nvtx had no effect in any CUDA build. Move the definition onto base and driver, and link CUDA::nvToolsExt for CUDA toolkits older than 12.9 (NVTX is header-only since 12.9). Verified with build_pw_gpu (USE_CUDA=ON): base/driver targets compile with NVTX symbols present in timer.cpp.o, driver.cpp.o references timer::enable_nvtx_, and the full abacus_pw_gpu executable links (v3.11.0-beta9). --- source/CMakeLists.txt | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/source/CMakeLists.txt b/source/CMakeLists.txt index ffd4272f71f..c142c948af5 100644 --- a/source/CMakeLists.txt +++ b/source/CMakeLists.txt @@ -517,7 +517,6 @@ set(ABACUS_BIN_PATH ${PROJECT_BINARY_DIR}/${ABACUS_BIN_NAME}) if(USE_CUDA) set_property(TARGET ${ABACUS_BIN_NAME} PROPERTY CUDA_ARCHITECTURES ${CMAKE_CUDA_ARCHITECTURES}) - target_compile_definitions(${ABACUS_BIN_NAME} PRIVATE __USE_NVTX) endif() if(ENABLE_COVERAGE) @@ -553,6 +552,20 @@ add_library( source_main/driver.cpp source_main/driver_run.cpp) +# __USE_NVTX gates the NVTX code in source_base/timer.cpp (target base) and +# source_main/driver.cpp (target driver), so it must be defined on those +# OBJECT libraries, not only on the final executable (whose sole TU, main.cpp, +# contains no NVTX code). Defining it only on the executable silently dropped +# every NVTX range and kept timer_enable_nvtx ineffective. +if(USE_CUDA) + target_compile_definitions(base PRIVATE __USE_NVTX) + target_compile_definitions(driver PRIVATE __USE_NVTX) + # NVTX is header-only since CUDA 12.9; older toolkits need libnvToolsExt. + if(CUDAToolkit_VERSION VERSION_LESS 12.9) + target_link_libraries(${ABACUS_BIN_NAME} PRIVATE CUDA::nvToolsExt) + endif() +endif() + list(APPEND device_srcs source_pw/module_pwdft/kernels/nonlocal_op.cpp source_pw/module_pwdft/kernels/veff_op.cpp