Skip to content

Reduce CAM initialization memory footprint that is important for ne1024 simulation- #4102 - #1643

Open
sjsprecious wants to merge 2 commits into
ESCOMP:cam_developmentfrom
sjsprecious:fix_ne1024_init_oom
Open

Reduce CAM initialization memory footprint that is important for ne1024 simulation- #4102#1643
sjsprecious wants to merge 2 commits into
ESCOMP:cam_developmentfrom
sjsprecious:fix_ne1024_init_oom

Conversation

@sjsprecious

Copy link
Copy Markdown
Collaborator

This PR reduces the memory usage during CAM initialization by adding a new module named cam_shmem_mod.F90, which allocates the read-only looku tables once per shared-memory node and maps it into every rank on that node.

Then the following modules are updated to reduce duplicated fields/tables:

  • mo_jlongxsqy, rsf_tab, xs_o2b/o3a/o3b. Also changes the bulk broadcast from all-ranks to leaders-only, cutting the inter-node broadcast volume.
  • phys_prop — the modal, hygroscopic and volcanic-radius aerosol-optics tables.
  • soil_erod_mod — the source-grid erodibility field, now read on masterproc only.
  • tracer_data — one per-file buffer holding the global lat-lon source field in flight, so the node leader reads each field once instead of every rank reading it redundantly. Files already on the unstructured physics grid and zonal-average files are unaffected.

A few additional updates:

  • New finalization path (jlong_final, photo_final, physprop_final) so the windows are freed collectively from chem_final/phys_final.
  • cam_shmem_init() is called early in cam_init so the first MPI_Comm_split_type(SHARED) is paid at a controlled collective point rather than mid-physprop.
  • Initial-condition and topo files are closed at the end of cam_init, after their last use in phys_init, instead of being held open until cam_final.
  • tracer_data carries the overlapping time record forward in memory when the bracketing window advances by one record, instead of re-reading it.

This PR is done by Claude code under my supervisory.

Fixes #1640

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cacraigucar @jtruesdal this file is generic and not specific to CAM. My suggestion is to merge this PR with it for now (after addressing all the review comments). After that I will merge this code with a similar one in CTSM (ESCOMP/CTSM#4102), and bring it into CESM_share instead. Once we have a new CESM_share tag and bring it into CAM, we can then safely remove this code later.

@sjsprecious sjsprecious changed the title Reduce CAM initialization memory footprint Reduce CAM initialization memory footprint that is important for ne1024 simulation- #4102 Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant