[codex] Embed CoLM2024 as MPAS land model#1482
Draft
zhongwangwei wants to merge 15 commits into
Draft
Conversation
CoLM2024 now builds inside the atmosphere core like Noah-MP, uses MPAS-owned ranks and cell subsets, keeps patch/PFT state internal to CoLM, and exposes fluxes through MPAS fields/streams. Constraint: CoLM2024 must run without a separate master/io/worker allocation or external -lcolm library. Rejected: standalone CoLM USEMPI-off adapter | it collapses parallel land/routing behavior and risks global duplicated state. Rejected: CaMa and sediment routing in embedded mode | not part of the requested GridRiverLakeFlow path. Confidence: medium Scope-risk: broad Directive: Keep MPAS cell == CoLM element; do not reintroduce global mesh gathers or CoLM-owned MPI roles. Tested: make gfortran CORE=atmosphere COLM2024=true AUTOCLEAN=true; cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8; cmake --install build-cmake-colm-debug; git diff --cached --check; otool/nm checks for external CoLM/CaMa/CMF symbols. Not-tested: full runtime case with real MPAS+CoLM input datasets.
MPAS-embedded CoLM consumes existing landdata/restart files and should not carry the standalone make-initial/make-surface generation chain. The runtime Land/PFT/Patch and SrfdataRestart loaders remain for MPAS cell-owned execution. Constraint: CoLM patch/PFT metadata still lives inside CoLM and is loaded from MPAS-owned landdata paths. Rejected: deleting all mksrfdata modules | MOD_SrfdataRestart and Land* modules are runtime loaders, not surface generators. Rejected: keeping LULCC driver sources | they depend on deleted make-surface transfer traces and MOD_Initialize. Confidence: high Scope-risk: moderate Directive: Keep CoLM source lists aligned between physics_colm2024/Makefile and core_atmosphere/CMakeLists.txt. Tested: rg found no remaining initial/surface/LULCC generator references; git diff --check; cmake -S . -B build-cmake-colm-debug; cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8; NETCDF=/opt/homebrew/Cellar/netcdf/4.10.0 NETCDFF=/opt/homebrew/Cellar/netcdf-fortran/4.6.2 PNETCDF=/opt/homebrew/Cellar/pnetcdf/1.14.1 make gfortran CORE=atmosphere COLM2024=true AUTOCLEAN=true -j 8. Not-tested: full runtime case with real MPAS+CoLM landdata/restart inputs.
MPAS owns CoLM decomposition now, so the embedded SPMD layer should expose root/active/compute names instead of standalone master/io/worker compatibility names. The compute push/remap API was renamed with the same boundary; local loop variables were left alone. Constraint: CoLM runtime still uses its existing MPI collectives and history routines, only the deprecated process-pool API names were removed. Rejected: renaming every local iworker/comment occurrence | that is churn without changing the exported process model. Confidence: high Scope-risk: moderate Directive: Do not reintroduce master/io/worker SPMD aliases in MOD_SPMD_Task. Tested: rg --no-ignore scans for old SPMD and WorkerPushData API names; git diff --check; cmake -S . -B build-cmake-colm-debug; cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8; NETCDF=/opt/homebrew/Cellar/netcdf/4.10.0 NETCDFF=/opt/homebrew/Cellar/netcdf-fortran/4.6.2 PNETCDF=/opt/homebrew/Cellar/pnetcdf/1.14.1 make gfortran CORE=atmosphere COLM2024=true AUTOCLEAN=true -j 8. Not-tested: real MPAS+CoLM runtime case with production landdata/restart inputs.
Constraint: MPAS embedded CoLM runs every MPAS rank as both active and compute, so legacy IO/worker scatter paths deadlock or reallocate local state. Rejected: Reintroducing standalone CoLM process pools | it conflicts with MPAS-owned decomposition and the cell==element integration target. Confidence: high Scope-risk: moderate Directive: Keep landdata reads on original block files; use rank-suffixed vector files only for MPAS-owned writes. Tested: cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8; NETCDF=/opt/homebrew/Cellar/netcdf/4.10.0 NETCDFF=/opt/homebrew/Cellar/netcdf-fortran/4.6.2 PNETCDF=/opt/homebrew/Cellar/pnetcdf/1.14.1 make gfortran CORE=atmosphere COLM2024=true AUTOCLEAN=true -j 8 Not-tested: Runtime MPAS+CoLM case with real landdata/restart inputs.
Constraint: CoLM2024 is now built and run only as the MPAS sf_colm2024 land-surface scheme in this repository. Rejected: Deleting CoLM core driver or mksrfdata modules | they remain in the MPAS adapter and build dependency chain. Confidence: high Scope-risk: narrow Directive: Keep future CoLM examples and forcing namelists outside the embedded MPAS source tree unless MPAS itself consumes them. Tested: rg found no remaining references to deleted run/evaluation files; rg found no standalone PROGRAM in physics_colm2024; git diff --cached --check; cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8; NETCDF=/opt/homebrew/Cellar/netcdf/4.10.0 NETCDFF=/opt/homebrew/Cellar/netcdf-fortran/4.6.2 PNETCDF=/opt/homebrew/Cellar/pnetcdf/1.14.1 make gfortran CORE=atmosphere COLM2024=true AUTOCLEAN=true -j 8 Not-tested: Runtime MPAS+CoLM case with real landdata/restart inputs.
Constraint: MPAS embedded CoLM still needs these surface/restart and land-unit modules, but no standalone mksrfdata directory should remain. Rejected: Deleting the modules outright | MPAS initialization and CoLM object lists still depend on them. Confidence: high Scope-risk: narrow Directive: Keep future embedded CoLM runtime/shared modules under share unless they are true MPAS driver code or main physics kernels. Tested: cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8; make -C src/core_atmosphere/physics/physics_colm2024 clean && NETCDF=/opt/homebrew/Cellar/netcdf/4.10.0 NETCDFF=/opt/homebrew/Cellar/netcdf-fortran/4.6.2 PNETCDF=/opt/homebrew/Cellar/pnetcdf/1.14.1 make gfortran CORE=atmosphere COLM2024=true AUTOCLEAN=true -j 8; git diff --check; git diff --cached --check; rg found no mksrfdata path references. Not-tested: Runtime MPAS+CoLM case with real landdata/restart inputs.
Constraint: MPAS embedded CoLM runs root and compute roles on the same MPAS ranks, so old standalone IO/worker point-to-point paths must not self-send or leave vector address tables unallocated. Rejected: Reintroducing a separate CoLM worker/IO process pool | MPAS must own the communicator and CoLM must stay rank-local to MPAS cells. Confidence: high Scope-risk: moderate Directive: Embedded CoLM helper paths should use local data access or self-skip collectives; do not add standalone master/worker handshakes under MPAS_EMBEDDED_COLM. Tested: git diff --check; cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8; NETCDF=/opt/homebrew/Cellar/netcdf/4.10.0 NETCDFF=/opt/homebrew/Cellar/netcdf-fortran/4.6.2 PNETCDF=/opt/homebrew/Cellar/pnetcdf/1.14.1 make gfortran CORE=atmosphere COLM2024=true AUTOCLEAN=true -j 8 Not-tested: Runtime MPAS+CoLM case with real landdata/restart/riverlake/reservoir inputs.
Constraint: MPAS embedded CoLM must use MPAS-owned cell decomposition while retaining CoLM patch/PFT state internally. Rejected: Coordinate fallback mapping | It can silently map multiple MPAS cells to one CoLM element and breaks the MPAS cell == CoLM element contract. Rejected: Embedded root-gather river/reservoir address tables | They recreate legacy process-pool assumptions and are not needed by MPAS-owned routing/restart paths. Confidence: high Scope-risk: moderate Directive: Keep CoLM MPI enabled through MPAS communicators; do not revive standalone process pools or global river-network replication. Tested: git diff --check; cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8 Not-tested: End-to-end multi-rank MPAS run with real landdata/restart files.
Embedded CoLM writes vector restart shards per MPAS rank to avoid block-file races. Reads must prefer the same rank-local shards while keeping original landdata fallback, and MPAS q2 must stay specific humidity. Constraint: MPAS cell-owned CoLM runs have no separate IO/worker pool and retain CoLM patch/PFT state internally. Rejected: Writing shared vector restart block files from all ranks | races/clobbers block files under MPAS-owned decomposition. Confidence: high Scope-risk: narrow Directive: Preserve original block-file fallback for initial landdata; rank-suffixed files are only an embedded restart path. Tested: git diff --check; cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8 Not-tested: multi-rank runtime restart with real CoLM landdata/restart files
Map MPAS microphysics precipitation into CoLM without truncating snow/graupel to the rainncv amount. CoLM still receives explicit rain/snow phase through forc_rain/forc_snow while forc_prc/forc_prl retain the water-balance total used by CoLM checks. Constraint: MPAS rainncv, snowncv, and graupelncv are separate diagnostics, matching the Noah-MP MPAS driver treatment. Rejected: deriving snow solely as a fraction of rainncv | drops explicit snowncv/graupelncv water when rainncv does not include frozen precipitation. Confidence: high Scope-risk: narrow Directive: Keep MPAS precipitation amount and phase fields separate when adapting future CoLM forcing changes. Tested: git diff --check; cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8 Not-tested: runtime integration case with real MPAS/CoLM landdata and restart files
CoLM now follows the MPAS block/cell decomposition while keeping patch and PFT state internal to CoLM, with cell-level flux/state exchange through the MPAS physics driver. Constraint: MPAS cell == CoLM element, and CoLM master/io/worker process pools are not valid inside MPAS. Rejected: standalone global CoLM landdata loading | it can deadlock or replicate global state under MPAS ranks. Confidence: high Scope-risk: broad Directive: Do not reintroduce standalone CoLM preprocess/make-surface paths or rank-specific IO-worker process ownership in MPAS embedded mode. Tested: git diff --check; cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8 Not-tested: runtime smoke test with real CoLM landdata/restart/river inputs; restart portability across changed MPI decompositions.
Constraint: CoLM2024 must stay MPAS-embedded while preserving patch/PFT internals. Rejected: Reopening standalone CoLM IO/worker paths | would reintroduce split ownership and deadlock risk. Confidence: high Scope-risk: moderate Directive: Keep runoff diagnostics cumulative in MPAS units; do not restore standalone role names in embedded paths. Tested: git diff --check; cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8; cmake --build build-cmake-colm-debug --target install -j 8; rg old CoLM role names under physics_colm2024 Not-tested: Real multi-rank MPAS+CoLM runtime case; river-system communicator refactor; zero-solve-cell rank support.
Constraint: MPAS ranks can own multiple CoLM river systems while CoLM still routes through the MPAS compute communicator. Rejected: Collapsing embedded GridRiverLakeFlow to one river system | couples unrelated rivers and forces a global routing timestep. Confidence: medium Scope-risk: moderate Directive: Preserve river-mouth keyed irivsys mapping in embedded mode; do not restore irivsys(:)=1 there. Tested: git diff --check; cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8; cmake --build build-cmake-colm-debug --target install -j 8 Not-tested: Real multi-rank MPAS+CoLM river-routing case with split river systems.
MPAS decomposition can leave some ranks with zero local land solve cells. Treat those ranks as valid empty CoLM owners while preserving MPAS communicator participation for river/restart collectives. Constraint: MPAS cell ownership defines CoLM element ownership; no rank may fall back to global landdata or standalone CoLM process pools. Rejected: Requiring at least one CoLM cell per MPAS rank | breaks normal MPAS partitioning and wastes MPI resources. Confidence: high Scope-risk: moderate Directive: Keep MPAS_EMBEDDED_COLM on subset-based landdata reads; do not reintroduce global mesh/pixelset fallback for empty ranks. Tested: git diff --check; cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8; cmake --build build-cmake-colm-debug --target install -j 8 Not-tested: end-to-end MPAS runtime with a real decomposition containing zero-land-cell ranks.
Constraint: CoLM must run as an MPAS land-surface option without standalone build, preprocess, run scripts, or dedicated CoLM rank pools. Rejected: Linking an external libcolm or keeping CoLM master/io/worker MPI roles | those paths fight MPAS decomposition and leave rank-local land state inconsistent. Confidence: high Scope-risk: broad Directive: Keep MPAS cell ownership as the source of truth; do not reintroduce CoLM standalone process pools or generated surface builders. Tested: git diff --check; cmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8; cmake --build build-cmake-colm-debug --target install -j 8; embedded CoLM Make dry-run; 201-object source table check; residual standalone/CaMa/master-worker scans. Not-tested: Real multi-rank MPAS+CoLM landdata/restart runtime smoke; full traditional Make compile.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why
CoLM2024 needs to behave like Noah-MP inside MPAS: same MPI ranks, same cell ownership, no dedicated CoLM master/io/worker process pool, and no separate standalone build product.
Validation
git diff --checkcmake --build build-cmake-colm-debug --target mpas_atmosphere -j 8cmake --build build-cmake-colm-debug --target install -j 8-lcolm/libcolmusageNot tested