Skip to content

Add regional calibration diagnostics (land cover and ecoregion breakdown) - #4069

Merged
mdietze merged 8 commits into
PecanProject:developfrom
Tejas7007:GH-ilamb-regional
Aug 25, 2026
Merged

Add regional calibration diagnostics (land cover and ecoregion breakdown)#4069
mdietze merged 8 commits into
PecanProject:developfrom
Tejas7007:GH-ilamb-regional

Conversation

@Tejas7007

Copy link
Copy Markdown
Contributor

Description

Adds a regional breakdown of the ensemble calibration assessment. Rather than a single domain-wide number, it reports the calibration of the ensemble spread per region, grouped two ways, by land cover class and by EPA/CEC ecoregion. It builds on the ensemble calibration diagnostics in the calibration module, computing the same spread-to-error ratio and coverage on subsets of sites.

Contents (in modules/benchmark/inst/ilamb/calibration/):

  • regional_diagnostics.py samples the per-member downscaled maps and the benchmark at each site and computes calibration per land cover class. Its stratified_calibration engine is grouping agnostic and is reused for the ecoregion breakdown.
  • ecoregion_join.py assigns each site an EPA/CEC North American ecoregion (Level 1 and Level 2) by point in polygon, then runs the same engine on that grouping.
  • regional_figures.py produces the choropleth ecoregion maps, the bar charts by land cover and by ecoregion, and the site map by land cover.
  • test_regional_diagnostics.py covers the stratified_calibration engine on synthetic data with known per-group calibration.
  • README_regional.md documents the inputs, the ecoregion shapefile source, and how to run everything.

The land cover class labels are the MODIS MCD12Q1 PFT classification used in the North American carbon reanalysis, with the class integers following the standard MODIS PFT coding, confirmed against site geography. The ecoregion breakdown uses the free CEC North American ecoregion shapefiles (Level 1 and Level 2), available from the EPA Ecoregions of North America page (https://www.epa.gov/eco-research/ecoregions-north-america) under its Downloads section.

Motivation and Context

Follow-on to the ensemble calibration work. A single domain-wide calibration number hides spatial structure. This reports calibration per land cover class and per ecoregion, showing where the ensemble spread is trustworthy and where it is not. The engine accepts any grouping and any benchmark, so future benchmarks and groupings plug into the same breakdown.

Review Time Estimate

  • Immediately
  • Within one week
  • When possible

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My change requires a change to the documentation.
  • My name is in the list of CITATION.cff
  • I agree that PEcAn Project may distribute my contribution under any or all of
    • the same license as the existing code,
    • and/or the BSD 3-clause license.
  • I have updated the CHANGELOG.md.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Breaks the ensemble calibration assessment down by region rather than a single
domain-wide number, grouping the diagnostics by land cover class and by EPA/CEC
ecoregion. Reuses the ensemble calibration diagnostics, so per-region numbers
match the domain-wide ones. The grouping engine accepts any labels and any
benchmark. Includes a notebook, tests, and figures (choropleth maps of
calibration by ecoregion and bar charts by land cover and ecoregion). Land cover
class labels are inferred from geography and the biomass and LAI signature and
are marked for confirmation.
Replace the inferred land-cover class names with the MODIS MCD12Q1 PFT
labels used in the North American reanalysis, confirmed against site
geography. Regenerate the land-cover figures with the corrected labels
and drop the narrative notebook.

@ankurdesai ankurdesai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good from my side

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this and the other png are Results, not code, and thus shouldn't be in the PR.

@mdietze
mdietze added this pull request to the merge queue Aug 25, 2026
Merged via the queue into PecanProject:develop with commit 3d0a777 Aug 25, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants