Add ArcGIS Online export script and embeddable chart pages#97
Open
michaelmfoley wants to merge 3 commits into
Open
Add ArcGIS Online export script and embeddable chart pages#97michaelmfoley wants to merge 3 commits into
michaelmfoley wants to merge 3 commits into
Conversation
Support building an ArcGIS StoryMap from the 2026-04 CSO/EJ analysis: - analysis/export_arcgis.py packages hosted-feature-layer-ready files into a gitignored arcgis_export/ directory: CSO outfall points (2022-2025 window), watershed/town/block-group polygons joined with EJSCREEN EJ indicators (as 0-100 percents) and discharge totals, an optional 2011 NECIR baseline, and a README data dictionary with suggested field aliases. Includes volume-conservation and geometry assertions, and repairs swapped lat/lon pairs found in the source outfall list (e.g. Haverhill HAV021B). - docs/embeds/ serves standalone, iframe-able wrapper pages for eight post charts (the chart includes are already self-contained HTML documents; Jekyll just doesn't serve _includes/). An index page documents usage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AGO's hosted-feature-layer publisher fails with "There was an error publishing the service" on the towns and watersheds layers as previously written: the MapShaper-simplified MassGIS sources carry non-RFC ring winding and one self-intersecting polygon (WAYLAND). Repair invalid geometries with buffer(0) and write all polygon layers with RFC7946=YES (enforced ring winding, 7-decimal precision, no legacy "crs" member). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ArcGIS Online's GeoJSON importer still rejects the MassGIS-derived towns and watersheds layers after RFC 7946 normalization and geometry repair (the block-group layer, from Census cartographic boundaries, publishes fine — including null attributes, ruling those out). Sidestep the GeoJSON parser entirely: write all three polygon layers into one amend_polygons.gdb.zip via GDAL's OpenFileGDB driver (promote_to_multi, since FileGDB layers hold a single geometry type), verify by read-back, and document it as the recommended AGO upload. GeoJSON outputs are kept for non-Esri tools. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tooling to build an ArcGIS StoryMap from the 2026-04 CSO × EJ analysis (companion to #96, but independent of it).
analysis/export_arcgis.pyPackages ArcGIS-Online-ready layers into a gitignored
arcgis_export/directory (run fromanalysis/, needs only pandas + geopandas, both already inamend_python; not part of CI):cso_outfalls_2022_2025.csv— 193 outfall points with clean field names (fixes thepermiteemisspelling at the export boundary) and total discharge/counts for the through-2025 windowwatersheds_ej_discharge.geojson/towns_ej_discharge.geojson/block_groups_ej_discharge.geojson— MassGIS/Census polygons joined with EJSCREEN indicators (converted to 0–100 percents) and discharge totals--include-2011NECIR baseline (renames the digit-leading columns AGO rejects)README.mddata dictionary with units, caveats, suggested AGO field aliases, and attributionBuilt-in checks: feature counts (193/32/351/4,982), CRS = EPSG:4326, MA bounding box, and volume conservation across aggregation levels (all agree at 19,348.3 Mgal; town totals are ~7% lower because some outfalls have unknown town assignments — documented in the README).
Data fix: the source outfall list has one row (Haverhill HAV021B) with latitude/longitude swapped (
lat=-71.08, lon=42.77); the export detects and repairs swapped pairs and logs them. This same row presumably renders off-map in the existing Plotly maps — may be worth fixing upstream in the outfall ingest at some point.docs/embeds/Standalone, iframe-able wrapper pages for eight of the post's charts (for embedding in ArcGIS StoryMaps or elsewhere). The chart includes in
docs/_includes/charts/are already complete self-contained HTML documents — Jekyll just doesn't serve_includes/— so each page is 4 lines: front matter + the include. Anindex.mddocuments usage and links all eight. Once merged these serve athttps://openamend.org/embeds/<name>.htmland update automatically whenever the pipeline regenerates the underlying charts.Testing
{% raw %}unwrapped) — each result is a complete HTML document with its Chart.js CDN reference; the include-path pattern is identical to what the posts already use in production🤖 Generated with Claude Code