Anirudh/s7 runtime and heap fixes - #60
Open
Anirudh-red11 wants to merge 8 commits into
Open
Conversation
Fixes the issue Dhirendra spotted: person ids in the demand file were sorted alphabetically as strings (0,1,10,100,1000,...) rather than numerically. Also lets us cut population size for faster dev-loop runs.
Generated with scripts/reduce_population.py from the original 40k demand
file. Not committing the .gz itself (matches the existing .gitignore
convention for demand-seasonal*.xml.gz in this folder) - regenerate with:
python3 scripts/reduce_population.py \
--input demand-seasonal-40k-50it-epsg32754-7886c91.xml.gz \
--output demand-seasonal-20k-50it-epsg32754-7886c91.xml.gz \
--max-agents 20000
Per Dhirendra: end the sim at 4pm instead of 23:59:59 to get closer to a 1-minute runtime. Fire ignites at 11:15 and the evacuate order goes out at 12:15, so this still covers the full event with headroom.
The maven-failsafe-plugin (which actually runs the *IT.java integration tests) had no argLine configured, so it forked with Java's default heap instead of anything close to the surefire config below it. That's what caused the EOFException/NullPointerException crash pattern on SCSMidweekInJanScenario7IT - the forked JVM was getting killed, and surefire/failsafe surfaced that as a misleading NPE at the Run.main() call site rather than a real error. Set -Xmx10g -Xms3g on both plugins: capped below what would risk the OS killing the process on a 16GB machine, and starting small (3g) so it only grows toward the cap if actually needed rather than eagerly reserving the max up front.
For loading the road network directly into QGIS alongside the zones and fire GeoJSON files already in scenarios/surf-coast-shire/.
Output of scripts/network_to_geojson.py against surf-coast-shire-network-2021-epsg32754.xml.gz - 5187 nodes, 10145 links. For visual QGIS work, not used by the simulation itself.
Second data point for the ~1min runtime target (per Dhirendra): try halving the already-reduced 20k population down to 10k, generated from the 20k file via scripts/reduce_population.py.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Still open
SCSMidweekInJanScenario7IT (unlike Scenario5IT) still fails on a separate, unrelated issue: an EditRoutes/disruption route-joining loop around simulated 09:00, tied to traffic-mgt-points.s3.json's impacted links and MATSim's within-day replanning. Diagnosed but not fixed, flagged to Dhirendra separately.
Test plan