FMI build once simulate many

[adrpo]

Fixes #78, the first task of #295: each FMU is built once and simulated with every tool the job asks for, instead of once per tool.

The duplication

The FMI jobs come in pairs. Both jobs of a pair build the same Model Exchange FMUs — about 19500 of them — and differ only in what simulates them:

OMSimulator job	FMPy job	compiler
v1.26-fmi	v1.26-fmi-fmpy	maintenance/v1.26
v1.27-fmi	v1.27-fmi-fmpy	maintenance/v1.27
master-fmi	master-fmi-fmpy	master

Building is what costs. Measured here on the twelve models of ExternData, nine of which build an FMU:

separate   164s building + 0.60s simulating   OMSimulator
           167s building + 3.09s simulating   FMPy       <- the same FMUs, rebuilt
merged     178s building + 0.74s + 2.58s      both       <- one build

48% of the run, three times over.

The three *-cs-fmu-cvode jobs are not part of this: they build Co-Simulation FMUs with --fmiFlags=s:cvode --fmuRuntimeDepends=modelica, which are different artifacts. Only the simulator may differ between results that share an FMU; anything that changes the FMU is a different job.

Using it

--fmisimulator can be given once per tool:

./test.py --branch=v1.27-fmi --fmi=true \
          --fmisimulator=/path/to/OMSimulator \
          --fmisimulator='python3 -m fmpy' \
          configs/conf.json

--branch names the job, and every simulator derives its own branch from it:

simulator	branch	published to
OMSimulator	v1.27-fmi	branches/v1.27-fmi
FMPy	v1.27-fmi-fmpy	branches/v1.27-fmi-fmpy

which is where those results have always been, so no report and no history has to move. The branch follows the tool, not the order, so a job given FMPy alone still fills v1.27-fmi-fmpy and leaves v1.27-fmi untouched — including where it publishes.

A branch directory looks exactly as it did before, file names included. A simulator writes <model>_<simulator>.sim in the workspace so that the tools of one job do not overwrite each other, but each branch is published from a directory where those appear under the plain <model>.sim. The .err is written by the build, so every simulator of a model publishes the same one; a copy rather than a link into another branch, since a job may fill only one of them, and 66 MB of 3.4 kB files against the 58 GB such a directory already holds is not worth the coupling.

Adding a simulator

The simulators are described in configs/fmi-simulators.json. Adding one is an entry there and no change to any script:

"fmusim": {
  "resultExtension": "csv",
  "versionArgument": "--version",
  "optionalArguments": { "stepSizeArgument": " --output-interval {stepSize:g}" },
  "arguments": "--interface-type ModelExchange --output-file {result} --start-time {startTime:g} --stop-time {stopTime:g}{stepSizeArgument} {fmu}"
}

arguments is a template over the model and the experiment; optionalArguments are the flags that have to disappear when there is nothing to put in them. That last one is not hypothetical: OMSimulator hangs on --stepSize=0 rather than ignoring it (OMSimulator#1622, found while doing this), while FMPy wants --output-interval 0 all the same. The entry for fmusim is included because it is the next tool we expect to want; it is marked untested, and asking for a simulator nobody has run yet says so rather than failing every model with a puzzling error. A Python package without a command line needs a small driver script that the entry points command at.

The README has both: how to run several simulators, and how to add one.

Jenkins

Six stages become three. The parameters keep their meaning:

• fmi_v1_27 asks for OMSimulator, fmpy_fmi_v1_27 for FMPy
• both ticked runs one job that builds every FMU once and simulates it with both
• either alone runs that tool alone and fills only its table

Which simulators a job runs is now something it is told rather than something guessed from its name, but a job that says nothing still gets the tool its name implies, so the cs-fmu-cvode jobs and every job that does not test FMI are untouched. OMSimulator is only cloned and built when a job actually asks for it.

How it was checked

Everything below was run here against real FMUs, with omc 1.28.0~dev-284, OMSimulator 3.0.0~154 and FMPy 0.3.30.

• The results are the same either way. Every model's phase, verification time, number of variables compared and number that differ is identical between a merged run and the two separate runs, for both tables.
• All three ways of asking: OMSimulator alone fills v1.27-fmi, FMPy alone fills v1.27-fmi-fmpy, both fill the two of them from a single build — results and published directories alike.
• The command lines are unchanged. The templates reproduce, character for character, what testmodel.py passed the two tools before, with a step size and without one.
• The published pages resolve: 21 links, 0 missing, in both branches' pages, each holding its own simulator's .sim.

That testing earned its keep: it caught FMPy's --output-file being blanked by OMSimulator's omit-the-flag rule, models failing before the FMU exists being reported as phase 0 instead of the phase the build stopped at, omc_new captured as a local when the verification became a function, and the first simulator's branch being the name of the job rather than of the tool. Each of those would have looked like a mass regression in one of the tables.

Before it runs for real

• This has never run in Jenkins. The Groovy is checked by reading and by evaluating the selection logic for every combination of the parameters, not by a build. Worth one run on a small library with both parameters ticked before the old jobs are retired.
• test.py is also touched by the PostgreSQL branch (Several improvements to LibraryTesting needed: network SQLDB and docker images to run the tests #295) in the same region. Whichever merges second needs a small merge.

---

Generated by Claude Code.