refactor(int32): per-model label dtype instead of sticky process-global#828
Open
FBumann wants to merge 5 commits into
Open
refactor(int32): per-model label dtype instead of sticky process-global#828FBumann wants to merge 5 commits into
FBumann wants to merge 5 commits into
Conversation
…ss-global Store the label dtype on the Model (snapshot of options['label_dtype'] at creation, exposed as Model.label_dtype) and widen only that model to int64 on overflow. Every cast site reads the dtype from its object's model reference; save_join takes it as an explicit fill_dtype parameter. This removes the sticky global state: the option and other models are unaffected by one model's widening, reset() keeps its usual semantics, and the widened dtype travels with the model through netCDF and pickle round-trips instead of relying on process-global stickiness. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Model(label_dtype=np.int64) is more direct than mutating the global option before construction; None falls back to options['label_dtype'] so embedders that construct the Model internally keep a process-wide default. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extract Model._allocate_labels (shared by variable and constraint label assignment) and config.validate_label_dtype (shared by the option setter and Model.__init__). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… argument
The option predated the constructor argument; with per-model storage it
only duplicated the knob. Embedders forward constructor kwargs (PyPSA:
n.optimize(model_kwargs={'label_dtype': np.int64})), so no process-wide
setting is needed. config.py returns to its pre-int32 state.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This implements int dtype control per Model instead of through a global config option (global kept).
This resolves all potential issues with the global for little complexity.
I think this is the right approach.
Note
The following content was generated by AI.
Stacked on #822. Keeps its auto-widening behavior but moves the label dtype from the sticky process-global option onto the
Modelitself.What changed
Model()argument:Model(label_dtype=np.int64), defaultnp.int32, exposed asModel.label_dtype. Thelinopy.options["label_dtype"]setting is removed entirely — embedders forward constructor kwargs instead (PyPSA:n.optimize(model_kwargs={"label_dtype": np.int64})), soconfig.pyreturns to its pre-perf: default integer arrays to int32 for ~25% memory reduction #566 state.int64when its labels would exceed the int32 maximum; theUserWarningrecommends passinglabel_dtype=np.int64upfront.common.save_jointakes it as an explicitfill_dtypeparameter instead of reading the global.read_netcdfrestores the widened dtype per model (from the counters, as before). Pickle round-trips preserve it automatically since it is an instance attribute.OptionSettings.widen_label_dtype()and its sticky-reset()semantics are removed, along with the test-suite fixture that guarded against global-state leakage.Why per-model
#822 chose the process-global because "the four float→int cast sites … have no reliable model reference (expressions can be detached)". That premise doesn't hold in the current codebase:
BaseExpression.__init__raises unlessmodelis aModelinstance, and everylabel_dtypecast site is a method on an object with a validated model reference (BaseExpression,Variable,Variables,Constraints,CSRConstraint). The only model-less site,save_join, is called withinteger_dtype=Truefrom exactly three container properties that all haveself.modelin scope.With a reliable model reference, a per-model monotonic widen gives the same no-narrowing guarantee at the same zero hot-path cost (an attribute read), and removes the global design's edge cases:
options["label_dtype"] = np.int32) while a widened model is alive lets the cast sites silently narrow>2^31labels —int64 → int32.astype()wraps to negative values without any error. Per-model, there is nothing to revert.Cast-site → model-reference verification
expressions.pyvars cast in initBaseExpression.__init__modelparam, enforcedisinstance(model, Model)(check moved above the cast)expressions.pysanitizeBaseExpressionself.modelexpressions.pysimplify (2×)LinearExpressionself.modelvariables.pyffill/bfill/sanitizeVariableself.model(init enforcesisinstance(model, Model))variables.pyflatremapVariablesself.modelconstraints.py_to_dataset(2×)CSRConstraintself._modelconstraints.pyflatremapConstraintsself.modelcommon.pysave_joinVariables.labels,Constraints.labels,Constraints.vars)Tests
test/test_dtypes.py: the widen tests assert other models stayint32; newtest_label_dtype_init_arg(+ invalid-dtype rejection),test_widen_applies_to_expressions,test_auto_widen_survives_pickle; the netcdf test asserts the loaded model is widened. Therestore_label_dtypefixture and the option tests are gone. Full suite passes.🤖 Generated with Claude Code