Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions doc/release_notes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ Upcoming Version
* ``add_variables(binary=True, ...)`` now accepts ``lower``/``upper`` bounds, as long as they are 0 or 1. Previously binary bounds could only be set via the ``.lower``/``.upper`` setters after creation. (https://github.com/PyPSA/linopy/issues/776)
* ``add_piecewise_formulation`` gained an ``active_fill`` parameter that gates a partial ``active`` (defined over a subset of the indexed dimension, or masked) as always-active (``1``) or always-off (``0``); without it, a partial ``active`` — which was previously zeroed silently — now raises. Useful when one formulation mixes gated and ungated entities (e.g. committable and non-committable units sharing a ``status``). ``active_fill`` is transitional and will be removed once v1 semantics make ``active.reindex(coords).fillna(value)`` sufficient. (https://github.com/PyPSA/linopy/issues/796)

*Documentation*

* The example notebooks now opt into the v1 arithmetic convention (``linopy.options["semantics"] = "v1"``). The coordinate-alignment and expression tutorials were reworked to teach strict label-based alignment: a mismatch on a shared dimension raises rather than silently filling or pairing by position, and is resolved explicitly with ``.sel`` / ``.reindex`` / ``.assign_coords`` or an explicit ``join=`` on the named ``.add`` / ``.mul`` / ``.le`` / … methods.

**Performance**

* ``LinearExpression.groupby(...).sum()`` now scatters terms directly into the padded result arrays via ``xarray.apply_ufunc``, avoiding intermediate copies and speeding up the grouping. A single kernel covers both numpy and chunked (dask) data, the latter staying lazy. On representative models this lowers build and export peak memory by up to ~3x.
Expand Down
88 changes: 55 additions & 33 deletions examples/coordinate-alignment.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"source": [
"# Coordinate Alignment\n",
"\n",
"Since linopy builds on xarray, coordinate alignment matters when combining variables or expressions that live on different coordinates. By default, linopy aligns operands automatically and fills missing entries with sensible defaults. This guide shows how alignment works and how to control it with the ``join`` parameter."
"Since linopy builds on xarray, coordinate alignment matters when combining variables or expressions that live on different coordinates. Under the **v1 convention**, linopy aligns strictly by coordinate *label*: operands that share a dimension must carry the same labels on it, and a genuine mismatch raises rather than being silently patched over. This guide shows how that strict alignment works and how to resolve mismatches explicitly with the ``join`` parameter."
]
},
{
Expand All @@ -19,16 +19,18 @@
"import pandas as pd\n",
"import xarray as xr\n",
"\n",
"import linopy"
"import linopy\n",
"\n",
"linopy.options[\"semantics\"] = \"v1\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Default Alignment Behavior\n",
"## Alignment by label\n",
"\n",
"When two operands share a dimension but have different coordinates, linopy keeps the **larger** (superset) coordinate range and fills missing positions with zeros (for addition) or zero coefficients (for multiplication)."
"When two operands **share a dimension**, linopy requires them to carry the same coordinate labels on it. Labels align by name, never by position, and non-shared dimensions broadcast freely. If the shared labels differ — a different set, or even the same labels in a different order — the operation raises a ``ValueError``; linopy never silently drops, reorders, or invents coordinates, because in an optimization model a dropped coordinate is a dropped term or constraint."
]
},
{
Expand All @@ -50,7 +52,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Adding ``x`` (5 time steps) and ``y`` (3 time steps) gives an expression over all 5 time steps. Where ``y`` has no entry (time 3, 4), the coefficient is zero — i.e. ``y`` simply drops out of the sum at those positions."
"``x`` spans ``time`` 0–4 and ``y`` spans 0–2. Because ``time`` is shared but the label sets differ, adding them directly raises:"
]
},
{
Expand All @@ -59,14 +61,17 @@
"metadata": {},
"outputs": [],
"source": [
"x + y"
"try:\n",
" x + y\n",
"except ValueError as e:\n",
" print(e)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The same applies when multiplying by a constant that covers only a subset of coordinates. Missing positions get a coefficient of zero:"
"The same holds for a constant that covers only a subset of the coordinates — a ``DataArray`` on ``time`` 0–2 shares the dimension but not the full label set, so multiplying by it raises too:"
]
},
{
Expand All @@ -76,14 +81,19 @@
"outputs": [],
"source": [
"factor = xr.DataArray([2, 3, 4], dims=[\"time\"], coords={\"time\": [0, 1, 2]})\n",
"x * factor"
"try:\n",
" x * factor\n",
"except ValueError as e:\n",
" print(e)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Adding a constant subset also fills missing coordinates with zero:"
"### Resolving a mismatch\n",
"\n",
"To combine mismatched operands, tell linopy how. Either bring them onto shared coordinates first — ``.sel`` / ``.isel`` / ``.reindex`` — relabel one side with ``.assign_coords``, or pass an explicit ``join=`` to the named ``.add`` / ``.mul`` / … methods (covered below). For instance, select ``x`` down to ``factor``'s coordinates and multiply on the shared range:"
]
},
{
Expand All @@ -92,16 +102,16 @@
"metadata": {},
"outputs": [],
"source": [
"x + factor"
"x.sel(time=[0, 1, 2]) * factor"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Constraints with Subset RHS\n",
"### Constraints follow the same rule\n",
"\n",
"For constraints, missing right-hand-side values are filled with ``NaN``, which tells linopy to **skip** the constraint at those positions:"
"A comparison (``<=``, ``>=``, ``==``) aligns its two sides exactly like ``+`` does. So a right-hand side that covers only a subset of the constraint's coordinates is a shared-dimension mismatch, and raises:"
]
},
{
Expand All @@ -111,24 +121,35 @@
"outputs": [],
"source": [
"rhs = xr.DataArray([10, 20, 30], dims=[\"time\"], coords={\"time\": [0, 1, 2]})\n",
"con = x <= rhs\n",
"con"
"try:\n",
" x <= rhs\n",
"except ValueError as e:\n",
" print(e)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The constraint only applies at time 0, 1, 2. At time 3 and 4 the RHS is ``NaN``, so no constraint is created."
"Build the constraint only where the RHS is defined by aligning first — either select the shared range, or use an explicit ``join`` (below). Selecting ``x`` down to the RHS coordinates:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"x.sel(time=[0, 1, 2]) <= rhs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Same-Shape Operands: Positional Alignment\n",
"### No positional shortcut\n",
"\n",
"When two operands have the **same shape** on a shared dimension, linopy uses **positional alignment** by default — coordinate labels are ignored and the left operand's labels are kept. This is a performance optimization but can be surprising:"
"Legacy linopy had a special case: two operands of the **same shape** on a shared dimension were paired by position, ignoring their labels. The v1 convention removes this — alignment is always by label, whatever the shapes. So an operand whose labels differ from ``x`` raises even when the shapes match:"
]
},
{
Expand All @@ -140,14 +161,17 @@
"offset_const = xr.DataArray(\n",
" [10, 20, 30, 40, 50], dims=[\"time\"], coords={\"time\": [5, 6, 7, 8, 9]}\n",
")\n",
"x + offset_const"
"try:\n",
" x + offset_const\n",
"except ValueError as e:\n",
" print(e)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Even though ``offset_const`` has coordinates ``[5, 6, 7, 8, 9]`` and ``x`` has ``[0, 1, 2, 3, 4]``, the result uses ``x``'s labels. The values are aligned by **position**, not by label. The same applies when adding two variables or expressions of identical shape:"
"If you genuinely want **positional** pairing — ignoring that the labels differ — ask for it explicitly with ``join=\"override\"``, which relabels the right operand onto the left's coordinates. It still requires the shared dimension to match in size:"
]
},
{
Expand All @@ -157,16 +181,14 @@
"outputs": [],
"source": [
"z = m.add_variables(lower=0, coords=[pd.RangeIndex(5, 10, name=\"time\")], name=\"z\")\n",
"x + z"
"x.add(z, join=\"override\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"``x`` (time 0–4) and ``z`` (time 5–9) share no coordinate labels, yet the result has 5 entries under ``x``'s coordinates — because they have the same shape, positions are matched directly.\n",
"\n",
"To force **label-based** alignment, pass an explicit ``join``:"
"``x`` (time 0–4) and ``z`` (time 5–9) share no labels; ``override`` pairs them by position and keeps ``x``'s labels. For **label-based** alignment over the union instead, use ``join=\"outer\"``:"
]
},
{
Expand All @@ -182,7 +204,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"With ``join=\"outer\"``, the result spans all 10 time steps (union of 0–4 and 5–9), filling missing positions with zeros. This is the correct label-based alignment. The same-shape positional shortcut is equivalent to ``join=\"override\"`` — see below."
"With ``join=\"outer\"``, the result spans all 10 time steps (union of 0–4 and 5–9); each non-overlapping position keeps whichever operand is present there, the missing side contributing its additive identity (``0``). The next section walks through every ``join`` value."
]
},
{
Expand Down Expand Up @@ -365,7 +387,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"With ``join=\"left\"``, the result covers all of ``a``'s coordinates (i=0, 1, 2). At i=2, where the RHS has no value, the RHS becomes ``NaN`` and the constraint is masked out.\n",
"With ``join=\"left\"``, the result covers all of ``a``'s coordinates (i=0, 1, 2). At i=2, where the RHS has no value, it is filled with ``0``, so the row becomes ``a[2] ≤ 0``.\n",
"\n",
"The same methods work on expressions:"
]
Expand Down Expand Up @@ -476,15 +498,15 @@
"source": [
"## Summary\n",
"\n",
"| ``join`` | Coordinates | Fill behavior |\n",
"| ``join`` | Coordinates | Non-overlapping positions |\n",
"|----------|------------|---------------|\n",
"| ``None`` (default) | Auto-detect (keeps superset) | Zeros for arithmetic, NaN for constraint RHS |\n",
"| ``\"inner\"`` | Intersection only | No fill needed |\n",
"| ``\"outer\"`` | Union | Fill with operation identity (0 for add, 0 for mul) |\n",
"| ``\"left\"`` | Left operand's | Fill right with identity |\n",
"| ``\"right\"`` | Right operand's | Fill left with identity |\n",
"| ``\"override\"`` | Left operand's (positional) | Positional alignment, ignore labels |\n",
"| ``\"exact\"`` | Must match exactly | Raises error if different |"
"| ``None`` (default) | Must match by label **and order** (v1 uses ``exact``) | Raises on any mismatch |\n",
"| ``\"inner\"`` | Intersection only | — (nothing to fill) |\n",
"| ``\"outer\"`` | Union | Missing side filled with the operation's identity |\n",
"| ``\"left\"`` | Left operand's | Right's extras dropped; right's gaps filled with identity |\n",
"| ``\"right\"`` | Right operand's | Left's extras dropped; left's gaps filled with identity |\n",
"| ``\"override\"`` | Left operand's (positional) | Requires equal size on shared dims |\n",
"| ``\"exact\"`` | Must match by label **and order** | Raises on mismatch |"
]
}
],
Expand Down
2 changes: 2 additions & 0 deletions examples/create-a-model-with-coordinates.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@
"source": [
"import linopy\n",
"\n",
"linopy.options[\"semantics\"] = \"v1\"\n",
"\n",
"m = linopy.Model()"
]
},
Expand Down
3 changes: 3 additions & 0 deletions examples/create-a-model.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,11 @@
"metadata": {},
"outputs": [],
"source": [
"import linopy\n",
"from linopy import Model\n",
"\n",
"linopy.options[\"semantics\"] = \"v1\"\n",
"\n",
"m = Model()"
]
},
Expand Down
3 changes: 3 additions & 0 deletions examples/creating-constraints.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,11 @@
"metadata": {},
"outputs": [],
"source": [
"import linopy\n",
"from linopy import Model\n",
"\n",
"linopy.options[\"semantics\"] = \"v1\"\n",
"\n",
"m = Model()"
]
},
Expand Down
Loading