Skip to content

fix(splines): bspline include_bias=True makes the design rank-deficient #33

Description

@ChrisW09

Summary

BSplineTransformer defaults to include_bias=True, and it is the only spline family that
does. A B-spline basis over a clamped knot vector is a partition of unity — every row
sums to exactly 1 — so the prepended intercept column is an exact linear combination of the
remaining columns. The design matrix is singular.

Preprocessor(numerical_method="bspline") inherits that default, so this is the
out-of-the-box behaviour, not an opt-in.

Reproduction

import numpy as np
from pretab.transformers import BSplineTransformer

X = np.linspace(0, 1, 200).reshape(-1, 1)
for include_bias in (True, False):
    D = BSplineTransformer(output_dim=8, include_bias=include_bias).fit_transform(X)
    print(include_bias, D.shape, "rank", np.linalg.matrix_rank(D), "cond", f"{np.linalg.cond(D):.2e}")
True  (200, 9) rank 8 cond 6.30e+15
False (200, 8) rank 8 cond 1.79e+01

The non-bias columns sum to exactly 1.0 on every row — i.e. they reproduce the bias column.

Through the Preprocessor:

import numpy as np, pandas as pd
from pretab import Preprocessor

rng = np.random.default_rng(0)
df = pd.DataFrame({"a": rng.uniform(0, 1, 300)})
y = np.sin(6 * df.a) + 0.1 * rng.normal(size=300)
A = Preprocessor(numerical_method="bspline", target_aware=False,
                 placement_strategy="uniform", categorical_method="none").fit(df, y).transform(
                 df, return_array=True)
print("rank", np.linalg.matrix_rank(A), "of", A.shape[1], "cond", f"{np.linalg.cond(A):.2e}")
rank 7 of 8 cond 9.62e+15

Expected

The emitted basis has full column rank, so a downstream linear model has an identifiable
intercept and a well-conditioned normal-equations matrix.

Actual

One redundant column and a condition number around 1e15 — numerically singular. Ordinary
least squares is unstable; a regularized fit silently spends a coefficient on an
unidentifiable direction.

Two further symptoms of the same cause

Because the bias column is counted in the output width, bspline is also the only numerical
method whose per-feature width does not equal output_dim:

output_dim=7  bspline        -> 8
output_dim=7  mspline        -> 7
output_dim=7  ispline        -> 7
output_dim=7  cubicspline    -> 7
output_dim=7  pspline        -> 7
output_dim=7  rbf            -> 7
output_dim=7  ple            -> 7

and it is the only one that can breach the adaptive window:

adaptive[4, 9]  bspline -> 10     <- exceeds max_output_dim=9
adaptive[4, 9]  mspline ->  9

include_bias is not a Preprocessor parameter ("include_bias" in Preprocessor().get_params()
is False), so there is no way to switch it off from the entry point.

Root cause

pretab/transformers/splines/bspline.py:42include_bias: bool = True, against
False in MSplineTransformer, ISplineTransformer and the shared
BaseSplineTransformer.

Suggested fix

Default BSplineTransformer.include_bias to False, matching its siblings and the base
class. That single change resolves all three symptoms: full rank, width equal to
output_dim, and the adaptive window respected.

This does change a documented default for the standalone class. Given 1.0.0 has not
shipped, now is the cheap moment; the alternative (keep the default and emit a
ConfigWarning when include_bias=True is requested on a partition-of-unity basis) leaves
the default path broken.

Worth noting the same collinearity applies to MSplineTransformer(include_bias=True)
M-splines are rescaled B-splines, so their span also contains the constant function
(rank 8/9). That is opt-in rather than default, but a warning would help there too.
ISplineTransformer(include_bias=True) is full rank and unaffected.

A regression test asserting matrix_rank(design) == design.shape[1] for each family would
pin this down.

Environment

  • pretab 0.1.0 (main @ 51c3043)
  • Python 3.11.15, numpy 2.4.6, pandas 2.3.3, scikit-learn 1.9.0, scipy 1.17.1
  • macOS (darwin 25.5.0)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions