Summary
Fitting on a DataFrame with duplicate column labels fails with an opaque
AttributeError: 'DataFrame' object has no attribute 'dtype' from deep inside
_detect_column_types, with nothing pointing at the real cause.
Reproduction
import numpy as np, pandas as pd
from pretab import Preprocessor
rng = np.random.default_rng(0)
dup = pd.DataFrame(np.column_stack([rng.normal(size=50)] * 2), columns=["a", "a"])
Preprocessor(numerical_method="minmax").fit(dup, rng.normal(size=50))
AttributeError: 'DataFrame' object has no attribute 'dtype'
... pretab/preprocessor.py, line 307, in _detect_column_types
Expected
A PretabDataError naming the duplicated labels and explaining that column names must be
unique — the ColumnTransformer this builds keys its transformers by column name, so
duplicates cannot be routed unambiguously even if detection were fixed.
Actual
AttributeError from numpy/pandas internals.
Root cause
pretab/preprocessor.py:288-312. The loop does X[col], which returns a DataFrame
rather than a Series when col is duplicated, so X[col].dtype does not exist. (nunique()
on the preceding line happens to work on a frame, which is why the failure surfaces on the
dtype access rather than earlier.)
Suggested fix
Check up front in fit / _detect_column_types, after the dict/ndarray coercion:
duplicated = X.columns[X.columns.duplicated()].unique().tolist()
if duplicated:
raise PretabDataError(
f"Duplicate column names are not supported: {duplicated}.\n"
"Fix: rename the columns so every name is unique."
)
Cheap to check and turns a confusing internal failure into an actionable one.
Environment
- pretab 0.1.0 (
main @ 51c3043)
- Python 3.11.15, numpy 2.4.6, pandas 2.3.3, scikit-learn 1.9.0, scipy 1.17.1
- macOS (darwin 25.5.0)
Summary
Fitting on a DataFrame with duplicate column labels fails with an opaque
AttributeError: 'DataFrame' object has no attribute 'dtype'from deep inside_detect_column_types, with nothing pointing at the real cause.Reproduction
Expected
A
PretabDataErrornaming the duplicated labels and explaining that column names must beunique — the
ColumnTransformerthis builds keys its transformers by column name, soduplicates cannot be routed unambiguously even if detection were fixed.
Actual
AttributeErrorfrom numpy/pandas internals.Root cause
pretab/preprocessor.py:288-312. The loop doesX[col], which returns a DataFramerather than a Series when
colis duplicated, soX[col].dtypedoes not exist. (nunique()on the preceding line happens to work on a frame, which is why the failure surfaces on the
dtype access rather than earlier.)
Suggested fix
Check up front in
fit/_detect_column_types, after the dict/ndarray coercion:Cheap to check and turns a confusing internal failure into an actionable one.
Environment
main@ 51c3043)