GH-50906: [Python] Reshape 1D tensors to 2D in SparseCSR/CSC matrix conversion

[pratyushadk]

Rationale for this change

SparseCSRMatrix and SparseCSCMatrix represent 2-dimensional sparse matrices. In the current codebase, passing a 1D SciPy sparse array (such as scipy.sparse.csr_array([1, 0, 2]) with ndim == 1) to SparseCSRMatrix.from_scipy or SparseCSCMatrix.from_scipy silently creates a 1D SparseCSRMatrix with shape (n,) without dimension validation.

This PR adds explicit 2D dimension checks to from_scipy in SparseCSRMatrix and SparseCSCMatrix to raise a ValueError for non-2D sparse objects, while ensuring legacy csr_matrix / csc_matrix objects (which normalize 1D input to (1, n)) continue to work and are tested.

What changes are included in this PR?

• python/pyarrow/tensor.pxi: Added if obj.ndim != 2: validation in SparseCSRMatrix.from_scipy and SparseCSCMatrix.from_scipy to raise ValueError("Expected 2-dimensional sparse input for ...").
• python/pyarrow/tests/test_sparse_tensor.py: Added a parametric unit test test_sparse_csx_matrix_from_1d verifying that 1D vector input passed via csr_matrix / csc_matrix produces the expected (1, n) 2D sparse matrix via from_scipy.

Are these changes tested?

Yes, tested by the new parametric unit test test_sparse_csx_matrix_from_1d in python/pyarrow/tests/test_sparse_tensor.py.

Are there any user-facing changes?

Yes. SparseCSRMatrix.from_scipy and SparseCSCMatrix.from_scipy now explicitly raise ValueError if passed a 1D sparse array instead of silently creating an invalid 1D sparse matrix.

• GitHub Issue: GH-50906: [Python] Reshape 1D tensors to 2D in SparseCSR/CSC matrix conversion #50906

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

⚠️ GitHub issue #50906 has been automatically assigned in GitHub to PR creator.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[rok]

@pratyushadk I've renamed the issue and the PR to reflect that this should be implemented in Python. Let's move the discussion here.
Please update this PR with the Python solution.
Please do not copy paste agent output here. The goal is that you understand what you're doing and that you learn from it.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[pratyushadk]

@rok Thanks for the direction. I'm new to open source and your feedback really helped me understand where the fix actually belongs. Updated the PR with the Python reshape approach. Looking forward to contributing more in the future!

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[rok]

@pitrou we've gone through several iterations here. Please review and feel free to merge.

@pratyushadk I understand you're trying to contribute and I appreciate it. But in the future please open issues and PRs with clear motivations as you're bound to waste time otherwise.

[rok]

Can we change this?

Suggested change

	scipy_array = sc_array_class(array.reshape(1, -1))
	scipy_array = sc_array_class(array)

[pratyushadk]

Thank you for the review and guidance, @rok. I'll make sure to open issues and PRs with clearer motivations in future. Regarding the suggested change, csc_array(array) raises ValueError: CSC arrays don't support 1D input. Use 2D, so the reshape(1, -1) is needed to construct both csr_array and csc_array with matching (1, n) shape.

[rok]

I see. This is annoying - so:

>>> scipy.sparse.csc_array([1, 0, 2, 0, 0, 3]).ndim
ValueError: CSC arrays don't support 1D input. Use 2D

>>> scipy.sparse.csc_matrix([1, 0, 2, 0, 0, 3]).ndim
2

csc_matrix accepts 1D input while csc_array array does not.

Let's then simply test that 1D csc_matrix/csr_matrixroundtrips PyArrow tensor and that PyArrow tensor roundtrips to csc_matrix/csr_matrix. Forbid reshape path in case we have another type of object.

[pratyushadk]

Sure. I've switched the test to verify both roundtrips with csr_matrix and csc_matrix:

1. 1D scipy matrix -> PyArrow -> compare dense
2. 1D PyArrow tensor -> sparse -> to_scipy -> compare to scipy matrix

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[rok]

I want to move these from from_tensor to from_scipy and only do this if we're coming from scipy.{csc,csr}_matrix and reject otherwise. scipy is moving away from allowing 1D input here for a reason.

[pratyushadk]

That makes sense, reverted the from_tensor changes. I've added validation in from_scipy to reject non-2D input (previously from_scipy(csr_array(1d_array)) would silently create an invalid 1D SparseCSRMatrix). The test now verifies the from_scipy roundtrip with csr_matrix/csc_matrix, which normalize 1D to (1, n).

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.