[Feature] Support distributed vector search on spark

[Stefanietry]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Currently, vector search operation is executed on a single node within the driver, which may lead to performance bottlenecks when dealing with large amounts of data. This issue aims to implement a distributed execution capability.

Solution

No response

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!