Add leanvec_primary_only build option to C++ runtime

[ibhati]

Summary

Adds a new leanvec_primary_only parameter (default false) to the LeanVec
build entry points in the C++ runtime API. When enabled, the LeanVec
secondary (full-precision) dataset is not allocated, roughly halving the
LeanVec memory footprint for workloads that don't need re-ranking.

API changes

A trailing bool leanvec_primary_only = false argument is added to:

• VamanaIndexLeanVec::build (both overloads)
• DynamicVamanaIndexLeanVec::build (the DynamicIndexParams overloads)

Existing call sites are source- and ABI-compatible: the parameter is
defaulted, and the legacy back-compat overloads are unchanged.

Plumbing

The flag is stored in {Vamana,DynamicVamana}IndexLeanVecImpl and forwarded
through init_impl → build_impl → StorageFactory<LeanVec>::init →
LeanDataset::reduce, where it skips secondary-dataset allocation and
disables re-ranking at search time.

[ethanglaser]

Is there a corresponding FAISS update?

[ibhati]

Is there a corresponding FAISS update?

Yes, I will publish it as well once we have this PR merged

[ethanglaser]

I'd recommend drafting a FAISS update and pointing the FAISS install in this PR to your FAISS branch, otherwise our CI will be failing

[ibhati]

I'd recommend drafting a FAISS update and pointing the FAISS install in this PR to your FAISS branch, otherwise our CI will be failing

Makes sense, I will add the corresponding FAISS update

[ethanglaser]

These changes to test-cpp-runtime-bindings.sh should not be needed - please restore based on suggestions and let's see if CI runs smoothly

[ethanglaser]

Suggested change

	# Install libsvs-runtime from the public release tarball (includes the
	# leanvec_primary_only API). Once a conda-forge package with this API is
	# published, this can revert to:
	# conda install -y /runtime_conda/libsvs-runtime-*.conda
	SVS_RUNTIME_URL="${SVS_RUNTIME_URL:-https://github.com/intel/ScalableVectorSearch/releases/download/nightly/svs_runtime-0.3.0-linux-x86_64-leanvec-primary-only-glibc228.tar.gz}"
	SVS_RUNTIME_PREFIX="${SVS_RUNTIME_PREFIX:-$HOME/svs_runtime}"
	mkdir -p "$SVS_RUNTIME_PREFIX"
	curl -fsSL "$SVS_RUNTIME_URL" | tar -xz -C "$SVS_RUNTIME_PREFIX"
	export CMAKE_PREFIX_PATH="$SVS_RUNTIME_PREFIX${CMAKE_PREFIX_PATH:+:$CMAKE_PREFIX_PATH}"
	export LD_LIBRARY_PATH="$SVS_RUNTIME_PREFIX/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
	# Install libsvs-runtime from local conda package
	conda install -y /runtime_conda/libsvs-runtime-*.conda

[ethanglaser]

Suggested change

	conda install -y -c conda-forge cmake=3.30.4 make=4.2 swig=4.0 "numpy>=2.0,<3.0" scipy=1.16 pytest=7.4 gflags=2.2 setuptools curl
	conda install -y -c conda-forge cmake=3.30.4 make=4.2 swig=4.0 "numpy>=2.0,<3.0" scipy=1.16 pytest=7.4 gflags=2.2 setuptools

[rfsaliev]

It seems like this API change is not convenient for existing API style:
Suggesting to use new values in StorageKind enum rather than adding extra arguments to index building routines.

[rfsaliev]

Why didn't you just extend the StorageKind enum with values aka: LeanVec4, LeanVec8?

[rfsaliev]

In case if:

• StorageKind enum would have LeanVec4, LeanVec8 values and
• svs::leanvec::LeanDataset supports void for the T2 param, all changes in CPP runtime would include just following new lines:

// LeanVec Primary-Only Storage support
template <
    size_t I1,
    typename Alloc,
    size_t LeanVecDims = svs::Dynamic,
    size_t Extent = svs::Dynamic>
using LeanPrimaryOnly = svs::leanvec::LeanDataset<
    svs::leanvec::UsingLVQ<I1>,
    void,
    LeanVecDims,
    Extent,
    Alloc>;

template <typename Alloc> struct StorageType<StorageKind::LeanVec4, Alloc> {
    using allocator_type = rebind_extracted_allocator_t<std::byte, Alloc>;
    using type = LeanPrimaryOnly<4, allocator_type>;
};
template <typename Alloc> struct StorageType<StorageKind::LeanVec8, Alloc> {
    using allocator_type = rebind_extracted_allocator_t<std::byte, Alloc>;
    using type = LeanPrimaryOnly<8, allocator_type>;
};

And no other changes needed.