Object store cache key drops the ABFS container, so two containers in one storage account share a store instance

[andygrove]

Describe the bug

prepare_object_store_with_configs in native/core/src/parquet/parquet_support.rs caches object store instances under (url_key, config_hash), where url_key is built from:

let url_key = format!(
    "{}://{}",
    scheme,
    &url[url::Position::BeforeHost..url::Position::AfterPort],
);

Position::BeforeHost starts after the URL userinfo. For ABFS, the container is the userinfo, not the host:

abfss://container@account.dfs.core.windows.net/path/to/file.parquet

So abfss://c1@acct.dfs.core.windows.net/... and abfss://c2@acct.dfs.core.windows.net/... both produce the key abfss://acct.dfs.core.windows.net. config_hash is computed over the per-session Hadoop config map, which does not vary by container, so it does not disambiguate them either.

Meanwhile MicrosoftAzureBuilder::parse_url bakes the container into the store instance from the URL userinfo, and the Path handed to that store is container-relative (the bucket/container is stripped by Path::from_url_path(url.path()) in PhysicalPlanner::get_partitioned_files).

The consequence: within one executor process, reading abfss://c2@acct/data/f.parquet after abfss://c1@acct/data/f.parquet should hit the cached c1 store and read c1's data.

Steps to reproduce

Not yet reproduced. This was found by code reading during review of an unrelated change. Confirming it needs an Azure storage account with two containers holding same-named paths, read from a single executor in one application.

Expected behavior

Each ABFS container resolves to its own object store instance, and a read from one container never returns another container's data.

Additional context

• S3, GCS, and HDFS are unaffected: their bucket/host lives in the URL host component, which url_key does keep.
• The same key derivation is now also used to key a shared Parquet metadata cache, where the impact is narrower (entries are additionally validated against file size and modification time). That is documented as a known limitation there; this issue is about the store cache itself, where no such validation exists.
• The object store cache is process-wide and unbounded by design, so a wrong entry persists for the executor's lifetime.

[peterxcli]

let me take a stab at this.