[VPEX][6/8] Add local-env uv backend and hidden CLI command

[rugpanov]

Why

• The engine needs a real PackageManager implementation (uv) and a CLI entry point so a user can actually run the feature.
• Wiring it in while keeping it Hidden lets the command be dogfooded and exercised by acceptance tests without becoming user-visible until the stack is complete.
• This is the first layer reachable from main, so it also makes the whole libs/localenv package live for the deadcode checker.

What

• libs/localenv/uv.go — the uv implementation of PackageManager: discover/install uv, install the Python minor, uv sync, seed pip into the venv, validate; plus the pip.conf → UV_INDEX_URL bridge for Databricks-managed machines.
• cmd/localenv/ — the command tree matching local-env python sync: a local-env group (Hidden: true), a python subgroup, and the sync verb. Parent nodes use root.ReportUnknownSubcommand; sync uses cobra.NoArgs, resolves flags/bundle target, builds the Pipeline with the uv manager, and renders text or --json. All Cobra Use values + the --json command field come from the libs/localenv constants.
• cmd/cmd.go — registers the group.

Testing strategy

• Unit tests for uv helper logic (discovery, pip.conf index-url, arg builders, stderr surfacing) (uv_test.go).
• Runtime smoke test of the hidden 3-level command: absent from top-level help; help works at each level; unknown subcommand exits non-zero; bare group shows help; flags + mutual-exclusion + NoArgs behave.
• Gates: go build ./..., go test, golangci-lint, deadcode (whole tree, no pragmas), gofmt — all green.
• cmd/localenv/ unit tests are intentionally deferred to the acceptance PR ([VPEX][7/8] Add local-env acceptance tests #5833), per the repo convention that user-visible CLI output is covered by acceptance tests.

---

About this stack

This is one of a series of small, stacked PRs that together add the databricks local-env python sync command — it provisions a local Python environment (Python version, databricks-connect pin, and dependency constraints) matched to a selected Databricks compute target. The work was split from one large branch into single-concern layers so each is independently reviewable; the command is kept hidden until the final PR so nothing is user-visible mid-stack.

Review bottom-up. Each PR targets the previous one as its base branch (this PR targets main), so its diff shows only that layer. Layers 1–5 have merged; the original layer-5 PR (#5828) was split into 5a/5b/5c during review.

#	PR	What	Status
1	#5823	foundation: result types + env-key mapping	merged
2	#5824	compute-target resolution	merged
3	#5826	constraint fetch + offline cache	merged
4	#5827	formatting-preserving pyproject.toml merge	merged
5a	#5850	package-manager interface + detection	merged
5b	#5851	six-phase pipeline orchestrator	merged
5c	#5854	--check cache purity, greenfield name, dbc insertion	merged
6	#5832 ← you are here	uv backend + CLI command (registered hidden)
7	#5833	acceptance tests
8	#5835	unveil (unhide + help + changelog)

This pull request and its description were written by Isaac.

[eng-dev-ecosystem-bot]

Integration test report

Commit: 11a1285

Run: 29345764311

	Env	🔄​flaky	💚​RECOVERED	🙈​SKIP	✅​pass	🙈​skip	Time
💚​	aws linux		4	4	230	1087	5:24
💚​	aws windows		4	4	232	1085	8:06
💚​	aws-ucws linux		4	4	316	1004	5:34
💚​	aws-ucws windows		4	4	318	1002	7:34
💚​	azure linux		4	4	230	1086	5:19
🔄​	azure windows	2	2	4	232	1084	9:07
💚​	azure-ucws linux		4	4	318	1001	5:54
💚​	azure-ucws windows		4	4	320	999	7:32
💚​	gcp linux		4	4	229	1088	5:06
💚​	gcp windows		4	4	231	1086	7:17

8 interesting tests: 4 SKIP, 2 RECOVERED, 2 flaky

	Test Name	aws linux	aws windows	aws-ucws linux	aws-ucws windows	azure linux	azure windows	azure-ucws linux	azure-ucws windows	gcp linux	gcp windows
💚​	TestAccept	💚​R	💚​R	💚​R	💚​R	💚​R	💚​R	💚​R	💚​R	💚​R	💚​R
🙈​	TestAccept/bundle/invariant/no_drift	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S
🙈​	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S
🙈​	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S
🙈​	TestAccept/ssh/connection	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S
💚​	TestFetchRepositoryInfoAPI_FromRepo	💚​R	💚​R	💚​R	💚​R	💚​R	💚​R	💚​R	💚​R	💚​R	💚​R
🔄​	TestFetchRepositoryInfoAPI_FromRepo/root	💚​R	💚​R	💚​R	💚​R	💚​R	🔄​f	💚​R	💚​R	💚​R	💚​R
🔄​	TestFetchRepositoryInfoAPI_FromRepo/subdir	💚​R	💚​R	💚​R	💚​R	💚​R	🔄​f	💚​R	💚​R	💚​R	💚​R

Top 10 slowest tests (at least 2 minutes):

duration	env	testname
6:39	aws windows	TestAccept
6:29	aws-ucws windows	TestAccept
6:24	azure-ucws windows	TestAccept
6:22	gcp windows	TestAccept
6:16	azure windows	TestAccept
2:58	gcp linux	TestAccept
2:57	azure linux	TestAccept
2:50	aws linux	TestAccept
2:48	aws-ucws linux	TestAccept
2:48	azure-ucws linux	TestAccept

[anton-107]

Approving — this is a clean, well-structured layer: it builds, libs/localenv unit tests pass, the error taxonomy is tidy, and the comments are unusually good at explaining why. Nothing here blocks merge.

One thing I'd genuinely like addressed before this command stops being hidden, plus a few smaller follow-ups (left as inline comments).

⚠️ Please address before unhide: silent curl | sh auto-install of uv

EnsureAvailable → installUv runs sh -c "curl -LsSf https://astral.sh/uv/install.sh | sh" (and powershell … irm … | iex on Windows) the first time uv isn't found — with no prompt, no opt-in flag, and not even a log.Debugf line before it fires. A user's first local-env python sync silently downloads and executes a remote installer that mutates ~/.local/bin. This is the one I'd push on:

• At minimum, log the install (and the URL) before running it.
• Better: prompt when interactive (the repo already has cmdio.IsPromptSupported / the prompt pattern in cmd/labs/project/login.go), and require an explicit opt-in / documented env var in non-interactive contexts. Note libs/versioncheck only ever suggests the equivalent curl string rather than executing it — worth staying consistent with that posture.

Other findings (inline)

• Bundle is loaded twice, and even when an explicit --cluster/--serverless/--job flag makes the bundle result irrelevant — can also double-print bundle load-time diagnostics.
• Text-mode failures print the error message twice (phase line + root's Error: line); the JSON path already avoids this via ErrAlreadyPrinted.
• Serverless --job always resolves to serverless-v4 even though the SDK exposes the environment version on the job.
• A few cleanups: triplicated index-URL branch, single-caller newUvManager wrapper, positional Validate stdout parsing.

Verified as NON-issues (so nobody re-litigates)

• JobClusters[0].NewCluster.SparkVersion — no nil panic; SDK JobCluster.NewCluster is a value type.
• errors.AsType with the stdlib errors import is correct on Go 1.26.
• resolveIndexURL returning "" when UV_INDEX_URL is already set is correct — process.Background merges env.All(ctx) first, so the existing value still propagates.
• pipIndexURLRe correctly ignores extra-index-url, comments, and index_url; redactURLCredentials strips userinfo correctly.
• The PR-description "runtime smoke test" wording isn't misleading given the adjacent line that explicitly defers cmd tests to #5833.

Nice work overall — the merge/backup safety comments in the earlier layers carry through well here.

[anton-107]

⚠️ Please address before this is unhidden. This silently runs curl -LsSf https://astral.sh/uv/install.sh | sh (and irm … | iex on Windows) the first time uv isn't found — no prompt, no opt-in flag, and not even a log.Debugf before it executes. The user's first local-env python sync downloads and runs a remote installer that mutates ~/.local/bin with no signal.

Suggestions, in order of preference:

1. Prompt when interactive (cmdio.IsPromptSupported + the prompt pattern in cmd/labs/project/login.go), and require an explicit opt-in / documented env var in non-interactive runs.
2. At the very least, log.Infof/log.Debugf the install and the URL before running it, so --debug shows what happened.

For comparison, libs/versioncheck only ever surfaces the equivalent curl command as a string to the user rather than executing it — worth matching that posture here.

[rugpanov]

Added a log.Debugf of the exact installer command + URL before it runs (visible under --debug). The interactive prompt + non-interactive opt-in env var will be addressed in a follow-up, before the command is unhidden.

[anton-107]

PreRunE = root.MustWorkspaceClient already calls TryConfigureBundle (running DefaultMutators + target selection + building the workspace client). configureBundle never stores the *bundle.Bundle on the context — bundle.Context() has no production callers — so bundle.GetOrNil returns nil here and this second TryConfigureBundle does a full second load.

Two consequences:

• Because logdiag renders immediately (Collect=false), any bundle load-time warning/recommendation is printed to the user twice.
• bundleTarget runs unconditionally, but ResolveTarget returns from the --cluster/--serverless/--job branch before ever consulting the bundle — so for the common explicit-flag case the whole second load is wasted work.

Suggest skipping bundleTarget when an explicit target flag is set, and/or reusing the bundle already resolved in PreRunE. (Impact is bounded — bundle load is fast/no-network and the command is hidden — so not a blocker.)

[rugpanov]

Good catch — deferring to a follow-up. Will skip bundleTarget when an explicit target flag is set / reuse the bundle from PreRunE.

[anton-107]

Text-mode failures print the message twice. pipeline.fail() sets the failing phase's Detail = pe.Error(), which the phase loop above prints inline (resolve error target resolution failed: <cause>); then returning the raw *PipelineError here means errors.Is(err, ErrAlreadyPrinted) is false in root.Execute (its Unwrap() yields the wrapped cause, not the sentinel), so root prints Error: <same message> again.

The JSON branch above already handles this by returning root.ErrAlreadyPrinted; the text path should do the same after emitting the phase lines + hint, so the failure shows up once.

[rugpanov]

Deferring to a follow-up: will read Environments[i].Spec.EnvironmentVersion for serverless jobs instead of defaulting to serverless-v4.

[anton-107]

For a serverless job this returns version="", and ResolveTarget treats an empty serverless version as "default to v4" — so every serverless --job resolves to serverless-v4, silently, even for a job pinned to another environment version. Unlike the bundle path (where the version genuinely isn't recorded), the data is present here: job.Settings.Environments[i].Spec.EnvironmentVersion.

I realize the v4 default is a documented stand-in from the original script (spec §4.3) and matches the bundle path, so this may be deliberate for P0 — but since the job path could do better, either read EnvironmentVersion or leave a short TODO noting the job path has more info than the bundle path.

[anton-107]

Minor/robustness: this parse is positional — lines[0] is assumed to be the Python version and the last line the databricks-connect version. CRLF and the dbc-absent case are handled fine, but any stray line uv writes to stdout before the interpreter output would make lines[0] wrong (e.g. "warning: …\n3.12\n17.2.0" → python "warning: …"). uv diagnostics normally go to stderr, so this is unlikely in practice — but a sentinel-delimited print (print("PYVER:" + …) and grep for the prefix) would make it robust rather than order-dependent.

[rugpanov]

Agreed, will make the Validate parse sentinel-delimited rather than positional in a follow-up.

[anton-107]

Nit: newUvManager is a single-caller wrapper whose body is just return &uvManager{}, and only NewUvManager calls it (tests build &uvManager{} literals directly). Could fold it into NewUvManager and move the lazy-binary-path doc comment there.

Separately (not worth its own thread): EnsurePython/Provision/PostProvision each repeat the identical resolveIndexURL + if indexURL != "" { WithEnv } else {} shape — collapsible into one runUv(ctx, args, dir) helper (WithDir("") and WithEnvs(emptyMap) are both no-ops, so behavior is preserved; the conditional itself must stay so an already-set UV_INDEX_URL isn't clobbered).

[rugpanov]

Will fold newUvManager into NewUvManager and collapse the repeated resolveIndexURL branch into a runUv helper in a follow-up.