Add generic environment-creation utilities (PEP 723 PR 5a/16) by StellaHuang95 · Pull Request #1651 · microsoft/vscode-python-environments

StellaHuang95 · 2026-07-23T22:19:38Z

Part of #1602 (PEP 723 inline script env support). Design doc: #1601.

Split for review (3 PRs). Reviewers flagged the original PR 5 as too large, so it is split into three stacked PRs grouped by dependency layer:

5a — generic env-creation utilities — this PR (Add generic environment-creation utilities (PEP 723 PR 5a/16) #1651). Based on main; independent; merges first.

5b — inline-script cache + interpreter utilities — Add inline-script cache and interpreter utilities (PEP 723 PR 5b/16) #1655. Stacked on 5a.

5c — create() happy path (manager + wiring) — Add PEP 723 inline script environment creation (PEP 723 PR 5c/16) #1656. Stacked on 5b.

Applied together the three PRs are byte-for-byte identical to the original single change. Merge order: 5a → 5b → 5c.

Roadmap context

This is the first slice of PR 5 of 16 in the PEP 723 inline-script roadmap. The full plan lives in #1602.

Phase	PR	Status
Phase 1: Foundation	PR 1: cache key hash utility	merged (#1634)
	PR 2: cache layout + `meta.json` sidecar	merged (#1635)
	PR 3: `requires-python` to interpreter selection	merged (#1636)
Phase 2: Manager	PR 4: `InlineScriptEnvManager` skeleton	merged (#1610)
	PR 5a: generic env-creation utilities	this PR (#1651)
	PR 5b: inline-script cache + interpreter utilities	#1655
	PR 5c: `create()` happy path (manager + wiring)	#1656
	PR 6: `create()` uv-install fallback	not started (needs 3, 5)
	PR 7: persistence with `get`, `set`, and Memento	not started (needs 4)
	PR 8: activation-time discovery	not started (needs 2, 4, 7)
Phase 3: Routing	PR 9: route PEP 723 scripts to the inline manager	not started (needs 4, 7)
	PR 10: per-script project registration	not started (needs 9)
Phase 4+: UX / lifecycle	PRs 11-16	not started

Why this PR

PR 5c implements InlineScriptEnvManager.create(). Before touching the manager, this PR lands the generic, reusable primitives it relies on — a cross-process file lock, a venv Python-path helper, a cancellation-hardened process runner, and two small createWithProgress options. None of this code is inline-script-specific, so it is reviewed on its own.

What this PR adds

Cross-process file lock (src/common/lockfile.apis.ts, new): acquireFileLock uses an atomic mkdir of a <path>.lock directory plus a per-owner marker file, returning AcquiredFileLock { release, retain }. retain() writes a retained marker so a later acquirer fails fast with ELOCKRETAINED instead of waiting out the 5-minute timeout — used when a build is cancelled mid-flight. Distinct error codes (ELOCKED, ELOCKRETAINED, ELOCKORPHANED, ECOMPROMISED, ERETAINFAILED) separate contention from corruption.

Shared getVenvPythonPath (src/common/utils/virtualEnvironment.ts, new): returns Scripts\python.exe on Windows, else bin/python. Replaces an inline copy in venvUtils and is reused by 5b/5c.

Hardened process helper (src/managers/builtin/helpers.ts): runUV and runPython now share one runProcess implementation whose cancellation guards kill() in try/catch and still emits a clean CancellationError if the process errors after a cancel. Per-caller options preserve existing behavior (collectStderr, logPrefix).

venvUtils.ts: createWithProgress gains CreateWithProgressOptions { trackUvEnvironment }, and CreateEnvironmentResult gains pkgInstallationCancelled so a caller can tell cancellation apart from a real install failure. Existing callers are unaffected (both are optional / additive).

Tests

lockfile.apis.unit.test.ts — 9 tests: contention, retain/fail-fast, orphaned and compromised locks, and timeout.
virtualEnvironment.unit.test.ts — 2 tests for getVenvPythonPath on Windows and POSIX.
helpers.cancellation.unit.test.ts — 4 tests for runProcess cancellation safety.
venvUtils.createWithProgress.unit.test.ts — 3 tests for trackUvEnvironment and pkgInstallationCancelled.

On this branch alone npm run compile-tests is clean and npm run unittest reports 1447 passing, 0 failing, 4 pending.

User impact

None. These are internal primitives with no new user-visible behavior. The refactors to helpers.ts and venvUtils.ts are behavior-preserving for existing callers.

edvilme · 2026-07-23T22:25:46Z

I feel we must have something like this somewhere in the codebase, though I could be wrong

There's one line code in src\managers\conda\condaUtils.ts that does something similar but it's not a helper, and the windows layouts are different, so I have this separate helper.

edvilme · 2026-07-23T23:52:08Z

+    return runProcess('uv', args, {
+        cwd,
+        displayName: 'uv',
+        log,
+        token,
+        timeout,
+        collectStderr: false,
+        logPrefix: '',


This is also used by other places of the codebase. Is there a reason for this change and the removal of the listeners and exceptions?

It's just refactoring to get rid of the duplicate code. The error handling stuff is moved to the share runProcess. Maybe I shouldn't include refactor of a hot path in this pr, let me think about it.

Cross-process file lock, venv Python-path helper, cancellation-safe process runner, and createWithProgress tracking options that inline-script environment creation builds on. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 39dcc6a3-0fbd-4f36-9d0f-68677de49c27

edvilme · 2026-07-24T00:08:49Z

+function isAlreadyExistsError(error: unknown): boolean {
+    return hasErrorCode(error, 'EEXIST');
+}
+
+function isFileNotFoundError(error: unknown): boolean {
+    return hasErrorCode(error, 'ENOENT');
+}
+


If these are not used anywhere else, I would just add the hasErrorCode(error, 'XYZ') inline

yeah sure it's only used within the file, I can make it inline.

edvilme · 2026-07-24T00:09:18Z

+export interface AcquireFileLockOptions {
+    readonly timeoutMs: number;
+    readonly retryIntervalMs: number;
+}
+
+export interface AcquiredFileLock {
+    readonly release: () => Promise<void>;
+    /** Keep the lock and make later acquisition attempts fail immediately. */
+    readonly retain: () => Promise<void>;
+}
+
+type LockState = 'held' | 'released' | 'retained';


Maybe move interfaces and types to the top of the file

sure, will do that.

edvilme · 2026-07-24T00:10:09Z

+    const retainedMarker = path.join(lockPath, 'retained');
+    const deadline = Date.now() + options.timeoutMs;
+
+    while (true) {


Just a question here: why do we need the while loop? Not saying it is wrong, just curious in case I am not understanding correctly

The loop prevents two VS Code processes from creating the same environment simultaneously.

For example:

Process A creates the lock and starts building the environment.

Process B tries to create the same lock but cannot because A owns it.

B waits briefly, then the while (true) loop tries again.

When A finishes, it removes the lock.

B’s next attempt succeeds. B can then reuse the environment A created.

The while(true) does not run forever. It either gets the lock, detects a retained lock, or reaches the timeout. Without the loop, Process B would fail immediately merely because Process A was still working.

edvilme · 2026-07-24T00:10:58Z

    venvRoot: Uri,
    envPath: string,
    packages?: PipPackages,
+    options?: CreateWithProgressOptions,


Why do we need this to be another object instead of just passing a trackUvEnvironment property?

It's mainly because so that the caller is more explicit about what's passed to the function and also it's open to extend to other options in the future. I could also make it a boolean value and add comment clarifying what that is in the caller.

StellaHuang95 marked this pull request as ready for review July 23, 2026 22:21

StellaHuang95 added the feature-request Request for new features or functionality label Jul 23, 2026

StellaHuang95 requested review from edvilme and eleanorjboyd July 23, 2026 22:22

StellaHuang95 force-pushed the pep723-pr5-create-happy-path branch from b1e9b95 to f005e16 Compare July 23, 2026 23:06

StellaHuang95 mentioned this pull request Jul 23, 2026

Add inline-script environment creation tests (PEP 723 PR 5/16 - tests) StellaHuang95/vscode-python-environments#2

Closed

StellaHuang95 changed the title ~~Add inline-script environment creation happy path (PEP 723 PR 5/16)~~ Add inline-script environment creation happy path - implementation (PEP 723 PR 5/16) Jul 23, 2026

StellaHuang95 marked this pull request as draft July 23, 2026 23:09

StellaHuang95 mentioned this pull request Jul 23, 2026

Add inline-script environment creation tests (PEP 723 PR 5/16 - tests) #1654

Closed

StellaHuang95 marked this pull request as ready for review July 23, 2026 23:14

StellaHuang95 force-pushed the pep723-pr5-create-happy-path branch from f005e16 to d36561b Compare July 23, 2026 23:30

StellaHuang95 changed the title ~~Add inline-script environment creation happy path - implementation (PEP 723 PR 5/16)~~ Add generic environment-creation utilities (PEP 723 PR 5a/16) Jul 23, 2026

This was referenced Jul 23, 2026

Add inline-script cache and interpreter utilities (PEP 723 PR 5b/16) #1655

Draft

Add PEP 723 inline script environment creation (PEP 723 PR 5c/16) #1656

Draft

StellaHuang95 marked this pull request as draft July 23, 2026 23:34

StellaHuang95 marked this pull request as ready for review July 23, 2026 23:35

edvilme reviewed Jul 23, 2026

View reviewed changes

edvilme reviewed Jul 24, 2026

View reviewed changes

StellaHuang95 force-pushed the pep723-pr5-create-happy-path branch from d36561b to 104166d Compare July 24, 2026 00:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add generic environment-creation utilities (PEP 723 PR 5a/16)#1651

Add generic environment-creation utilities (PEP 723 PR 5a/16)#1651
StellaHuang95 wants to merge 1 commit into
microsoft:mainfrom
StellaHuang95:pep723-pr5-create-happy-path

StellaHuang95 commented Jul 23, 2026 •

edited

Loading

Uh oh!

edvilme Jul 23, 2026

Uh oh!

StellaHuang95 Jul 24, 2026

Uh oh!

edvilme Jul 23, 2026

Uh oh!

StellaHuang95 Jul 24, 2026

Uh oh!

edvilme Jul 24, 2026

Uh oh!

StellaHuang95 Jul 24, 2026

Uh oh!

edvilme Jul 24, 2026

Uh oh!

StellaHuang95 Jul 24, 2026

Uh oh!

edvilme Jul 24, 2026

Uh oh!

StellaHuang95 Jul 24, 2026

Uh oh!

edvilme Jul 24, 2026

Uh oh!

StellaHuang95 Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

StellaHuang95 commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Roadmap context

Why this PR

What this PR adds

Tests

User impact

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

StellaHuang95 commented Jul 23, 2026 •

edited

Loading