Skip to content

[GR-76653][GR-76899] Add a Windows native os backend.#1032

Merged
graalvmbot merged 38 commits into
masterfrom
tim/graalpython-asy-gr-76653
Jul 21, 2026
Merged

[GR-76653][GR-76899] Add a Windows native os backend.#1032
graalvmbot merged 38 commits into
masterfrom
tim/graalpython-asy-gr-76653

Conversation

@graalvmbot

Copy link
Copy Markdown
Collaborator

Add a native Windows implementation of GraalPy’s POSIX-support layer so that Windows can use the native backend instead of relying on Unix-oriented or Java-emulated behavior. The change aims to improve CPython compatibility for core OS functionality—files and paths, sockets, mmap, subprocesses, multiprocessing, and Windows-specific modules—while preserving GraalPy’s common POSIX abstraction.

Conceptual changes

  • Windows-native POSIX shim python-libposix now provides Windows implementations for the POSIX-support operations GraalPy uses. It translates between CRT (a.k.a. Win stdlib) file descriptors, Win32 handles, and Winsock sockets; uses wide-character Windows file APIs; and implements or emulates Windows equivalents for operations such as stat, rename/replace, pipes, select, mmap, process handling, and semaphores.

  • Adds Windows modules and platform-specific builtins (e.g. nt, msvcrt, _winapi, _overlapped)

  • New param os for annotations @CoreFunctions and @Builtin

  • Separate filesystem paths from ordinary native C strings since they have different encoding contracts on Windows. Filesystem paths must be passed to Windows as UTF-16 wide strings, while APIs such as networking, environment handling on non-Windows, and command/argument interfaces may require a narrow, NUL-terminated byte string.

  • Windows error capture and the GraalPy POSIX error bridge
    Windows has three relevant thread-local error channels:
    - errno for CRT/POSIX-style APIs;
    - GetLastError() for Win32 APIs;
    - WSAGetLastError() for Winsock APIs.

    Native wrappers capture all three immediately after a failing call and record which channel is authoritative.
    When GraalPy turns a native failure into a Python OSError, the Java POSIX error bridge checks the captured source. CRT failures retain errno; Win32 and Winsock failures are mapped to their closest POSIX errno values, while the original Windows value is retained as OSError.winerror. This follows CPython’s general winerror-to-errno model and includes compatibility-specific mappings, such as treating non-blocking Winsock connect’s WSAEWOULDBLOCK as EINPROGRESS, and ReleaseSemaphore’s too many posts error as EOVERFLOW.

    Unlike CPython, which performs much of this flow directly within its C runtime and exception helpers, GraalPy must explicitly preserve error state across a C-to-Java boundary before constructing the Python exception. The mapping is intentionally explicit and currently covers the Windows errors used by the new backend; unknown Win32 errors retain their numeric value rather than receiving a broader fallback mapping.

  • Windows-compatible mmap

  • Windows descriptor model for sockets: Windows sockets are not CRT file descriptors. The backend introduces a mapping from GraalPy-visible integer FDs to native SOCKETs, backed by placeholder CRT descriptors. This lets Python code continue to use a unified FD model while native operations dispatch correctly to either CRT file APIs or Winsock APIs. It also centralizes socket close, blocking mode, inheritance, and handle lookup.

  • os.replace is separated from os.rename because Windows distinguishes behavior more strictly.

  • Binary-mode native I/O
    Windows CRT descriptors are opened or normalized in binary mode where Python expects byte-exact I/O. This prevents CRT newline translation from changing data written to pipes, files, or standard streams.

Other fixes

  • LZMA decompression now copies only the logical input range (offset + avail_in) to native memory, rather than the entire Java buffer capacity. This prevents the native decoder from seeing stale or unrelated bytes beyond the valid compressed input.

  • lzma.LZMAFile no longer forces unbuffered file access. That avoided Windows-specific short/partial-read behavior that could interfere with stream decompression.

  • Windows stat/fstat now populate access, modification, and creation timestamps from FILETIME values using UTC conversion. Previously those timestamp fields could be missing or incorrect.

  • Windows stat fallback behavior now matches CPython more closely. When opening metadata through the primary path fails because a file is inaccessible or sharing-locked, the backend falls back to attribute lookup; it does not incorrectly mask unrelated errors such as missing files, invalid paths, or unavailable network locations.

  • fstatat now preserves the real Windows error from its fallback path. Previously a failing metadata lookup could lose or overwrite the meaningful GetLastError() value before it reached OSError.

  • Successful named mmap creation now clears the visible Windows last-error state. This matches the expectation that _winapi.GetLastError() does not report a stale ERROR_ALREADY_EXISTS after a successful mapping operation.

  • Standard-stream text wrappers now use normal newline translation. This fixes Windows output so \n is emitted according to Windows text-stream conventions rather than being written unchanged.

  • Launcher robustness was improved on Windows:
    - invalid executable-path strings no longer abort program-path discovery;
    - native executable discovery falls back to the current process command when the initially reported path is unusable;
    - non-interactive console read failures can be treated as EOF;
    - the venv launcher removes inherited GRAAL_PYTHON_ARGS before launching the child, preventing arguments from being duplicated or unintentionally propagated.

  • Bytecode reparsing resolves source and bytecode files to normalized absolute paths, fixing failures when files are deleted or accessed through differing relative-path forms—particularly visible on Windows.

timfel and others added 30 commits July 21, 2026 14:36
@oracle-contributor-agreement oracle-contributor-agreement Bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Jul 21, 2026
@graalvmbot graalvmbot closed this Jul 21, 2026
@graalvmbot
graalvmbot merged commit 17d7b9b into master Jul 21, 2026
38 of 42 checks passed
@graalvmbot
graalvmbot deleted the tim/graalpython-asy-gr-76653 branch July 21, 2026 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Verified All contributors have signed the Oracle Contributor Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants