[GR-76653][GR-76899] Add a Windows native os backend.#1032
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a native Windows implementation of GraalPy’s POSIX-support layer so that Windows can use the native backend instead of relying on Unix-oriented or Java-emulated behavior. The change aims to improve CPython compatibility for core OS functionality—files and paths, sockets, mmap, subprocesses, multiprocessing, and Windows-specific modules—while preserving GraalPy’s common POSIX abstraction.
Conceptual changes
Windows-native POSIX shim
python-libposixnow provides Windows implementations for the POSIX-support operations GraalPy uses. It translates between CRT (a.k.a. Win stdlib) file descriptors, Win32 handles, and Winsock sockets; uses wide-character Windows file APIs; and implements or emulates Windows equivalents for operations such asstat,rename/replace, pipes,select,mmap, process handling, and semaphores.Adds Windows modules and platform-specific builtins (e.g.
nt,msvcrt,_winapi,_overlapped)New param
osfor annotations@CoreFunctionsand@BuiltinSeparate filesystem paths from ordinary native C strings since they have different encoding contracts on Windows. Filesystem paths must be passed to Windows as UTF-16 wide strings, while APIs such as networking, environment handling on non-Windows, and command/argument interfaces may require a narrow, NUL-terminated byte string.
Windows error capture and the GraalPy POSIX error bridge
Windows has three relevant thread-local error channels:
-
errnofor CRT/POSIX-style APIs;-
GetLastError()for Win32 APIs;-
WSAGetLastError()for Winsock APIs.Native wrappers capture all three immediately after a failing call and record which channel is authoritative.
When GraalPy turns a native failure into a Python
OSError, the Java POSIX error bridge checks the captured source. CRT failures retainerrno; Win32 and Winsock failures are mapped to their closest POSIX errno values, while the original Windows value is retained asOSError.winerror. This follows CPython’s general winerror-to-errno model and includes compatibility-specific mappings, such as treating non-blocking Winsock connect’sWSAEWOULDBLOCKasEINPROGRESS, andReleaseSemaphore’s too many posts error asEOVERFLOW.Unlike CPython, which performs much of this flow directly within its C runtime and exception helpers, GraalPy must explicitly preserve error state across a C-to-Java boundary before constructing the Python exception. The mapping is intentionally explicit and currently covers the Windows errors used by the new backend; unknown Win32 errors retain their numeric value rather than receiving a broader fallback mapping.
Windows-compatible
mmapWindows descriptor model for sockets: Windows sockets are not CRT file descriptors. The backend introduces a mapping from GraalPy-visible integer FDs to native
SOCKETs, backed by placeholder CRT descriptors. This lets Python code continue to use a unified FD model while native operations dispatch correctly to either CRT file APIs or Winsock APIs. It also centralizes socket close, blocking mode, inheritance, and handle lookup.os.replaceis separated fromos.renamebecause Windows distinguishes behavior more strictly.Binary-mode native I/O
Windows CRT descriptors are opened or normalized in binary mode where Python expects byte-exact I/O. This prevents CRT newline translation from changing data written to pipes, files, or standard streams.
Other fixes
LZMA decompression now copies only the logical input range (
offset + avail_in) to native memory, rather than the entire Java buffer capacity. This prevents the native decoder from seeing stale or unrelated bytes beyond the valid compressed input.lzma.LZMAFileno longer forces unbuffered file access. That avoided Windows-specific short/partial-read behavior that could interfere with stream decompression.Windows
stat/fstatnow populate access, modification, and creation timestamps fromFILETIMEvalues using UTC conversion. Previously those timestamp fields could be missing or incorrect.Windows stat fallback behavior now matches CPython more closely. When opening metadata through the primary path fails because a file is inaccessible or sharing-locked, the backend falls back to attribute lookup; it does not incorrectly mask unrelated errors such as missing files, invalid paths, or unavailable network locations.
fstatatnow preserves the real Windows error from its fallback path. Previously a failing metadata lookup could lose or overwrite the meaningfulGetLastError()value before it reachedOSError.Successful named mmap creation now clears the visible Windows last-error state. This matches the expectation that
_winapi.GetLastError()does not report a staleERROR_ALREADY_EXISTSafter a successful mapping operation.Standard-stream text wrappers now use normal newline translation. This fixes Windows output so \n is emitted according to Windows text-stream conventions rather than being written unchanged.
Launcher robustness was improved on Windows:
- invalid executable-path strings no longer abort program-path discovery;
- native executable discovery falls back to the current process command when the initially reported path is unusable;
- non-interactive console read failures can be treated as EOF;
- the venv launcher removes inherited
GRAAL_PYTHON_ARGSbefore launching the child, preventing arguments from being duplicated or unintentionally propagated.Bytecode reparsing resolves source and bytecode files to normalized absolute paths, fixing failures when files are deleted or accessed through differing relative-path forms—particularly visible on Windows.