[GR-76653][GR-76899] Add a Windows native os backend.

CHANGELOG.md

@@ -3,6 +3,9 @@
 This changelog summarizes major changes between GraalVM versions of the Python
 language runtime. The main focus is on user-observable behavior of the engine.
 
+## Version 25.2.4
+* Windows standalone builds now use a native OS backend by default, improving compatibility for files, sockets, subprocesses, `mmap`, and Windows-specific modules compared to the Java-based backend.
+
 ## Version 25.1.3
 * The standalone artifacts now include the Python version name before the Graal version. The new artifacts now start with `graalpy<PYTHON_VERSION>-<GRAAL_VERSION>-<OPERATING_SYSTEM>-<ARCHITECTURE>`.
 * Standalone JVM artifacts are no longer released as separate distributions. For standalone deployments, use the GraalPy native artifacts. If you require Java interoperability, use our jbang launcher (`jbang graalpy@oracle/graalpython -c "print('hello from GraalPy')"`) or a custom embedding.

docs/contributor/IMPLEMENTATION_DETAILS.md

@@ -91,6 +91,44 @@ The patches are regular POSIX `patch` command compatible diffs located in
 `lib-graalpython/patches`. The directory has a `README.md` file describing
 how the patches are applied.
 
+## Operating System Backends
+
+GraalPy exposes operating system functionality through a common POSIX-support abstraction with native and Java implementations.
+The native implementation is provided by the `python-libposix` shim.
+On Windows, the shim translates between CRT file descriptors, Win32 handles, and Winsock sockets, and implements or emulates Windows equivalents for operations such as `stat`, `rename`, `replace`, pipes, `select`, `mmap`, process handling, and semaphores.
+Windows modules and platform-specific builtins include `nt`, `msvcrt`, `_winapi`, and `_overlapped`.
+
+### Native Strings and Filesystem Paths
+
+Filesystem paths are kept separate from ordinary native C strings because they have different encoding contracts on Windows.
+Paths are passed to Windows as UTF-16 wide strings, while interfaces such as networking, environment handling on non-Windows platforms, and command or argument APIs may require narrow, NUL-terminated byte strings.
+
+### Windows Error Handling
+
+Windows has three relevant thread-local error channels:
+
+- `errno` for CRT and POSIX-style APIs
+- `GetLastError()` for Win32 APIs
+- `WSAGetLastError()` for Winsock APIs
+
+Native wrappers capture all three immediately after a failing call and record which channel is authoritative.
+When GraalPy turns a native failure into a Python `OSError`, the Java POSIX error bridge checks the captured source.
+CRT failures retain `errno`; Win32 and Winsock failures are mapped to their closest POSIX error numbers, while the original Windows value is retained as `OSError.winerror`.
+This follows CPython's general `winerror`-to-`errno` model and includes compatibility-specific mappings, such as treating `WSAEWOULDBLOCK` from a non-blocking Winsock connect as `EINPROGRESS`, and the "too many posts" error from `ReleaseSemaphore` as `EOVERFLOW`.
+
+Unlike CPython, which performs much of this flow directly within its C runtime and exception helpers, GraalPy must explicitly preserve error state across a C-to-Java boundary before constructing the Python exception.
+The mapping is intentionally explicit and currently covers the Windows errors used by the backend; unknown Win32 errors retain their numeric value rather than receiving a broader fallback mapping.
+
+### Windows Descriptor Model
+
+Windows sockets are not CRT file descriptors.
+The backend maps GraalPy-visible integer file descriptors to native `SOCKET` values backed by placeholder CRT descriptors.
+Python code therefore retains a unified file-descriptor model while native operations dispatch to either CRT file APIs or Winsock APIs.
+The mapping also centralizes socket closing, blocking mode, inheritance, and handle lookup.
+
+The backend provides a Windows-compatible `mmap` implementation and keeps `os.replace` separate from `os.rename`, since Windows distinguishes their behavior more strictly.
+CRT descriptors are opened or normalized in binary mode where Python expects byte-exact I/O, preventing CRT newline translation from changing data written to pipes, files, or standard streams.
+
 ## The GIL
 
 We always run with a GIL, because C extensions in CPython expect to do so and

docs/user/Embedding-Permissions.md

@@ -22,7 +22,8 @@ The `PosixModuleBackend` option determines which backend is used: `native` or `j
 
 ### Native Backend
 
-The native backend directly calls the POSIX API in mostly the same way as CPython (the reference Python implementation).
+The native backend directly calls the operating system API in mostly the same way as CPython (the reference Python implementation).
+On POSIX platforms this means POSIX APIs; on Windows this means a Windows-native backend that maps Python's OS interface to the CRT, Win32, and Winsock APIs.
 
 This approach is the most compatible with CPython and provides bare access to the underlying OS interface without an intermediate emulation layer.
 

docs/user/Platform-Support.md

@@ -4,4 +4,13 @@ GraalPy is mostly written in Java and Python, but the Python package ecosystem i
 The main operating system is Oracle Linux, the CPU architectures are AMD64 and ARM, and the primary JDK is Oracle GraalVM.
 **Linux is recommended for getting started with GraalPy.** Windows and macOS with GraalVM JDK are less well tested, and outside of those combinations only basic test coverage is provided.
 As macOS and other platforms are not prioritized, some GraalPy features may not work on these platforms.
-See [Test Tiers](../user/Test-Tiers.md) for a detailed breakdown.
+See [Test Tiers](../user/Test-Tiers.md) for a detailed breakdown.
+
+## Windows
+
+GraalPy standalone builds on Windows use a native OS backend by default.
+The backend maps Python's OS interface to the Windows CRT, Win32, and Winsock APIs instead of relying on the Java-based backend.
+It provides native or emulated Windows implementations for files and paths, sockets, pipes, `select`, `mmap`, subprocesses, multiprocessing semaphores, and Windows-specific modules such as `nt`, `msvcrt`, `_winapi`, and `_overlapped`.
+
+Filesystem paths are passed to wide-character Windows APIs as UTF-16, while other native interfaces continue to use narrow strings where required.
+Errors from the CRT, Win32, and Winsock APIs are translated to Python `OSError` instances using CPython-compatible error mappings, with the original Windows error retained as `OSError.winerror`.

graalpython/com.oracle.graal.python.shell/src/com/oracle/graal/python/shell/DefaultConsoleHandler.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2017, 2025, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2017, 2026, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -48,16 +48,21 @@
 class DefaultConsoleHandler extends ConsoleHandler {
 
     private final BufferedReader in;
+    private final boolean eofOnIOException;
 
-    DefaultConsoleHandler(InputStream in) {
+    DefaultConsoleHandler(InputStream in, boolean eofOnIOException) {
         this.in = new BufferedReader(new InputStreamReader(in));
+        this.eofOnIOException = eofOnIOException;
     }
 
     @Override
     public String readLine(String prompt) {
         try {
             return in.readLine();
         } catch (IOException e) {
+            if (eofOnIOException) {
+                return null;
+            }
             throw new RuntimeException(e);
         }
     }

graalpython/com.oracle.graal.python.shell/src/com/oracle/graal/python/shell/GraalPythonMain.java

@@ -620,7 +620,12 @@ private String getLauncherExecName() {
      * @return The absolute path to the program or {@code null}.
      */
     private String calculateProgramFullPath(String program, Predicate<Path> isExecutable, String envPath) {
-        Path programPath = Paths.get(program);
+        Path programPath;
+        try {
+            programPath = Paths.get(program);
+        } catch (InvalidPathException e) {
+            return null;
+        }
 
         // If this is an absolute path, we are already fine.
         if (programPath.isAbsolute()) {
@@ -877,10 +882,6 @@ protected void launch(Builder contextBuilder) {
 
         setOptionIfNotSetViaCommandLine(contextBuilder, "AllowSignalHandlers", "true");
 
-        if (IS_WINDOWS) {
-            contextBuilder.option("python.PosixModuleBackend", "java");
-        }
-
         setOptionIfNotSetViaCommandLine(contextBuilder, "WarnExperimentalFeatures", "false");
         setOptionIfNotSetViaCommandLine(contextBuilder, "UnicodeCharacterDatabaseNativeFallback", "true");
 
@@ -1023,6 +1024,9 @@ private static void applyRaisedJitThresholds(Map<String, String> polyglotOptions
     }
 
     private static String toAbsolutePath(String executable) {
+        if (executable.isEmpty()) {
+            return executable;
+        }
         if (executable.contains(":")) {
             // this is either already an absolute windows path, or not a single executable
             return executable;
@@ -1285,7 +1289,7 @@ protected void collectArguments(Set<String> options) {
 
     private static ConsoleHandler createConsoleHandler(InputStream inStream, OutputStream outStream) {
         if (!isTTY()) {
-            return new DefaultConsoleHandler(inStream);
+            return new DefaultConsoleHandler(inStream, IS_WINDOWS);
         } else {
             return new JLineConsoleHandler(inStream, outStream);
         }
@@ -1470,7 +1474,7 @@ protected String getResolvedExecutableName() {
         if (ImageInfo.inImageRuntimeCode()) {
             // This is the fastest, most straightforward way to get the name of the actual
             // executable, i.e., with symlinks resolved all the way down to graalpy
-            String executableName = ProcessProperties.getExecutableName();
+            String executableName = getNativeExecutableName();
             if (executableName != null) {
                 return executableName;
             }
@@ -1499,7 +1503,7 @@ protected String getResolvedExecutableName() {
     private List<String> getCmdline(List<String> args, List<String> subProcessDefs) {
         List<String> cmd = new ArrayList<>();
         if (isAOT()) {
-            cmd.add(ProcessProperties.getExecutableName());
+            cmd.add(getNativeExecutableName());
             for (String subProcArg : subProcessDefs) {
                 assert subProcArg.startsWith("D");
                 cmd.add("--native." + subProcArg);
@@ -1530,6 +1534,20 @@ private List<String> getCmdline(List<String> args, List<String> subProcessDefs)
         return cmd;
     }
 
+    private static String getNativeExecutableName() {
+        String executableName = ProcessProperties.getExecutableName();
+        if (executableName != null) {
+            try {
+                if (Files.isExecutable(Path.of(executableName))) {
+                    return executableName;
+                }
+            } catch (InvalidPathException e) {
+                // Try the process handle below.
+            }
+        }
+        return ProcessHandle.current().info().command().orElse(executableName);
+    }
+
     private static final class ExitException extends RuntimeException {
         private static final long serialVersionUID = 1L;
         private final int code;

graalpython/com.oracle.graal.python.test/src/com/oracle/graal/python/test/SocketTests.java

@@ -1060,11 +1060,11 @@ private byte[] getNulTerminatedTempFileName() throws IOException {
     }
 
     private Object s2p(String s) {
-        return lib.createPathFromString(posixSupport, toTruffleStringUncached(s));
+        return lib.createCStringFromString(posixSupport, toTruffleStringUncached(s));
     }
 
     private String p2s(Object p) {
-        return lib.getPathAsString(posixSupport, p).toJavaStringUncached();
+        return lib.getCStringAsString(posixSupport, p).toJavaStringUncached();
     }
 
     private static void expectErrno(ThrowingRunnable runnable, OSErrorEnum... expectedErrorCodes) {

graalpython/com.oracle.graal.python.test/src/com/oracle/graal/python/test/builtin/modules/PathConversionNodeTests.java

@@ -67,7 +67,7 @@
 import com.oracle.graal.python.builtins.objects.PNone;
 import com.oracle.graal.python.builtins.objects.common.SequenceStorageNodes;
 import com.oracle.graal.python.builtins.objects.tuple.PTuple;
-import com.oracle.graal.python.runtime.PosixSupportLibrary.Buffer;
+import com.oracle.graal.python.runtime.PosixSupportLibrary;
 import com.oracle.graal.python.runtime.PythonContext;
 import com.oracle.graal.python.runtime.object.PFactory;
 import com.oracle.graal.python.test.PythonTests;
@@ -91,10 +91,9 @@ public static String[] params() {
     public void setUp() {
         org.junit.Assume.assumeTrue(backendName.equals("java") || !IS_WINDOWS);
         PythonTests.enterContext(Collections.singletonMap("python.PosixModuleBackend", backendName), new String[0]);
-        pathToString = backendName.equals("java") ? p -> (String) p.value : p -> {
-            Buffer b = (Buffer) p.value;
-            return new String(b.data, 0, (int) b.length);
-        };
+        Object posixSupport = PythonContext.get(null).getPosixSupport();
+        PosixSupportLibrary posixLib = PosixSupportLibrary.getUncached();
+        pathToString = p -> posixLib.getPathAsString(posixSupport, p.value).toJavaStringUncached();
     }
 
     @After

graalpython/com.oracle.graal.python.test/src/com/oracle/graal/python/test/shell/TestPathResolution.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2024, 2025, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2024, 2026, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -53,12 +53,15 @@
 
 public class TestPathResolution {
     private Method calculateProgramFullPathMethod;
+    private Method toAbsolutePathMethod;
 
     @Before
     public void setup() throws NoSuchMethodException {
         // Use reflection to avoid exposing the method in public API
         calculateProgramFullPathMethod = GraalPythonMain.class.getDeclaredMethod("calculateProgramFullPath", String.class, Predicate.class, String.class);
         calculateProgramFullPathMethod.setAccessible(true);
+        toAbsolutePathMethod = GraalPythonMain.class.getDeclaredMethod("toAbsolutePath", String.class);
+        toAbsolutePathMethod.setAccessible(true);
     }
 
     public String calculateProgramFullPath(String executable, Predicate<Path> isExecutable, String path) {
@@ -69,6 +72,14 @@ public String calculateProgramFullPath(String executable, Predicate<Path> isExec
         }
     }
 
+    public String toAbsolutePath(String executable) {
+        try {
+            return (String) toAbsolutePathMethod.invoke(null, executable);
+        } catch (Throwable e) {
+            throw new RuntimeException(e);
+        }
+    }
+
     @Test
     public void testProgramNameResolutionFullPath() {
         Assert.assertEquals("/absolute/path/graalpy",
@@ -78,6 +89,11 @@ public void testProgramNameResolutionFullPath() {
                         calculateProgramFullPath("./blah/graalpy", path -> true, null));
     }
 
+    @Test
+    public void testEmptyExecutableStaysEmpty() {
+        Assert.assertEquals("", toAbsolutePath(""));
+    }
+
     @Test
     public void testProgramNameResolutionEndOfPath() {
         Assert.assertEquals("/last/in/path/graalpy",

graalpython/com.oracle.graal.python.test/src/tests/__init__.py

@@ -144,14 +144,39 @@ def get_setuptools(setuptools='67.6.1'):
     ensure_packages(setuptools=setuptools)
 
 
+def _system_python_for_venv():
+    python = os.environ.get("MX_PYTHON") or "python"
+    resolved = shutil.which(python)
+    if resolved and (not sys.platform.startswith('win32') or Path(resolved).suffix.lower() == ".exe"):
+        return resolved
+    if sys.platform.startswith('win32'):
+        version = os.environ.get("MX_PYTHON_VERSION")
+        if not version and python.startswith("python") and len(python) > len("python"):
+            version = python[len("python"):]
+        if version:
+            version_dir = version.replace(".", "")
+            for candidate in (
+                f"C:/Python{version_dir}/python.exe",
+                f"C:/Python{version_dir}/python3.exe",
+                f"C:/Python{version_dir}/{python}.exe",
+            ):
+                if Path(candidate).is_file():
+                    return candidate
+        for executable in ("python.exe", "python3.exe", "python"):
+            fallback = shutil.which(executable)
+            if fallback and Path(fallback).suffix.lower() == ".exe":
+                return fallback
+    return python
+
+
 def install_venv(venv_path: Path) -> bool:
     """Installs a virtual environment at the given path."""
     if not sys.executable or (sys.platform.startswith('win32') and sys.implementation.name == "graalpy"):
         # When running in a PolyBench benchmark context sys.executable is unset
         # And thus we must defer to the system's python
         # Deferring to the system's python is fine as it will only be used to install setuptools
         import subprocess
-        subprocess.run(["python", "-m", "venv", str(venv_path)], check=True)
+        subprocess.run([_system_python_for_venv(), "-m", "venv", str(venv_path)], check=True)
         return True
     else:
         import venv

graalpython/com.oracle.graal.python.test/src/tests/cpyext/test_tuple.py

@@ -391,16 +391,19 @@ def test_memoryview_cast_shape_native_tuple(self):
 
     def test_utime_ns_divmod_native_tuple(self):
         class NativeDivmod:
+            def __init__(self, value):
+                self.value = value
+
             def __divmod__(self, other):
                 assert other == 1000000000
-                result = TupleSubclass(1, 234567890)
+                result = TupleSubclass(*divmod(self.value, other))
                 assert is_native_object(result)
                 return result
 
         with tempfile.NamedTemporaryFile() as f:
-            os.utime(f.name, ns=(NativeDivmod(), NativeDivmod()))
+            expected = 1700000001234567890
+            os.utime(f.name, ns=(NativeDivmod(expected), NativeDivmod(expected)))
             stat = os.stat(f.name)
-            expected = 1234567890
             # We don't care about the exact value. It's about if native tuples are accepted.
             tolerance = 1_000_000_000
             assert abs(stat.st_atime_ns - expected) <= tolerance

graalpython/com.oracle.graal.python.test/src/tests/test_autopatch_capi.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2024, 2025, Oracle and/or its affiliates. All rights reserved.
+# Copyright (c) 2024, 2026, Oracle and/or its affiliates. All rights reserved.
 # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
 #
 # The Universal Permissive License (UPL), Version 1.0
@@ -38,6 +38,7 @@
 # SOFTWARE.
 
 import sys
+import os
 import tempfile
 import textwrap
 
@@ -48,13 +49,17 @@
     def check_autopatched(source, expected):
         source = textwrap.dedent(source).rstrip()
         expected = textwrap.dedent(expected).rstrip()
-        with tempfile.NamedTemporaryFile('r+') as f:
+        with tempfile.NamedTemporaryFile('w+', delete=False) as f:
             f.write(source)
             f.flush()
-            autopatch_capi.auto_patch(f.name, False)
-            f.seek(0)
-            actual = f.read()
+            name = f.name
+        try:
+            autopatch_capi.auto_patch(name, False)
+            with open(name) as f:
+                actual = f.read()
             assert actual == expected, f"Autopatch didn't make expected changes. Expected:\n{expected}\nActual:\n{actual}"
+        finally:
+            os.unlink(name)
 
 
     def test_replace_field_access():