Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 18 additions & 2 deletions .agents/skills/third-party-package-patches/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: third-party-package-patches
description: Create or update GraalPy third-party package compatibility patches under graalpython/lib-graalpython/patches, including PyPI source preparation, rebasing existing patches, metadata.toml updates, license checks, version-range validation, and verify_patches.py validation.
description: Create or update GraalPy third-party package and Rust crate compatibility patches under graalpython/lib-graalpython/patches, including PyPI source preparation, Cargo crate autopatching, rebasing existing patches, metadata.toml updates, license checks, version-range validation, and verify_patches.py validation.
---

# Third-Party Package Patches
Expand All @@ -11,6 +11,8 @@ Use this skill when creating or updating compatibility patches for packages inst
- Source preparation: `scripts/get_pypi_source.py`
- Patch metadata: `graalpython/lib-graalpython/patches/metadata.toml`
- Patch directory: `graalpython/lib-graalpython/patches/`
- Cargo autopatcher: `graalpython/lib-graalpython/modules/autopatch_cargo.py`
- Crate patch metadata: `graalpython/lib-graalpython/patches/crates/metadata.toml`
- Metadata verifier: `mx.graalpython/verify_patches.py`

## Workflow
Expand All @@ -20,7 +22,7 @@ Use this skill when creating or updating compatibility patches for packages inst
```bash
python scripts/get_pypi_source.py package==version
```
The script prints `Prepared source at: ...`. Use that directory as the working tree. It is already a temporary git repository with an initial commit, and it has already been processed by `graalpython/lib-graalpython/modules/autopatch_capi.py`.
The script prints `Prepared source at: ...`. Use that directory as the working tree. It is already a temporary git repository with an initial commit, and it has already been processed by `autopatch_capi.py` and `autopatch_cargo.py`.

3. Inspect `graalpython/lib-graalpython/patches/metadata.toml` for existing `[[package.rules]]` entries.
- If a matching patch exists for the requested version, apply it first.
Expand Down Expand Up @@ -74,6 +76,19 @@ python mx.graalpython/verify_patches.py graalpython/lib-graalpython/patches

12. If you were asked to build or test the patched package, you need to rebuild GraalPy with `mx python-jvm` to pick up the changes. Create a venv with `mx python -m venv venv_name` and use it for building and testing.

## Rust Crate Patches

Use `autopatch_cargo.py` when the incompatibility is in a transitive crates.io dependency rather than the Python package source. It finds matching crates in `Cargo.lock`, copies only those crates into the source tree, applies registered patches, and adds Cargo path overrides.

1. Add rules to `graalpython/lib-graalpython/patches/crates/metadata.toml`. Use the crate package name as the table key and specify `version`, `patch`, and `license` for every rule.
2. Generate crate patch files from a clean checkout of the published crate. Patch paths must be relative to the crate root. Do not hand-edit generated patch files.
3. Run the tool directly when validating a prepared source tree:
```bash
python graalpython/lib-graalpython/modules/autopatch_cargo.py /path/to/source
```
4. Test every crate release covered by the version range, then build representative Python packages that resolve those releases.
5. Use the Python package rule's `autopatch = false` only when both `autopatch_capi` and `autopatch_cargo` must be disabled for that package version.

## Metadata Reference
Rule keys accepted by the verifier are:
- `version`
Expand All @@ -83,6 +98,7 @@ Rule keys accepted by the verifier are:
- `dist-type`
- `install-priority`
- `note`
- `autopatch`

Allowed `dist-type` values are `wheel` and `sdist`.

Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ repos:
name: Copyright check
language: system
require_serial: true
entry: bash scripts/pre-commit-checkcopyrights.sh
entry: env VERIFY_PATCHES_GIT=1 bash scripts/pre-commit-checkcopyrights.sh
types: [text]
- id: eclipseformat
name: Eclipse formatter
Expand Down
3 changes: 0 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,9 +67,6 @@ It consists of: Java (Truffle) + C (CPython C-API compatibility) + Python stdlib
`mx graalpytest TEST-SELECTOR`
* Run JUnit tests
`mx python-gate --tags python-junit`
* Style / formatting
`mx python-style --fix`
`mx python-gate --tags style`
* Building standalones for benchmarking
- use `mx --env native-ee sforceimports && mx --env native-ee checkout-downstream compiler graal-enterprise` to get the right revisions
- use `mx --env jvm-ee-libgraal` and `mx --env native-ee` to build the JAVA and NATIVE standalone distributions
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2017, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2017, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* The Universal Permissive License (UPL), Version 1.0
Expand Down Expand Up @@ -74,8 +74,11 @@
import org.graalvm.shadowed.org.jline.terminal.Terminal;
import org.graalvm.shadowed.org.jline.terminal.TerminalBuilder;
import org.graalvm.shadowed.org.jline.utils.InputStreamReader;
import org.graalvm.shadowed.org.jline.utils.Signals;

public class JLineConsoleHandler extends ConsoleHandler {
private static final String[] SIGNALS = {"INT", "QUIT", "TSTP", "CONT", "WINCH"};

private final InputStream inputStream;
private final OutputStream outputStream;
private LineReader reader;
Expand All @@ -93,10 +96,13 @@ public class JLineConsoleHandler extends ConsoleHandler {
@SuppressWarnings("unused")
public void initializeReadline(Value readlineModule) {
Terminal terminal;
Object[] signalHandlers = stashSignalHandlers();
try {
terminal = TerminalBuilder.builder().jna(false).streams(inputStream, outputStream).system(true).signalHandler(Terminal.SignalHandler.SIG_IGN).build();
terminal = TerminalBuilder.builder().jna(false).streams(inputStream, outputStream).system(true).build();
} catch (IOException ex) {
throw new RuntimeException("unexpected error opening console reader", ex);
} finally {
restoreSignalHandlers(signalHandlers);
}
final Value getCompleter = readlineModule.getMember("get_completer");
final Value shouldRecord = readlineModule.getMember("get_auto_history");
Expand Down Expand Up @@ -226,6 +232,7 @@ public String readLine(String prompt) {
}
}
if (lineBuffer.isEmpty()) {
Object[] signalHandlers = stashSignalHandlers();
try {
String lines = reader.readLine(prompt);
Collections.addAll(lineBuffer, lines.split("\n"));
Expand All @@ -235,11 +242,27 @@ public String readLine(String prompt) {
return null;
} catch (Exception ex) {
throw new RuntimeException(ex);
} finally {
restoreSignalHandlers(signalHandlers);
}
}
return lineBuffer.poll();
}

private static Object[] stashSignalHandlers() {
Object[] handlers = new Object[SIGNALS.length];
for (int i = 0; i < SIGNALS.length; i++) {
handlers[i] = Signals.registerDefault(SIGNALS[i]);
}
return handlers;
}

private static void restoreSignalHandlers(Object[] handlers) {
for (int i = 0; i < SIGNALS.length; i++) {
Signals.unregister(SIGNALS[i], handlers[i]);
}
}

// Used via interop
@SuppressWarnings("unused")
public CompletionState getCompletionState() {
Expand Down
10 changes: 3 additions & 7 deletions graalpython/com.oracle.graal.python.test/src/runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@
CURRENT_PLATFORM_KEYS = frozenset({CURRENT_PLATFORM})

RUNNER_ENV = {}
DISABLE_JIT_ENV = {'GRAAL_PYTHON_VM_ARGS': '--experimental-options --engine.Compilation=false'}
DISABLE_JIT_ENV = {'GRAAL_PYTHON_VM_ARGS': '-X jit=0'}
# The worker transport sends pickled data, so keep it on loopback only.
WORKER_SERVER_HOST = '127.0.0.1'

Expand All @@ -93,12 +93,8 @@

# We leave the JIT enabled for the tests themselves, but disable it for subprocesses
# noinspection PyUnresolvedReferences
if IS_GRAALPY and __graalpython__.is_native and 'GRAAL_PYTHON_VM_ARGS' not in os.environ:
try:
subprocess.check_output([sys.executable, '--version'], env={**os.environ, **DISABLE_JIT_ENV})
RUNNER_ENV = DISABLE_JIT_ENV
except subprocess.CalledProcessError:
pass
if IS_GRAALPY and 'GRAAL_PYTHON_VM_ARGS' not in os.environ:
RUNNER_ENV = DISABLE_JIT_ENV


class Logger:
Expand Down
Loading
Loading