[Hotspot] Refactor Reference and Cleaner handling to fix Unsafe nativ…

[superajun-wsj]

…e link errors

Refactor the Reference and Cleaner handling logic in CVM to simplify the reference dispatch mechanism and remove unnecessary dependencies on JDK 17 Reference/Cleaner implementations. This change eliminates excessive coupling with JDK 17-specific classes and fundamentally resolves the Unsafe native link errors introduced by jdk.internal.misc.Cleaner.

[Copilot]

Pull request overview

Refactors HotSpot and CVM class-library integration around Reference/Cleaner handling, aiming to reduce coupling to JDK 17-specific implementations and address Unsafe-related native link issues by removing/rewiring several alt-kernel classes and adding regression tests.

Changes:

• Gates HotSpot known-method initialization for jdk.internal.misc.Unsafe throw helpers and stack-walker support behind HOTSPOT_TARGET_CLASSLIB == 17.
• Removes a set of JDK 17-derived reference/cleaner/unsafe support classes from cvm/alt_kernel/src17u and updates Reference/Thread behavior accordingly.
• Adds CVM tests covering reference forwarding, cleaner dispatch, and direct-buffer cleanup behavior.

Reviewed changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/hotspot/share/memory/universe.cpp	Conditionalizes known-method cache initialization for JDK17-specific helpers.
src/hotspot/share/classfile/vmClassMacros.hpp	Adjusts VM class list for classlib 8 vs non-8 targets (removes internal_Unsafe_klass from the 8 list).
cvm/test/java/nio/Buffer/DirectBufferCleanerTest.java	New test validating direct-buffer cleaner behavior and direct-memory accounting.
cvm/test/java/lang/ref/ReferenceForwardingTest.java	New test validating mixed pending-list forwarding for finalizers and phantom references.
cvm/test/java/lang/ref/CleanerReferenceForwardingTest.java	New test validating cleaner dispatch does not break phantom forwarding.
cvm/alt_kernel/src8u/share/classes/java/nio/Bits.java	Removes alt-kernel JDK8 Bits implementation.
cvm/alt_kernel/src17u/share/classes/sun/nio/cs/ArrayEncoder.java	Removes alt-kernel JDK17 charset fastpath interface.
cvm/alt_kernel/src17u/share/classes/sun/nio/cs/ArrayDecoder.java	Removes alt-kernel JDK17 charset fastpath interface.
cvm/alt_kernel/src17u/share/classes/sun/misc/JavaLangRefAccess.java	Removes alt-kernel sun.misc reference-access hook.
cvm/alt_kernel/src17u/share/classes/jdk/internal/util/ArraysSupport.java	Removes alt-kernel JDK17 array utilities.
cvm/alt_kernel/src17u/share/classes/jdk/internal/ref/PhantomCleanable.java	Removes alt-kernel JDK17 cleaner infrastructure.
cvm/alt_kernel/src17u/share/classes/jdk/internal/ref/CleanerImpl.java	Removes alt-kernel JDK17 cleaner runtime.
cvm/alt_kernel/src17u/share/classes/jdk/internal/ref/CleanerFactory.java	Removes alt-kernel JDK17 common-cleaner factory.
cvm/alt_kernel/src17u/share/classes/jdk/internal/misc/Unsafe.java	Removes alt-kernel JDK17 jdk.internal.misc.Unsafe copy.
cvm/alt_kernel/src17u/share/classes/jdk/internal/misc/InnocuousThread.java	Removes alt-kernel JDK17 innocuous thread implementation.
cvm/alt_kernel/src17u/share/classes/java/lang/Thread.java	Updates WeakClassKey.equals to avoid refersTo(...).
cvm/alt_kernel/src17u/share/classes/java/lang/ref/ReferenceQueue.java	Removes alt-kernel ReferenceQueue implementation.
cvm/alt_kernel/src17u/share/classes/java/lang/ref/Reference.java	Reworks reference processing, pending handling, and SharedSecrets wiring.
cvm/alt_kernel/src17u/share/classes/java/lang/ref/FinalReference.java	Removes alt-kernel FinalReference.
cvm/alt_kernel/src17u/share/classes/java/lang/ref/Cleaner.java	Removes alt-kernel java.lang.ref.Cleaner.

Suppressed comments (2)

src/hotspot/share/memory/universe.cpp:909

• Wrapping initialization of _throw_illegal_access_error_cache/_throw_no_such_method_error_cache in #if HOTSPOT_TARGET_CLASSLIB == 17 leaves these caches uninitialized for other targets, but the returned methods are still used unconditionally elsewhere (e.g., klassVtable.cpp:1342 installs Universe::throw_illegal_access_error() into itables, and javaClasses.cpp:4356 / resolvedMethodTable.cpp:356-359 use Universe::throw_no_such_method_error() for deleted methods). If the caches are never initialized, these accessors return NULL, which changes behavior (itable entries remain empty; deleted method fallback becomes NULL) and can lead to incorrect exceptions or later crashes.

#if HOTSPOT_TARGET_CLASSLIB == 17
  initialize_known_method(_throw_illegal_access_error_cache,
                          vmClasses::internal_Unsafe_klass(),
                          "throwIllegalAccessError",
                          vmSymbols::void_method_signature(), true, CHECK);

cvm/alt_kernel/src17u/share/classes/java/lang/ref/Reference.java:260

• SharedSecrets.setJavaLangRefAccess(...) is wiring an API (tryHandlePendingReference) that does not exist on the jdk.internal.access.JavaLangRefAccess interface in this repo (it expects waitForReferenceProcessing() and runFinalization()). This will not compile once the imports are corrected, and it also breaks Bits/other callers that rely on the existing waitForReferenceProcessing contract.

        SharedSecrets.setJavaLangRefAccess(new JavaLangRefAccess() {
            @Override
            public boolean tryHandlePendingReference() {
                return tryHandlePending(false);
            }

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.