[Bug]: Different output upon decoding invalid utf-8

[vytas7]

Describe the bug

Not sure whether it is a bug or just a different behaviour.
Not critical either way, but thought filing it for reference anyway.

• When decoding invalid UTF-8 with errors='replace', CPython (and PyPy for that matter) decodes every invalid byte as �.
• GraalPy, OTOH, treats invalid UTF-8 triplets as one character.

Operating system

Linux

CPU architecture

x86_64

GraalPy version

GraalPy 3.12.8 (GraalVM CE Native 25.0.2)

JDK version

No response

Context configuration

No response

Steps to reproduce

GraalPy:

$ graalpy -c "print(b'\xed\xae\x80\xed\xb0\x80'.decode(errors='replace'))"
��

CPython:

$ python -c "print(b'\xed\xae\x80\xed\xb0\x80'.decode(errors='replace'))"
������

Expected behavior

To match CPython unless there is a good reason not to.

Stack trace

Additional context

No response

[msimacek]

Hi @vytas7, we currently use the Java codecs for encodings and this is just what the Java codec reports. We do have another codec provided by TruffleString that would report error ranges closer to CPython for UTF-8, but it currently doesn't provide all the error handling APIs we need. So this is fixable, but far from trivial.

[vytas7]

No worries @msimacek , I just wanted to report this discrepancy so that it's tracked. It was one of the few things left in Falcon's basic test suite failing on Graal.
It can be easily skipped by detecting the platform/interpreter.

[saisharan0103]

GraalPy 3.12.8 is grouping each invalid UTF-8 triplet in b'\xed\xae\x80\xed\xb0\x80' into one replacement character, while CPython and PyPy replace each invalid byte. The UTF-8 errors='replace' path is the part I would inspect first, especially where invalid surrogate-range sequences advance the input cursor. For a PR, I would start by adding this byte sequence as a compatibility test against CPython behavior.