Skip to content

Fix precision of real32quant for 32 bit float - #86

Merged
hahnjo merged 1 commit into
root-project:mainfrom
undefined-panda:float-precision-real32quant
Aug 12, 2026
Merged

Fix precision of real32quant for 32 bit float#86
hahnjo merged 1 commit into
root-project:mainfrom
undefined-panda:float-precision-real32quant

Conversation

@undefined-panda

Copy link
Copy Markdown
Collaborator

No description provided.

@undefined-panda
undefined-panda force-pushed the float-precision-real32quant branch from 3734f5d to 6e3fe69 Compare August 6, 2026 15:13
@hahnjo
hahnjo self-requested a review August 10, 2026 06:37

@hahnjo hahnjo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change would benefit from more details in the commit message: what does ROOT do (compute in double), what does the macro request (single-precision float so the value is cast), why is that not possible in JavaScript (no single precision float type), and how does the change solve it (Math.fround "cutting off" the mantissa).

Also you need to update jsroot/types/fundamental/README.md to remove the caveat documented so far.

Comment thread jsroot/types/fundamental/real32quant/read.mjs Outdated
Comment thread jsroot/types/fundamental/real32quant/read.mjs Outdated
@undefined-panda
undefined-panda force-pushed the float-precision-real32quant branch from 6e3fe69 to 3008012 Compare August 10, 2026 08:46
@hahnjo

hahnjo commented Aug 10, 2026

Copy link
Copy Markdown
Member

regarding the commit message:

ROOT computes quantized floats in double precision but can write them with single- or double-precision.

Not quite, the quantization is eventually written as integers with a fixed number of bits. The important point is that the in-memory field type can be single- or double-precision, depending on the requested template argument.

Since JS only has double-precision floats, Math.fround is used to cut off the mantissa to 32 bits for Float32 fields.

The mantissa of 32-bit floating-point numbers are 23 bits. I think the important aspect of Math.fround is that it "returns the nearest 32-bit single precision float representation of a number."

@undefined-panda
undefined-panda force-pushed the float-precision-real32quant branch from 3008012 to d5e2232 Compare August 10, 2026 14:47
@hahnjo

hahnjo commented Aug 11, 2026

Copy link
Copy Markdown
Member

@undefined-panda can you please rebase to resolve the (trivial) conflict?

Proposal for the commit message:

Real32Quant is stored as integers on disk and then transformed back
to floating-point numbers using double-precision arithmetic. In the
ROOT (C++) implementation, the value is then cast according to the
requested field type, which can have either single-precision (float)
or double-precision.
Since JavaScript only has double-precision floats, Math.fround is used
to get the nearest 32-bit single-precision number.

@undefined-panda
undefined-panda force-pushed the float-precision-real32quant branch 5 times, most recently from e20e1b4 to 4a34699 Compare August 12, 2026 08:37
Comment thread jsroot/types/fundamental/real32quant/read.mjs
Real32Quant is stored as integers on disk and then transformed back
to floating-point numbers using double-precision arithmetic. In the
ROOT (C++) implementation, the value is then cast according to the
requested field type, which can have either single-precision (float)
or double-precision.
Since JavaScript only has double-precision floats, Math.fround is used
to get the nearest 32-bit single-precision number.
@undefined-panda
undefined-panda force-pushed the float-precision-real32quant branch from 4a34699 to a2dfbaa Compare August 12, 2026 09:46
@hahnjo
hahnjo merged commit f91c1c7 into root-project:main Aug 12, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants