Issues with numeric field variables and OFMT/CONVFMT

[stephane-chazelas]

Similar issues were reported for GNU awk in the thread at https://lists.gnu.org/archive/html/bug-gawk/2026-06/msg00001.html

Per POSIX, $0 is meant to be recomputed by joining field variables with OFS at the point any of those field variables or NF are being assigned.

And similarly, field variables are meant to be recomputed by splitting $0 based on FS at the point $0 is assigned (explicitly or via reading a new record, or any other way).

The problem here is that onetrueawk like gawk implements some performance optimisations by which that joining/splitting is only done later if necessary (or if OFS, respectively FS are modified) but when field variables (respectively $0) are numeric, those optimisations overlook the fact that it's the value of CONVFMT at the time of the field variable (resp $0) were modified that should be used to convert those numbers to string prior to joining/splitting.

In the case of onetrueawk, there also seems to be some confusion between OFMT and CONVFMT in some cases.

$ echo 5 3 | onetrueawk '{CONVFMT="%.3f"; OFMT="%03e"; $0 = $1/$2; CONVFMT="%.0f"; OFMT="%.0e"; print $1}'
2

Upon $0 = $1/$2, the string representation of $0 should have been 1.667 (as CONVFMT contained %.3f at that point), which should have been split into $1 == "1.667". Here, as an optimisation, that splitting was done at the point $1 was referenced but used the wrong value of CONVFMT, the one that was set after $0 was assigned.

$ echo 5 3 | onetrueawk '{CONVFMT="%.3f"; OFMT="%03e"; $0 = $1/$2; CONVFMT="%.0f"; OFMT="%.0e"; print $0, $1}'
2e+00 2e+00

It's worse here as it's the value of OFMT that was used, which is only meant to be used for output, while it's the value of CONVFMT from the time $0 was modified that should have been used to convert $0 to string prior to splitting to compute the field variables.

For the symmetric case of $0 computed out of field variables:

$ echo 5 3 | onetrueawk '{CONVFMT="%.3f"; OFMT="%03e"; $1 = $1/$2; CONVFMT="%.0f"; OFMT="%.0e"; print $0}'
2 3

Same thing, it used the wrong value of CONVFMT.

$ echo 5 3 | onetrueawk '{CONVFMT="%.3f"; OFMT="%03e"; $1 = $1/$2; CONVFMT="%.0f"; OFMT="%.0e"; print $1, $0}'
2e+00 2e+00 3

Same confusion with OFMT.

$ echo 5 3 | onetrueawk '{CONVFMT="%.3f"; OFMT="%03e"; $1 = $1/$2; OFS="+"; CONVFMT="%.0f"; OFMT="%.0e"; print $0}'
1.667 3

Correct in this case, but only because assigning OFS presumably triggers the joining (with the previous value of OFS), so here before CONVFMT is modified.

$ echo 5 3 | onetrueawk '{CONVFMT="%.3f"; OFMT="%03e"; $1 = $1/$2; CONVFMT="%.0f"; OFS="+"; OFMT="%.0e"; print $0}'
2 3

Back to being wrong as that's now done after CONVFMT was modified.

As also noted for gawk, this kind of issue is unlikely to be hit in practice as if CONVFMT/OFMT are being set, that's generally once and for all at the start.

[arnoldrobbins]

The current denouement of the discussion for gawk is here. I disagree with Stephane as to what's "correct" behaviour, but it's clear that this is very much "dark corner" territory. Just for whatever it's worth.

Thanks @stephane-chazelas for raising the issue(s).

[stephane-chazelas]

The current denouement of the discussion for gawk is here. I disagree with Stephane as to what's "correct" behaviour, but it's clear that this is very much "dark corner" territory. Just for whatever it's worth.
[...]

Hi @arnoldrobbins. For the record here, can you please clarify why you disagree with my interpretation of the standard or why that could be seen as anything other than an incorrect/incomplete optimisation by gawk/onetrueawk?

In the gawk thread, in https://lists.gnu.org/archive/html/bug-gawk/2026-06/msg00042.html you say (for gawk 'BEGIN{CONVFMT = "%.6g"; $0 = 10.1; CONVFMT = "%.6e"; print "" $0, $1}' outputting 1.010000e+01 1.010000e+01 instead of 1.010000e+01 10.1):

It makes no sense to me that when there is only field to have $0 and $1 be different

But here that's:

• $1, which is the result of splitting $0 at the time it was assign the 10.1 numeric value and converted to string at that point with the value of CONVFMT which then was %.6g.
• vs "" $0 (not $0 for which OFMT would be involved in that print statement) where the value of $0 (which is still numeric) is converted to string at the point of that print statement, so with the new %.6e value of CONVFMT.

[stephane-chazelas]

In any case, here, even if we disagree on that particular aspect and are OK for awk to use any of the values CONVFMT had between the time $0 was assigned a numeric value and the time $1 is referenced for $0's conversion to string for splitting (or the reverse for joining field variables into $0), onetrueawk has that separate issue of confusing OFMT with CONVFMT which I think we can agree is a bug.

[arnoldrobbins]

The OFMT and CONVFMT confusion is an issue. Gawk apparently has a similar problem. My comment that

It makes no sense to me that when there is only [one] field to have $0 and $1 be different

was incorrect, and as can be seen in the item I linked, with FS = "e", gawk now produces 2 fields.

Dealing with this more thoroughly in gawk may end up waiting, as I would really like to make a release very soon.

By the way, while I respect the POSIX work, there are many places where POSIX is silent when it shouldn't be, as well as places where it's just wrong. The way I'm doing things may even be incompatible with POSIX, but it makes the most sense for the gawk code base. In other words, it's entirely possible you're correct in how you think things should be done; there's lots of room for interpretation here.