Add support for BEGINFILE and ENDFILE rules#515
Conversation
Implement gawk's BEGINFILE/ENDFILE special patterns, the nextfile statement, and the ERRNO and ARGIND special variables: - Parser: BEGINFILE/ENDFILE/nextfile keywords, per-file input-loop tuple layout (NEXT_FILE / CONSUME_FILE_INPUT scaffolding emitted only when these features are used, so existing programs keep the flat layout), and gawk-compatible compile-time restrictions (next rejected in BEGINFILE/ENDFILE, nextfile rejected in BEGIN/END/ENDFILE). - Runtime: per-file stepping in StreamInputSource with FILENAME, FNR=0, cleared $0, ARGIND, and ERRNO on open failure; open failures become non-fatal only when a BEGINFILE rule skips the file with nextfile; nextfile unwinds user-defined function calls like exit does; non-redirected getline is fatal inside BEGINFILE/ENDFILE. - New opcodes appended at the end of the Opcode enum to preserve precompiled tuple (-L) compatibility. - BEGINFILE/ENDFILE are not special in POSIX mode (--posix), as in gawk. - Lexer: a slash inside a regexp bracket expression (/[/]/) no longer terminates the literal, which beginfile1.awk requires; this also makes the vendored gawk regexpbrack test pass. - FNR now counts records read from standard input (POSIX: FNR is the record number in the current input file); custom InputSource implementations keep the documented isFromFilenameList() contract. - Tests: BeginFileEndFileTest (25 cases), re-enabled the POSIX nextfile conformance test, new stdin-FNR conformance tests, and wired the vendored beginfile1 fixture plus six beginfile2 sub-tests into GawkExtensionIT (beginfile1 matches beginfile1.ok byte for byte). - Docs: README, cli.md, cli-reference.md, extensions.md, java.md, java-input.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 697ebee809
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
gawk resets ERRNO to the empty string whenever the next main input (file operand or stdin) is selected successfully, not only on the BEGINFILE per-file path. Mirror that in the ordinary input flow and pin the behavior with unit tests. Addresses review feedback on #515. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@codex Please review again |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4622583816
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…E/ENDFILE rules exist A non-redirected getline (or getline var) in an ordinary action could open the next ARGV file directly through the auto-advancing input path, skipping the ENDFILE rules of the old file and the BEGINFILE rules of the new one, and later running ENDFILE with the wrong FILENAME. While the per-file main input loop is active, only the loop itself may now cross file boundaries: getline in an action reports end-of-input at the end of the current file, and the loop then fires ENDFILE/BEGINFILE in order before processing the next file's records. In BEGIN and END rules getline keeps streaming across the input, and programs without BEGINFILE/ENDFILE rules (including nextfile-only programs) keep the classic AWK file-crossing behavior. Addresses review feedback on #515. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@codex Please review again. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: df930a4194
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…p brackets - A name=value operand between two input files no longer increments NR: per POSIX (and gawk), NR only counts records actually read. The bump existed in both the streaming and the per-file input paths and made NR off by one per assignment operand; the unit test that pinned the old behavior now asserts the gawk-compatible count. - The regexp lexer keeps bracket mode through POSIX character class, collating element, and equivalence subexpressions ([:digit:], [.x.], [=e=]), so a slash after such a class (e.g. /[[:digit:]/]/) no longer terminates the regexp literal early. Addresses review feedback on #515. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@codex please review once more. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 48e6017f74
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Like gawk --posix: ERRNO and ARGIND are gawk extensions, so POSIX mode compiles them as plain global variables — usable as function parameters, seeded normally by -v and name=value assignments, and untouched by the input machinery at file boundaries. Outside POSIX mode they remain JRT-managed special variables. Addresses review feedback on #515. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 participant
Closes #457.
What changed
Jawk now parses and executes gawk's
BEGINFILE/ENDFILEspecial patterns, together with thenextfilestatement and theERRNO/ARGINDspecial variables.AwkParser): newBEGINFILE/ENDFILE/nextfilekeywords and rule ASTs. When any of these features is used, the main input loop is compiled with the per-file tuple layout from the issue (NEXT_FILEbeforebegin_file,CONSUME_FILE_INPUT → end_fileat each file's EOF, ENDFILE rules, thenNEXT_FILE → no_more_inputfalling into the END blocks). Programs that don't use these features keep the exact previous flat layout, so there is zero overhead for existing scripts.AVM,JRT,StreamInputSource): per-file stepping with gawk-compatibleFILENAME,FNR = 0, cleared$0,ARGIND, andERRNOon open failure (No such file or directory/Is a directory, synthesized deterministically cross-platform). A file that cannot be opened is non-fatal only if a BEGINFILE rule skips it withnextfile; otherwise the usual gawk-style fatal error fires.nextfileworks in ordinary rules and BEGINFILE — including from user-defined functions (it unwinds the runtime/operand stacks the same wayexitdoes) — and runs ENDFILE for successfully opened files while skipping it for failed ones, matching the vendored.okfixtures.exitinside the hooks jumps to the END blocks.nextis rejected inside BEGINFILE/ENDFILE andnextfileinside BEGIN/END/ENDFILE (compile time when direct, runtime when reached through a function); non-redirectedgetlineis a fatal error inside both hooks, redirected forms are allowed.NEXT_FILE,CONSUME_FILE_INPUT,EXEC_NEXTFILE,SET_ENDFILE_ADDRESS,SET_NEXTFILE_ADDRESS,PUSH/ASSIGN_ERRNO,PUSH/ASSIGN_ARGIND— appended at the end of theOpcodeenum becausefromIdresolves by ordinal, preserving precompiled tuple (-L) compatibility.BEGINFILEandENDFILEare not special under--posix/setPosix(true)and lex as ordinary identifiers.nextfilestays available in both modes (POSIX.1-2024 standardizes it)./inside a regexp bracket expression (/.*[/]/, used bybeginfile1.awk) no longer terminates the literal. This also flipped the vendored gawkregexpbrackIT from error to pass.printf 'a\nb\n' | jawk '{print FNR}'now prints1 2instead of0 0. CustomInputSourceimplementations keep the documentedisFromFilenameList()contract.Tests
BeginFileEndFileTest(25 cases): execution order across files, empty files, stdin, ERRNO +nextfileskip, fatal open failures,nextfilefrom functions and without hooks, all restriction diagnostics, ARGIND,$0clearing,exitin hooks, POSIX-mode non-specialness, bracket-slash regexp.nextfileconformance test; added stdin-FNR conformance tests and anInputSourceTestguard for the custom-source FNR contract.GawkExtensionIT#test_beginfile1now runs the vendored fixture for real and matchesbeginfile1.okbyte for byte (including the ERRNO lines for.and a missing file). Sixbeginfile2sub-tests (2, 5, 9a, 9b, 12, 15) extract their programs and expected transcripts from the vendoredbeginfile2.in/.okand pass. The remaining sub-tests rely on non-redirectedgetlinein BEGIN/END driving the BEGINFILE/ENDFILE hooks mid-statement (or compare non-zero gawk CLI transcripts) — that reentrancy isn't reproducible in the tuple VM; the umbrella skip documents it.getfile/readfile2need gawk's dynamic extension API (@load,get_file()), which Jawk doesn't provide; their skip reasons now say so.Verification
mvn verify site: BUILD SUCCESS; checkstyle / PMD / SpotBugs at 0 findings; 543 unit tests pass.regexpbrackturning green; zero regressions across the 939 pre-existing cases, including after the FNR change.Notes for reviewers
test_beginfile2skip): non-redirectedgetlinein BEGIN/END reads ahead in the main input without firing BEGINFILE/ENDFILE; the loop then adopts the already-open file instead of reopening it.$0and the fields are cleared in BEGINFILE.🤖 Generated with Claude Code