OPENNLP-547: Add a dependency parser component

[krickert]

Summary

• add a dependency parser API and immutable dependency graph model
• provide classical transition parsing and a pure Java feedforward training and inference tier
• add CoNLL-U reading, evaluation, reproducible model persistence, tests, and user documentation

Base

Rebased on main. The Document annotation container (#1182) is merged, so the branch adds only the parser: 34 files, all new, with no change to existing code.

Review round of 2026-09-04

The branch carries the review conventions from the sibling PRs: Javadoc on every method including private helpers and constructors, @throws on each validating path, validation at the public boundary with IllegalArgumentException, named constants, the final-sigma rule in vocabulary normalization, bounded model dimensions, and failing-test commits ahead of each fix.

Verification

opennlp-api, opennlp-runtime, and opennlp-formats build with checkstyle and forbidden APIs; the parser and CoNLL-U test classes pass (128 tests), and the manual chapter's examples are mirrored by ConlluDependencyParserUsageTest.

[mawiesne]

@krickert Can you rebase this one onto main? Currently changes show up here that are now already included in main.

[krickert]

Rebased on main .. good to go

#1182 merged the diff is the parser alone, 34 new files and no change to existing code.

Also a run of the review conventions.

[jzonthemtn]

Thanks @krickert. A long open JIRA ticket for sure!

The API and graph validation look solid. I found two issues worth checking into:

• The CoNLL-U reader accepts DEPREL=_ as a label instead of skipping the incomplete annotation.
• Training with a tag such as UNK overwrites a reserved vocabulary entry, producing a model that serializes successfully but fails to reload.

What do you think about held-out accuracy tests for the neural parser, similar to the treebank evaluation for the classical parser?

[krickert]

Thanks Jeff. Both are fixed, each with a failing test committed ahead of the fix. The CoNLL-U reader now skips a sentence whose relation column is the underscore placeholder, the same way it already skipped an underscore head. The feedforward trainer no longer lets a tag or label spelled like a reserved symbol take over that symbol's embedding row; it shares the row, as words already did, so the model reloads.

On held-out accuracy: I added UniversalDependencyParserEval in opennlp-eval-tests, following the AbstractEvalTest and OPENNLP_DATA_DIR pattern. It trains both parsers on the UD 2.0 English train split and pins the dev-split scores: the transition parser at UAS 0.8182 and LAS 0.7861, the feedforward parser at UAS 0.8413 and LAS 0.8174, on 25,148 words. The numbers reproduce exactly across runs. It replaces the property-gated evaluation test that lived in opennlp-formats.

[krickert]

Eval build for this head (afd68fb) is running on Jenkins: https://ci-builds.apache.org/job/OpenNLP/job/eval-tests-configurable/74/ (eval-tests-configurable, branch OPENNLP-547-dependency-parser). It uses the opennlp-data.zip published on nightlies today and includes the new UniversalDependencyParserEval, so expect it to take longer than a regular eval run.

[krickert]

Evals passed for this, going to re-run it as I added double precision scoring.

[rzo1]

Evals passed for this, going to re-run it as I added double precision scoring.

Moved to draft for now. Please re-open as ready for review once all your additional checks and stuff are done.

[krickert]

https://ci-builds.apache.org/job/OpenNLP/job/eval-tests-configurable/76/console

Eval build is running

[krickert]

Eval ran successfully

[rzo1]

Thanks. jzonthemtn's two issues are fixed with tests, and the eval pattern is right.

My main concern is the model. FeedforwardDependencyModel is not a BaseModel: it has its own binary format (ONLP-FFDP-1) with no manifest, language, version check or artifact serializer. Once released we have to support that format. The same goes for the rest of the component shape: training takes a Settings record instead of TrainingParameters, and there's no DependencyParserFactory extends BaseToolFactory, no CLI tools, and no stream factory registration. Every other component in OpenNLP works this way, and I don't want a second model format with 3.0.0.

I'd suggest splitting it:

1. This PR: API (DependencyParser, DependencySample, DependencyGraph), the CoNLL-U reader, evaluator and arc-standard parts, aligned with the other components (Sample, serialVersionUID, TrainingParameters, factory, CLI).
2. Follow-up: the feedforward model as a BaseModel, with a new eval run.

If you'd rather keep it in one PR, the model format has to be settled before we look at the details.

Other points:

• OPENNLP-547 has no fix version. Please set 3.0.0.
• There are 147 commits, some without a JIRA key. We will squash-merge.
• Duplication, see inline comments.
• dev/download-ud-treebank.sh, its README and UdTreebankDownloadScriptTest: the build doesn't use the script, since the eval reads from opennlp-data, and the unit test starts bash. Please remove all three and the reference in dependency.xml.

[rzo1]

@krickert Can you fixed up this branch plz? It has 147 commits and I don't think they belong here.

[krickert]

R4R @rzo1