Skip to content

Allow getting bench from a file in the repo#335

Open
jgilchrist wants to merge 1 commit into
AndyGrant:masterfrom
tcheran-chess:bench-from-file
Open

Allow getting bench from a file in the repo#335
jgilchrist wants to merge 1 commit into
AndyGrant:masterfrom
tcheran-chess:bench-from-file

Conversation

@jgilchrist

@jgilchrist jgilchrist commented Jun 17, 2026

Copy link
Copy Markdown

Currently benches are stored in commits which means:

  • Every commit needs to include a bench number, even ones that clearly can't change the bench (e.g. typo fixes, documentation updates, etc.).

    With the bench stored in a file, the last bench is carried across by default.

  • Commit messages need to be parsed to extract bench

    With the bench stored in a file, the contents can be used directly with no need to parse it.

    At least for myself, I've found this makes automation around bench easier. For example, a CI job that checks the engine's bench matches currently needs to replicate OB's parsing logic but could, with these changes, just compare directly with the contents of the file. Similarly, updating can be done by just piping into the file.

  • Bench changes aren't included in diffs

    This is a small one, but I have found it helpful to be able to see,
    without referencing previous commits, if a given diff changed bench.

I've had this patched into my own instance for roughly a year now and have preferred the experience, so thought I would see if there is any appetite for getting it merged upstream.

Currently benches are stored in commits which means:

* Every commit needs to include a bench number, even ones that clearly
  can't change the bench (e.g. typo fixes, documentation updates, etc.).

  With the bench stored in a file, the last bench is carried across by
  default.

* Commit messages need to be parsed to extract bench

  With the bench stored in a file, the contents can be used directly
  with no need to parse it.

  At least for myself, I've found this makes automation around bench
  easier. For example, a CI job that checks the engine's bench matches
  currently needs to replicate OB's parsing logic but could, with these
  changes, just compare directly with the contents of the file.
  Similarly, updating can be done by just piping into the file.

* Bench changes aren't included in diffs

  This is a small one, but I have found it helpful to be able to see,
  without referencing previous commits, if a given diff changed bench.

I've had this patched into my own instance for roughly a year now and
have preferred the experience.
@AndyGrant

Copy link
Copy Markdown
Owner

I don't have any particular desire to do that. Easy enough to push commits and auto append the bench. Anytime I push a commit locally, the engine gets built, benched, and that gets appended to the commit message.

@jgilchrist

jgilchrist commented Jun 17, 2026

Copy link
Copy Markdown
Author

For what it's worth I was motivated to file this after seeing Discord messages about people having forgotten to add bench to their most recent commit. So even though you've got that automation, it might help others who haven't.

But fair enough - as I said it's just something I've personally found helpful so I thought it was at least worth seeing if there was any appetite for adding it as an option.

If you're not convinced, feel free to close this.

@AndyGrant

Copy link
Copy Markdown
Owner

If one forgets to put a bench in a commit, can't they also forget to update a local file? Can just put benches into OpenBench as well without deferring to a commit. Generally useful for NNUE tweaks.

@jgilchrist

Copy link
Copy Markdown
Author

If one forgets to put a bench in a commit, can't they also forget to update a local file?

The commit in question added a license file which couldn't have changed bench, so adding bench to the commit in this case (in my view at least) is superfluous. Same goes for README updates, typo fixes, etc. For any of these commits, there'd be no need to touch the file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants