Allow getting bench from a file in the repo#335
Conversation
4860771 to
21a115e
Compare
Currently benches are stored in commits which means: * Every commit needs to include a bench number, even ones that clearly can't change the bench (e.g. typo fixes, documentation updates, etc.). With the bench stored in a file, the last bench is carried across by default. * Commit messages need to be parsed to extract bench With the bench stored in a file, the contents can be used directly with no need to parse it. At least for myself, I've found this makes automation around bench easier. For example, a CI job that checks the engine's bench matches currently needs to replicate OB's parsing logic but could, with these changes, just compare directly with the contents of the file. Similarly, updating can be done by just piping into the file. * Bench changes aren't included in diffs This is a small one, but I have found it helpful to be able to see, without referencing previous commits, if a given diff changed bench. I've had this patched into my own instance for roughly a year now and have preferred the experience.
21a115e to
bfdcb74
Compare
|
I don't have any particular desire to do that. Easy enough to push commits and auto append the bench. Anytime I push a commit locally, the engine gets built, benched, and that gets appended to the commit message. |
|
For what it's worth I was motivated to file this after seeing Discord messages about people having forgotten to add bench to their most recent commit. So even though you've got that automation, it might help others who haven't. But fair enough - as I said it's just something I've personally found helpful so I thought it was at least worth seeing if there was any appetite for adding it as an option. If you're not convinced, feel free to close this. |
|
If one forgets to put a bench in a commit, can't they also forget to update a local file? Can just put benches into OpenBench as well without deferring to a commit. Generally useful for NNUE tweaks. |
The commit in question added a license file which couldn't have changed bench, so adding bench to the commit in this case (in my view at least) is superfluous. Same goes for README updates, typo fixes, etc. For any of these commits, there'd be no need to touch the file. |
Currently benches are stored in commits which means:
Every commit needs to include a bench number, even ones that clearly can't change the bench (e.g. typo fixes, documentation updates, etc.).
With the bench stored in a file, the last bench is carried across by default.
Commit messages need to be parsed to extract bench
With the bench stored in a file, the contents can be used directly with no need to parse it.
At least for myself, I've found this makes automation around bench easier. For example, a CI job that checks the engine's bench matches currently needs to replicate OB's parsing logic but could, with these changes, just compare directly with the contents of the file. Similarly, updating can be done by just piping into the file.
Bench changes aren't included in diffs
This is a small one, but I have found it helpful to be able to see,
without referencing previous commits, if a given diff changed bench.
I've had this patched into my own instance for roughly a year now and have preferred the experience, so thought I would see if there is any appetite for getting it merged upstream.