Skip to content

Benchmarks Update: Generation testing Scoring Rework - #1717

Merged
jlarson4 merged 3 commits into
devfrom
bug/p4-scoring-rework
Aug 22, 2026
Merged

Benchmarks Update: Generation testing Scoring Rework#1717
jlarson4 merged 3 commits into
devfrom
bug/p4-scoring-rework

Conversation

@jlarson4

Copy link
Copy Markdown
Collaborator

Description

The system is now sensitive to multilingual models, encoder-decoder models, code models, Chat models, etc. The goal is to improve the score of the generation test, to give a more accurate read of the models output

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

@jlarson4
jlarson4 merged commit 79ae2f4 into dev Aug 22, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant