feat: add lettucedetect CLI entry point (#47)#59
Conversation
- Add lettucedetect/cli.py with argparse-based CLI wrapping HallucinationDetector - Support --model, --context, --answer, --question, --format, --method flags - Context and answer accept file paths, literal strings, or '-' for stdin - Multi-passage context split on blank lines - Register lettucedetect = lettucedetect.cli:main in pyproject.toml - Add tests/test_cli_pytest.py with 35 unit tests Closes KRLabsOrg#47
adaamko
left a comment
adaamko
left a comment
There was a problem hiding this comment.
Thanks — the CLI shape is right (entry point wiring, file/stdin/literal inputs, exit codes), but three things block merge:
-
The test suite fails as shipped (16/35).
tests/test_cli_pytest.pypatcheslettucedetect.cli.HallucinationDetector, butcli.pyimports it lazily insidemain(), so the attribute never exists on the module andmock.patchraisesAttributeError. Patchlettucedetect.models.inference.HallucinationDetectorinstead (works with the lazy import), and please run the suite before pushing. -
--method llmcan't work.HallucinationDetector(method="llm", model_path=...)—LLMDetectortakesmodel=, notmodel_path=; the unknown kwarg gets forwarded to the OpenAI client and TypeErrors, and--modelwould be silently ignored anyway. Either scope the CLI totransformer(matches #47) or map the flag per method. -
Lint/format.
ruff checkfails (RUF100 on the two# noqa: BLE001lines, RUF005 in the tests) andruff format --checkwants to reformat both files; both are missing trailing newlines.
Happy to merge once tests pass and lint is clean.
|
Heads-up: the repo now requires a contribution-rights attestation on PRs (new required CI check). When you push the fixes, please add this line to your PR description and tick it:
|
- Patch lettucedetect.models.inference.HallucinationDetector instead of lettucedetect.cli.HallucinationDetector in tests, matching the lazy import in main(). Fixes 16/35 failing tests (AttributeError on patch). - Map --model to the correct constructor kwarg per --method: model_path for transformer, model for llm. Previously method=llm always passed model_path=, which forwarded as an unknown kwarg to the OpenAI client and raised TypeError. Add test_llm_method_uses_model_kwarg_not_model_path to cover this path. - Handle non-UTF-8 --context/--answer files: read_text(encoding='utf-8') raises an unhandled UnicodeDecodeError on invalid encoding, surfacing as a raw traceback instead of a clean CLI error. Wrap both _read_source() calls in try/except (OSError, UnicodeDecodeError) -> parser.error(), matching the exit-code-2 convention. Covered by test_exits_on_non_utf8_context_file and test_exits_on_non_utf8_answer_file. - Remove two unused # noqa: BLE001 (BLE isn't in the enabled ruff rule set, so RUF100 flagged them), fix RUF005 list-concat in tests, and run ruff format on both files (missing trailing newlines). - Update --model help text to reflect it now applies to both detection methods, not just transformer. 38 tests pass (Windows/Python 3.14 and Linux/Python 3.12, independently verified). ruff check and ruff format --check both clean.
|
Thanks for the feedback! Fixed all three, plus one more edge case I found while testing:
38 tests pass, ruff check . and ruff format --check . both clean. |
2 participants
Closes #47
Adds a
lettucedetectconsole script wrappingHallucinationDetector.lettucedetect/cli.py--> argparse CLI, supports file/stdin/literal for context and answerpyproject.toml--> register entry point under[project.scripts]tests/test_cli_pytest.py--> 35 unit testsI certify that I have the right to submit this code and that it may be distributed under the repository's MIT license