Skip to content

feat: create sampling preset for python code generation#1265

Open
akihikokuroda wants to merge 2 commits into
generative-computing:mainfrom
akihikokuroda:issue1123
Open

feat: create sampling preset for python code generation#1265
akihikokuroda wants to merge 2 commits into
generative-computing:mainfrom
akihikokuroda:issue1123

Conversation

@akihikokuroda

@akihikokuroda akihikokuroda commented Jun 12, 2026

Copy link
Copy Markdown
Member

Pull Request

Issue

Fix: #1123

Description

This PR implements issue #1123 by creating reusable sampling presets that combine Mellea requirements with optimized feedback strategies for Python code generation tasks.

What's included

New feedback strategy system (mellea/stdlib/sampling/feedback.py):

  • PythonCodeFeedbackStrategy — converts validation failures (syntax errors, import restrictions, output size limits) into targeted repair instructions the model can act on
  • Structured feedback messages that include the error, its context, and concrete next steps
  • Integration with the existing interactive validation repair loop

Sampling presets (mellea/stdlib/sampling/presets.py):

  • python_code_generation_preset() — factory for general Python code generation (syntax validation, execution safety, size constraints)
  • python_plotting_preset() — specialized factory for matplotlib/visualization code (adds plotting-specific requirements like figure closure and size limits)
  • Both combine requirements + feedback strategy into a ready-to-use Sampling object

Examples:

  • docs/examples/sampling/python_code_generation_preset.py — demonstrates the general preset and feedback flow
  • docs/examples/sampling/python_plotting_preset.py — shows specialized plotting preset with repair strategies

Tests (test/stdlib/sampling/test_presets.py):

  • Feedback generation for each error type
  • Preset factory configuration
  • Integration with the validation repair loop

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code was added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

  • AI coding assistants used

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

  • Component
  • Requirement
  • Sampling Strategy
  • Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
@akihikokuroda akihikokuroda requested a review from a team as a code owner June 12, 2026 22:43
@github-actions github-actions Bot added the enhancement New feature or request label Jun 12, 2026
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Requirements Library: Create sampling preset for Python code generation

1 participant