Skip to content

feat: add auto-commit option#130

Open
andrewklatzke wants to merge 2 commits intoaklatzke/AIC-2154/optimize-for-durationfrom
aklatzke/AIC-1793/output-name-option
Open

feat: add auto-commit option#130
andrewklatzke wants to merge 2 commits intoaklatzke/AIC-2154/optimize-for-durationfrom
aklatzke/AIC-1793/output-name-option

Conversation

@andrewklatzke
Copy link
Copy Markdown
Contributor

@andrewklatzke andrewklatzke commented Apr 9, 2026

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions

Describe the solution you've provided

Implements the "auto commit" functionality that allows you to push a variation back to LaunchDarkly when the optimization is finished. "Auto commit" is assumed true when using the from_config method, but must be passed explicitly if just passing options (open to changing that).

Describe alternatives you've considered

This is pushing the optimized variation back to LaunchDarkly. No alternatives considered here.

Additional context

This implements the following:

  • When auto_commit is passed as true, will automatically attempt to write the generated variation back to LD. The project key and api key are required for this, so there is validation to ensure they're present if auto commit is passed (before any LLM executions)
  • Since the optimization needs to insert a valid variation, it needs a way to fetch model configs and ensure we're conforming to the provider.modelName keys. Adds a method to fetch the model configs from the API to achieve this
  • For the from_options path, adds the baseURL so that we have 1:1 with the from_config path that allows you to specify the environment URL
  • Adds the coolname dependency which generates random names in the format requested in the PRD (ie, heroku-style naming adjective-noun)
  • The user can also specify an output_key field and it will attempt to write with that key. In cases where the key is already taken, a random suffix is generated and appended to the key
  • Very naive/basic retry logic on the variation create endpoint that will try up to 3 times
  • Adds 3 methods to the ld_api_client (get variation - for checking key validity, list model configs - for finding the correct provider/model combo, create variation - self explanatory)

Here's an example output for the following options for the from_options method:

# ...
        auto_commit=True,
        project_key="default",
        output_key="winning-variation",         
        base_url="https://ld-stg.launchdarkly.com/"
# ...
Screenshot 2026-04-09 at 12 46 19 PM

And here is one with an auto-generated name (run via the from_config method):

Screenshot 2026-04-09 at 12 06 09 PM

Note

Medium Risk
Adds new write-path behavior that can create AI Config variations via the LaunchDarkly REST API, so misconfiguration (API key/project key/base_url) or API failures could cause unexpected side effects or retries.

Overview
Adds an optional auto_commit capability to optimization runs so a successful result can be written back to LaunchDarkly as a new AI Config variation (with optional output_key, collision handling, and basic retry).

Updates the options dataclasses to expose auto_commit, project_key, output_key, and base_url (and defaults auto_commit=True for optimize_from_config), and tracks last-run success in the client to decide whether to commit.

Extends LDApiClient with AI Config endpoints (get_ai_config, get_model_configs, create_ai_config_variation) and adds the coolname dependency to generate default variation keys.

Reviewed by Cursor Bugbot for commit 4cb8859. Bugbot is set up for automated code reviews on this repo. Configure here.

@andrewklatzke andrewklatzke requested a review from a team as a code owner April 9, 2026 20:50
@andrewklatzke andrewklatzke marked this pull request as draft April 9, 2026 20:54
@andrewklatzke andrewklatzke marked this pull request as ready for review April 9, 2026 20:56
@andrewklatzke andrewklatzke requested a review from jsonbailey April 9, 2026 20:56
import uuid
from typing import Any, Dict, List, Literal, Optional, Union

from coolname import generate_slug
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any more popular libraries we could use here? Do we need it to be name based?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants