Skip to content

fix(implications): move per-call target to end of prompt to enable prompt caching - #61

Open
fashionmascherine-svg wants to merge 1 commit into
chainstacklabs:mainfrom
fashionmascherine-svg:fix/implications-prompt-caching
Open

fix(implications): move per-call target to end of prompt to enable prompt caching#61
fashionmascherine-svg wants to merge 1 commit into
chainstacklabs:mainfrom
fashionmascherine-svg:fix/implications-prompt-caching

Conversation

@fashionmascherine-svg

@fashionmascherine-svg fashionmascherine-svg commented Jul 15, 2026

Copy link
Copy Markdown

Problem

IMPLICATION_PROMPT places the per-call {target_title} before the large, run-static {group_titles_text} block and the instruction body. Prompt caching keys on the longest common prefix, so a target that changes every call makes the whole ~35–44k-token static section uncacheable — every implications call is billed full input.

Fix

Move the ## TARGET EVENT block to the end of the prompt (after the available-events list and the instructions). The static prefix (events list + instructions) becomes a stable, cacheable prefix across all calls in a run. No change to the task semantics or output schema.

Measured (DeepSeek deepseek-chat / v4-flash)

Same input, controlled before/after run:

before after
input cache-hit ratio ~0.2% ~40% aggregate (~80% on the implications step alone)
run cost (57-group tag) baseline ~−40%
  • Larger tags benefit more (the static prefix scales with the number of groups).
  • Output unchanged in order of magnitude — identical coverage distribution on the re-run.
  • The validation prompt is untouched, which is why the aggregate cache-hit sits below the per-step figure.

Diff

Two hunks in backend/core/steps/implications.py:

 IMPLICATION_PROMPT = """Find ONLY logically necessary relationships between prediction market events.
 
-## TARGET EVENT:
-"{target_title}"
-
 ## AVAILABLE EVENTS:
 {group_titles_text}
@@ ## CRITICAL RULES:
 5. When in doubt, LEAVE IT OUT
+
+## TARGET EVENT
+Find the necessary relationships between the TARGET event below and the AVAILABLE EVENTS listed above. Use the exact titles from the AVAILABLE EVENTS list.
+
+TARGET: "{target_title}"
+
+Output ONLY the JSON described in OUTPUT FORMAT above.
 """

Summary by CodeRabbit

  • Improvements
    • Enhanced market-event implication analysis with clearer relationship-matching instructions.
    • Improved precision by requiring exact event titles and strictly formatted JSON output.

IMPLICATION_PROMPT places the per-call {target_title} before the large, run-static {group_titles_text} block and the instructions. Prompt caching keys on the common prefix, so the changing target at the top makes the ~35-44k static tokens uncacheable and every call is billed full input (~0.2% cache hit measured).

Moving the target block to the end makes the static prefix cacheable. Measured on DeepSeek v4-flash: cache-hit 0.2% -> ~40% aggregate (~80% on the implications step), ~40% lower run cost on a small tag; larger tags benefit more. Output unchanged in order of magnitude (coverage identical).
@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: bf46ddbc-84d9-4877-bbd7-e5e1ec9f243d

📥 Commits

Reviewing files that changed from the base of the PR and between 65aea7a and 2c813ab.

📒 Files selected for processing (1)
  • backend/core/steps/implications.py

📝 Walkthrough

Walkthrough

The implication prompt is reorganized so target-event instructions appear later, require exact titles from available events, use a stricter target format, and require only the specified JSON output.

Changes

Implication prompt update

Layer / File(s) Summary
Target event instruction flow
backend/core/steps/implications.py
The earlier target-event block is removed, and a later section adds exact-title relationship instructions, a tightened target placeholder, and JSON-only output requirements.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Suggested reviewers: smypmsa

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: moving the per-call target block to the end of the prompt to improve caching.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant