docs: dev note for workflow chaining#775
Conversation
|
Fern preview: https://nvidia-preview-pr-775.docs.buildwithfern.com/nemo/datadesigner
|
andreatgretel
changed the title
| The new part is the boundary. Because the boundary has a name, you can stop there, inspect the selected output, replace it with an approved artifact, and resume downstream without rerunning the trusted upstream work. | ||
|
|
||
| ```python | ||
| from data_designer.interface import ResumeMode |
There was a problem hiding this comment.
this makes me wonder if ResumeMode should be moved to config so we can just do dd.ResumeMode ....
|
|
||
| In that shape, `quality_gate` is both a normal stage and a contract. Upstream work promises a schema. Downstream work consumes that schema. A reviewer, evaluator, dashboard, or cleanup script only has to preserve the contract. | ||
|
|
||
|  |
There was a problem hiding this comment.
Something seems off here....
| The demo task is document field extraction: generate synthetic invoices and forms, propose boxes around the fields to extract, and turn those boxes into structured rows. A weak detector proposes boxes on each page and assigns uncertainty. The workflow pauses at `review_candidates`, where all rows are still present but only the uncertain rows are marked for review. | ||
|
|
||
| The reviewer corrects the proposed boxes for that uncertain slice. They do not relabel the whole dataset, and they do not change the workflow shape. They write a replacement artifact with the same row count and schema, then the downstream stages use human-corrected boxes where they exist and calibrated detector boxes everywhere else. |
There was a problem hiding this comment.
I believe conditional generation could achieve the same goal within a same DD stage? Just calling it out. Might be interesting to offer as an alternative?
Signed-off-by: Andre Manoel <amanoel@nvidia.com>
Greptile SummaryThis PR adds a workflow-chaining docs story and runnable review-gate recipe. The main changes are:
|
| Filename | Overview |
|---|---|
| docs/assets/recipes/workflow_chaining/document_review_gate.py | Adds the runnable workflow-chaining recipe; repeat runs can reuse stale metadata, and zero-record CLI input reaches a workflow setup error. |
| packages/data-designer/tests/docs/test_document_review_gate_recipe.py | Adds happy-path tests for page generation, review artifact creation, and final dataset export. |
| fern/versions/latest/pages/devnotes/posts/annotate-hard-pages-review-gates.mdx | Adds the workflow-chaining dev note with images and a link to the recipe. |
| fern/versions/latest/pages/recipes/workflow_chaining/document_review_gate.mdx | Adds the Document Review Gate recipe page and download link. |
| fern/versions/latest.yml | Adds navigation entries for the new dev note and recipe. |
| fern/docs.yml | Adds a redirect from the older dev-note path to the new page. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
A[Generate or load page metadata] --> B[Run to review_candidates]
B --> C[Export review candidates]
C --> D[Write reviewed artifact]
D --> E[Resume with stage output override]
E --> F[Calibrate extractor]
F --> G[Extract remaining rows]
G --> H[Export final dataset]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart LR
A[Generate or load page metadata] --> B[Run to review_candidates]
B --> C[Export review candidates]
C --> D[Write reviewed artifact]
D --> E[Resume with stage output override]
E --> F[Calibrate extractor]
F --> G[Extract remaining rows]
G --> H[Export final dataset]
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 2
docs/assets/recipes/workflow_chaining/document_review_gate.py:219-222
**Cached Metadata Ignores Run Arguments**
When the same artifact directory is reused without `--overwrite`, this branch returns the existing metadata parquet without checking the requested `count` or `seed`. A user can run once with 12 records, rerun with `--num-records 24`, and still get a final dataset with the old 12 rows because `build_workflow()` sizes the workflow from the cached dataframe.
### Issue 2 of 2
docs/assets/recipes/workflow_chaining/document_review_gate.py:790
**Zero Records Crash Workflow Setup**
`--num-records 0` is accepted by the CLI and reaches `build_workflow()`, where the generated page dataframe has length 0 and the first stage is added with `num_records=0`. The workflow API rejects that value, so this advertised runnable recipe exits with an engine error instead of handling the CLI input at the recipe boundary.
Reviews (1): Last reviewed commit: "docs: clarify workflow chaining value" | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if path.exists() and not force: | ||
| df = pd.read_parquet(path) | ||
| validate_metadata_rows(df) | ||
| return df |
There was a problem hiding this comment.
When the same artifact directory is reused without --overwrite, this branch returns the existing metadata parquet without checking the requested count or seed. A user can run once with 12 records, rerun with --num-records 24, and still get a final dataset with the old 12 rows because build_workflow() sizes the workflow from the cached dataframe.
Context Used: Do not suggest defensive coding patterns such as a... (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: docs/assets/recipes/workflow_chaining/document_review_gate.py
Line: 219-222
Comment:
**Cached Metadata Ignores Run Arguments**
When the same artifact directory is reused without `--overwrite`, this branch returns the existing metadata parquet without checking the requested `count` or `seed`. A user can run once with 12 records, rerun with `--num-records 24`, and still get a final dataset with the old 12 rows because `build_workflow()` sizes the workflow from the cached dataframe.
**Context Used:** Do not suggest defensive coding patterns such as a... ([source](greptile.json))
How can I resolve this? If you propose a fix, please make it concise.| def build_arg_parser() -> argparse.ArgumentParser: | ||
| parser = argparse.ArgumentParser(description="Headless workflow chaining review gate recipe.") | ||
| parser.add_argument("--model-alias", default="unused", help="Accepted for recipe runner compatibility.") | ||
| parser.add_argument("--num-records", type=int, default=12) |
There was a problem hiding this comment.
--num-records 0 is accepted by the CLI and reaches build_workflow(), where the generated page dataframe has length 0 and the first stage is added with num_records=0. The workflow API rejects that value, so this advertised runnable recipe exits with an engine error instead of handling the CLI input at the recipe boundary.
Context Used: Do not suggest defensive coding patterns such as a... (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: docs/assets/recipes/workflow_chaining/document_review_gate.py
Line: 790
Comment:
**Zero Records Crash Workflow Setup**
`--num-records 0` is accepted by the CLI and reaches `build_workflow()`, where the generated page dataframe has length 0 and the first stage is added with `num_records=0`. The workflow API rejects that value, so this advertised runnable recipe exits with an engine error instead of handling the CLI input at the recipe boundary.
**Context Used:** Do not suggest defensive coding patterns such as a... ([source](greptile.json))
How can I resolve this? If you propose a fix, please make it concise.
|
Thanks for putting this together, @andreatgretel — this is a genuinely nice dev note, and the runnable recipe makes the workflow-chaining story concrete. SummaryThis is a docs + recipe PR: it adds the "Pause, Inspect, Resume" dev note, a headless FindingsSuggestions — Take it or leave it
What Looks Good
Notes
Structural Impact (graphify, 2.4s)Risk: LOW (localized change)
This confirms a localized, additive change with no god-node or cross-package blast radius — consistent with a docs + standalone-recipe PR that imports the public API surface without modifying it. VerdictShip it (with nits) — Only optional suggestions above; nothing blocking. Worth a quick reply to the two existing inline comments before merge, but the change itself is in good shape. This review was generated by an AI assistant. |
2 participants
📋 Summary
Adds a dev note and runnable recipe for workflow chaining as a pause/inspect/resume control surface. The post focuses on what named stage boundaries enable, including human review, reusable intermediate artifacts, downstream resume, and future workflow shapes.
🔗 Related Issue
N/A
🔄 Changes
review_candidates, writes a reviewed artifact, and resumes downstream from that artifact.🧪 Testing
.venv/bin/ruff check ..venv/bin/ruff format --check ..venv/bin/pytest packages/data-designer/tests/docs/test_document_review_gate_recipe.pyenv npm_config_cache=/private/tmp/npm-cache-datadesigner make check-fern-docs(0 errors, 1 existing Fern warning)✅ Checklist