chore(operators): Add proper descriptions to text source fields#6804
chore(operators): Add proper descriptions to text source fields#6804Neilk1021 wants to merge 11 commits into
Conversation
Automated Reviewer SuggestionsBased on the
|
|
@mengw15 Could you please look over this when you get the chance? |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #6804 +/- ##
=========================================
Coverage 77.30% 77.30%
Complexity 3524 3524
=========================================
Files 1161 1161
Lines 45922 45922
Branches 5101 5101
=========================================
Hits 35501 35501
Misses 8840 8840
Partials 1581 1581
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
✅ No material benchmark regressions detected🟢 6 better · 🔴 0 worse · ⚪ 9 noise (<±5%) · 0 without baseline
Baseline detailsLatest main
Raw CSVconfig_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,346.83,200,128000,577,0.352,16617.12,24204.29,24204.29
1,100,10,64,20,1634.49,2000,1280000,1224,0.747,79420.03,105424.47,105424.47
2,1000,10,64,20,13931.84,20000,12800000,1436,0.876,702617.91,722343.41,722343.41 |
|
Good direction. One consistency note: the existing ~100 @JsonPropertyDescriptions are short declarative phrases ("Data column for box plot"), whereas these are questions / multi-sentence ("What is the name of the column…?"). Worth aligning to the short-phrase style and keeping only the ones that add real info (the per-line vs whole-file note on attributeType is useful; "Attribute Name" probably doesn't need a sentence). @aglinxinyuan for input. |
|
Please fix the format of the file. |
There was a problem hiding this comment.
Pull request overview
This PR improves operator documentation for the text source operators by adding richer field-level descriptions (via @JsonPropertyDescription) and updating the property editor styling so multi-line descriptions render cleanly in the Angular UI.
Changes:
- Added
@JsonPropertyDescriptionannotations toTextSourceOpDesc(attribute type/name, limit, offset) andTextInputSourceOpDesc(text input). - Tweaked the property editor SCSS to preserve newlines in form “extra” text (
.ant-form-item-extra) rendered vianzExtra. - Updated some field titles (e.g., “Limit (lines)”, “Offset (lines)”) to be more explicit.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| frontend/src/app/workspace/component/property-editor/operator-property-edit-frame/operator-property-edit-frame.component.scss | Preserve newlines and improve spacing for multi-line property descriptions in the form UI. |
| common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/source/scan/text/TextSourceOpDesc.scala | Add field descriptions/titles for shared text source properties (attribute type/name, limit, offset). |
| common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/source/scan/text/TextInputSourceOpDesc.scala | Add a description for the text input field shown in the operator property editor. |
Comments suppressed due to low confidence (1)
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/source/scan/text/TextSourceOpDesc.scala:66
- Same as above for Offset: "large binary" is a single-tuple type but the HideAnnotation regex does not hide Offset for it, even though the executor ignores offset/limit when attributeType.isSingle.
@JsonPropertyDescription("Number of lines to skip from the start before reading. " +
"(Not used when reading the whole file as one row.)")
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Neil Ketteringham <53205839+Neilk1021@users.noreply.github.com>
|
The deferred To recover: re-run the failed jobs in the run above, or manually dispatch the Codecov Upload workflow with |
Yicong-Huang
left a comment
There was a problem hiding this comment.
LGTM with comments on the description itself.
What changes were proposed in this PR?
Added descriptions to fields in
TextSourceOpDescandTextInputSourceOpDesc. Currently these operators suffer heavily from a lack of information.Any related issues, documentation, discussions?
Closes #6802. Related idea in #6798
How was this PR tested?
Deployed changes locally via docker and
./bin/local-dev.sh upWas this PR authored or co-authored using generative AI tooling?
No.