Skip to content

feat(elt-pipelines): Port opralogweb pipeline to elt - #414

Open
WHTaylor wants to merge 2 commits into
mainfrom
321-port-opralogweb-pipeline
Open

feat(elt-pipelines): Port opralogweb pipeline to elt#414
WHTaylor wants to merge 2 commits into
mainfrom
321-port-opralogweb-pipeline

Conversation

@WHTaylor

Copy link
Copy Markdown
Contributor

ref #321

Ports the opralogweb pipeline to elt-pipelines.

@WHTaylor
WHTaylor requested a review from a team as a code owner July 29, 2026 10:42
@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

Adds an OPRA logbook database extractor with table watermarking, merge/upsert configuration, entry-linked related-column queries, HTML-to-Markdown comment conversion, and optional runtime dependencies.

Changes

OPRA logbook extraction

Layer / File(s) Summary
Resource extraction contract
elt-pipelines/facility_ops/ingest/accelerator/opralogweb/opralogweb.py
Defines append-only watermark mappings and configures Entries and MoreEntryColumns extraction properties.
Entry-linked processing
elt-pipelines/facility_ops/ingest/accelerator/opralogweb/opralogweb.py, elt-pipelines/pyproject.toml
Converts entry comments to Markdown, tracks EntryId values, batches related-column queries, and adds the required optional dependencies.

Possibly related PRs

Suggested reviewers: martyngigg, ambolt314, bashanlam

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarises the main change: porting the opralogweb pipeline into elt-pipelines.
Description check ✅ Passed The description directly matches the change by stating the opralogweb pipeline is being ported to elt-pipelines.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@elt-pipelines/facility_ops/ingest/accelerator/opralogweb/opralogweb.py`:
- Around line 32-55: Update the resource generator around the Entries and
MoreEntryColumns ResourceProperties so each deferred extractor opens and owns
its database connection when invoked, rather than capturing conn from the
enclosing with block. Ensure _extract_entries and _extract_more_entry_columns
execute while their respective connections remain open, and remove the outer
connection scope that closes before iteration.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8d27adaa-669b-479e-a598-603701455a60

📥 Commits

Reviewing files that changed from the base of the PR and between e2558eb and 4a6c769.

⛔ Files ignored due to path filters (1)
  • elt-pipelines/uv.lock is excluded by !**/*.lock
📒 Files selected for processing (2)
  • elt-pipelines/facility_ops/ingest/accelerator/opralogweb/opralogweb.py
  • elt-pipelines/pyproject.toml

@WHTaylor

WHTaylor commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Forgot to add, env variables should be:

  • OPRALOGWEB__DATABASE, OPRALOGWEB__HOST, OPRALOGWEB__PASSWORD, and OPRALOGWEB__USERNAME with the values from the opralogweb secrets
  • OPRALOGWEB__DRIVERNAME=mssql+pymssql

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant