Skip to content

UN-3636 [DEV] Build images and run e2e tier in one workflow#2151

Merged
chandrasekharan-zipstack merged 40 commits into
mainfrom
feat/un-3636-e2e-build-test
Jul 10, 2026
Merged

UN-3636 [DEV] Build images and run e2e tier in one workflow#2151
chandrasekharan-zipstack merged 40 commits into
mainfrom
feat/un-3636-e2e-build-test

Conversation

@chandrasekharan-zipstack

@chandrasekharan-zipstack chandrasekharan-zipstack commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

What

Enable the e2e test tier in CI and restructure ci-test.yaml into a multi-job
workflow that runs every tier and posts one unified test report.

  • Delete ci-container-build.yaml and ci-test-e2e.yaml (the latter never ran —
    nothing built/pulled images and VERSION was never exported, so every recorded
    run was skipped).
  • ci-test.yaml becomes four jobs in one workflow: changes (path filter) →
    parallel test matrix (unit, integration on testcontainers) + e2e
    (builds SHA-tagged images, boots the full platform via compose) → report
    (needs all three, merges every tier's junit into one critical-path evaluation
    and posts a single sticky PR comment).
  • Add the first real e2e coverage: the auth-login critical path
    (tests/e2e/auth/test_login.py).

Part of UN-3636 / parent UN-3635.

Why

  • Images can't travel across jobs (fresh runner VMs), so the e2e tier must
    build and run on the same runner — the e2e job does both. The rig's boot
    (docker compose up -d --wait + endpoint readiness in tests/rig/runtime.py)
    is a strict superset of the old ci-container-build "any exited containers?"
    check, so that signal is preserved and stronger.
  • One report, not three. e2e coverage (e.g. auth-login) must show up
    alongside unit/integration in the same PR comment. A single report job that
    needs all tiers is the natural aggregation point — it runs last and sees
    everything, so the critical-path table is reconciled once.
  • Keep parallelism + isolation. unit/integration run as parallel matrix legs
    on their own runners (testcontainers); e2e runs on its own runner (compose).
    Merging execution into one job was tried and reverted — it serialised the tiers
    onto one 7 GB VM (~17 min vs ~7 min) and mixed compose + testcontainers infra in
    a single rig invocation, which pointed the backend at the compose DB hostname
    (unreachable from host-side pytest).
  • e2e is not path-gated. docker/** edits don't count as "relevant" for
    unit/integration but must still be e2e-tested, so the e2e job runs on every
    non-draft PR regardless of the changes filter.

How

  • ci-test.yaml (rewritten):
    • changesdorny/paths-filter, job-level (an untriggered workflow reports
      no check; a skipped job passes). Gates only unit/integration.
    • test — matrix [unit, integration], fail-fast: false, testcontainers,
      tox -e <tier> -- --fail-on-critical-gap, uploads reports-<tier>.
    • e2e — Docker login → run-platform.sh -edocker compose -f docker/docker-compose.build.yaml build (VERSION=${{ github.sha }}) →
      tox -e e2e -- --fail-on-critical-gap (UNSTRACT_E2E_RUNTIME=compose,
      UNSTRACT_TEST_VERSION=${{ github.sha }}). Captures compose logs on failure,
      uploads reports-e2e.
    • reportneeds: [changes, test, e2e], downloads reports-*,
      tox -e rig -- report combine, posts one sticky test-results comment,
      manages a single baseline (unstract-test-baseline-main-*, this job is the
      sole reader+writer), and a final "Fail if any tier failed" step makes it the
      single branch-protection gate (red if unit/integration/e2e failed; a
      path-filter skipped test counts as pass).
  • e2e login coverage: tests/e2e/auth/test_login.py (happy-path login +
    bad-credential rejection, @pytest.mark.critical_path("auth-login"));
    tests/e2e/conftest.py adds the authed_session fixture (mock-login → set
    active org, guards an empty org list); tests/groups.yaml adds e2e-login;
    tests/critical_paths.yaml points auth-login at POST /api/v1/login with
    covered_by: [e2e-login], proof marker.
  • Rig plumbing: tests/rig/runtime.py adds shared health_targets() (single
    source for the readiness probe + smoke test, correct per-service health paths);
    tests/e2e/smoke/test_smoke.py checks all configured endpoints;
    tests/rig/cli.py + pyproject.toml add requests to the rig's per-group
    pytest env (e2e HTTP client); tox.ini adds VERSION to passenv so the
    rig's compose subprocess resolves ${VERSION} image tags.
  • tests/compose/docker-compose.test.yaml: comment points at the e2e job
    in ci-test.yaml.

Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)

No production/runtime code is touched — this is a CI + test-harness change. The
build coverage of the old build job is preserved (same
docker-compose.build.yaml, all services). The e2e tier that now runs on PRs
contains only e2e-smoke (rig sentinel + service /health) and e2e-login;
the remaining e2e groups are optional placeholders that skip. Added failure
surface is minimal and deliberate.

Database Migrations

  • None.

Env Config

  • None. (VERSION / UNSTRACT_TEST_VERSION are CI-only, set per-run to the
    commit SHA.)

Relevant Docs

  • tests/README.md, tests/rig/runtime.py, tests/critical_paths.yaml,
    tests/compose/docker-compose.test.yaml

Related Issues or PRs

  • UN-3636 (e2e enablement), parent UN-3635 (make ci-test.yaml a required merge
    gate).

Dependencies Versions

  • None.

Notes on Testing

  • Green run on this PR: all four jobs pass in ~7 min (unit/integration parallel
    to the ~7 min e2e leg; report ~20 s). 728 passed / 0 failed; the unified
    comment reconciles both covered critical paths — auth-login (e2e-login) and
    adapter-register-llm (integration-backend).
  • python -m tests.rig validate → OK.

Branch protection (UN-3635)

The single required check is now report. The old required-check names
(build, and any per-lane test/e2e names) are replaced by it and should be
dropped from main's protection.

Checklist

I have read and understood the Contribution Guidelines.

🤖 Generated with Claude Code

https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU

chandrasekharan-zipstack and others added 26 commits June 30, 2026 15:36
… main runs green

The rig's unit/integration CI job had never passed on main. `--fail-on-critical-gap`
(passed only on the main push) failed the build on EVERY uncovered critical path,
including e2e-only paths run during the unit/integration tier and paths with no test
anywhere. Every group-level failure was already non-gating (optional groups, or exit
5 = no tests collected), so the gap gate was the sole cause of redness.

- Split critical-path gaps into in-scope vs out-of-scope (add `in_scope` to
  CriticalPathStatus; evaluate() already computed it). --fail-on-critical-gap now
  gates only on in-scope gaps — a declared in-tier covering group that didn't run
  green (real coverage regressed). Out-of-scope gaps (covered only by an unrun tier,
  or no declared coverage) are reported + logged but never gate that tier.
- Wire honest coverage that exists today: adapter-register-llm -> unit-sdk1. Gives
  the path teeth: if sdk1 regresses, it flips to an in-scope gap and fails.
- Drop unit-tool-registry group (component slated for removal; can't even collect).
- Delete 3 dead tests referencing removed code: core test_pandora_account.py
  (account_services removed), core test_pubsub_helper.py (LogHelper -> LogPublisher),
  platform-service test_auth_middleware.py (platform_service.main removed; also made
  live Postgres calls). unit-platform-service is green via its hermetic memory-leak
  test; unit-core has no valid tests left and skips as a placeholder.

New self-tests cover the in-scope/out-of-scope split at both evaluate() and cmd_run().

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XJqp7xMdd1kjLUKbvrJsq4
…kend tests

Follow-up cleanup on the rig manifests (UN-3636):

- Park unit-runner and unit-prompt-service (commented out, not run by
  default) with a TODO to delete when those services are removed — both are
  being decommissioned; no value testing components on their way out.
- Drop the unit-tool-registry NOTE block entirely.
- Prune 6 unit-backend paths that collect zero tests (account_v2,
  api_deployment_v2, connector_v2, file_management, project,
  tenant_account_v2 — dirs missing or empty); optional skip-if-missing was
  hiding them and implying coverage that was never written.
- Wire two real backend tests into unit-backend:
    * middleware/test_exception.py — 5 hermetic tests, pass now.
    * prompt_studio/prompt_studio_core_v2/tests — pins the executor
      _handle_ide_index async path (no prompt-service coupling); runs once
      unit-backend is un-gated, skips safely until then.
- Comment out the tool-sandbox-exec critical path (TODO: remove with
  tool-registry/runner) — its covering group unit-runner is now parked.

`python -m tests.rig validate` → OK (13 groups, 9 critical paths).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
…jango setting

unit-core ran 0 tests / 2 collection errors (ModuleNotFoundError: No module
named 'unstract'). Root cause: _prepare_group_env did `uv pip install -e .`
for install_editable groups, but _pytest_command runs `uv run`, which
re-syncs the venv every call and wipes that install before pytest imports the
package — the same hazard the code already flagged for plugins.

Inject the package via `uv run --with-editable <workdir>` (survives the
re-sync, same mechanism as the RIG_PYTEST_PLUGINS `--with` specs) and drop the
wiped `uv pip install -e .`. unit-core now 27 passed, 0 errors.

Also remove DJANGO_SETTINGS_MODULE from the repo-root [tool.pytest.ini_options]:
it's a pytest-django option that warns "Unknown config option" for every
non-Django group, points at a non-existent module (backend.settings.test_cases),
and Django settings don't belong at the polyglot repo root. The rig injects it
per-group via groups.yaml env for unit-backend only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
The unit-backend group pointed at a non-existent settings module
(backend.settings.test_cases) and ran without pytest-django, so the
DB-backed tests errored at collection (Django uninitialised) and the
django_db tests had no test-DB lifecycle. Make the group actually runnable:

- Add pytest-django to the backend test group (bootstraps Django before
  collection; provides the test-DB + django_db fixtures).
- Point DJANGO_SETTINGS_MODULE at the existing backend.settings.test and
  inject the import-time-required settings via the group env — base.py reads
  them before any dotenv load, so they must exist in the process env, not a
  settings module. ENCRYPTION_KEY is an all-zero (valid, zero-entropy) Fernet
  placeholder, not a real secret.
- Set DB_SCHEMA=public: the app's fixed schema doesn't exist in the fresh
  test DB and tenancy is row-level, so migrations run in public.
- Drop workflow_manager/endpoint_v2/tests from the wired paths: its
  destination-connector tests import the enterprise `plugins` package,
  absent in OSS.
- Add the missing utils/file_storage{,/helpers}/__init__.py so those
  modules import as a package under pytest.
- Stop test_build_index_payload's sys.modules stubs from leaking into
  sibling collection: record + restore the originals once the helper is
  imported (a stubbed account_v2.models was breaking other modules'
  real imports).

unit-backend now collects clean; 126 passed, 4 skipped. The remaining 6
failures (usage_v2 helper stubs, dashboard_metrics cleanup tasks) are
pre-existing test bugs tracked separately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
dashboard_metrics: organization FK targets Organization's int PK, but the
tests passed a UUID string as organization_id, and verified rows through
the org-scoped default manager (empty without a UserContext). Create a
real Organization and read via _base_manager.

usage_v2: drop the fragile "stub usage_v2.models into sys.modules before
import" trick — under pytest-django Django imports the real module first,
so the stub never took and the helper hit the DB. Rebind the Usage symbol
the helper resolved instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
The connector suite had 12 reds that needed real external services or
per-developer credentials no one has by default. Two were genuine bugs;
the rest are integration tests masquerading as unit tests.

Fixes (not credential-related):
- mariadb: assertion text drifted from the connector's actual message
  ("SSL SETTINGS", not "ssl-settings").
- sharepoint: skip test_json_schema_has_is_personal — is_personal is read
  from settings in code but was never exposed in json_schema.json (personal
  vs site is inferred from an empty site_url). Whether the schema should
  expose it is a product decision; tracked under UN-3414.

Gating (skipUnless, mirrors the existing SharePoint integration tests):
- filesystems (box, gdrive, minio, pcs, dropbox) already read creds from
  env; add the missing skip guard so they SKIP instead of failing.
- databases (mssql, mysql, postgresql, redshift, snowflake) had hardcoded
  personal creds (incl. a live-looking neon.tech URL and a Snowflake
  account) querying bespoke tables. Move creds to *_TEST_* env vars and
  skip unless provided, removing the secrets from the repo.

CI can run these by injecting the corresponding secrets as env vars in a
dedicated integration job; by default they skip cleanly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
Both now run green and standalone (no external services; integration
tests skip cleanly when credentials are absent), so drop optional: true
to make them blocking merge gates per UN-3635.

unit-backend stays optional until the rig provisions a reachable DB_HOST
for it (UN-3636 follow-up).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
- in_scope defaults False on CriticalPathStatus so a future evaluate()
  regression that forgets it under-gates (warning) rather than over-gates
  (spurious build block). [greptile]
- widen connector integration skip guards (redshift, snowflake, gdrive,
  minio, pcs) to require every env var the test hard-references, so a
  partially configured env skips cleanly instead of failing. [coderabbit]
- usage_v2 test_helper: swap Usage via an autouse monkeypatch fixture
  instead of a module-level rebind that leaks FakeUsage into later tests.
- build_index_payload test: evict the helper module from sys.modules after
  binding it, so later importers in the same process get a real copy.
- drop dead tox `runner` alias (its unit-runner group was removed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
… out of unit

Make `requires_services` actually provision instead of being cosmetic. The rig
now brings up testcontainers infra (Postgres/MinIO) for any runnable group that
declares `requires_services`, and injects connection env into the group's pytest
subprocess (Postgres URL -> discrete DB_* vars; MinIO endpoint/creds). Previously
django_db tests fell back to the compose hostname `backend-db-1`, unreachable
from host-side pytest, so unit-backend had to be `optional`.

Reclassify infra-dependent tests by the rig's own tier taxonomy (unit = no
external services, integration = real infra but not the full platform):

- backend: split unit-backend into pure `unit-backend` (gates unit tier, no
  infra) and `integration-backend` (django_db tests: dashboard_metrics +
  prompt_studio_registry_v2; provisioned Postgres; gates integration tier).
- connectors: marker-based split (tests are interleaved within files). Credential
  + MinIO tests marked `@pytest.mark.integration`; `unit-connectors` runs
  `-m "not integration"`, new `integration-connectors` runs `-m "integration"`.
  test_minio actually runs against provisioned MinIO; external-credential tests
  skip. Skip-guard test_http_fs (was hitting a live URL unguarded in CI).

Both groups are non-optional and gate their tiers. Unit tier stays infra-free.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
…ation, cut rig complexity

- integration-backend declares requires_services: [postgres, redis] but the rig
  only injected Postgres/MinIO env, so Redis-backed tests bypassed the
  testcontainer and hit localhost:6379. Inject REDIS_HOST/PORT +
  CELERY_BROKER_BASE_URL from the provisioned endpoint (CodeRabbit).
- test_http_fs was skip-guarded but unmarked, so the connector marker split
  (-m "not integration") could still run it in the unit tier. Mark the module
  integration (CodeRabbit).
- Extract _inject_infra_env and _pytest_base_cmd to bring both functions under
  SonarCloud's cognitive-complexity threshold. NOSONAR the test's DB_PASSWORD
  placeholder (not a real credential).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
The MinIO endpoint is a throwaway testcontainer with no TLS, so http is
expected. Suppress the SonarCloud insecure-protocol hotspot that otherwise
blocks the quality gate on new code.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
The module stubs adapter_processor_v2.models to import PromptStudioHelper
without the full Django app, but only provided AdapterInstance. The helper
also imports UserDefaultAdapter, so the import failed and all 4 tests in the
module self-skipped via the _IMPORT_ERROR guard. Add the missing stub so the
tests actually execute.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
Removed from the e2e test overlay; the platform brought up for e2e no longer
provisions a standalone prompt-service.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
…r tests

- test_miniofs: drop the permanently-skipped test_s3 (hardcoded AWS S3 smoke).
  MinIO/S3 is covered by test_minio (integration), the TestAccessFilteredS3
  unit tests, and the connectorkit registry check.
- database + filesystem tests: replace print()-in-loop debug with assertions
  (or drop redundant prints) so integration runs don't spam the logs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
…tion tests

unit-backend was a hand-kept file allowlist that had to grow with every new
test dir. Collect the whole backend tree instead and let markers decide:
tests needing live infra carry `@pytest.mark.integration` and are excluded
from the unit tier via `-m "not integration"`.

- register the `integration` marker in backend/pyproject.toml
- mark the two DB-bound suites (dashboard_metrics, prompt_studio_registry_v2)
  that were previously grouped by path only
- add a conftest marking the endpoint_v2 destination-connector subtree
  integration (uses django.test.TestCase -> needs Postgres). Kept out of the
  gating integration-backend group for now: 3 postgres destination tests are
  pre-existing failures and need a skip-guard/fix before they can gate.
- remove the dead SharePoint `test_json_schema_has_is_personal` skip: the
  schema never exposes `is_personal`, so the test only ever asserted-then-
  skipped; drop it rather than carry a permanent skip.

Verified: unit tier 116 passed / 50 deselected; integration-backend collects
its marked suites; rig self-tests 54 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C5HQX5CSoMR6RzHtXcfwJt
…gration API test

Replace the scattered integration-marking (per-file pytestmark, per-app
endpoint_v2 conftest) with a single backend/conftest.py hook that auto-marks
any Django TestCase/TransactionTestCase or django_db item as integration —
tests declare their DB need by how they're written, not a hand-kept marker.

Cover the adapter-register-llm critical path honestly: unit-sdk1 only
exercises the SDK provider classes, not the HTTP endpoint, so map it to a new
integration-backend APITestCase that POSTs /adapter/ (SDK context-window call
mocked, everything else real). Trim comments that would go stale.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ation tier

Follow-up completing 6d61a56, which staged only the adapter API test and the
deleted per-dir conftest, leaving the rest of the batch uncommitted.

- backend/conftest.py: central pytest_collection_modifyitems hook auto-marks
  every Django TestCase/TransactionTestCase/django_db test as `integration`, so
  unit-backend (-m "not integration") and integration-backend (-m integration)
  are exact complements. Drops now-redundant per-app pytestmark in
  dashboard_metrics and prompt_studio_registry_v2.
- critical_paths.yaml: adapter-register-llm now covered_by integration-backend
  (real API test), reverting the earlier unit-sdk1 placeholder mapping.
- groups.yaml: integration-backend gains the adapter API test and the
  destination-connectors DB-writer tests (BE orchestration over the connector
  lib — superset of the connector-lib DB tests). Adds WORKFLOW_EXECUTION_DIR_PREFIX
  to the shared backend test env (ExecutionFileHandler builds paths from it).
- destination-connector postgres test: read DB_SCHEMA (was hardcoded "test"),
  use a lowercase table name (connector lowercases on read-back), drop the
  error-record case (hits a latent product edge: data=None serialized as the
  string 'None' into a jsonb column).
- Delete 7 connector-lib DB tests (databases/test_*_db.py) — superseded by the
  backend DB-writer tests; keep test_sql_safety.py, filesystems, connectorkit.
- rig cli.py / critical_paths.py: comment + docstring cleanups.

Verified (testcontainers Postgres/Redis): unit-backend 116 passed,
integration-backend 24 passed / 26 skipped (external-DB engines skip w/o creds),
unit-connectors 53 passed, rig validate OK (15 groups, 9 paths).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…connector_v2 tests

- groups.yaml: integration-backend now selects paths ["."] with -m integration
  — the exact complement of unit-backend over the same tree. New backend DB
  tests auto-join the group with no manifest edit. Verified equivalent:
  collect-only shows the same 50/166 tests (116 deselected); full run against
  testcontainers Postgres/Redis gives the same 24 passed / 26 skipped as the
  hand-listed paths on CI.
- Delete backend/connector_v2/tests (connector_tests.py, conftest.py, fixture):
  written for the pre-v2 schema — the fixture loads removed models
  (account.org, account.user, project) so loaddata errors immediately, and the
  filename never matched pytest's test_*.py pattern, so these tests have not
  been collected anywhere. Same dead-test cleanup as the rest of this branch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU
The previous version stubbed cross-app imports in sys.modules (only when
absent) — written for a bare no-Django env. Under the rig, pytest-django
boots Django and the real modules are already imported, so the stubs were
skipped and the tests called reset_mock() on real classes (AttributeError).

Same control-flow assertions, now via mock.patch.multiple on the imported
module: no sys.modules mutation, no import-order dependence, safe in a
shared pytest session. Stays in the unit tier (no DB touched).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU
…ss flask check

- test_build_index_payload.py: import the real prompt_studio_helper (Django is
  loaded by the rig env) instead of installing ~30 stub modules in sys.modules
  and restoring them post-import. The clobber/restore dance was import-order
  dependent and silently skipped all tests when the stub surface drifted.
  Tests and assertions unchanged; patches now managed via ExitStack.
- test_legacy_executor_scaffold.py: test_no_flask_import now runs the check in
  a fresh subprocess. The in-place importlib.reload swapped class identities
  under sibling tests, and scanning sys.modules in-process fails if any other
  test imported flask.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU
…compat

Trigger-level paths-ignore means docs/frontend/docker-only PRs never start
the workflow, so no check run is created and a required 'test' status check
would block those PRs forever. A 'changes' job now always runs and gates
'test' via needs + if; a skipped job still reports a check run (skipped =
passing for required checks), while any executed test failure still fails.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TGcZHkM4jbUmrx4YNYewZK
ci-container-build.yaml built + booted the stack, checked nothing exited,
and threw it all away; ci-test-e2e.yaml had e2e plumbing but no images to
run against (VERSION never exported), so it never executed. Merge them:
build once, then let the rig boot its compose project against the just-
built SHA-tagged images and run the e2e tier. up -d --wait plus endpoint
readiness subsumes the old exited-container check.

VERSION joins tox passenv so the rig's compose subprocess can resolve the
${VERSION} image tags of the base compose file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TGcZHkM4jbUmrx4YNYewZK
@coderabbitai

coderabbitai Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

This PR removes separate container-build and e2e workflow files, consolidates CI into one workflow, and adds login-critical-path e2e coverage with shared readiness and authenticated-session helpers.

Changes

CI, e2e login, and health-gate updates

Layer / File(s) Summary
CI workflow consolidation
.github/workflows/ci-test.yaml
Reworks the main CI workflow into one job that runs unit, integration, and e2e tiers sequentially, restores and saves the baseline cache, captures compose logs on failure, and uploads combined reports.
Login gate configuration
pyproject.toml, tests/critical_paths.yaml, tests/groups.yaml, tox.ini, tests/compose/docker-compose.test.yaml
Adds the critical_path(path_id) marker, points auth-login at POST /api/v1/login with covered_by: [e2e-login], defines the new e2e-login group, passes VERSION into tox, and updates the compose test comment about image pinning.
Shared health checks
tests/rig/runtime.py, tests/e2e/smoke/test_smoke.py
Adds shared health-target generation for platform services and rewrites the smoke test to check all configured health endpoints.
Authenticated e2e login tests
tests/e2e/conftest.py, tests/e2e/auth/test_login.py, tests/rig/cli.py
Adds an authenticated session fixture, includes requests in the rig plugin environment, and adds login tests for a usable session and rejected bad credentials.

Estimated code review effort: 4 (Complex) | ~60 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 62.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title matches the main change: merging image build and e2e execution into one workflow.
Description check ✅ Passed The description covers the required template sections and includes the main implementation, impact, testing, and related issue details.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/un-3636-e2e-build-test

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Comment thread .github/workflows/ci-e2e-build-test.yaml Fixed
Sonar/GHAS flag mutable tag refs — a moved tag can silently swap the
action code. Pinned to the newest release within each currently-used
major, so no behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TGcZHkM4jbUmrx4YNYewZK
chandrasekharan-zipstack and others added 2 commits July 8, 2026 15:49
test_no_flask_import, the test_miniofs comment, and the groups.yaml Fernet
comment were intentionally dropped or shortened in #2115; keep main's version
rather than the e2e branch's re-additions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU
Add the first real e2e critical-path test: OSS mock-login over real HTTP.

- tests/e2e/auth/test_login.py: POST form creds to /api/v1/login, assert 302 +
  sessionid cookie, confirm the session is usable via /api/v1/session; a
  negative test asserts bad creds don't redirect. Marked
  @pytest.mark.critical_path("auth-login").
- New non-optional e2e-login group (gates on failure) covering auth-login with
  proof: marker. Fix the path's stale entry (/api/v1/auth/login -> /api/v1/login).
- Shared authed_session fixture (login + org handshake, session-scoped) so
  future e2e tests don't re-implement auth.
- Extend the smoke gate to health-check every service, and fix _wait_ready:
  it probed the removed prompt-service (connection refused -> guaranteed 300s
  TimeoutError) and used wrong paths for runner/x2text. Both now use a single
  health_targets() source of truth.
- Run --fail-on-critical-gap on e2e PRs too (out-of-scope gaps stay warn-only);
  --update-baseline remains main-only. Matches the unit/integration lane.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU
@chandrasekharan-zipstack chandrasekharan-zipstack marked this pull request as ready for review July 9, 2026 08:45

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
tests/e2e/conftest.py (1)

55-65: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider failing fast on a missing csrftoken cookie.

The fallback session.cookies.get("csrftoken", "") silently sends an empty X-CSRFToken header if the GET to /api/v1/organization didn't seed the cookie. The subsequent resp.raise_for_status() would then surface a 403, but the root cause (missing csrftoken) is obscured. An explicit assert on the cookie presence would produce a clearer failure message.

♻️ Suggested refactor
     # GET /organization sets the csrftoken cookie and returns the org id, which
     # the following POST needs both as a cookie and echoed in the CSRF header.
     orgs = session.get(f"{base}/api/v1/organization", timeout=10)
     orgs.raise_for_status()
     org_id = orgs.json()["organizations"][0]["id"]
+    csrftoken = session.cookies.get("csrftoken")
+    assert csrftoken, "GET /organization did not set csrftoken cookie"
     resp = session.post(
         f"{base}/api/v1/organization/{org_id}/set",
-        headers={"X-CSRFToken": session.cookies.get("csrftoken", "")},
+        headers={"X-CSRFToken": csrftoken},
         timeout=10,
     )
     resp.raise_for_status()
     return session
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/e2e/conftest.py` around lines 55 - 65, Fail fast when the CSRF cookie
is missing in the organization setup flow. In tests/e2e/conftest.py, after
session.get to /api/v1/organization, explicitly verify the csrftoken cookie
exists before calling session.post in the organization setup sequence, rather
than defaulting to an empty X-CSRFToken header. Use the existing setup logic
around orgs, org_id, and session.cookies.get to locate the change and make the
failure message clearly indicate the missing csrftoken.
.github/workflows/ci-e2e-build-test.yaml (1)

83-88: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low value

Consider using an env var instead of direct ${{ github.ref }} interpolation in shell.

zizmor flags this as a potential template-injection vector. While github.ref is GitHub-controlled and not attacker-controllable (it's refs/heads/<branch> for pushes and refs/pull/<number>/merge for PRs), using an environment variable is the recommended hardening pattern and silences the warning.

♻️ Proposed refactor
       - name: Run e2e tier (rig boots the platform via compose)
         env:
           UNSTRACT_E2E_RUNTIME: compose
+          GITHUB_REF: ${{ github.ref }}
         # Gap check on PRs too: main is kept gap-free and PRs test the merge
         # ref, so an in-scope gap here is PR-introduced. --update-baseline is
         # main-only — a PR must not write main's regression baseline.
         run: |
           flags="--fail-on-critical-gap"
-          if [ "${{ github.ref }}" = "refs/heads/main" ]; then
+          if [ "$GITHUB_REF" = "refs/heads/main" ]; then
             flags="$flags --update-baseline"
           fi
           tox -e e2e -- $flags
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci-e2e-build-test.yaml around lines 83 - 88, The shell
block in the CI e2e workflow is interpolating github.ref directly, which
triggers the template-injection warning. Move that value into an environment
variable for the step and have the script read the env var instead of
referencing github.ref inline; update the run block that builds flags and the
tox invocation to use the new env-based reference.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/ci-e2e-build-test.yaml:
- Around line 34-37: The Checkout repository step in the ci-e2e-build-test
workflow persists the GITHUB_TOKEN by default, which can leak through uploaded
artifacts. Update the actions/checkout usage in that workflow to disable
credential persistence by setting persist-credentials to false alongside the
existing fetch-depth setting, and keep the rest of the job behavior unchanged
since no later git operations need the token.

---

Nitpick comments:
In @.github/workflows/ci-e2e-build-test.yaml:
- Around line 83-88: The shell block in the CI e2e workflow is interpolating
github.ref directly, which triggers the template-injection warning. Move that
value into an environment variable for the step and have the script read the env
var instead of referencing github.ref inline; update the run block that builds
flags and the tox invocation to use the new env-based reference.

In `@tests/e2e/conftest.py`:
- Around line 55-65: Fail fast when the CSRF cookie is missing in the
organization setup flow. In tests/e2e/conftest.py, after session.get to
/api/v1/organization, explicitly verify the csrftoken cookie exists before
calling session.post in the organization setup sequence, rather than defaulting
to an empty X-CSRFToken header. Use the existing setup logic around orgs,
org_id, and session.cookies.get to locate the change and make the failure
message clearly indicate the missing csrftoken.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 71b1df5f-ae5f-4241-97a9-771ac81c70a8

📥 Commits

Reviewing files that changed from the base of the PR and between 913bb6a and 7607562.

📒 Files selected for processing (13)
  • .github/workflows/ci-container-build.yaml
  • .github/workflows/ci-e2e-build-test.yaml
  • .github/workflows/ci-test-e2e.yaml
  • pyproject.toml
  • tests/compose/docker-compose.test.yaml
  • tests/critical_paths.yaml
  • tests/e2e/auth/__init__.py
  • tests/e2e/auth/test_login.py
  • tests/e2e/conftest.py
  • tests/e2e/smoke/test_smoke.py
  • tests/groups.yaml
  • tests/rig/runtime.py
  • tox.ini
💤 Files with no reviewable changes (2)
  • .github/workflows/ci-container-build.yaml
  • .github/workflows/ci-test-e2e.yaml

Comment thread .github/workflows/ci-e2e-build-test.yaml Outdated
@greptile-apps

greptile-apps Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR restructures CI into a single ci-test.yaml workflow with four jobs (changes \u2192 parallel test matrix + e2e \u2192 report), adds the first real e2e coverage for the auth-login critical path, and deletes the two now-redundant workflow files. The e2e job builds SHA-tagged images and boots the full platform on its own runner so images don\u2019t need to travel across jobs, and the unified report job collects JUnit output from all tiers into one PR comment.

  • ci-test.yaml is rewritten: test (unit + integration matrix) is path-gated; e2e runs on every non-draft push/PR; report is the single branch-protection gate and sole baseline reader/writer.
  • e2e test coverage added in tests/e2e/auth/test_login.py covering the auth-login critical path (happy-path 302 + cookie validation and bad-credential rejection); tests/e2e/conftest.py adds a reusable authed_session fixture with the org-handshake flow.
  • Rig plumbing improvements: health_targets() extracted as a single source of truth for both the compose readiness probe and the smoke test; requests added to RIG_PYTEST_PLUGINS; VERSION added to tox.ini passenv.

Confidence Score: 5/5

This PR is safe to merge — no production or runtime code is touched, only CI workflow YAML and test-harness Python.

The workflow restructure is well-reasoned: draft gating, path-filter scoping, matrix fail-fast:false, single-writer baseline, and the Fail-if-any-tier-failed gate all handle the edge cases (skipped test, docker-only push to main, fork PRs) correctly. The e2e test additions are self-contained and the rig plumbing changes are additive. The two observations are minor style points that do not affect correctness.

No files require special attention. The two minor observations are in .github/workflows/ci-test.yaml (unnecessary fetch-depth: 0) and tests/e2e/smoke/test_smoke.py (strict == 200 health check).

Important Files Changed

Filename Overview
.github/workflows/ci-test.yaml Rewritten into a 4-job workflow; logic for draft gating, path filtering, baseline ownership, and the Fail-if-any-tier-failed gate is well-structured and handles skipped/cancelled results correctly.
tests/rig/runtime.py Extracts health_targets() as a shared source of truth; health paths for runner and x2text are corrected; prompt service intentionally removed from readiness probing per docstring.
tests/e2e/auth/test_login.py Self-contained login test; correctly builds its own session; covers happy-path (302 + sessionid cookie + /session identity check) and bad-credential rejection.
tests/e2e/conftest.py Adds session-scoped authed_session fixture with full org-handshake flow and CSRF token handling; org list guard is now an explicit assertion.
tests/e2e/smoke/test_smoke.py Expanded from a single backend check to iterating all health_targets(); collects all failures before asserting. Strict 200 check could fail if any health endpoint returns a different 2xx code.
tests/rig/cli.py Adds requests>=2.31.0 to RIG_PYTEST_PLUGINS so the HTTP client is available in every group's ephemeral venv.
tox.ini Adds VERSION to passenv so ComposeRuntime's docker compose up resolves ${VERSION} image tags from the caller's environment.
tests/groups.yaml Adds e2e-login group (non-optional, critical, depends on e2e-smoke) that gates on the new login tests.
tests/critical_paths.yaml Updates auth-login entry: covered_by: [e2e-login] and proof: marker, closing the gap previously declared with covered_by: [].

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    trigger["push:main / pull_request / workflow_dispatch"] --> changes
    changes["changes job\ndorny/paths-filter"] -->|relevant == true| test
    changes -->|relevant == false| test_skip["test: skipped"]
    test["test job\nmatrix: unit + integration\ntestcontainers\ntox -e {tier} --fail-on-critical-gap"]
    test_skip --> report
    changes --> e2e
    trigger --> e2e
    e2e["e2e job\nDocker login -> run-platform.sh -e\ndocker compose build VERSION=sha\ntox -e e2e --fail-on-critical-gap"]
    test -->|always| report
    e2e -->|always| report
    report["report job\nDownload reports-*\ntox -e rig -- report combine\nSticky PR comment\nSave baseline on green main"]
    report --> gate{"Fail if any tier failed"}
    gate -->|test==success OR skipped AND e2e==success| green["Branch protection: PASS"]
    gate -->|otherwise| red["Branch protection: FAIL"]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    trigger["push:main / pull_request / workflow_dispatch"] --> changes
    changes["changes job\ndorny/paths-filter"] -->|relevant == true| test
    changes -->|relevant == false| test_skip["test: skipped"]
    test["test job\nmatrix: unit + integration\ntestcontainers\ntox -e {tier} --fail-on-critical-gap"]
    test_skip --> report
    changes --> e2e
    trigger --> e2e
    e2e["e2e job\nDocker login -> run-platform.sh -e\ndocker compose build VERSION=sha\ntox -e e2e --fail-on-critical-gap"]
    test -->|always| report
    e2e -->|always| report
    report["report job\nDownload reports-*\ntox -e rig -- report combine\nSticky PR comment\nSave baseline on green main"]
    report --> gate{"Fail if any tier failed"}
    gate -->|test==success OR skipped AND e2e==success| green["Branch protection: PASS"]
    gate -->|otherwise| red["Branch protection: FAIL"]
Loading

Reviews (9): Last reviewed commit: "Merge branch 'main' into feat/un-3636-e2..." | Re-trigger Greptile

Comment thread tests/e2e/conftest.py Outdated
chandrasekharan-zipstack and others added 5 commits July 9, 2026 14:31
The e2e groups run pytest via `uv run --with ...`, whose ephemeral venv only
carries the specs the rig injects — it does not sync the root project's
`test-rig` dependency group where `requests` lives. So the e2e conftest's
`import requests` failed with ModuleNotFoundError, killing both e2e groups at
conftest load (pytest exit 4, no tests). Inject requests alongside the pytest
plugins so it survives the per-call venv re-sync.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU
A fresh DB without seed data would make organizations[0] raise a bare
IndexError as a fixture error across every consumer. Assert first so the
failure names the cause.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU
- ci-e2e-build-test.yaml: persist-credentials: false on checkout (reports/
  is uploaded as an artifact; nothing after checkout needs the token).
- Trim verbose comments in the workflow header, RIG_PYTEST_PLUGINS, and
  _wait_ready to concise WHY-only form.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU
Collapse ci-test.yaml (unit+integration matrix + report job) and
ci-e2e-build-test.yaml into a single `test` job that builds the service
images and runs every tier in one `rig run all` invocation. Because all
tiers are evaluated together, e2e coverage (auth-login) now shows in the
same PR comment as unit/integration instead of being invisible to it.

- One shared baseline cache (unstract-test-baseline-main-*); drops the
  ut/e2e namespacing that kept e2e coverage out of the posted report.
- Single sticky `test-results` comment, rendered by `rig run all` itself
  (write_summary emits combined-test-report.md).

Tradeoff (accepted for now): tiers run serially in one job, no path-filter
skip — a docs-only PR still builds images + runs e2e (as the old e2e
workflow already did). Revisit if wall-time bites.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU
Companion to the previous commit's workflow deletion: the merged single-job
workflow content (build images + rig run all + one sticky report) belongs in
ci-test.yaml. The prior commit only staged the ci-e2e-build-test.yaml removal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (3)
.github/workflows/ci-test.yaml (3)

8-16: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Consider a concurrency group to cancel superseded runs.

With a 45-minute single-runner job triggered on every PR synchronize, rapid pushes stack up redundant in-flight builds. A concurrency group scoped to the ref cancels stale runs and saves runner time.

⚙️ Suggested concurrency block
   workflow_dispatch:

+concurrency:
+  group: ci-test-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
+
 jobs:
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci-test.yaml around lines 8 - 16, Add a workflow-level
concurrency setting to the CI workflow so redundant runs are canceled when new
commits arrive on the same ref. Update the workflow near the existing
on/push/pull_request triggers in the CI YAML to use a ref-scoped concurrency
group and enable cancel-in-progress, so superseded PR runs on synchronize do not
keep consuming runner time.

18-29: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Add permissions: for the PR-comment job.

marocchino/sticky-pull-request-comment needs pull-requests: write; without an explicit block, this job can run with read-only token scopes and the report comment may not post.

Suggested block
+permissions:
+  contents: read
+  pull-requests: write
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci-test.yaml around lines 18 - 29, Add an explicit
permissions block for the workflow job that posts the PR comment so the
`marocchino/sticky-pull-request-comment` step can write to pull requests. Update
the `test` job in the workflow to include `permissions` with `pull-requests:
write` while keeping the rest of the job unchanged, so the comment/report step
has the required token scope.

57-64: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low value

Pin actions/cache@v5 to a commit SHA for consistency.
suggestions here are already pinned elsewhere in this workflow; using a SHA for the cache actions keeps the supply-chain posture consistent.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci-test.yaml around lines 57 - 64, The Cache tox
environments step is still using the floating actions/cache@v5 tag, while the
rest of the workflow is pinned to commit SHAs. Update the actions/cache
reference in this workflow to a specific commit SHA to match the existing
supply-chain hardening pattern. Keep the rest of the cache configuration
unchanged, and make the update in the Cache tox environments step so it stays
consistent with the other pinned action usages.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/workflows/ci-test.yaml:
- Around line 8-16: Add a workflow-level concurrency setting to the CI workflow
so redundant runs are canceled when new commits arrive on the same ref. Update
the workflow near the existing on/push/pull_request triggers in the CI YAML to
use a ref-scoped concurrency group and enable cancel-in-progress, so superseded
PR runs on synchronize do not keep consuming runner time.
- Around line 18-29: Add an explicit permissions block for the workflow job that
posts the PR comment so the `marocchino/sticky-pull-request-comment` step can
write to pull requests. Update the `test` job in the workflow to include
`permissions` with `pull-requests: write` while keeping the rest of the job
unchanged, so the comment/report step has the required token scope.
- Around line 57-64: The Cache tox environments step is still using the floating
actions/cache@v5 tag, while the rest of the workflow is pinned to commit SHAs.
Update the actions/cache reference in this workflow to a specific commit SHA to
match the existing supply-chain hardening pattern. Keep the rest of the cache
configuration unchanged, and make the update in the Cache tox environments step
so it stays consistent with the other pinned action usages.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e23e7129-43c1-471f-a5f1-f29325fab2ff

📥 Commits

Reviewing files that changed from the base of the PR and between d1fb262 and bf90689.

📒 Files selected for processing (1)
  • .github/workflows/ci-test.yaml

`rig run all` picks one infra mode per invocation ("platform wins"), so with
e2e (compose) and integration (testcontainers) groups in one call it skips the
testcontainers Postgres integration-backend needs — the backend then resolves
the compose DB hostname, unreachable from host-side pytest (78 DNS errors →
integration-backend red → adapter-register-llm gap → build fails).

Run one rig invocation per tier instead (unit/integration on testcontainers,
e2e on compose; compose up only for e2e), then `report combine` unifies every
tier's junit into a single evaluation + PR comment. Both auth-login (e2e) and
adapter-register-llm (integration) now show covered in one report.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
.github/workflows/ci-test.yaml (1)

106-116: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Avoid inline template expansion in shell scripts — use $GITHUB_REF instead.

zizmor flags line 113 for template injection: ${{ github.ref }} is expanded by the workflow evaluator before the shell runs, so any shell metacharacters in the value would be injected directly into the script. While github.ref is GitHub-controlled in this workflow's trigger context (refs/heads/main for push, refs/pull/N/merge for PR) and not attacker-controllable, the GITHUB_REF environment variable is available by default in the runner and eliminates the injection surface entirely.

🔒️ Proposed fix
         run: |
           flags=""
-          if [ "${{ github.ref }}" = "refs/heads/main" ] && [ "${TIERS_RC:-1}" = "0" ]; then
+          if [ "$GITHUB_REF" = "refs/heads/main" ] && [ "${TIERS_RC:-1}" = "0" ]; then
             flags="--update-baseline"
           fi
           tox -e rig -- report combine $flags
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci-test.yaml around lines 106 - 116, The shell step in the
CI workflow uses inline template expansion via github.ref, which creates an
unnecessary injection surface. Update the Combine tiers into one report step to
rely on the runner-provided GITHUB_REF environment variable instead, and keep
the conditional logic in the same block so the tox report combine command still
only adds --update-baseline on main when TIERS_RC is 0.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/workflows/ci-test.yaml:
- Around line 106-116: The shell step in the CI workflow uses inline template
expansion via github.ref, which creates an unnecessary injection surface. Update
the Combine tiers into one report step to rely on the runner-provided GITHUB_REF
environment variable instead, and keep the conditional logic in the same block
so the tox report combine command still only adds --update-baseline on main when
TIERS_RC is 0.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 53a0cc3a-97f1-4b1c-97cf-b42a23625771

📥 Commits

Reviewing files that changed from the base of the PR and between bf90689 and 93dc1cc.

📒 Files selected for processing (1)
  • .github/workflows/ci-test.yaml

chandrasekharan-zipstack and others added 2 commits July 9, 2026 17:19
Revert the single-job merge. unit/integration run as parallel matrix legs on
testcontainers; e2e builds images and boots compose on its own runner. A report
job needs all three, merges every tier's junit into one critical-path
evaluation, and posts a single sticky PR comment.

Restores unit+integration parallelism and per-tier resource isolation the merged
job lost, while keeping the unified report. One baseline (report is sole
reader+writer) replaces the old per-lane baselines, killing the cross-lane
regression false-positive. e2e is not gated by the path filter — docker/**
changes must be e2e-tested.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU
ci-e2e-build-test.yaml was folded into the e2e job of ci-test.yaml.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017UVfw7aAocC3KCJaEyZ5GU
Comment thread .github/workflows/ci-container-build.yaml Outdated

@ritwik-g ritwik-g left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — CI-only change, verified: build coverage is preserved (folded into the e2e job, runs on every non-draft PR), the smoke gate is strengthened (exact == 200 across all services vs the old < 500), and the new health paths + rig manifest changes check out against source.

Nit (PR body, non-blocking): the description still describes creating ci-e2e-build-test.yaml and says the required check becomes e2e-build-test — but at head the build is folded into ci-test.yaml and there's no e2e-build-test job. The actual branch-protection gate is report. Worth fixing that line so whoever updates required checks for UN-3635 targets report (and removes the now-defunct build check).

@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

Unstract test results

Per-group results

Status Group Tier Passed Failed Errors Skipped Duration (s)
e2e-login e2e 2 0 0 0 1.8
e2e-smoke e2e 2 0 0 0 1.1
integration-backend integration 56 0 0 27 48.4
integration-connectors integration 1 0 0 7 8.6
unit-backend unit 118 0 0 0 23.7
unit-connectors unit 63 0 0 0 10.8
unit-core unit 27 0 0 0 1.5
unit-platform-service unit 9 0 0 0 1.5
unit-rig unit 69 0 0 0 3.9
unit-sdk1 unit 381 0 0 0 25.9
unit-workers unit 0 0 0 0 30.0
TOTAL 728 0 0 34 157.1

Critical paths

⚠️ Critical paths not yet covered

  • workflow-create-execute — Create a workflow, configure source+destination, execute, poll, fetch result. (entry: POST /api/v1/workflow/{id}/execute/; declared coverage: e2e-workflow)
  • api-deployment-run — Deploy a workflow as an API, POST a document, receive structured JSON. (entry: POST /deployment/api/{org}/{name}/; declared coverage: e2e-api-deployment)
  • prompt-studio-fetch-response — Prompt Studio: create project, add prompt, run single-pass, get response. (entry: POST /api/v1/prompt-studio/prompt-studio-tool/{id}/fetch_response/; declared coverage: e2e-prompt-studio)
  • pipeline-etl-execute — Run an ETL pipeline from source connector to destination. (entry: POST /api/v1/pipeline/{id}/execute/; declared coverage: no groups declared)
  • usage-token-tracking — Per-execution token usage is recorded and retrievable. (entry: GET /api/v1/usage/get_token_usage/; declared coverage: no groups declared)
  • workflow-execution-fan-out — Multi-file workflow execution fans out to file-processing workers and rejoins. (entry: internal: backend → rabbitmq → workers/file_processing; declared coverage: no groups declared)
  • callback-result-delivery — Async results are posted back via the callback worker. (entry: internal: workers/callback → backend /internal endpoints; declared coverage: no groups declared)
✅ Covered critical paths
  • auth-login — covered by e2e-login
  • adapter-register-llm — covered by integration-backend

@chandrasekharan-zipstack chandrasekharan-zipstack merged commit c525d1a into main Jul 10, 2026
12 checks passed
@chandrasekharan-zipstack chandrasekharan-zipstack deleted the feat/un-3636-e2e-build-test branch July 10, 2026 05:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants