diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index b14043c94b89..302583e6251a 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -99,12 +99,18 @@ jobs: # Retry action: https://github.com/marketplace/actions/retry-action # Codecov action: https://github.com/codecov/codecov-action - name: Upload coverage to Codecov.io + # Best-effort: a Codecov upload failure (e.g. missing CODECOV_TOKEN on this branch/fork) + # must not fail the build — required unit + integration tests determine status. + continue-on-error: true uses: Wandalen/wretry.action@v3.8.0 with: action: codecov/codecov-action@v4 - # Ensure codecov-action throws an error when it fails to upload + # Do NOT fail the build when the Codecov upload fails. On this fork/branch the + # CODECOV_TOKEN secret is not configured, so the upload always errors; with + # fail_ci_if_error:true that reddened the whole build even though all unit + integration + # tests pass. Coverage upload stays best-effort. (dtq-dev drops the codecov job entirely.) with: | - fail_ci_if_error: true + fail_ci_if_error: false token: ${{ secrets.CODECOV_TOKEN }} # Try re-running action 5 times max attempt_limit: 5 diff --git a/CLARIN_DSPACE_V9_PROGRESS.md b/CLARIN_DSPACE_V9_PROGRESS.md new file mode 100644 index 000000000000..639f8ed1e80c --- /dev/null +++ b/CLARIN_DSPACE_V9_PROGRESS.md @@ -0,0 +1,1117 @@ +# CLARIN-DSpace → DSpace 9 Upgrade — Progress (source of truth) + +> Read this file before starting or resuming. Update it after every meaningful step +> and before any context compaction/handoff. This is the durable state of the effort. +> Last updated: 2026-06-18 (session 1 — initial ground-truth + inventory). + +## 0. TL;DR / current status + +> Last refreshed: 2026-06-22 (context cleanup vs. original mandate). The §0 below is the +> authoritative current state; earlier sections retain methodology/history. + +- **Both PRs are MERGEABLE** (conflicts resolved): PR bases were re-pointed from `dtq-dev` to + **`dtq-dev-9-base`** (= vanilla `dspace-9.3`, BE `e0fae432ff` / FE `a2141979`), so each PR diff = + pure CLARIN additions on v9, no conflicts. Final landing into `dtq-dev` = Phase-2 reconciliation + merge (`-s ours`) after the port completes (see §7b). +- **Backend PR #1339 — 2 tranches, CI-GREEN, MERGEABLE — but COMPILE-ONLY skeleton, NOT runtime-functional:** + 1. `bb7a599cd2` migrations (19) + ~107 entities/services/DAOs/factories. + 2. `8836cc632a` Spring bean wiring. + Unit+Integration+CodeRabbit ✅ (`codecov` red = pre-existing infra). + - Tranche 3 `85b260f567`: 24 ADDED CLARIN config files (clarin-dspace.cfg, OAI crosswalks, emails, + registries) + entity columns (WorkspaceItem.shareToken, EPerson.welcomeInfo/canEditSubmissionMetadata). + - Tranche 4 `7219832193`: **CLARIN License REST** (17 files: model/hateoas/converter/repository) → + /server/api/core/clarinlicenses(+labels/resourcemappings/lruallowances) endpoints. compile+checkstyle ✅. + - Tranche 5 `060e4fc6e8`: **handle/epic-handle/user-metadata/user-registration/verification-token/ + featured-service REST** (31 files incl. HandleRestRepository + link repositories). compile+checkstyle ✅. + Deferred: ExternalHandleRestRepository (needs RandomStringGenerator bean). + - Tranche 6 (plural bean-name fix): **RUNTIME-VALIDATED**. v9's REST framework resolves + `getBean(category + "." + modelPlural)` (Utils.getResourceRepositoryByCategoryAndModel) — CLARIN + repos were registered with singular `NAME` (7.x pattern) → every CLARIN endpoint 404'd. Added + `PLURAL_NAME = NAME + "s"` to 8 Rest models + switched 8 RestRepositories' `@Component` to PLURAL_NAME. + **RUNTIME VALIDATION (2026-06-23, the milestone reviewers demanded — "does it actually boot/work?"):** + Built full reactor `mvn -o package` → **BUILD SUCCESS** (deployable installer, WAR + boot jar). Deployed + via `ant fresh_install` to a local runtime, started a local Postgres (pgcrypto) container on :54321. Ran + `dspace database migrate` → **ALL CLARIN Flyway migrations applied cleanly on real Postgres** (previously + only h2): tables license_definition/label/label_extended_mapping/resource_mapping/resource_user_allowance, + clarin_token, user_metadata, user_registration, previewcontent + columns workspaceitem.share_token, + eperson.welcome_info/can_edit_submission_metadata, handle.url/dead/dead_since. Booted `server-boot.jar` + (v9.3) → Spring context wires all CLARIN beans; **CLARIN REST endpoints serve from Postgres**: + `/server/api/core/clarinlicenses|clarinlicenselabels|handles` → 200 (valid HAL, empty because + insert_default_licenses.sql is an intentionally-commented template); `clarinlruallowances| + clarinusermetadatas` → 401 (correctly auth-protected). This is the proof BE compile+CI could not give. + Full endpoint sweep: clarinlicenses/clarinlicenselabels/clarinlicenseresourcemappings/handles=200, + clarinlruallowances/clarinusermetadatas/clarinuserregistrations=401, clarinlicenses/search=200. + KNOWN QUIRK (faithful to 7.x, NOT a regression): `clarinverificationtokens` returns **500** for + anonymous because the service throws `AuthorizeException("You must be an admin...")` which the repo + wraps as RuntimeException (it returns 200 with admin auth). Left as-is to preserve port fidelity; + could be improved to 403 via `@PreAuthorize("hasAuthority('ADMIN')")` like the sibling repos. Also: + CLARIN endpoints are not advertised in the discoverable `/api/core` index (no DiscoverableEndpoints + registration) — harmless, the FE calls them by path. + - Tranche 7 `98771fe034` (2nd runtime-diagnosed bug): v9 `ConverterService.toRest` enforces a + `@PreAuthorize` SpEL read from the repository's most-derived `findOne` for every BaseObjectRest; + CLARIN `findOne` overrides had none (faithful to 7.x; v7's ConverterService didn't do this) → + converting any real CLARIN object threw `'expressionString' must not be null or blank` (400/500). + Empty GETs passed only because there was no row to convert; surfaced on POST(create). FIX: add + `@PreAuthorize` to findOne of the 8 CLARIN repos (permitAll() for license/label/mapping/handle, + hasAuthority('ADMIN') for allowance/usermetadata/userregistration/verificationtoken) — mirrors + vanilla CommunityRestRepository.findOne. compile+checkstyle clean. + WRITE-PATH VALIDATION: **PROVEN 2026-06-24** (full BE built clean -> deployed -> booted v9.3 on :18080 + against Postgres :54321). Admin login (XSRF + JWT) OK; **POST /server/api/core/clarinlicenselabels -> + 201 Created** (id 23, returned HAL with type clarinlicenselabel); **GET back -> totalElements 1** + (persisted "PUB | Publicly Available"). Full CRUD cycle works: auth -> create -> Postgres -> converter + serialize -> read. Runtime-proves the findOne @PreAuthorize fix (tranche 7) on a real object. (Earlier + attempts had stalled only because the host was RAM-starved by the user's docker stack; RAM freed + overnight -> 8GB.) Local validation setup persists: pg container `clarin-pg` :54321 (pwd dspace, admin + admin@clarin.test/adminpass), runtime `dspace-runtime/`, boot `java -Xmx2g -Ddspace.dir=... -jar + webapps/server-boot.jar -Dserver.port=18080`; redeploy via full `mvn -o package` then cp + dspace/target/dspace-installer/webapps/server-boot.jar to dspace-runtime/webapps/. + - Tranche 8 `02f123fd9`: 26 more REST files — ConfigFile REST, authorization (CanManageLicense + + test controller), ClarinAutoRegistration/UserInfo controllers, License/Handle import controllers, + submission steps (ClarinLicenseDistribution/Resource/Notice + 2 validations + SubmissionUtils), + refbox DTOs, ClarinDataLicense, BigMultipartFile. javax.mail->jakarta.mail. clean compile+checkstyle. + DEFERRED 14 (need unported vanilla methods Util.replaceLast/normalizeDiscoverQuery + libs + org.json.simple/com.hp.hpl.jena, in `_deferred/`): ClarinUserMetadataRestController, + Clarin{Item,EPerson,UserMetadata}ImportController, ClarinGroupRestController, SubmissionController, + SuggestionRestController, ClarinRefBoxController, ClarinShibbolethLoginFilter, SolrOAIReindexer, + Authrn{Rest,Resource}+AuthorizationRestController, DBConnectionStatisticsController. + - Tranche 9 `33172a376`: vanilla-file methods — Item.isHidden()+isDiscoverable() tweak, + WorkspaceItemService.findByShareToken (+Impl/DAO/DAOImpl). clean compile+checkstyle on dspace-api. + - LESSON LEARNED: tranche 7 first pushed a duplicate-@PreAuthorize (7 of 8 CLARIN repos ALREADY had + findOne @PreAuthorize in 7.x; only ClarinLicenseLabel lacked it) — a STALE incremental compile + masked it -> CI red. Fixed `d28db8cab` by restoring originals. ALWAYS verify with `clean compile`. + - Tranche 10 `64feedacce`: vanilla Utils methods (ONLY the ones v9.3 lacks — it already had maskEmail/ + getAllowedTemplateConfig/getSecureVelocityProperties/getMaxTimestamp; full delta would duplicate): + core.Utils replaceLast/getTransactionPid/fetchUUIDFromUrl; rest.utils.Utils normalizeDiscoverQuery + (+helpers)/encodeNonAsciiCharacters/disableCertificateValidation/distinctByKey/ + getCanonicalHandleUrlNoProtocol. Un-deferred 7 controllers (Authorization+AuthrnRest[+getTypePlural]/ + Resource, ClarinUserMetadata REST+Import, Submission, Suggestion) + json-simple 1.1.1 pom. + Verified combined reactor clean compile (api+webapp) + checkstyle. STILL DEFERRED (7, deep + v9-migration in _deferred/): ClarinRefBoxController (ancient com.hp.hpl.jena), SolrOAIReindexer + + Clarin{Item,EPerson}ImportController (Date->Instant), ClarinShibbolethLoginFilter (StatelessLoginFilter + ctor changed), ClarinGroupRestController (GroupRest.GROUPS), DBConnectionStatisticsController + (getHibernateStatistics). Each needs individual v9 API adaptation. + **STILL TODO for full function:** 57 MODIFIED config files (dspace.cfg include of clarin-dspace.cfg, + item-submission.xml, shibboleth auth), remaining vanilla-file method additions, import/submission-step/ + OAI REST, CLARIN tests, then full Docker stack (BE+FE+Solr) + seed data + Playwright. +- **Frontend PR #1316 — 7 tranches, MERGEABLE** (head `d8511814d1`): + 1. assets (947 imgs) · 2. core models+data-services (49) · 3. **clarin-licenses** ✅green · + 4. **handle-page** ✅green · 5. **epic-handle** ✅green · 6. **share-submission+change-submitter** + (re-running after e2e-Docker flake; 22.x green) · 7. **contact-page+static-page** (CI running). + Pre-validation `npm run build` + `npm run lint:nobuild -- --quiet` reliably predicts CI. +- **Porting method fully proven + documented** (BE §4/§6, FE standalone recipe + v9 gotcha catalog §6e). +- **REMAINING vs Definition-of-Done (none silently skipped):** + - FE feature modules: login/shibboleth+discojuice, license-contract, **item-page (155 files)**, + submission steps, bitstream-page, entity-groups, admin, info, accessibility, clarin-navbar-top. + - FE i18n keys (all langs). + - BE deferred features (S3/sync, preview, report/health, matomo, PID/EPIC clients, versioning CLI) — + in `_deferred/` (§5 list); CLARIN BE unit/IT tests; REST layer (`dspace-server-webapp`, ~295 files); config. + - **Docker:** bring up full CLARIN v9 stack locally (not yet done). + - **Playwright** (`dspace-ui-tests`) against local stack + **manual specs** (#55, #411) — needs seeded data (§6b). + - **Independent review agents** to challenge completeness (required by mandate; not yet run). + - Phase-2 `dtq-dev` reconciliation merge. + +## 1. Objective + +Port all CLARIN-DSpace (LINDAT/CLARIN, a.k.a. UFAL) features from the customized +fork (`dtq-dev`, DSpace **7.6.5**) onto vanilla **DSpace 9.3** backend and +**dspace-angular 9.x** frontend, on branch `ufal/clarin-dspace-upgrade-v9` in both +repos, preserving compatibility with the `dtq-dev` workflow, CI, Docker and local dev. +Get the full stack running locally in Docker and the `dataquest-dev/dspace-ui-tests` +Playwright suite passing. + +## 2. Environment (verified this session) + +| Tool | Version | Notes | +|------|---------|-------| +| Docker | 29.4.3 | daemon up, 12 CPUs, 15.58 GiB RAM | +| Docker Compose | v5.1.4 | | +| Java | Temurin 17.0.16 | DSpace 9 requires JDK 17 ✓ | +| Maven | 3.8.3 | | +| Node | 20.19.0 | | +| npm | 10.8.2 | | +| gh CLI | 2.67.0 | authed as `milanmajchrak` (ssh, repo scope) | +| git | 2.45.1 | SSH remotes; push allowed only to the 2 PR branches | + +Workspace (do not leave it): `C:\workspace\clarin-dspace-v9-upgrade\` +- `DSpace/` — backend repo, branch `ufal/clarin-dspace-upgrade-v9` +- `dspace-angular/` — frontend repo, branch `ufal/clarin-dspace-upgrade-v9` + +## 3. Ground-truth branch map + +| Ref | What it is | +|-----|-----------| +| `origin/dtq-dev` (BE) | CLARIN-DSpace **7.6.5** — THE SOURCE of CLARIN features. Active (last commit 2026-06-17). | +| `origin/dtq-dev` (FE) | dspace-angular **7.6.5** CLARIN fork — SOURCE for FE features. | +| `origin/dtq-dev-9` (BE) | plain vanilla DSpace **9.1** tracking branch — *no CLARIN work*. Not useful except as reference. | +| `ufal/clarin-dspace-upgrade-v9` (BE) | TARGET — currently vanilla **9.3** (`e0fae432ff`). | +| `ufal/clarin-dspace-upgrade-v9` (FE) | TARGET — currently vanilla **9.x**. | +| tags `dspace-7.6.5`, `dspace-9.3` (BE) / `dspace-7.6.3`, `dspace-9.0` (FE) | vanilla baselines used to compute fork-delta. | + +## 4. Methodology — fork-delta porting + +The 7.6.5→9.3 PR diff mixes DSpace's own evolution with CLARIN changes. To isolate the +**CLARIN customization (fork-delta)**: + +``` +# backend CLARIN delta = vanilla 7.6.5 -> dtq-dev +git diff dspace-7.6.5 origin/dtq-dev +# frontend CLARIN delta = vanilla 7.6.3 -> dtq-dev +git diff dspace-7.6.3 origin/dtq-dev # (merge-base is 7.6.1/7.6.3 era) +``` + +Then re-apply that delta on top of v9, adapting for API changes. ADDED files port +near-verbatim (just package/API tweaks); MODIFIED vanilla files are the hard ports +(v9 refactored many APIs). + +### Fork-delta size (measured this session) + +**Backend (`dspace-7.6.5` → `dtq-dev`): 838 files, +107009 / −2677** +- 521 **added** files (187 Java classes named `Clarin*`/in `clarin/` pkgs, + migrations, + config) +- 314 **modified** vanilla files → **218 modified .java** (need v9 API adaptation) + 52 config/xml/properties +- By area: dspace-api/src 357, dspace-server-webapp/src 295, dspace/config 85, dspace-oai/src 32, .github/workflows 10, scripts ~25 + +**Frontend (`dspace-7.6.3` → `dtq-dev`): 1985 files, +191114 / −31092** +- ~959 are `src/assets/images` (license/label icons — copy verbatim) → ~1000 real code files +- By area: app/shared 199, item-page 155, core 85, submission 48, handle-page 28, clarin-licenses 24, epic-handle 23, login-page 21, entity-groups 21, bitstream-page 25, discojuice 7, license-contract-page 6, share-submission 6 … + +## 5. CLARIN feature inventory (from wiki + delta) and porting status + +Source: https://github.com/ufal/clarin-dspace/wiki/Features (+ linked pages). +Status legend: ⬜ not started · 🟦 in progress · ✅ ported+verified · ⛔ blocked · ➖ N/A. + +### Licensing & Access Control +| # | Feature | BE | FE | Notes / source classes | +|---|---------|----|----|------------------------| +| L1 | CLARIN Licenses (label framework, confirmation) | ⬜ | ⬜ | `content/clarin/ClarinLicense*`, `app/clarin-licenses` | +| L2 | License Agreement Dialog (user details at download) | ⬜ | ⬜ | `ClarinLicenseResourceUserAllowance*`, `ClarinUserMetadata*` | +| L3 | Creative Commons license submission step | ⬜ | ⬜ | | +| L4 | Field-Level Permissions (ACL) | ⬜ | ⬜ | wiki FineGrainedFieldPermissions | +| L5 | Bitstream Download Tokens (time-limited) | ⬜ | ⬜ | `ClarinBitstreamServiceImpl`, download-token | + +### Persistent Identifiers +| # | Feature | BE | FE | Notes | +|---|---------|----|----|-------| +| P1 | PIDs & Handles (per-community prefixes, external handles, content negotiation) | ⬜ | ⬜ | handle-page, epic-handle | +| P2 | EPIC PID API v2 integration | ⬜ | ⬜ | | +| P3 | Handle management GUI | ⬜ | ⬜ | | +| P4 | DOI registration (DataCite: reserve/register/update) | ⬜ | ⬜ | | +| P5 | DOI config per community | ⬜ | ⬜ | | +| P6 | ORCID authority (no Solr) | ⬜ | ⬜ | | +| P7 | ROR authority | ⬜ | ⬜ | | + +### Submission Workflow +| # | Feature | BE | FE | Notes | +|---|---------|----|----|-------| +| S1 | Sharing a submission (share token) | ⬜ | ⬜ | `Add_share_token_to_workspaceitem` migration, app/share-submission | +| S2 | Complex fields (contact_person, funding, sizeInfo) | ⬜ | ⬜ | | +| S3 | Admin-only fields | ⬜ | ⬜ | | +| S4 | Autocomplete (Solr / static JSON) | ⬜ | ⬜ | | +| S5 | CMDI metadata file upload (METADATA bundle) | ⬜ | ⬜ | | +| S6 | CLARIN license steps (Distribution + Resource) | ⬜ | ⬜ | | +| S7 | CLARIN notice step | ⬜ | ⬜ | | + +### Authentication & User Management +| # | Feature | BE | FE | Notes | +|---|---------|----|----|-------| +| A1 | Shibboleth AAI + DiscoJuice | ⬜ | ⬜ | `authenticate/clarin/ClarinShibAuthentication`, aai/discojuice | +| A2 | Shibboleth auto-registration | ⬜ | ⬜ | `ClarinUserRegistration*`, `ClarinVerificationToken*` | +| A3 | User registration + email verification | ⬜ | ⬜ | | +| A4 | Personal Access Tokens (PAT) | ⬜ | ⬜ | `ClarinToken*`, `Clarin_token` migration | + +### Item Display & Services +| # | Feature | BE | FE | Notes | +|---|---------|----|----|-------| +| I1 | Featured services / Refbox | ⬜ | ⬜ | `ClarinFeaturedService*` | +| I2 | File previews (+directory tree) | ⬜ | ⬜ | `Added_Preview_Tables` migration | +| I3 | Item tombstones (withdrawn/replaced) | ⬜ | ⬜ | | +| I4 | ZIP download (all bitstreams) | ⬜ | ⬜ | | +| I5 | Item versioning (CLARIN handles/DOIs) | ⬜ | ⬜ | `ItemVersionLinker` | +| I6 | WebLicht integration (CMDI/OAI) | ⬜ | ⬜ | dspace-oai delta | + +### Analytics & Monitoring +| # | Feature | BE | FE | Notes | +|---|---------|----|----|-------| +| M1 | Matomo analytics (bitstream + OAI tracking, report subscriptions) | ⬜ | ⬜ | `ClarinMatomo*Tracker`, `MatomoReportSubscription`, migration | +| M2 | Health report + report diff (DB snapshots, email) | ⬜ | ⬜ | `report_result` migration | +| M3 | Google Dataset Search structured data | ⬜ | ⬜ | | + +### Operations & Admin Tools +| # | Feature | BE | FE | Notes | +|---|---------|----|----|-------| +| O1 | S3 storage integration (presigned URLs) | ⬜ | ⬜ | | +| O2 | File downloader CLI | ⬜ | ⬜ | `administer/FileDownloader` | +| O3 | Curation tasks (requiredmetadata, metadataqa, checklinks, checkhandles, registerdoi, profileformats) | ⬜ | ⬜ | | +| O4 | CLI scripts (clarin-token, process-cleaner, item-version-linker, file-downloader, file-preview, health-report, report-diff) | ⬜ | ⬜ | `administer/Clarin*`, `ItemVersionLinker` | +| O5 | Bulk import REST API (handles, users, licenses, metadata, bitstreams, logos) | ⬜ | ⬜ | import endpoints | +| O6 | Configuration file admin API | ⬜ | ⬜ | `app/configuration/service/ConfigFileService` | +| O7 | CMDI metadata export REST endpoint | ⬜ | ⬜ | | + +## 6. Foundational: CLARIN DB migrations to port (Flyway) + +Both `h2` (tests) and `postgres` (runtime) copies needed. v9 may already include some of +these upstream (e.g. orcid/supervision/system-wide-alerts were 7.x upstream additions and +are likely already in 9.x — DO NOT double-add). **Verify each against v9 before porting.** + +CLARIN-specific (must port): +- `V7.2_2022.07.28__Upgrade_to_Lindat_Clarin_schema.sql` ← the big CLARIN schema (license tables, handles, user metadata, etc.) +- `V7.6_2024.08.05__Added_Preview_Tables.sql` + `V7.6_2025.06.09__Added_Indexes_To_Preview_Tables.sql` +- `V7.6_2024.09.30__Add_share_token_to_workspaceitem.sql` +- `V7.6_2024.10.25__insert_default_licenses.sql` +- `V7.6_2024.01.25__insert_checksum_result.sql` +- `V7.6_2025.06.03__Create_table_report_result.sql` +- `V7.6_2025.07.29__Matomo_report_registry_table.sql` +- `V7.6_2025.09.18__Clarin_token.sql` +- `V7.6_2025.10.30__7z_bitstream_format.sql` + +> NOTE: version-numbering. These are `V7.x` prefixed. On v9 the Flyway baseline differs; +> CLARIN migrations may need renumbering to `V9.x_...` or placed so they run after the v9 +> schema. Decide & document (see Decisions). + +## 6b. Playwright suite (dataquest-dev/dspace-ui-tests) — how to run + +Cloned to `C:\workspace\clarin-dspace-v9-upgrade\dspace-ui-tests` (private, branch `master`). +- **Stack:** Playwright 1.57 + TypeScript, npm. 6 spec files under `tests/tests/` + (homePage, itemPage, loginPage, searchPage, submissionPage, universalPage) with page + objects under `tests/pages/`. +- **Config:** `playwright.config.ts` (`globalSetup: ./scripts/merge-config.ts` merges + `customer-constants/config.default.json` + any other `*.json`). No baseURL hardcoded; + uses env. `ignoreHTTPSErrors:true`, 3 browser projects (chromium/firefox/webkit). +- **Run recipe (from CI `.github/workflows`):** env `HOME_URL=https://dev-5.pc:8443/repository/`, + `NAME=DEFAULT`; `cd scripts && ./test.sh`. Credentials via `.env` + (admin `dspace.admin.dev@dataquest.sk`/`admin`, user `dspace.user.dev@dataquest.sk`/`user`). +- **⚠ Hard dependency — seeded test data:** tests assert against FIXED production data: + - handles: restricted_download `11234/1-2683`, bitstreams `11234/1-5419`, + doi `20.500.12801/3901359-01`, new_version `11234/1-5677`, restricted collection + `11234/1-f26b6363`, icons `11234/1-4875`, version redirect `11858/00-097C-...`. + - branding: home title "LINDAT/CLARIAH-CZ Repository Home"; URL prefix `/repository`. + - OAI endpoints, license manage-table, handle-table, bulk-access pages. + → To run locally we must (a) point HOME_URL at the local stack, (b) seed matching + CLARIN items/handles/licenses/DOIs/versions, (c) set `NAME`/locators for our config, + or relax data-specific assertions. **This is a substantial test-data task, tracked separately.** + +## 6c. Manual test specs +- **dspace-customers#55** "[TEST] Testing scenarios" (OPEN): step-by-step manual scenarios — + create EPerson (→ `user_registration` row), restricted-download redirect to login, + download from restricted item, license manage-table, handle-table, bulk-access, OAI cmdi + exposure, item versions, icons, etc. Mirrors the Playwright assertions. +- **dspace-customers#411**: (to summarize next session) additional manual test spec. + +## 6e. Frontend (dspace-angular) port plan — STARTED 2026-06-18 + +Repo `dspace-angular`, branch `ufal/clarin-dspace-upgrade-v9` = vanilla **9.3.0** (port not started). +PR #1316, base branch `dtq-dev-9-base` (= `a2141979` = vanilla `dspace-9.3`) ready. + +**FE fork-delta (`dspace-7.6.3` → `dtq-dev`): 1497 added files + 266 modified `.ts`** (the hard +angular 7→9 ports). Added by module: item-page 101, shared 56, **core 53** (data-services/models — +foundational), handle-page 28, **clarin-licenses 24**, epic-handle 23, bitstream-page 18, login-page 15, +submission 14, admin 8, static-page 7, contact-page 7, share-submission 6, license-contract-page 6, +clarin-navbar-top 5, change-submitter 4, accessibility 4, info 3, statistics 2 (+~959 image assets). + +v9 angular migration concerns: standalone components / new control-flow, `@dspace` API changes, +SSR build. Same proven method as BE: baseline build → port module → lint/build → coherent commit → CI. + +**FE tranches:** +1. ✅ Baseline: `npm ci` OK; vanilla 9.3 builds (`npm run build`, ~4 min, dist produced). +2. ✅ **Assets pushed** (commit `31f660b4e9`, 947 net-new images: mime/flags/item-types/footer/ + static-pages/LINDAT branding). FE PR #1316 now has 1 commit → base switchable to `dtq-dev-9-base`. +3. ✅ **`core` clarin models + data services (49 files) PUSHED** (commit `9c8dfed2bd`): TYPE-CLEAN + LINT-CLEAN (`tsc -p tsconfig.json`, + zero errors in CLARIN files; only pre-existing vanilla noise: grecaptcha global + a `.spec`). + **Key v9 FE API delta:** `@dataService` decorator MOVED from `core/data/base/data-service.decorator` + → `core/cache/builders/build-decorators` (NOT removed — still used for HAL resolution). Fix = repoint + import path per file (kept the decorator). Deferred from this tranche: `bitstream-url-serializer` + + `shared/clarin-shared-util` (pull in the `clarin-item-box-view` feature — port with that). + FE validation loop: `tsc --noEmit -p tsconfig.json` (full project; `tsconfig.app.json`/ng build only + checks files reachable from main, so unreferenced new files need the full tsconfig). Lint = `npm run lint` + (`build:lint && ng lint`; direct `npx eslint` fails — needs the custom-rule build). Lint validating now. +4. ✅ **`clarin-licenses` module ported to v9 standalone & PUSHED** (`b3a7ff33c0` + lint fix `b5e2a6cbd5`): + `ng build` + full `ng lint --quiet` both clean. First push went CI-red on template lint (scoped + eslint missed `.html`); fixed via control-flow migration + `dsBtnDisabled` + `===` (see recipe step 5/6). + Lesson baked into recipe: always run full `npm run lint:nobuild -- --quiet` before pushing FE. +5. ✅ **handle-page module PUSHED** (`e0c1270b28`, FE tranche 4): 6 standalone components, /handle-table + admin route. build+lint clean. +6. ✅ **epic-handle module PUSHED** (`7c5a6aa29b`, FE tranche 5): 5 standalone components, /epic-handle-table + admin route. build+lint clean. +7. ✅ **share-submission + change-submitter PUSHED** (`5ece6b02ce`, FE tranche 6): 2 standalone + components, /share-submission route. build+lint clean. (Fixes: chart.js→hasNoValue, instanceof generic.) +8. ⬜ i18n keys (`src/assets/i18n/*.json5` — additive, merge into all langs + watch i18n lint). +9. ✅ **contact-page + static-page PUSHED** (`d8511814d1`, FE tranche 7): themed contact page + static + HTML pages. Ported ClarinSafeHtmlPipe + HtmlContentService. build+lint clean. +10. ⬜ Remaining modules: login/shibboleth+discojuice, license-contract, item-page additions (155 files, + large), submission steps, bitstream-page, entity-groups, admin, info, accessibility, clarin-navbar-top. + +### Themed-component v9 pattern (resolved): +`Themed*Component extends ThemedComponent` — NO `standalone`/`imports` field, `templateUrl: +'../shared/theme-support/themed.component.html'`, methods `getComponentName()`/`importThemedComponent()`/ +`importUnthemedComponent()`. The 7.x CLARIN themed wrappers already match — no conversion needed. Base +component is converted to standalone normally; route points at the Themed wrapper. + +### More v9 API changes (learned porting contact/static — apply to all remaining modules): +- `LocaleService.getCurrentLanguageCode()` now returns `Observable` (was `string`) → + `await firstValueFrom(...)` in async methods. +- `@nguniversal/express-engine/tokens` REMOVED → `REQUEST`/`RESPONSE` now in `src/express.tokens` + (relative e.g. `../../express.tokens`). +- CLARIN-added services (e.g. HtmlContentService) need `@Injectable({ providedIn: 'root' })` (no NgModule + provides them now). +- `ds-themed-loading` element → `ds-loading` (ThemedLoadingComponent). +- `no-negated-async`: `!(obs | async)` → `(obs | async) === null/false/undefined` or `?.length === 0`. +- Other CLARIN pipes ported standalone so far: 6 license pipes (clarin-licenses) + ClarinSafeHtmlPipe. + +### FE module CI confirmations +- clarin-licenses (`b5e2a6cbd5`): ✅ green. handle-page (`e0c1270b28`): ✅ green. epic-handle + (`7c5a6aa29b`): ✅ green. share-submission (`5ece6b02ce`): tests(22.x) ✅ but tests(20.x) ❌ on a + TRANSIENT infra step **"Start DSpace REST Backend via Docker (for e2e)"** (not code — 22.x passed same + commit) → re-ran. contact/static (`d8511814d1`): CI running. +- **CI flake pattern:** FE `tests` job e2e step "Start DSpace REST Backend via Docker" is occasionally + flaky (like BE `net.handle` repo flake). If only that step fails (and the other Node version passes), + it's infra — `gh run rerun --failed`. Pre-validation (`npm run build` + `npm run lint:nobuild + -- --quiet`) remains reliable for CODE correctness. + +### More v9 FE gotchas (learned porting handle/epic): +- `ds-loading` selector = `ThemedLoadingComponent` (shared/loading/themed-loading.component). +- `*ngVar` = `VarDirective` (shared/utils/var.directive) — control-flow migration does NOT convert it. +- `standalone: true` is the DEFAULT now → eslint rule `dspace-angular-ts/no-default-standalone-value` + strips it (eslint --fix removes the line; keep `imports: []`). +- rxjs `catchError`/error callbacks: keep param `(error: unknown)` (rule + `@smarttools/rxjs/no-implicit-any-catch` forbids `any`), cast `(error as any)` at access sites. +- Most lint errors are auto-fixable (`eslint --fix`): import sort/newlines, standalone-import sort, + no-default-standalone-value, disabled→dsBtnDisabled (html). Manual: eqeqeq, missing imports. + +### v9 standalone feature-module migration RECIPE (proven on clarin-licenses) +1. `git checkout origin/dtq-dev -- ` (exclude `*.spec.ts`). +2. Each component: add `standalone: true` + `imports: [...]` to `@Component`. Imports = template deps: + `CommonModule` (ngIf/ngFor/async), `TranslateModule` (translate pipe), `ReactiveFormsModule`/`FormsModule` + (forms/ngModel), `NgbXModule` pieces, v9 standalone components (`ThemedLoadingComponent` + `shared/loading/themed-loading.component`, `PaginationComponent` `shared/pagination/pagination.component`), + sibling components, and any custom CLARIN pipes (make those `@Pipe({ standalone: true })` too). +3. Replace `*.module.ts` + `*-routing.module.ts` with `*-routes.ts` exporting `ROUTES: Route[]`; use v9 + FUNCTION resolvers/guards (`i18nBreadcrumbResolver`, `siteAdministratorGuard`, lowercase) not the 7.x classes. +4. Add any route-path constants to `app-routing-paths.ts`; wire `loadChildren: () => import('./x/x-routes').then(m => m.ROUTES)` into `app-routes.ts` (+ import the path const). +5. **v9 template migration (REQUIRED — CI gate `ng lint --quiet` checks `.html`!):** run + `npx ng generate @angular/core:control-flow --path=src/app/ --interactive=false` + to convert `*ngIf/*ngFor` → `@if/@for` (CI rule `@angular-eslint/template/prefer-control-flow`). + Also fix `==`→`===` (`template/eqeqeq`) and `disabled`/`[disabled]` on `