Skip to content

test: de-flake ORCID cache tests and ZIP-download IT#1344

Merged
milanmajchrak merged 3 commits into
dtq-devfrom
fix/flaky-orcid-zip-tests
Jun 25, 2026
Merged

test: de-flake ORCID cache tests and ZIP-download IT#1344
milanmajchrak merged 3 commits into
dtq-devfrom
fix/flaky-orcid-zip-tests

Conversation

@milanmajchrak

@milanmajchrak milanmajchrak commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Why

dtq-dev's scheduled build.yml keeps going red intermittently even after #1321 merged. The same
commit SHA passes some scheduled runs and fails others — a pure flakiness signal. Investigating the recent
failing runs (via the job logs) pinpointed four independent, genuinely-flaky tests. All four are fixed
here at the root cause — test-only changes, no retries / no symptom-masking.

# Test Job Symptom Root cause
1 CachingOrcidRestConnectorTest.testCachable / testCacheableWithError Unit expected:<Connor, John> but was:<null> live ORCID sandbox call
2 MetadataBitstreamControllerIT.downloadAllZip IT one ZIP byte differs DOS timestamp in byte-exact compare
3 AuthorizationRestRepositoryIT.findByObjectSSRTest IT expected:<200> but was:<400> config auto-reload drops setProperty
4 StatisticsRestRepositoryIT.topCountriesReport_Community_Visited IT report points: [] Solr commit waitSearcher=false race

1. CachingOrcidRestConnectorTest — live ORCID sandbox dependency

#1321 mocked the HTTP layer for getLabel/search/search_fail, but left testCachable and
testCacheableWithError hitting the live ORCID sandbox
(https://pub.sandbox.orcid.org/v3.0). They use the
real Spring bean (not a Mockito spy) on purpose — to exercise the real @Cacheable CGLIB proxy — so the
HTTP layer couldn't be stubbed. Every CI run made a real network call; when the sandbox was slow/unavailable/
changed, getLabel returned null.

Fix: point the real bean's apiURL at a local okhttp3.mockwebserver.MockWebServer serving the canned
orcid-expanded-search.xml. Keeps the real @Cacheable proxy + HTTP transport under test, removes the
network dependency. Mirrors the existing OpenAIRERestConnectorTest. testCachable now also asserts
getRequestCount() == 1 (proves the 2nd call is cache-served). Verified locally (class 8/8 green, repeated
offline runs).

2. MetadataBitstreamControllerIT.downloadAllZip — time-dependent ZIP bytes

The test compared the server response to a locally-built ZIP byte-for-byte. A ZIP entry's DOS timestamp
(2-second resolution) defaults to "now" on both sides and differs across a 2-second boundary.

Fix: assert the unzipped entry name + content (and exactly one entry) instead of raw bytes — removes the
timestamp dependency and is a stronger assertion.

3. AuthorizationRestRepositoryIT.findByObjectSSRTest — config auto-reload drops setProperty

The test sets dspace.server.ssr.url (used by Utils.getBaseObjectRestFromUri to resolve the request URI) and
AlwaysThrowExceptionFeature.turnoff via configurationService.setProperty(...). Those in-memory overrides are
silently dropped when the combined config is rebuilt by the auto-reload listener (fires on any reloadable
*.cfg mtime change mid-run). When ssr.url is dropped the URI no longer resolves → 400; and because the
/search/object path enumerates all SITE features (incl. alwaysexception, no try/catch), a dropped turnoff
would let it throw → 500.

Fix: set both values via JVM system properties (+ reloadConfig()) so they live in the
highest-precedence <system/> override layer and are re-read on every rebuild → survive auto-reload; cleared in
@After. Exactly the pattern #1321 used for the Shibboleth WWW-Authenticate leak
(AuthenticationRestControllerIT#setAuthenticationMethodSequence). Applied to all three SSR tests.

4. StatisticsRestRepositoryIT.topCountriesReport_Community_Visited — Solr searcher race

After posting two view events the test immediately queried the TopCountries report and intermittently got
points: []. SolrLoggerServiceImpl.postView commits with solr.commit(false, false) (waitSearcher=false),
so it can return before the new searcher is registered and the report query hits the stale searcher (and a
dropped solr-statistics.autoCommit=false override would skip the commit entirely).

Fix: after posting the view events, force solrLoggerService.commit() (no-arg → waitSearcher=true) so the
events are flushed and visible before the report query — robust to both the searcher race and a dropped
autoCommit override.


Verification & review

  • ORCID fix verified locally (offline, repeated runs + full class green); all four files compile against the
    current dspace-api; dspace-api checkstyle clean.
  • Each fix independently reviewed by a senior Java/Spring pass (root cause confirmed against production code;
    the SSR "both overrides must survive" and the Solr "commit covers both causes" points were validated).
  • The Fix flaky tests in IT pipeline #1321 Hibernate-CME and live-search flakes show no recurrence post-merge — those fixes are holding.

All changes are confined to test code (*Test.java / *IT.java).

Summary by CodeRabbit

  • Tests
    • Improved automated coverage for ORCID lookup caching using a local mock server, verifying successful reuse, error handling (no caching on failure), and expected request behavior.
    • Strengthened ZIP download validation by parsing the returned archive and asserting expected entry names and decoded contents.
    • Updated REST repository integration tests to configure SSR object-by-URI resolution via JVM system properties with cleanup after execution.
    • Ensured Solr-based top countries reporting tests force a commit so newly posted view events are reflected in the generated report.

Two independent flaky tests keep turning the dtq-dev pipeline red after #1321:

1. CachingOrcidRestConnectorTest.testCachable / testCacheableWithError still
   hit the live ORCID sandbox (#1321 only mocked getLabel/search/search_fail).
   They use the real Spring @Cacheable bean (to exercise the CGLIB caching
   proxy), so they could not be spied. Point the bean's apiURL at a local
   MockWebServer serving the canned orcid-expanded-search.xml instead -- keeps
   the real caching proxy and HTTP transport under test, removes the network
   dependency. No production change.

2. MetadataBitstreamControllerIT.downloadAllZip compared the response to a
   locally-built ZIP byte-for-byte; a ZIP entry's DOS timestamp (2s resolution)
   defaults to "now" on both sides and differs across a 2s boundary. Assert the
   unzipped entry name + content instead of raw bytes.

Test-only changes. Verified locally (ORCID class 8/8 green, repeated offline
runs; webapp test module compiles; checkstyle clean).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8442be65-43f6-47e1-972e-67f6bde3d0ec

📥 Commits

Reviewing files that changed from the base of the PR and between d10c833 and 27bad9a.

📒 Files selected for processing (2)
  • dspace-server-webapp/src/test/java/org/dspace/app/rest/AuthorizationRestRepositoryIT.java
  • dspace-server-webapp/src/test/java/org/dspace/app/rest/StatisticsRestRepositoryIT.java

📝 Walkthrough

Walkthrough

Tests now use local mocking or explicit commits to make ORCID, ZIP, SSR, and statistics assertions depend on test-controlled state.

Changes

Test isolation and timing updates

Layer / File(s) Summary
Mock ORCID response setup
dspace-api/src/test/java/org/dspace/external/CachingOrcidRestConnectorTest.java
Imports add MockWebServer support, and a helper builds canned ORCID XML responses from the stored search result.
Caching behavior tests
dspace-api/src/test/java/org/dspace/external/CachingOrcidRestConnectorTest.java
testCachable and testCacheableWithError point the connector at a MockWebServer and assert cache state, return values, and request count across repeated calls.
Shared bitstream fixture
dspace-server-webapp/src/test/java/org/dspace/app/rest/MetadataBitstreamControllerIT.java
Imports, the dummy content constant, and setup code switch the ZIP test fixture away from BitstreamService-backed reference generation.
ZIP response assertions
dspace-server-webapp/src/test/java/org/dspace/app/rest/MetadataBitstreamControllerIT.java
The downloadAllZip test reads the response ZIP with ZipInputStream and checks entry names and UTF-8 decoded content against the expected bitstream text.
SSR test configuration helper
dspace-server-webapp/src/test/java/org/dspace/app/rest/AuthorizationRestRepositoryIT.java
The authorization integration test adds SSR system-property setup and cleanup, reloads configuration for enabled resolution, and updates the SSR-focused test methods to use the helper.
Statistics report commit
dspace-server-webapp/src/test/java/org/dspace/app/rest/StatisticsRestRepositoryIT.java
The statistics integration test commits the Solr statistics logger after posting view events before running the TopCountries report query.

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested labels

REVIEW-done

Suggested reviewers

  • vidiecan
🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is detailed but does not follow the repository template and omits the required section headings and checklist items. Rewrite it using the template headings: Problem description, Analysis, Problems, Manual Testing, and Copilot review, with the checklist items filled out.
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and matches real changes in the PR, especially the ORCID cache tests and ZIP-download IT fixes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@dspace-server-webapp/src/test/java/org/dspace/app/rest/MetadataBitstreamControllerIT.java`:
- Around line 91-101: The ZIP assertion in MetadataBitstreamControllerIT is
masking duplicate entries because the ZipInputStream loop stores them in a Map
keyed by entry name, so repeated filenames overwrite each other. Update the test
to track each ZIP entry independently in the same loop (for example by counting
entries or collecting them in a list) and assert that exactly one entry is
returned before checking its name and content. Use the existing ZipInputStream
iteration and the bts.getName()/BITSTREAM_CONTENT assertions to locate the test
logic.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 96ae7c62-ae49-4926-ad24-0c0cadd8b02d

📥 Commits

Reviewing files that changed from the base of the PR and between ee37116 and 3880fb8.

📒 Files selected for processing (2)
  • dspace-api/src/test/java/org/dspace/external/CachingOrcidRestConnectorTest.java
  • dspace-server-webapp/src/test/java/org/dspace/app/rest/MetadataBitstreamControllerIT.java

milanmajchrak and others added 2 commits June 25, 2026 14:38
Address CodeRabbit: a Map keyed by entry name could mask duplicate ZIP entries
(same filename overwrites). Track the entry count separately and assert it is 1,
so an unexpected extra entry fails the test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two more intermittent failures on dtq-dev, both addressed at the root cause
(test-only changes):

1. AuthorizationRestRepositoryIT.findByObjectSSRTest flaked with 400 instead of
   200. The test sets dspace.server.ssr.url (used by Utils.getBaseObjectRestFromUri
   to resolve the request URI) and the AlwaysThrowExceptionFeature.turnoff flag via
   configurationService.setProperty(...). Such in-memory overrides are silently
   dropped when the combined config is rebuilt by the auto-reload listener (fires on
   any reloadable cfg file mtime change mid-run). When ssr.url is dropped the URI no
   longer resolves -> 400; if turnoff were also dropped the /search/object path would
   let alwaysexception throw -> 500. Fix: set both via JVM system properties (+
   reloadConfig) so they sit in the highest-precedence override layer and survive
   auto-reload, cleared in @after. Same pattern as #1321's Shibboleth fix
   (AuthenticationRestControllerIT#setAuthenticationMethodSequence). Applied to all
   three SSR tests.

2. StatisticsRestRepositoryIT.topCountriesReport_Community_Visited flaked with an
   empty report (points: []). postView() commits with waitSearcher=false, so the
   just-posted view events can be invisible to the immediately-following report query
   (and a dropped solr-statistics.autoCommit override would skip the commit entirely).
   Fix: after posting the view events, force solrLoggerService.commit() (waitSearcher
   =true) so the events are flushed and visible before the report is queried.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@milanmajchrak milanmajchrak merged commit f6f1356 into dtq-dev Jun 25, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant