PD-6176 Index version-of and funded-by external identifiers in Solr

[cryptalith]

Closes PD-6176 — https://orcid.clickup.com/t/9014437828/PD-6176

Problem

Activity external identifiers are indexed into the profile Solr core as dynamic fields named <id-type>-<relationship>, so people can search:

https://pub.orcid.org/v3.0/csv-search/?q=<ID-TYPE>-self:<ID>&fl=orcid

Only self and part-of actually work. The two halves of this ticket turned out to be different problems:

• funded-by was never implemented at any layer — no constant, no document field, no Solr field.
• version-of already had a Solr field and indexer support for research resources and peer reviews, but the works loop only ever checked SELF and PART_OF. Since works are the dominant activity type, every *-version-of field is empty in production.

Measured with a field-existence query (q=<field>:[* TO *]) against /v3.0/search/. Note /csv-search/ silently swallows Solr errors and returns empty for any garbage field name, so it can't be used to test this.

Field	Sandbox	Production
doi-self	9,200	7,667,297
doi-part-of	163	273,478
doi-version-of	0	0
*-funded-by (all types)	HTTP 500 undefined field	HTTP 500 undefined field

Changes

• DYNAMIC_FUNDED_BY in both SolrConstants copies
• fundedByIds map on OrcidSolrDocument, including equals/hashCode
• VERSION_OF + FUNDED_BY indexed for works
• FUNDED_BY indexed for research resources, resource items and peer reviews
• *-funded-by dynamic field declared in the profile schema
• test coverage for the relationship maps, which had none

No orcid-model change is needed — Relationship already ships all four values.

Fundings are deliberately out of scope

Fundings contribute no relationship based identifiers at all, so funding external identifiers are invisible to self and part-of queries too. That is a real gap, but fixing it would change the results of queries that already work today across millions of records, which is a different risk profile from adding brand new fields and is not what this ticket asked for. Funding grant numbers remain searchable through the existing grant-numbers field.

Raised separately on the ticket for product to consider. A test asserts the exclusion so it is not reintroduced by accident.

⚠️ Deployment ordering

The Solr dynamic field must be applied to the running cores BEFORE this is deployed.

The profile core uses a managed schema (ClassicIndexSchemaFactory is commented out in solrconfig.xml), so the schema.xml change in this PR does not update an already provisioned core. Verified locally: on startup Solr converts schema.xml into managed-schema and renames the original to schema.xml.bak.

There is no AddSchemaFieldsUpdateProcessorFactory, so unknown fields are not auto-created. Verified locally against two Solr 8.0.0 instances with the same document:

• with the field → HTTP 200
• without it → HTTP 400 undefined field

Because the listener sends the whole document via addBean, that 400 fails the entire record, not just the unknown field. Deploying this before the schema would break indexing for every record carrying a funded-by identifier.

Applying the field is hot, auto-reloads the core, and needs no reindex:

curl -X POST -H 'Content-type:application/json' --data-binary '{"add-dynamic-field":{"name":"*-funded-by","type":"string","indexed":true,"stored":false,"multiValued":true}}' http://<solr-write-host>:6983/solr/profile/schema

Backfill

Solr documents are fully replaced on update, so existing documents only gain the new fields once re-indexed. Backfill script is in a companion ORCID-Internal PR: https://github.com/ORCID/ORCID-Internal/pull/736

Testing

• 155 tests pass across orcid-utils and orcid-message-listener
• new tests confirmed to fail without the fix (expected:<[10.1000/version-of]> but was:<null>)
• end to end against a real Solr 8.0.0 loaded with this schema: ran the actual converter through SolrJ addBean, then queried — doi-version-of, doi-funded-by, grant_number-funded-by and doi-self all returned the record

Notes

*-funded-by values will also flow into the catch-all text field via the existing <copyField source="*" dest="text"/>, consistent with how the other three relationships already behave.

[amontenegro]

Task linked: PD-6176 Index funded-by and version-of identifiers