Unexpected context_ref_mismatch warnings

[jennifergward]

Checked for duplicates

No - I haven't checked

🐛 Describe the bug

When I run validate I get a context_ref_mismatch warning even though I'm using the correct values.

I'm using "NASA/Keck Reflectance Experiment Laboratory" and "NASA Reflectance Experiment Laboratory Bidirectional Visible and Near Infrared Spectrometer" for in <Observing_System_Component> for urn:nasa:pds:context:facility:laboratory.relab and urn:nasa:pds:context:instrument:relab.bd-vnir, respectively.

These are both listed in registered_context_products.json:
{
"name": [
"NASA/Keck Reflectance Experiment Laboratory (RELAB), Brown University, Providence, Rhode Island, United States",
"NASA/Keck Reflectance Experiment Laboratory (RELAB)",
"NASA/Keck Reflectance Experiment Laboratory"
],
"type": [
"Laboratory"
],
"lidvid": "urn:nasa:pds:context:facility:laboratory.relab::1.1"
},

{
"name": [
"Bi-directional Visible and Near Infrared Spectrometer at RELAB",
"NASA Reflectance Experiment Laboratory Bidirectional Visible and Near Infrared Spectrometer",
"RELAB Bidirectional Spectrometer"
],
"type": [
"N/A"
],
"lidvid": "urn:nasa:pds:context:instrument:relab.bd-vnir::1.2"
},

however previous versions are also listed in registered_context_products.json:
{
"name": [
"NASA/Keck Reflectance Experiment Laboratory (RELAB), Brown University, Providence, Rhode Island, United States",
"NASA/Keck Reflectance Experiment Laboratory (RELAB)"
],
"type": [
"Laboratory"
],
"lidvid": "urn:nasa:pds:context:facility:laboratory.relab::1.0"
},

{
"name": [
"Bi-directional Visible and Near Infrared Spectrometer at RELAB",
"RELAB Bidirectional Spectrometer"
],
"type": [
"Spectrometer"
],
"lidvid": "urn:nasa:pds:context:instrument:relab.bd-vnir::1.0"
},

It seems validate doesn't know to skip the older version.

🕵️ Expected behavior

Validate should know to skip the older versions or the older versions should be removed from registered_context_products.json.

Also note that there are other entries in registered_context_products.json which would likely produce the same warnings:
These 3 LIDVIDs are repeated:
urn:esa:psa:context:instrument:vex.mag::1.2
urn:esa:psa:context:instrument:vex.virtis::1.2
urn:esa:psa:context:investigation:mission.venus_express::1.1
and these 5 LIDs (and none other) show up with multiple VIDs
urn:esa:psa:context:instrument_host:spacecraft.vex
urn:nasa:pds:context:instrument:relab.bc-ftir1
urn:nasa:pds:context:instrument:relab.bcf-ftir2
urn:nasa:pds:context:instrument:relab.bd-vnir
urn:nasa:pds:context:investigation:other_investigation.relab_speclib

📜 To Reproduce

🖥 Environment Info

Windows

📚 Version of Software Used

Version 4.2.0

🩺 Test Data / Additional context

Rhyodacite-glass_VNIR_measured_reflectance_photometric_correction_63-106um_g20.xml

🦄 Related requirements

🦄 #xyz

⚙️ Engineering Details

No response

🎉 Integration & Test

No response

[jordanpadams]

Thanks for the detailed report — we've been able to reproduce and trace the root cause.

Investigation findings

When validate --latest-json-file (-u) runs, it queries the PDS Search API (https://pds.nasa.gov/api/search/1/products) and downloads all registered Product_Context entries. The API is returning multiple versions of the same LID for a small set of products — meaning older versions (with fewer/different names) end up alongside newer versions in registered_context_products.json.

The 9 affected LIDs we identified:

LID	Versions in registry
urn:nasa:pds:context:facility:laboratory.relab	1.0, 1.1
urn:nasa:pds:context:instrument:relab.bd-vnir	1.0, 1.2
urn:nasa:pds:context:instrument:relab.bc-ftir1	1.0, 1.2
urn:nasa:pds:context:instrument:relab.bcf-ftir2	1.0, 1.2
urn:nasa:pds:context:investigation:other_investigation.relab_speclib	1.0, 2.1
urn:esa:psa:context:instrument_host:spacecraft.vex	1.1, 1.2
urn:esa:psa:context:investigation:mission.venus_express	1.1, 1.1 (two registry nodes!)
urn:esa:psa:context:instrument:vex.mag	1.2, 1.2 (two registry nodes!)
urn:esa:psa:context:instrument:vex.virtis	1.2, 1.2 (two registry nodes!)

This matches exactly the 8 problematic LIDs you identified in the issue.

Why this causes the warning

When a label uses a lid_reference (no explicit version — valid and common PDS4 practice), validate looks up the product by LID alone and matches against the first entry in the JSON for that LID. Because the API returns results sorted by harvest date, older versions appear first. The ::1.0 entry for relab.bd-vnir has names ["Bi-directional Visible and Near Infrared Spectrometer at RELAB", "RELAB Bidirectional Spectrometer"] — it does not include "NASA Reflectance Experiment Laboratory Bidirectional Visible and Near Infrared Spectrometer", which was added in ::1.2. Validate checks the name against the older entry and reports a mismatch.

Two contributing issues

1. Registry data quality — The vex.mag, vex.virtis, and mission.venus_express entries are the same LIDVID harvested from two different nodes (PDS_ENG and PSA). This should be reported to the registry team to clean up.

2. Validate bug — Validate should deduplicate by LID and keep only the highest version when building registered_context_products.json. This is something we can fix on the validate side regardless of the registry state. We're working on that fix now.

In the meantime, a workaround is to re-run validate -u to refresh your JSON file once the registry data is cleaned up, or to manually remove the older duplicate entries from your local registered_context_products.json.

🤖 Generated with Claude Code