Skip to content

Include user_metadata in rule fingerprints - #1719

Draft
adkinsty wants to merge 1 commit into
databrickslabs:mainfrom
adkinsty:fix/fingerprint-user-metadata
Draft

adkinsty wants to merge 1 commit into
databrickslabs:mainfrom
adkinsty:fix/fingerprint-user-metadata

Conversation

@adkinsty

@adkinsty adkinsty commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Changes

compute_rule_fingerprint now includes user_metadata. Before this change, a save that changed only user_metadata matched the stored fingerprint and was skipped. The checks table kept the old values and the caller got no error (#1476).

  • user_metadata is hashed only when it is non-empty, so rules without it keep their current fingerprint. Key order does not change the hash.
  • message_expr is still left out of the fingerprint.
  • The storage guide and the save docstrings for Delta and Lakebase storage now describe the new behaviour, with a note on how it differed in 0.17.2 and earlier.

Notes for reviewers

  • Design change: the old docs said user_metadata was left out on purpose, so that a tag edit such as changing an owner would not create a new version. This PR reverses that. Any change to a rule, including its metadata, is now a new version.
  • Upgrade effect: rules that carry user_metadata get a new fingerprint. The first save_checks after upgrading writes a new version even if nothing changed, and results written before the upgrade won't match the newly stored fingerprints. Matching on rule name still works. This should go in the release notes.
  • DQX Studio: Studio adds run_mode and binding_version to each check's user_metadata when it submits a run, so draft and published runs of the same rules now get different rule_set_fingerprint values. Nothing in Studio compares them today. This PR replaces the one Studio test that asserted they matched and updates a docstring. Deployed Studio pins core 0.17.0 (dqx_version in app/databricks.yml), so it is unaffected until that pin is bumped.
  • Not included: save_checks([], mode="overwrite") still does nothing. That part of [BUG]: TableChecksStorageHandler.save() in overwrite mode #1686 will be a separate fix.

Two questions for the Studio maintainers before the pin is bumped:

  1. Should draft and published runs of the same rules share a fingerprint? If so, the run tags need to move out of per-check user_metadata.
  2. Studio stores a fingerprint for each rule in dq_resolved_rules. Rows for rules with metadata were computed the old way, so the rule_set_fingerprint in the dq_rules_core view won't match core until those rows are recomputed. Should that be a one-time backfill?

Linked issues

Resolves #1476
Relates to #1686

Tests

  • manually tested
  • added unit tests
  • added integration tests
  • added end-to-end tests
  • added performance tests

Unit tests: updated or removed the tests that asserted the old behaviour, and added tests for a change to metadata alone, key order, and empty or missing metadata. test_fingerprint_byte_parity pins seven known hashes; the one for the rule with user_metadata was re-captured. The other six are unchanged, which confirms that rules without metadata keep their fingerprint.

Integration tests: new Delta and Lakebase tests save a rule, change only its user_metadata, save again in overwrite mode, and check that the new value loads back.

Run locally: make fmt (no changes), make lint, make test and make app-test (Studio backend). Integration tests on serverless: all of test_save_and_load_checks_from_table.py (including the new test) and test_apply_checks_with_rule_fingerprint.py, 33 passed. The new Lakebase test has not been run locally.

Documentation and Demos

  • added/updated demos
  • added/updated docs
  • added/updated agent skills (no skill describes fingerprint contents)

This pull request and its description were written by Isaac.

compute_rule_fingerprint now hashes user_metadata when it is non-empty,
so a save that changes only user_metadata is written as a new version
instead of being skipped. Rules without user_metadata keep their
existing fingerprint.

Updates the storage docs and save docstrings, flips the tests that
asserted the old behaviour, and adds Delta and Lakebase integration
tests for a metadata-only overwrite. In DQX Studio, replaces the test
that required draft and published runs to share a fingerprint.

Resolves databrickslabs#1476
Relates to databrickslabs#1686

Co-authored-by: Isaac <no-reply@databricks.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

compute_rule_fingerprint doesn't include user_metadata

1 participant