Incorrect results when multiple policies have the same target/action combinations.

[joachimvh]

I'm getting an unexpected result with the following input, using new ODRLEvaluator(new ODRLEngineMultipleSteps()).

Request:

@prefix odrl: <http://www.w3.org/ns/odrl/2/> .

<urn:ucp:policy:2a797ad7-232a-4e1f-853f-81388969e4a1> a odrl:Request;
    odrl:permission <urn:ucp:rule:e3ba21f7-57b0-4a43-988a-3221aba858ef>.
<urn:ucp:rule:e3ba21f7-57b0-4a43-988a-3221aba858ef> a odrl:Permission;
    odrl:action odrl:read;
    odrl:target <http://localhost:3000/alice/other/resource.txt>;
    odrl:assignee <https://modify.pod.knows.idlab.ugent.be/profile/card#me>.

Policy:

@prefix ex: <http://example.org/>.
@prefix odrl: <http://www.w3.org/ns/odrl/2/> .

ex:usagePolicy1 a odrl:Agreement ;
  odrl:permission ex:permission1 .
ex:permission1 a odrl:Permission ;
  odrl:action odrl:modify, odrl:read ;
  odrl:target <http://localhost:3000/alice/other/resource.txt> ;
  odrl:assignee <https://both.pod.knows.idlab.ugent.be/profile/card#me> .

ex:usagePolicy2 a odrl:Agreement ;
  odrl:permission ex:permission2 .
ex:permission2 a odrl:Permission ;
  odrl:action odrl:modify ;
  odrl:target <http://localhost:3000/alice/other/resource.txt> ;
  odrl:assignee <https://modify.pod.knows.idlab.ugent.be/profile/card#me> .

State of the world is just the current time

The resulting report is the following:

@prefix ex: <http://example.org/>.
@prefix odrl: <http://www.w3.org/ns/odrl/2/>.
@prefix report: <http://example.com/report/temp/>.

<urn:uuid:94a5f2fc-d910-470f-b1c4-3c08d9ece998> a report:PolicyReport;
    <http://purl.org/dc/terms/created> "2025-04-08T12:52:10.986Z"^^<http://www.w3.org/2001/XMLSchema#dateTime>;
    report:policy ex:usagePolicy1;
    report:policyRequest <urn:ucp:policy:428b6a05-40cd-49ac-83aa-e128e64ac337>;
    report:ruleReport <urn:uuid:83de2fb9-dc8c-4c7a-958e-19056e40eec1>.
<urn:uuid:2cda4dbd-abf6-4c69-a4bc-3ea6aeff4bf1> a report:PolicyReport;
    <http://purl.org/dc/terms/created> "2025-04-08T12:52:10.986Z"^^<http://www.w3.org/2001/XMLSchema#dateTime>;
    report:policy ex:usagePolicy2;
    report:policyRequest <urn:ucp:policy:428b6a05-40cd-49ac-83aa-e128e64ac337>;
    report:ruleReport <urn:uuid:a93a6aa8-1c4f-43bf-85a8-89f66af7fae5>.
<urn:uuid:83de2fb9-dc8c-4c7a-958e-19056e40eec1> a report:PermissionReport;
    report:attemptState report:Attempted;
    report:rule ex:permission1;
    report:ruleRequest <urn:ucp:rule:dcfcc0ac-12c8-4c94-85c0-f52b90270d07>;
    report:premiseReport <urn:uuid:b5b481b7-2b3f-4ed5-9c03-fba6cb606cf5>, <urn:uuid:396fb126-0cda-4efe-a290-3f5e54be7d57>, <urn:uuid:34f5d001-4ea5-416d-be6d-fc054a940e6f>, <urn:uuid:f85721b7-4131-4a91-8d35-6e0fbe7e94f8>.
<urn:uuid:a93a6aa8-1c4f-43bf-85a8-89f66af7fae5> a report:PermissionReport;
    report:attemptState report:Attempted;
    report:rule ex:permission2;
    report:ruleRequest <urn:ucp:rule:dcfcc0ac-12c8-4c94-85c0-f52b90270d07>;
    report:premiseReport <urn:uuid:b5b481b7-2b3f-4ed5-9c03-fba6cb606cf5>, <urn:uuid:f8ea227e-f9e2-41ed-af36-a2f900031004>, <urn:uuid:34f5d001-4ea5-416d-be6d-fc054a940e6f>.
<urn:uuid:b5b481b7-2b3f-4ed5-9c03-fba6cb606cf5> a report:TargetReport;
    report:satisfactionState report:Satisfied.
<urn:uuid:396fb126-0cda-4efe-a290-3f5e54be7d57> a report:PartyReport.
<urn:uuid:f8ea227e-f9e2-41ed-af36-a2f900031004> a report:PartyReport;
    report:satisfactionState report:Satisfied.
<urn:uuid:34f5d001-4ea5-416d-be6d-fc054a940e6f> a report:ActionReport;
    report:satisfactionState report:Satisfied.
<urn:uuid:f85721b7-4131-4a91-8d35-6e0fbe7e94f8> a report:ActionReport;
    report:satisfactionState report:Satisfied.
<urn:uuid:a93a6aa8-1c4f-43bf-85a8-89f66af7fae5> report:activationState report:Active.
<urn:uuid:83de2fb9-dc8c-4c7a-958e-19056e40eec1> report:activationState report:Inactive.
<urn:uuid:396fb126-0cda-4efe-a290-3f5e54be7d57> report:satisfactionState report:Unsatisfied.

According to this result the second policy should be active, even though the action value is incorrect.
If the first policy is not present in the file, or does not have both the read and modify actions, the second policy will not be active.

[woutslabbinck]

This is indeed strange.

Especially the fact that both Action reports are satisfied, which does not make sense.

I've played a around a bit with these inputs and what I found is that if the three policies are atomic policies themselves, the problem disappears.

A simple fix is to implement policy decomposition properly within the ODRL Evaluator (as was designed).
Perhaps some code from https://github.com/joachimvh/policy-conflict-resolver might help?

Atomic policies from your input:

@prefix ex: <http://example.org/>.
@prefix odrl: <http://www.w3.org/ns/odrl/2/> .

ex:usagePolicy1 a odrl:Agreement ;
  odrl:permission ex:permission1 .
ex:permission1 a odrl:Permission ;
  odrl:action odrl:read;
  odrl:target <http://localhost:3000/alice/other/resource.txt> ;
  odrl:assignee <https://both.pod.knows.idlab.ugent.be/profile/card#me> .

ex:usagePolicy2 a odrl:Agreement ;
  odrl:permission ex:permission2 .
ex:permission2 a odrl:Permission ;
  odrl:action odrl:modify ;
  odrl:target <http://localhost:3000/alice/other/resource.txt> ;
  odrl:assignee <https://modify.pod.knows.idlab.ugent.be/profile/card#me> .

ex:usagePolicy3 a odrl:Agreement ;
  odrl:permission ex:permission3 .
ex:permission3 a odrl:Permission ;
  odrl:action odrl:modify ;
  odrl:target <http://localhost:3000/alice/other/resource.txt> ;
  odrl:assignee <https://both.pod.knows.idlab.ugent.be/profile/card#me> .

Compliance Report

@prefix ex: <http://example.org/> .
@prefix dct: <http://purl.org/dc/terms/> .
@prefix report: <https://w3id.org/force/compliance-report#> .

<urn:uuid:4494f94b-7d1f-41c4-ad2f-0464a1822636> a report:PolicyReport ;
  dct:created "2024-02-12T11:20:10.999Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> ;
  report:policy ex:usagePolicy2 ;
  report:policyRequest <urn:ucp:policy:2a797ad7-232a-4e1f-853f-81388969e4a1> ;
  report:ruleReport <urn:uuid:25d79c77-0149-408a-809b-ac97068ab5b6> .

<urn:uuid:25d79c77-0149-408a-809b-ac97068ab5b6> a report:PermissionReport ;
  report:attemptState report:Attempted ;
  report:rule ex:permission2 ;
  report:ruleRequest <urn:ucp:rule:e3ba21f7-57b0-4a43-988a-3221aba858ef> ;
  report:premiseReport <urn:uuid:83b7943d-3e58-4d30-ba2f-cdda548f0d3a>, <urn:uuid:b95626fa-9780-48e7-b988-655432274b32>, <urn:uuid:88d04c1a-92d5-452c-9d03-8dba6b254285> ;
  report:activationState report:Inactive .

<urn:uuid:29fac280-3893-45cd-bdd0-942a0be7f4b8> a report:PolicyReport ;
  dct:created "2024-02-12T11:20:10.999Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> ;
  report:policy ex:usagePolicy1 ;
  report:policyRequest <urn:ucp:policy:2a797ad7-232a-4e1f-853f-81388969e4a1> ;
  report:ruleReport <urn:uuid:8b0e07d5-b8e4-47c8-b670-a11b40ee7670> .

<urn:uuid:8b0e07d5-b8e4-47c8-b670-a11b40ee7670> a report:PermissionReport ;
  report:attemptState report:Attempted ;
  report:rule ex:permission1 ;
  report:ruleRequest <urn:ucp:rule:e3ba21f7-57b0-4a43-988a-3221aba858ef> ;
  report:premiseReport <urn:uuid:83b7943d-3e58-4d30-ba2f-cdda548f0d3a>, <urn:uuid:a1a9c2b9-d16a-42c1-a52c-45e1ca61162c>, <urn:uuid:3f9c51df-c2ae-41a2-9905-2d566457ddbf> ;
  report:activationState report:Inactive .

<urn:uuid:9602a478-ff93-4ace-a771-e96c4552aed1> a report:PolicyReport ;
  dct:created "2024-02-12T11:20:10.999Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> ;
  report:policy ex:usagePolicy3 ;
  report:policyRequest <urn:ucp:policy:2a797ad7-232a-4e1f-853f-81388969e4a1> ;
  report:ruleReport <urn:uuid:6163ed7f-c7ed-481b-b072-a8cd0af6fcfe> .

<urn:uuid:6163ed7f-c7ed-481b-b072-a8cd0af6fcfe> a report:PermissionReport ;
  report:attemptState report:Attempted ;
  report:rule ex:permission3 ;
  report:ruleRequest <urn:ucp:rule:e3ba21f7-57b0-4a43-988a-3221aba858ef> ;
  report:premiseReport <urn:uuid:83b7943d-3e58-4d30-ba2f-cdda548f0d3a>, <urn:uuid:88d04c1a-92d5-452c-9d03-8dba6b254285>, <urn:uuid:a1a9c2b9-d16a-42c1-a52c-45e1ca61162c> ;
  report:activationState report:Inactive .

<urn:uuid:83b7943d-3e58-4d30-ba2f-cdda548f0d3a> a report:TargetReport ;
  report:satisfactionState report:Satisfied .

<urn:uuid:b95626fa-9780-48e7-b988-655432274b32> a report:PartyReport ;
  report:satisfactionState report:Satisfied .

<urn:uuid:88d04c1a-92d5-452c-9d03-8dba6b254285> a report:ActionReport ;
  report:satisfactionState report:Unsatisfied .

<urn:uuid:a1a9c2b9-d16a-42c1-a52c-45e1ca61162c> a report:PartyReport ;
  report:satisfactionState report:Unsatisfied .

<urn:uuid:3f9c51df-c2ae-41a2-9905-2d566457ddbf> a report:ActionReport ;
  report:satisfactionState report:Satisfied .

[woutslabbinck]

This bug does not happen when using the CompositeODRLEvaluator, the evaluator that deals with compact rules.

The reason for the bug is that the following happens by the N3 evaluation:

1. for each action, a premisreport is generated
2. the satisfaction states are calculated through matching requested action and actual action in a rule.
   The problem is that there was an assumption that there would only be one action per rule.

In this sitation, there are more actions in the first rule. So for the two premise reports generated, the reasoner finds a match and in turn the premise is satisfied.

However, if in the report, the action of the rule is added as well while creating the premise report, this bug does not happen anymore.

It is just a matter of question regarding whether such explanations should be added to premise report and if so what is the meaning.

NOTE: example of change to the report.n3 rules that addresses this issue (but, reading and modifying by https://both.pod.knows.idlab.ugent.be/profile/card#me becomes impossible. Both premisereports can never be satisfied at the same time. Thats why we recommend to just use CompositeODRLEvaluator)

# Action report -> simple action
# Create action report
{
   ?ruleReport a ?ruleReportType ;
       report:rule ?permission ;
       report:ruleRequest ?requestPermission .
   ?ruleReportType list:in (report:PermissionReport report:RuleReport report:ProhibitionReport) .

   ?permission odrl:action ?action .
   
   ( ?action ) :getUUID ?urnUuid .
}
=> 
{
   ?ruleReport report:premiseReport ?urnUuid .
   ?urnUuid a report:ActionReport .
   ?urnUuid report:explanation ?action .
} .
# Calculate satisfaction state (simple action)
{
   ?ruleReport a ?ruleReportType ;
       report:rule ?permission ;
       report:ruleRequest ?requestPermission ;
       report:premiseReport ?actionReport .
   ?ruleReportType list:in (report:PermissionReport report:RuleReport report:ProhibitionReport) .

   ?actionReport a report:ActionReport .
   ?actionReport report:explanation ?action .
   ?permission odrl:action ?action .
   ?requestPermission odrl:action ?requestedAction .

   ?action log:equalTo ?requestedAction .
}
=> 
{
   ?actionReport a report:ActionReport ;
       report:satisfactionState report:Satisfied .
} .
# Calculate satisfaction state (included in)
{
   ?ruleReport a ?ruleReportType ;
       report:rule ?permission ;
       report:ruleRequest ?requestPermission ;
       report:premiseReport ?actionReport .
   ?ruleReportType list:in (report:PermissionReport report:RuleReport report:ProhibitionReport) .
      
   ?actionReport a report:ActionReport .
   ?actionReport report:explanation ?action .
   ?permission odrl:action ?action .
   ?requestPermission odrl:action ?requestedAction .

   ?requestedAction odrl:includedIn ?action .
} => 
{
   ?actionReport a report:ActionReport ;
       report:satisfactionState report:Satisfied .
} .
# Calculate satisfaction state (exact match)
{
   ?ruleReport a ?ruleReportType ;
       report:rule ?permission ;
       report:ruleRequest ?requestPermission ;
       report:premiseReport ?actionReport .
   ?ruleReportType list:in (report:PermissionReport report:RuleReport report:ProhibitionReport) .
      
   ?actionReport a report:ActionReport .
   ?actionReport report:explanation ?action .
   ?permission odrl:action ?action .
   ?requestPermission odrl:action ?requestedAction .

   ?requestedAction skos:exactMatch ?action .
} => 
{
   ?actionReport a report:ActionReport ;
       report:satisfactionState report:Satisfied .
} .