[Shopify] Migrate Shopify Connector to Expiring Offline Access Tokens#9327
[Shopify] Migrate Shopify Connector to Expiring Offline Access Tokens#9327onbuyuka wants to merge 19 commits into
Conversation
Adds support for Shopify expiring offline access tokens (public apps must
migrate by 2027-01-01):
- Persist token/refresh expiry on "Shpfy Registered Store New" and store the
refresh token in IsolatedStorage.
- Request expiring tokens on install (expiring=1), refresh before expiry, and
migrate legacy non-expiring tokens via token exchange (best-effort).
- Orchestrate on-demand from "Shpfy Communication Mgt." (GetAccessToken) with a
reactive 401 refresh-and-retry.
- Scheduled backstop job ("Shpfy Token Refresh" + per-shop worker) to keep
tokens and 90-day refresh tokens alive; registered via installer/upgrade.
- Shop Card reconnect notification when the refresh token has expired.
- Tests for the refresh-token expiry decision logic.
NOTE: "Shpfy Token Dev Tools" (page 30440) is temporary test scaffolding and
must be removed before merge.
Fixes AB#637954
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
- Remove the temporary "Shpfy Token Dev Tools" page (30440) and its permission-set entry. - Clear the new telemetry event IDs (set to '') so they can be assigned by the tagging script. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
Refresh the living docs (al-docs update) to cover slice 637954: - business-logic.md: new "Authentication and token lifecycle" section. - data-model.md: Shpfy Registered Store New token/expiry storage + ER entry. - patterns.md: Job Queue dispatcher/worker per-shop isolation pattern. - CLAUDE.md (app + Base): auth overview and "things to know". Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
Remove the LogTokenTelemetry helper and inline Session.LogMessage('', ...) at
each call site so the telemetry tagging script can assign event IDs (it only
fills literal Session.LogMessage first-args). Matches the existing connector
convention. Tags left empty pending the script.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
Fill the token lifecycle telemetry event IDs (0000UIV-0000UJ1) via the tagging script. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
- Mark SaveInstalledToken [NonDebuggable] (it holds the token-bearing response body). - Give the token-refresh backstop its own Job Queue category (SHPFYAUTH) so it is not serialized behind long-running SHPFY sync jobs (pricing/inventory) and can refresh 1-hour access tokens promptly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
Copilot PR ReviewIteration 11 · Outcome: completed Knowledge source: https://github.com/microsoft/BCQuality@186d8a131465475c79244d994acb872cd5c0d4bf Findings by domainFindings split into Knowledge-backed (cite a BCQuality article) and Agent (the agent's own judgement, no matching BCQuality rule).
Totals: 2 knowledge-backed · 0 agent findings. Orchestrator pre-filter (2 file(s) excluded)
Findings produced by the AL review agent v1.7.3. Reply 👎 on any inline comment to flag false positives. |
- Encrypt the refresh token at rest: IsolatedStorage.SetEncrypted instead of Set
(90-day credential; Get transparently decrypts).
- Mark ResponseHasAccessToken [NonDebuggable] (handles the token-bearing body).
- Remove the redundant per-iteration Commit() in the token-refresh dispatcher
loop; the per-shop worker (Codeunit.Run) already commits on success.
- Guard Codeunit.Run("Job Queue - Enqueue") in ScheduleRefreshJob so an enqueue
failure is logged instead of aborting the install/upgrade.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
SaveTokenResponse (new in this PR) stores the Shopify offline access token via RegisteredStoreNew.SetAccessToken(), which persists it with IsolatedStorage.Set (unencrypted at rest), while the refresh token introduced by this same PR is correctly stored via SetRefreshToken() using IsolatedStorage.SetEncrypted.Both are OAuth credentials for the same store and belong in the same encrypted class. This inconsistency becomes load-bearing now: every store going forward will have an encrypted refresh token sitting next to an unencrypted, equally sensitive access token in the same record. Change SetAccessToken (Shpfy Registered Store New table) to use SetEncrypted for parity with the refresh token. Knowledge: Line mapping was unavailable, so this was posted as an issue comment. 👍 useful · ❤️ especially valuable · 👎 wrong - reply with why |
Fill 0000UJ6 (enqueue-failure warning) via the tagging script. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
A legacy store has no refresh token, so EnsureValidAccessToken never hits the fast path and re-attempts migration on every API call -- in a sync loop over many records a failing migration means many redundant lock+HTTP+commit round trips. Add a "Last Migration Attempt" timestamp on Shpfy Registered Store New and only re-attempt migration once per hour (ShouldAttemptMigration/TryMigrate, shared by EnsureValidAccessToken and ForceTokenRefresh). Docs updated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
- Do not interpolate GetLastErrorText() into the telemetry message string (LogRefreshFailure and the schedule-failure log). Use a generic message, move the error detail and shop code to custom dimensions, and classify the event as CustomerContent so the platform can handle it appropriately. - Move the shared telemetry Labels to the codeunit object scope. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
Codeunit 139635 is already used by "E-Doc. Receive Files" in the EDocument test app; BCApps validates test object IDs globally. Move Shpfy Token Refresh Test to 139613, which is free repo-wide within the Shopify test ID ranges. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
…riant API Test) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
The new SetRefreshToken procedure correctly uses IsolatedStorage.SetEncrypted for the 90-day refresh token, but the pre-existing SetAccessToken (line 68 of the same table) still uses the unencrypted IsolatedStorage.Set for the access token, sitting right next to the new encrypted call.Both values are Shopify credentials of comparable sensitivity per the referenced guidance ('anything that would harm the tenant if leaked — API keys, tokens, connection strings, OAuth client secrets — uses SetEncrypted'). Since this PR is already touching this table's secret-handling code, it's a good opportunity to align SetAccessToken with SetEncrypted for consistency. Knowledge: Line mapping was unavailable, so this was posted as an issue comment. 👍 useful · ❤️ especially valuable · 👎 wrong - reply with why |
The country builds compile the test app with CodeCop treating AA0137 as an error. DaysFromNow used exit() with a named return variable 'Result' left unassigned; drop the name. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
Correct the doc comment: the table lock serializes refresh/migration per company only (the store table is DataPerCompany), not across companies. Document that the lock is deliberately held across the token request and that cross-company safety relies on Shopify's idempotent-response window. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
In ShpfyCommunicationMgt, after HandleUnauthorizedResponse forces a token refresh on a 401, the code Clears HttpResponseMessage and re-sends the request but discards the boolean return value of the retry's HttpClient.Send(HttpRequestMessage, HttpResponseMessage) call - unlike the original Send() a few lines above, whose return value is checked.If the retry Send fails to obtain a response (e.g. a transient network failure right after the 401), HttpResponseMessage is left as the just-Cleared, empty default instance. The subsequent while-loop condition (IsBlockedByEnvironment / EvaluateResponse, both driven by HttpStatusCode()) then evaluates a status code of 0, which matches none of EvaluateResponse's retry branches (429, 500-599), so the loop silently exits and the caller receives a blank/default response instead of a clear signal that the retried request never completed. Recommendation:
Line mapping was unavailable, so this was posted as an issue comment. 👍 useful · ❤️ especially valuable · 👎 wrong - reply with why |
Declare JobQueueCategoryLbl and JobDescriptionTxt in the codeunit's top-level var block so they are reliably extracted for XLIFF/translation and visible at object-level review. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
The access token is an OAuth credential of the same sensitivity as the refresh token, which this PR already stores encrypted. SetAccessToken now uses SetEncrypted for parity; existing unencrypted values remain readable via IsolatedStorage.Get, and future writes are encrypted. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
After a 401 forces a token refresh, the retried Send's boolean result was discarded; a transient failure then left an empty response that the retry loop would evaluate as status 0. Track the send result and gate the retry loop on it so a failed (re)send exits cleanly instead of evaluating an empty response. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
|
Addressed the latest review findings:
The cross-company-same-store serialization finding remains intentionally open for reviewer input (a tenant-wide token store is a larger change than this migration slice). |
Codeunit.Run cannot nest inside an open write transaction. On a fresh install AddRetentionPolicyAllowedTables (and on upgrade the earlier per-company steps) write before ScheduleRefreshJob runs Codeunit.Run(Job Queue - Enqueue), which would throw at runtime. Commit before the guarded enqueue; this is a one-time install/upgrade action, not a loop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
…esh job COMMIT (explicit or implicit) is not allowed inside install/upgrade triggers; the Commit added in 2260086 crashed OnInstallAppPerCompany (fresh install). Codeunit 'Job Queue - Enqueue' performs no implicit commit, so no prior commit is needed - matching the platform's own JobQueueEntry.EnqueueTask and the existing ShpfyBackgroundSyncs pattern. The guarded Codeunit.Run still prevents an enqueue failure from aborting the install/upgrade. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5fe0c18c-7f03-4f1f-880f-9adb9eb1dfcc
| begin | ||
| if RefreshTokenExpired(RegisteredStoreNew) then begin | ||
| Session.LogMessage('0000UIY', TokenRefreshExpiredTxt, Verbosity::Normal, DataClassification::SystemMetadata, TelemetryScope::ExtensionPublisher, 'Category', CategoryTok); | ||
| Error(RefreshTokenExpiredErr, Store); |
There was a problem hiding this comment.
RefreshAccessToken raises a plain Error(RefreshTokenExpiredErr, Store) whose message explicitly names the fix ('Open the Shopify Shop card and reconnect the store to continue') but offers no way to get there.
The PR already built the pieces for a Show-it/Fix-it action for this exact scenario (Shpfy Authentication Mgt.ReconnectFromNotification and the Shop Card notification action), so when this error surfaces in an interactive context it dead-ends the user instead of using ErrorInfo with AddNavigationAction (Page::"Shpfy Shop Card") or AddAction pointing at the existing reconnect handler. Wrapping the three Error(RefreshTokenExpiredErr, Store) call sites in an ErrorInfo with a navigation/fix-it action would make the error self-service instead of a dead end.
Knowledge:
👍 useful · ❤️ especially valuable · 👎 wrong - reply with why
Background
As of December 2025, Shopify supports expiring offline access tokens with refresh-token rotation. Public apps created before 2026-04-01 must migrate to expiring tokens by 2027-01-01; after that date, REST/GraphQL Admin API requests made with non-expiring tokens are rejected. The Shopify Connector currently uses non-expiring offline tokens and must be migrated.
Token characteristics (Shopify):
expires_in: 3600)refresh_token_expires_in: 7776000)Refs: About offline access tokens - Migrating from non-expiring to expiring tokens
Approach
Token validity is centralized in a single orchestrator,
EnsureValidAccessToken(Store), invoked fromShpfy Communication Mgt.GetAccessToken- so every API call (interactive and background) transparently migrates a legacy token or refreshes an expiring one before use. A scheduled backstop job additionally keeps idle shops access/refresh tokens alive and completes migrations for shops that never make an on-demand call.Changes
Token storage -
Shpfy Registered Store New(table 30138)Token Expires At/Refresh Token Expires At; refresh token kept in IsolatedStorage.Acquisition, refresh & migration -
Shpfy Authentication Mgt.(codeunit 30199)expiring=1).EnsureValidAccessTokenorchestrates migrate-if-legacy / refresh-if-near-expiry, serialized per shop.RefreshAccessToken(grant_type=refresh_token) with transient retry and a terminal reconnect error.MigrateToExpiringToken(one-time token exchange, best-effort - keeps the working token on failure).ForceTokenRefresh,IsRefreshTokenExpired, and a Shop Card reconnect notification.Consumption -
Shpfy Communication Mgt.GetAccessTokeninvokes the orchestrator; reactive 401 -> refresh -> retry inExecuteWebRequest.Scheduled backstop -
Shpfy Token Refresh(30431) +Shpfy Token Refresh Shop(30432)TableNo = Job Queue Entry) iterates enabled shops and runs the per-shop worker (TableNo = Shpfy Shop) viaCodeunit.Runfor isolation; recurring Job Queue entry registered via installer + upgrade.UX -
Shpfy Shop CardDocs & tests
Shpfy Token Refresh Testunit tests; living docs refreshed (business-logic, data-model, patterns, CLAUDE.md).Telemetry
Emitted for migration success/failure, refresh success/transient/terminal and backstop per-shop failures, under event IDs
0000UIV-0000UJ1.Test plan
Shpfy Token Refresh Testunit tests for the expiry decisions.Fixes AB#637954