[Shopify] Migrate Shopify Connector to Expiring Offline Access Tokens

src/Apps/W1/Shopify/App/.docs-updated

@@ -1,4 +1,4 @@
 # Documentation last updated
-commit: 343aa21d1e5737d250aabf05cd9233d15c411cdc
-date: 2026-04-08
+commit: ca6ec0d1b28e22419d9f83f4101ae83b74067c29
+date: 2026-07-11
 scope: full

src/Apps/W1/Shopify/App/CLAUDE.md

@@ -19,6 +19,10 @@ All Shopify API communication goes through GraphQL. The `ShpfyCommunicationMgt.C
 
 *Updated: 2026-03-24 -- GraphQL resource file refactoring*
 
+Authentication uses Shopify **expiring offline access tokens**. On install the connector requests an expiring token (`expiring=1`) and stores it -- with its `Token Expires At` and 90-day `Refresh Token Expires At` -- on `Shpfy Registered Store New`, keyed by store URL (access and refresh tokens live in IsolatedStorage as `SecretText`). `ShpfyAuthenticationMgt.EnsureValidAccessToken`, called from `ShpfyCommunicationMgt` before every request, transparently refreshes a near-expiry token and performs the one-time (irreversible) migration of legacy non-expiring tokens via token exchange. A recurring backstop job (`Shpfy Token Refresh`) keeps tokens and their refresh tokens alive for idle shops.
+
+*Updated: 2026-07-11 -- Expiring offline access token support (slice 637954)*
+
 Mapping strategies are interface-driven throughout. Customer mapping (`ICustomerMapping`) selects between by-email/phone, by-bill-to, or default-customer strategies. Company mapping (`ICompanyMapping`) can match by email/phone or tax ID. Stock calculation uses `IStockAvailable` and `IStockCalculation` interfaces. Product status on creation, removal actions for blocked items, county resolution, and customer name formatting are all interface-backed enums. The Shop record's enum fields (e.g., `"Customer Mapping Type"`, `"Stock Calculation"`, `"Status for Created Products"`) select which implementation to use at runtime.
 
 Sync is incremental via the Synchronization Info table (`ShpfySynchronizationInfo.Table.al`), which stores the last sync timestamp per shop and sync type. An empty/zero date falls back to a sentinel value of `2004-01-01` (see `GetEmptySyncTime()`). Products use hash-based change detection -- the Product table stores `"Image Hash"`, `"Tags Hash"`, and `"Description Html Hash"` fields, computed via a custom hash algorithm in `ShpfyHash`, to avoid unnecessary API calls when nothing has actually changed.
@@ -62,6 +66,7 @@ Records link to BC entities via SystemId (GUID), not Code/No. For example, `Shpf
 
 - The Shop table is the god object -- nearly every configuration setting lives there, with over 100 fields controlling sync directions, mapping strategies, account mappings, plan-based feature flags, webhook config, and more. The `"Advanced Shopify Plan"` field gates features requiring Plus/Advanced plans (currently staff members). B2B features are now unconditionally available on all plans.
 - All API calls go through GraphQL, never REST. Queries are `.graphql` resource files in `.resources/graphql/{Area}/`, loaded via `NavApp.GetResourceAsText()` and dispatched through the `ShpfyGraphQLType` enum.
+- Authentication uses expiring offline access tokens: the connector requests `expiring=1` on install, refreshes the 1-hour access token before expiry (rotating the 90-day refresh token), and migrates legacy non-expiring tokens on first use. `EnsureValidAccessToken` (from `ShpfyCommunicationMgt.GetAccessToken`) handles this on demand; the recurring `Shpfy Token Refresh` job is the backstop. If the refresh token lapses (90 days idle), the merchant must reconnect the shop from the Shop Card. Migration is one-time and irreversible per shop.
 - Products use hash-based change detection (`"Image Hash"`, `"Tags Hash"`, `"Description Html Hash"`) via a custom hash algorithm to skip unnecessary API calls when nothing has changed.
 - Records link to BC entities via SystemId (GUID), not Code/No. -- FlowFields like `"Item No."` display the human-readable values via CalcFormula lookup. Renumbering BC items does not break Shopify links.
 - Orders store every monetary amount in dual currency: shop currency fields (`"Total Amount"`, `"VAT Amount"`) and presentment/customer-facing currency fields (`"Presentment Total Amount"`, `"Presentment VAT Amount"`). The `"Currency Handling"` setting on Shop controls which is used for BC documents.

src/Apps/W1/Shopify/App/docs/business-logic.md

@@ -2,6 +2,22 @@
 
 This document covers the major processing flows in the Shopify Connector, focusing on decision points, non-obvious behavior, and what can go wrong.
 
+## Authentication and token lifecycle
+
+The connector authenticates to Shopify with **expiring offline access tokens** (public apps must migrate to these by 2027-01-01). An access token lives for 1 hour; a refresh token lives for 90 days and is rotated on each refresh. Tokens are held on `Shpfy Registered Store New` (keyed by store URL) -- the access and refresh tokens as `SecretText` in IsolatedStorage, with `Token Expires At` and `Refresh Token Expires At` as fields.
+
+`ShpfyCommunicationMgt.GetAccessToken` calls `ShpfyAuthenticationMgt.EnsureValidAccessToken` before every API request, so the following happens transparently in any session, interactive or background:
+
+- **Fast path** -- a valid expiring token is used as-is.
+- **Refresh** -- when the access token is within a 5-minute buffer of expiry, `RefreshAccessToken` exchanges the refresh token (`grant_type=refresh_token`) for a new access + refresh token. Transient failures are retried with the *same* refresh token (Shopify returns the same response for ~1 hour); a 401 or a lapsed refresh token is terminal and surfaces a "reconnect the store" error.
+- **Migration** -- a legacy non-expiring token (no refresh token stored) is migrated once via token exchange. This is best-effort: if it fails the existing token is kept (it still works until the deadline); on success Shopify revokes the old token, so the exchange is irreversible per shop. Because a legacy store has no refresh token, the fast-path check never short-circuits it, so migration is throttled by a `Last Migration Attempt` timestamp (retried at most once per hour) to avoid re-attempting a failing migration on every API call in a sync loop.
+
+Refresh and migration are serialized with a table lock plus a double-checked re-read, because Shopify allows only one refreshable token per app and store -- this keeps concurrent sessions (or multiple BC companies sharing a shop) from thrashing the token. As a safety net, an unexpected 401 from a normal API call triggers a single forced refresh and retry.
+
+The `Shpfy Token Refresh` job (a recurring Job Queue entry registered on install and upgrade) is the proactive backstop. It iterates enabled shops and runs a per-shop worker so one shop's failure does not abort the run, keeping access tokens and 90-day refresh tokens alive for shops that are otherwise idle. When a refresh token has fully lapsed, the Shop Card shows a reconnect notification.
+
+*Updated: 2026-07-11 -- Expiring offline access token support (slice 637954)*
+
 ## Product synchronization
 
 Product sync is bi-directional, controlled by the Shop's `"Sync Item"` setting (To Shopify, From Shopify, or disabled). The two directions have fundamentally different architectures.

src/Apps/W1/Shopify/App/docs/data-model.md

@@ -14,12 +14,17 @@ The Shop table includes plan-based feature flags. The `"Advanced Shopify Plan"`
 
 Shop Location maps Shopify fulfillment locations to BC warehouse locations. Each mapping includes a stock calculation enum that determines how to compute available stock for that pairing.
 
+Registered Store (`Shpfy Registered Store New`) holds the OAuth state for each connected shop, keyed by store URL (not Shop Code): the requested and actual scopes, plus expiring-token metadata (`Token Expires At`, `Refresh Token Expires At`, and `Last Migration Attempt` for migration throttling). The access and refresh tokens themselves are not table fields -- they are stored as `SecretText` in IsolatedStorage (module scope), keyed by the record's SystemId (the refresh token via `SetEncrypted`). This is a per-app+store credential, so multiple BC companies connected to the same shop each keep their own copy. See business-logic.md for the token lifecycle.
+
+*Updated: 2026-07-11 -- Expiring offline access token metadata (slice 637954)*
+
 ```mermaid
 erDiagram
     SHOP ||--o{ SYNCHRONIZATION_INFO : tracks
     SHOP ||--o{ SHOP_LOCATION : maps
     SHOP ||--o{ PRODUCT : contains
     SHOP ||--o{ ORDER_HEADER : contains
+    SHOP ||..|| REGISTERED_STORE : "OAuth tokens (by URL)"
 ```
 
 The Shop's `GetEmptySyncTime()` returns `2004-01-01` as a sentinel for "never synced" -- not `0DT`. This date is far enough in the past to import all data on first sync but avoids edge cases with zero-date handling in AL. The `SetLastSyncTime()` method stores `CurrentDateTime` after a successful sync. When the next sync runs, it passes this timestamp to Shopify's `updated_at` filter to fetch only changed records.

src/Apps/W1/Shopify/App/docs/patterns.md

@@ -159,6 +159,14 @@ CustomerNo := ICustomerMapping.DoMapping(...);
 
 This pattern repeats for stock calculation, product status, removal actions, customer name formatting, county resolution, company mapping, return/refund processing, metafield types, metafield owners, bulk operations, and document link handlers. Once you understand one instance, you understand them all.
 
+## Job Queue dispatcher and per-shop worker
+
+Background work that must run per shop with error isolation uses a two-codeunit split. The scheduled `Shpfy Token Refresh` codeunit has `TableNo = "Job Queue Entry"` and iterates the enabled shops. For each shop it calls `Codeunit.Run` on a separate worker, `Shpfy Token Refresh Shop`, which has `TableNo = "Shpfy Shop"`. Running the worker via `Codeunit.Run` gives each shop its own transaction, so a failure for one shop is caught and logged without aborting the whole run.
+
+The `TableNo` on the dispatcher is not optional: the Job Queue dispatcher passes the `Job Queue Entry` record to the codeunit it runs, so a Job-Queue-scheduled codeunit must use `TableNo = "Job Queue Entry"` (or none). Giving it the record it actually processes (e.g. `"Shpfy Shop"`) fails at runtime with "Record(472) is not compatible". Keep the dispatcher on `"Job Queue Entry"` and delegate record-specific work to a worker codeunit.
+
+*Updated: 2026-07-11 -- Token refresh backstop (slice 637954)*
+
 ## Legacy patterns
 
 ### Config Template Header (removed in v25)

src/Apps/W1/Shopify/App/src/Base/Codeunits/ShpfyCommunicationMgt.Codeunit.al

@@ -212,6 +212,7 @@ codeunit 30103 "Shpfy Communication Mgt."
         HttpRequestMessage: HttpRequestMessage;
         HttpResponseMessage: HttpResponseMessage;
         RetryCounter: Integer;
+        Sent: Boolean;
     begin
         FeatureTelemetry.LogUptake('0000HUV', 'Shopify', Enum::"Feature Uptake Status"::Used);
         if CheckOutgoingRequest then
@@ -230,17 +231,25 @@ codeunit 30103 "Shpfy Communication Mgt."
                 Sleep(Wait);
         end;
 
-        if HttpClient.Send(HttpRequestMessage, HttpResponseMessage) then begin
+        Sent := HttpClient.Send(HttpRequestMessage, HttpResponseMessage);
+        if Sent then begin
+            if HandleUnauthorizedResponse(HttpResponseMessage) then begin
+                Clear(HttpClient);
+                Clear(HttpRequestMessage);
+                Clear(HttpResponseMessage);
+                CreateHttpRequestMessage(Url, Method, Request, HttpRequestMessage);
+                Sent := HttpClient.Send(HttpRequestMessage, HttpResponseMessage);
+            end;
             Clear(RetryCounter);
-            while (not HttpResponseMessage.IsBlockedByEnvironment) and (EvaluateResponse(HttpResponseMessage)) and (RetryCounter < MaxRetries) do begin
+            while Sent and (not HttpResponseMessage.IsBlockedByEnvironment) and (EvaluateResponse(HttpResponseMessage)) and (RetryCounter < MaxRetries) do begin
                 RetryCounter += 1;
                 Sleep(1000);
                 LogShopifyRequest(Url, Method, Request, HttpResponseMessage, Response, RetryCounter);
                 Clear(HttpClient);
                 Clear(HttpRequestMessage);
                 Clear(HttpResponseMessage);
                 CreateHttpRequestMessage(Url, Method, Request, HttpRequestMessage);
-                HttpClient.Send(HttpRequestMessage, HttpResponseMessage);
+                Sent := HttpClient.Send(HttpRequestMessage, HttpResponseMessage);
             end;
         end;
         if GetContent(HttpResponseMessage, Response) then;
@@ -374,6 +383,7 @@ codeunit 30103 "Shpfy Communication Mgt."
         NoAccessTokenErr: label 'No Access token for the store "%1".\Please request an access token for this store.', Comment = '%1 = Store';
         ChangedScopeErr: Label 'The application scope is changed, please request a new access token for the store "%1".', Comment = '%1 = Store';
     begin
+        AuthenticationMgt.EnsureValidAccessToken(Store);
         if RegisteredStoreNew.Get(Store) then
             if RegisteredStoreNew."Requested Scope" = AuthenticationMgt.GetScope() then begin
                 AccessToken := RegisteredStoreNew.GetAccessToken();
@@ -528,6 +538,22 @@ codeunit 30103 "Shpfy Communication Mgt."
         end;
     end;
 
+    local procedure HandleUnauthorizedResponse(HttpResponseMessage: HttpResponseMessage): Boolean
+    var
+        AuthenticationMgt: Codeunit "Shpfy Authentication Mgt.";
+        Store: Text;
+    begin
+        // An expiring offline token may have been retired unexpectedly. Force a single refresh
+        // (or migration) so the request can be retried with a fresh token.
+        if HttpResponseMessage.HttpStatusCode() <> 401 then
+            exit(false);
+        Store := Shop.GetStoreName();
+        if Store = '' then
+            exit(false);
+        AuthenticationMgt.ForceTokenRefresh(Store);
+        exit(true);
+    end;
+
     /// <summary> 
     /// Description for SetShop.
     /// </summary>

src/Apps/W1/Shopify/App/src/Base/Codeunits/ShpfyInstaller.Codeunit.al

@@ -21,6 +21,14 @@ codeunit 30273 "Shpfy Installer"
     begin
         AddRetentionPolicyAllowedTables();
         AddShopifyCueSetup();
+        ScheduleTokenRefreshJob();
+    end;
+
+    local procedure ScheduleTokenRefreshJob()
+    var
+        TokenRefresh: Codeunit "Shpfy Token Refresh";
+    begin
+        TokenRefresh.ScheduleRefreshJob();
     end;
 
     procedure AddRetentionPolicyAllowedTables()

src/Apps/W1/Shopify/App/src/Base/Codeunits/ShpfyUpgradeMgt.Codeunit.al

@@ -45,6 +45,7 @@ codeunit 30106 "Shpfy Upgrade Mgt."
         OrderTransactionShopCodeUpgrade();
         HasAdvancedShopifyPlanUpgrade();
         ItalianSardinianProvinceRenameUpgrade();
+        ScheduleTokenRefreshJobUpgrade();
     end;
 
     internal procedure UpgradeTemplatesData()
@@ -535,6 +536,19 @@ codeunit 30106 "Shpfy Upgrade Mgt."
         UpgradeTag.SetUpgradeTag(GetHasAdvancedShopifyPlanUpgradeTag());
     end;
 
+    local procedure ScheduleTokenRefreshJobUpgrade()
+    var
+        UpgradeTag: Codeunit "Upgrade Tag";
+        TokenRefresh: Codeunit "Shpfy Token Refresh";
+    begin
+        if UpgradeTag.HasUpgradeTag(GetScheduleTokenRefreshJobUpgradeTag()) then
+            exit;
+
+        TokenRefresh.ScheduleRefreshJob();
+
+        UpgradeTag.SetUpgradeTag(GetScheduleTokenRefreshJobUpgradeTag());
+    end;
+
     internal procedure GetAllowOutgoingRequestseUpgradeTag(): Code[250]
     begin
         exit('MS-445989-AllowOutgoingRequestseUpgradeTag-20220816');
@@ -615,6 +629,11 @@ codeunit 30106 "Shpfy Upgrade Mgt."
         exit('MS-630316-HasAdvancedShopifyPlanUpgrade-20260408');
     end;
 
+    local procedure GetScheduleTokenRefreshJobUpgradeTag(): Code[250]
+    begin
+        exit('MS-637954-ScheduleTokenRefreshJob-20260711');
+    end;
+
     local procedure ItalianSardinianProvinceRenameUpgrade()
     var
         UpgradeTag: Codeunit "Upgrade Tag";
@@ -674,5 +693,6 @@ codeunit 30106 "Shpfy Upgrade Mgt."
         PerCompanyUpgradeTags.Add(GetOrderTransactionShopCodeUpgradeTag());
         PerCompanyUpgradeTags.Add(GetHasAdvancedShopifyPlanUpgradeTag());
         PerCompanyUpgradeTags.Add(GetItalianSardinianProvinceRenameUpgradeTag());
+        PerCompanyUpgradeTags.Add(GetScheduleTokenRefreshJobUpgradeTag());
     end;
 }

src/Apps/W1/Shopify/App/src/Base/Pages/ShpfyShopCard.Page.al

@@ -1225,6 +1225,8 @@ page 30101 "Shpfy Shop Card"
         ApiVersion: Text;
         ApiVersionExpiryDate: Date;
         ScopeChangeConfirmLbl: Label 'The access scope of shop %1 for the Shopify connector has changed. Do you want to request a new access token?', Comment = '%1 - Shop Code';
+        RefreshTokenExpiredNotificationLbl: Label 'The connection to Shopify shop %1 has expired. Reconnect the shop to continue synchronizing.', Comment = '%1 - Shop Code';
+        ReconnectActionLbl: Label 'Reconnect';
         ConnectionSuccessfulMsg: Label 'Connection successful.';
         ConnectionAndWebhooksSuccessfulMsg: Label 'Connection successful and auto synchronize orders is configured correctly.';
         OrderCreatedWebhookNotConfiguredMsg: Label 'Connection successful, but auto synchronize orders is misconfigured. Reactivate Auto Sync Orders setting.';
@@ -1235,6 +1237,7 @@ page 30101 "Shpfy Shop Card"
         AuthenticationMgt: Codeunit "Shpfy Authentication Mgt.";
         CommunicationMgt: Codeunit "Shpfy Communication Mgt.";
         ShopReview: Codeunit "Shpfy Shop Review";
+        RefreshTokenExpiredNotification: Notification;
         ApiVersionExpiryDateTime: DateTime;
     begin
         FeatureTelemetry.LogUptake('0000HUU', 'Shopify', Enum::"Feature Uptake Status"::Discovered);
@@ -1257,6 +1260,13 @@ page 30101 "Shpfy Shop Card"
             Rec.UpdateFulfillmentService();
 #endif
             ShopReview.MaybeShowReviewReminder(Rec.GetStoreName());
+
+            if AuthenticationMgt.IsRefreshTokenExpired(Rec.GetStoreName()) then begin
+                RefreshTokenExpiredNotification.Message(StrSubstNo(RefreshTokenExpiredNotificationLbl, Rec.Code));
+                RefreshTokenExpiredNotification.SetData('ShopCode', Rec.Code);
+                RefreshTokenExpiredNotification.AddAction(ReconnectActionLbl, Codeunit::"Shpfy Authentication Mgt.", 'ReconnectFromNotification');
+                RefreshTokenExpiredNotification.Send();
+            end;
         end;
     end;
 

src/Apps/W1/Shopify/App/src/Base/docs/CLAUDE.md

@@ -21,6 +21,18 @@ GraphQL queries. `Shpfy Communication Events` publishes internal events
 for every API interaction (`OnClientSend`, `OnClientPost`, `OnClientGet`,
 `OnGetContent`, `OnGetAccessToken`) -- tests use these to mock responses.
 
+`GetAccessToken` ensures a valid expiring offline access token before each
+request via `ShpfyAuthenticationMgt.EnsureValidAccessToken` (refresh a
+near-expiry token or migrate a legacy non-expiring one, on demand), and
+`ExecuteWebRequest` forces a single token refresh and retry on an
+unexpected 401. The `Shpfy Token Refresh` dispatcher plus
+`Shpfy Token Refresh Shop` worker are a recurring backstop that keeps
+tokens and their 90-day refresh tokens alive; `Shpfy Installer` and
+`Shpfy Upgrade Mgt.` register that Job Queue entry. See the app-level
+business-logic.md for the full token lifecycle.
+
+*Updated: 2026-07-11 -- Expiring offline access token support (slice 637954)*
+
 `Shpfy Background Syncs` orchestrates all sync operations via Job Queue,
 splitting between background-allowed and foreground-only shops.
 
@@ -61,5 +73,6 @@ syncs companies and catalog prices regardless of plan.
 - The `Shpfy Cue` table uses FlowFields for role center counts: unmapped
   customers/products/companies, unprocessed orders/shipments, sync errors.
 - Empty sync time sentinel is `2004-01-01` (`GetEmptySyncTime()`), not `0DT`.
+- Authentication uses expiring offline access tokens: `EnsureValidAccessToken` (from `GetAccessToken`) refreshes before expiry and migrates legacy non-expiring tokens; the `Shpfy Token Refresh` dispatcher + `Shpfy Token Refresh Shop` worker are the proactive backstop, registered as a Job Queue entry on install/upgrade. A lapsed 90-day refresh token requires reconnecting the shop from the Shop Card.
 - Three page extensions embed Shopify Activities into standard role centers.
 - `ShpfyConnectorGuide` and `ShpfyInitialImport` provide first-time setup.