test(e2e): wait for dashboard metrics before asserting layout - #2434
Conversation
The two dashboard layout tests read `dashboard-metric` cards immediately after `waitForLoading()`, which only waits on the text "Loading" and `.animate-spin`. The dashboard renders Kumo `SkeletonLine` placeholders while the stats query is in flight, so neither selector matches and the wait returns straight away. `expect(await count())` takes a single snapshot with no retry, so a slow stats response fails the run -- observed on the Cloudflare E2E shard as "Expected: >= 3, Received: 0" with the page snapshot still showing the skeleton state. Use `expect.poll` for the card count and a web-first visibility assertion before the tracking test's `evaluate`, both of which retry. Verified by delaying the dashboard stats response 3s locally: both tests fail without these changes and pass with them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
github-actions
Bot
added
size/XS
review/needs-review
![emdashbot[bot]](https://avatars.githubusercontent.com/in/3255304?s=60&v=4){kind=small}
There was a problem hiding this comment.
This is the right change for the reported flake. The race happens because waitForLoading() only waits for the literal "Loading" text and .animate-spin to disappear, while the dashboard renders SkeletonLine placeholders during the separate dashboard-stats fetch. Turning the snapshot-style assertion into a polling web-first assertion (expect.poll for the metric count, toBeVisible() before the typography evaluate) makes the tests wait for the actual DOM they inspect instead of locking in a moment while skeletons are still displayed.
The fix is localized to the two failing tests in e2e/tests/accessibility.spec.ts, which is correct: waitForLoading() is a generic helper used across many pages, so teaching it dashboard-specific skeletons would be the wrong scope. No product code is changed, so the PR correctly omits a changeset and skips the i18n/Discussion checklist items.
I checked the changed file, the fixture helper, and the dashboard component. The new assertions use Playwright's expect re-exported from @playwright/test, so expect.poll is available. The await on expect.poll(...) and on the toBeVisible() call are both present. No other E2E tests reference dashboard-metric, so there are no sibling assertions left with the same race. No AGENTS.md conventions are violated: no new UI strings, SQL, API routes, auth, localization issues, logged-out queries, indexes, or comment problems.
Clean test-only fix.
@emdash-cms/admin
@emdash-cms/auth
@emdash-cms/auth-atproto
@emdash-cms/blocks
@emdash-cms/cloudflare
@emdash-cms/contentful-to-portable-text
emdash
create-emdash
@emdash-cms/gutenberg-to-portable-text
@emdash-cms/plugin-cli
@emdash-cms/plugin-types
@emdash-cms/registry-client
@emdash-cms/registry-lexicons
@emdash-cms/registry-verification
@emdash-cms/sandbox-workerd
@emdash-cms/x402
@emdash-cms/plugin-ai-moderation
@emdash-cms/plugin-atproto
@emdash-cms/plugin-audit-log
@emdash-cms/plugin-color
@emdash-cms/plugin-embeds
@emdash-cms/plugin-field-kit
@emdash-cms/plugin-forms
@emdash-cms/plugin-webhook-notifier
commit: |
2 participants
What does this PR do?
Fixes a race in the two dashboard layout E2E tests that fails runs on the Cloudflare shard.
Both tests read
dashboard-metriccards immediately afteradmin.waitForLoading(). That helper only waits for the textLoadingand for.animate-spinto disappear (e2e/fixtures/admin.ts:171), but the dashboard renders KumoSkeletonLineplaceholders while thedashboard-statsquery is in flight (packages/admin/src/components/Dashboard.tsx:133-152) — the skeleton cards carry nodata-testid. Neither selector matches, both waits time out silently into.catch(() => {}), and the helper returns immediately.expect(await metricCards.count())is a plain assertion rather than a web-first one, so it takes a single snapshot with no retry. When the stats fetch is slow the count is0:The failure artifact's page snapshot confirms the loading state —
maincontained only the<h1>, quick-action links, and the "Content"/"Recent Activity" headings.The fix uses assertions that retry:
expect.pollfor the card count, and a web-first visibility check before the tracking test'sevaluate(that one throwsDashboard typography is missingunder the same race).No product code changes — the dashboard behaves correctly; only the tests were reading it too early.
Type of change
Checklist
pnpm typecheckpassespnpm lintpassespnpm testpasses (or targeted tests for my change)pnpm formathas been runAI-generated code disclosure
Screenshots / test output
Verified by temporarily delaying the dashboard stats response by 3s locally, which makes the race deterministic.
Before (delay, original assertions) — both tests fail, reproducing CI exactly:
After (same 3s delay, with this change) — both pass.
Clean run of all three dashboard tests without the delay:
The temporary delay was local-only and is not part of the diff.