Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 25 additions & 44 deletions .claude/skills/playwright-triage/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,39 +1,23 @@
---
name: playwright-triage
description: Triage and fix one microsoft/playwright-python issue, or do a maintainer's task, unattended inside the Issue triage GitHub Actions workflow (.github/workflows/triage.yml), which a needs-triage, needs-triage-mac or needs-triage-win label starts, or a maintainer starts by hand with an issue, a task or both. Reproduces the issue on the runner, then leaves a verified fix proposal for the workflow to open as a draft PR, an investigation note for it to post on the issue, or an upstream bug report for a maintainer to file in microsoft/playwright. Only for that workflow.
description: Triage and fix an issue in microsoft/playwright-python. How to use the upstream checkout, how to run a published version, how to tell a bug of the Python client from an upstream one, the upstream bug report, and what not to run from a PR of a third party.
---

# Playwright Python Issue Triage

You run unattended in the [Issue triage](../../../.github/workflows/triage.yml) workflow of
`microsoft/playwright-python`. First read the shared guide, `$AGENT_GUIDES_DIR/triage.md`, a copy of
[microsoft/playwright/.github/actions/run-agent/guides/triage.md](https://github.com/microsoft/playwright/blob/main/.github/actions/run-agent/guides/triage.md),
and then [AGENTS.md](../../../AGENTS.md), both in full. The guide says how to triage an issue in any
Playwright repo. AGENTS.md says how to build, test and lint this one, and where a regression test
goes. This file covers what neither has: the checkouts, how to run a published version, and the
outcome that this repo adds, an upstream bug report.

## The checkouts

`$REPO_DIR` is the checkout of `microsoft/playwright-python` that the guide describes. Its requirements
are installed (`pip install -r local-requirements.txt`, `pip install -r requirements.txt`,
`pip install -e .`), the wheel is built (`python -m build --wheel`, which builds the driver into
`playwright/driver/`), and Chromium, Firefox and WebKit are installed with
`python -m playwright install --with-deps`. It is the working directory, and the checkout that the
guides refer to. The clone is shallow: run `git fetch --unshallow --tags origin` in it before you
bisect or use `git tag --contains`.

`$PLAYWRIGHT_DIR` is `$REPO_DIR/.upstream/`, a full clone of `microsoft/playwright` at the commit
in `DRIVER_VERSION`, created by `python scripts/upstream.py checkout`. The driver was built from it.
Read upstream code there. Do not edit, commit or switch branches in it: the wheel build and
`./scripts/lint.sh` stop when it is not at that commit or has local changes. Before you run upstream
tests in it, it needs `npm ci`, `npm run build` and `npx playwright install`. The wheel build
(`scripts/build_driver.py`) ran the first two there; run them again when `node_modules/` is missing.
## Upstream

`.upstream/` is the clone of `microsoft/playwright` at the commit in `DRIVER_VERSION`, as
[AGENTS.md](../../../AGENTS.md) says. The driver is built from it. Read upstream code there. Do not
edit, commit or switch branches in it: the wheel build and `./scripts/lint.sh` stop when it is not
at that commit or has local changes. Before you run upstream tests in it, it needs `npm ci`,
`npm run build` and `npx playwright install`. The wheel build (`scripts/build_driver.py`) runs the
first two there; run them again when `node_modules/` is missing.

## Published versions

To run a published version, for example to bisect, create `$SCRATCH_DIR/playwright-python-<n>/<version>/`
and run there:
To run a published version, for example to bisect, create an empty directory outside the repo for
each version, and run there:

```bash
python -m venv env && source env/bin/activate # env\Scripts\activate on Windows
Expand All @@ -43,41 +27,38 @@ python -m playwright install

The fixtures of the repo are not available there. Write the condensed repro as a plain script with
`sync_playwright()` or `async_playwright()` in that directory, with `page.set_content()` in place of
the test server, and run it from there. In `$REPO_DIR`, the source tree shadows the installed package.
the test server, and run it from there. In this repo, the source tree shadows the installed
package.

The driver of a release is in `DRIVER_VERSION` at its tag: `git show v<version>:DRIVER_VERSION`
prints the upstream commit it was built from, or, for releases up to 1.64, the `playwright-core`
version of the driver; `npm view playwright-core@<core version> gitHead` prints its commit. Match a
regression in the driver to commits with `git -C $PLAYWRIGHT_DIR log <good>..<bad> -- <path>`.
regression in the driver to commits with `git -C .upstream log <good>..<bad> -- <path>`.

## Upstream bugs

The client talks to the Node driver that is built from `$PLAYWRIGHT_DIR`, and `microsoft/playwright`
The client talks to the Node driver that is built from `.upstream/`, and `microsoft/playwright`
and this repo are one team. Reproduce the bug in Python first. Then write the same calls as an
upstream test and run it in `$PLAYWRIGHT_DIR`. The triage skill of `microsoft/playwright`, in
`$PLAYWRIGHT_DIR/.claude/skills/playwright-triage/`, says where such a test goes, which fixtures it
uses and how to run it. The rest of that skill is not for this run. Put the test in a new spec
file, and delete the file when you are done, so that the checkout is clean.
upstream test and run it in `.upstream/`. The triage skill of `microsoft/playwright`, in
`.upstream/.claude/skills/playwright-triage/`, says where such a test goes, which fixtures it uses
and how to run it. Use only those parts of it. Put the test in a new spec file, and delete the file
when you are done, so that the checkout is clean.

- The Node test passes: the bug is in the Python client. Fix it here.
- The Node test fails too: the root cause is upstream, and the fix lands in `microsoft/playwright`.
Do not work around it in `playwright/_impl/`. Run `git -C $PLAYWRIGHT_DIR fetch origin main`.
When `git -C $PLAYWRIGHT_DIR log HEAD..origin/main -- <path>` shows a fix that already landed,
the note says that it reaches this repo with the next roll. Otherwise write the report.
Do not work around it in `playwright/_impl/`. Run `git -C .upstream fetch origin main`. When
`git -C .upstream log HEAD..origin/main -- <path>` shows a fix that already landed, it reaches
this repo with the next roll: say so. Otherwise write the report.
- The Python API lacks a member or an option: the generated API has everything that the upstream
docs in `$PLAYWRIGHT_DIR/docs/src/api` give Python (`* langs:` decides; no `langs` means every
docs in `.upstream/docs/src/api` give Python (`* langs:` decides; no `langs` means every
language). The gap is in the docs, so it is an upstream change too. Do not edit a generated file,
and do not add to `scripts/codegen/python_rules.js` to go around the docs.

The report is for an engineer of `microsoft/playwright` who did not read the Python issue: what
fails, expected versus actual, the versions (the Python package, the commit in `DRIVER_VERSION`, the
browser, the platform), the root cause when you found it, and the condensed Node test as the repro.
The title has at most 256 characters, for example `route.fulfill() drops the charset of
Content-Type`.

The bot never files it in `microsoft/playwright`: this run allows no `issue` call (the calls are
in `$AGENT_GUIDES_DIR/calls.d.ts`). Put the report at the end of your comment, for a maintainer to
file, in this form:
Content-Type`. A maintainer files it in `microsoft/playwright`. Give it in this form:

`````markdown
Upstream bug report, for a maintainer to file in microsoft/playwright:
Expand All @@ -94,7 +75,7 @@ Upstream bug report, for a maintainer to file in microsoft/playwright:
</details>
`````

Write the rest of the comment as the note: the verdict, the evidence, and that the bug is upstream.
With it, give the verdict, the evidence, and that the bug is upstream.

## PRs from third parties

Expand Down
51 changes: 0 additions & 51 deletions .github/actions/setup-triage-runner/action.yml

This file was deleted.

33 changes: 33 additions & 0 deletions .github/workflows/review.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Hands the pull request to the Playwright bot when a maintainer applies agent-review. Nothing
# else of the bot lives in this repo. Apply the label again to start another run.
name: "Agent: review"

on:
# Runs from the default branch, for PRs from forks too. It never checks out the PR.
pull_request_target:
types: [labeled]

permissions: {}

jobs:
pr:
name: Hand the PR to the bot
if: github.repository_owner == 'microsoft' && github.event.label.name == 'agent-review'
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0
id: app-token
with:
client-id: ${{ vars.PLAYWRIGHT_APP_CLIENT_ID }}
private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }}
owner: microsoft
repositories: devops.playwright.dev
permission-actions: write
- name: Start the bot
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
NUMBER: ${{ github.event.pull_request.number }}
run: >-
gh workflow run pr.yml --repo microsoft/devops.playwright.dev --ref main
-f repo="$GITHUB_REPOSITORY" -f pr="$NUMBER"
133 changes: 0 additions & 133 deletions .github/workflows/revision.yml

This file was deleted.

Loading
Loading