AI test automation with Playwright: generate, heal and trust E2E tests

AI test automation with Playwright means letting a coding agent explore your app in a real browser, write Playwright tests from what it sees and repair tests when they break. This guide covers Playwright's built-in test agents, the MCP and CLI options, prompts that produce maintainable tests, handling flaky tests and a safe CI setup.

What is AI test automation with Playwright?

AI test automation with Playwright is using an AI agent to write, run and maintain Playwright end-to-end (E2E) tests. The agent opens your app in a real browser, explores a flow such as sign-up or checkout, writes a test file in Playwright's normal syntax and runs it to confirm it passes. When the UI changes and tests break, the agent can investigate and repair them.

The result is ordinary Playwright code in your repository. Nothing about the test runner changes, which matters: you can read, review, debug and run the tests without any AI involved.

What AI changes is the expensive part of E2E testing:

• Writing the first version of a test for a flow nobody has covered yet.
• Finding stable locators by inspecting the real page instead of guessing.
• Repairing tests after a deliberate UI change renames a button or adds a step.
• Diagnosing failures from traces, logs and screenshots.

What AI does not change is the need to decide what is worth testing and to check that each test asserts something meaningful. For unit-level testing with agents, see our guide to writing tests with AI.

What are Playwright's planner, generator and healer agents?

Playwright ships three agent definitions that split test automation into steps. The Playwright test agents docs describe them this way:

You add the definitions to your project with one command, choosing the agent loop you use:

npx playwright init-agents --loop=claude

The documented loops are vscode, claude, codex and opencode. The planner works from a seed test (seed.spec.ts), which sets up a ready-to-use page, for example signed in and on the dashboard, so every generated test starts from a known state. The docs recommend regenerating the agent definitions whenever you update Playwright, so they pick up new tools and instructions.

The three agents can run on their own or as a chain. A typical first pass is: ask the planner for a plan covering one feature, edit the plan by hand (this is where your judgment goes), then let the generator write the tests and the healer fix any that fail on the first run.

How do agents control the browser: MCP or CLI?

A coding agent needs a way to drive a real browser so it can see the page it is testing. Microsoft provides two: the Playwright MCP server and the Playwright CLI for agents. Both let the agent navigate, click, fill forms and read the page through structured snapshots rather than screenshots.

Playwright MCP exposes browser actions as MCP tools. It works from Playwright's accessibility tree, so no vision model is needed. Adding it to Claude Code is one command, from the Playwright MCP README:

claude mcp add playwright npx @playwright/mcp@latest

Useful options include --headless, --isolated (keep the browser profile in memory) and --browser to choose Chrome, Firefox, WebKit or Edge.

The Playwright CLI for agents (playwright-cli, from the @playwright/cli package) gives the agent shell commands instead of MCP tools, plus skill files that teach it the workflow. Install the skills into your project for Claude Code with:

playwright-cli install --skills

Which one to use:

The Playwright MCP README itself suggests the CLI for coding agents, because it avoids loading large tool schemas and verbose accessibility trees into context. If MCP is new to you, our explainer on what MCP is covers servers, tools and security basics.

How do you prompt an agent to write good Playwright tests?

Give the agent the flow, the expected outcome, the conventions your suite follows and an instruction to run the test before declaring it done. Without conventions, agents tend to produce long tests with brittle selectors and fixed waits that pass once and break next week.

Put the conventions in your agent instruction file (CLAUDE.md or AGENTS.md) so every session follows them. They come straight from Playwright's best practices:

## E2E tests (Playwright)- Locators: getByRole, getByLabel, getByText or getByTestId. Never CSS classes or XPath.- Assertions: web-first only, e.g. await expect(locator).toBeVisible().  Never expect(await locator.isVisible()).toBe(true).- No page.waitForTimeout(). Wait for a visible state or a network response.- Each test is independent. Use the auth setup project for signed-in state.- Mock third-party services with page.route(); only test what we control.- Run `npx playwright test <file>` and make it pass before finishing.

Then a task prompt for one flow:

Write a Playwright test for the password reset flow.Start from tests/seed.spec.ts (signed out, on the home page). Use the browserto walk through the flow yourself first: click "Forgot password", submit theemail test-user@example.com, and confirm the success message.The test should assert: the form validates an empty email, a valid email shows"Check your inbox", and the request to /api/auth/reset returns 200. Mock theemail service with page.route(). Save it as tests/auth/password-reset.spec.ts,run it three times, and show me the final file.

What a good result looks like:

import { test, expect } from "@playwright/test";test("user can request a password reset", async ({ page }) => {  await page.goto("/");  await page.getByRole("link", { name: "Forgot password" }).click();  await page.getByRole("button", { name: "Send reset link" }).click();  await expect(page.getByText("Enter your email")).toBeVisible();  const resetRequest = page.waitForResponse("**/api/auth/reset");  await page.getByLabel("Email").fill("test-user@example.com");  await page.getByRole("button", { name: "Send reset link" }).click();  expect((await resetRequest).status()).toBe(200);  await expect(page.getByText("Check your inbox")).toBeVisible();});

When you review, check the assertions before anything else. A test that clicks through five screens and asserts only that the page has a title is worse than no test, because it creates false confidence.

How do you handle flaky and broken tests with AI?

Separate the two cases first. A broken test fails every time because the app changed; a flaky test fails sometimes because of timing, shared state or test order. Agents handle broken tests well. For flaky tests, use the agent to find the root cause, not to hide it.

For broken tests, the healer agent or a plain coding agent can rerun the failing test, read the trace and update the locator or flow. Two rules keep this safe:

1. Healing is a code change. Review the diff. An agent under pressure to make a test pass may delete an assertion, loosen it or wrap it in a condition.
2. A failing test can mean a real bug. Playwright's healer can return a skipped test when it believes the functionality is broken. Treat that as a bug report, not a cleanup.

For flaky tests, reproduce the flake and ask for a diagnosis:

npx playwright test tests/checkout.spec.ts --repeat-each=20 --trace on

tests/checkout.spec.ts fails about 1 in 20 runs. Traces are intest-results/. Find the root cause from the failing traces. Do not add retries,increase timeouts or add waitForTimeout. Explain the cause in two sentences,propose the smallest fix, then rerun with --repeat-each=20 to prove it.

The usual causes an agent finds are a missing await, an assertion that does not wait (the isVisible() pattern above), two tests sharing one account, or a click that happens before a network response settles. Playwright's best practices also recommend the @typescript-eslint/no-floating-promises lint rule to catch missing awaits before they become flakes. Our guide to debugging with AI covers the reproduce, isolate, hypothesize loop in more detail.

How do you run AI-generated Playwright tests in CI?

Run AI-written tests in CI exactly like hand-written ones: on every pull request, on Linux runners, with traces kept for failures. The AI part happens before the pull request; CI only runs the resulting code.

A sensible setup:

• Run on every pull request so broken flows never merge.
• Keep traces on the first retry rather than for every test, since Playwright's best practices warn that tracing every test on CI is performance-heavy. Open them with npx playwright show-report.
• Shard large suites across machines with npx playwright test --shard=1/3.
• Gate new tests on stability: before merging a new or healed test, run it with --repeat-each=10.
• Lint and type-check: run tsc --noEmit and the floating-promises rule.
• Isolate test data: give the suite its own seeded database and test accounts, never production data. Agents that explore a live environment can create, change or delete real records, so point both the agent and CI at a disposable environment you can reset between runs.

You can also put the agent itself in CI. With the Claude Code GitHub Action, a team member can comment on a pull request asking Claude to write tests for the changed flow or fix a failing one, and it pushes commits to the branch for review. Our guide to Claude Code GitHub Actions covers setup and permissions.

Test generation and healing are good tasks to run while you are away, since the agent's loop of exploring, writing and rerunning takes a while. Orkio runs coding agents such as Claude Code on an always-on cloud box, so you can queue "cover the checkout flow" in the evening and review the pull request in the morning.

What should AI-generated E2E tests cover?

Point the agent at the flows where a failure costs money or trust, and leave the rest to cheaper unit and integration tests. E2E tests are slow and the most expensive kind to maintain, even with AI help.

Good candidates:

• Sign-up, login, password reset.
• Checkout and payment (with the payment provider mocked).
• The two or three actions your product exists for.
• Flows that broke in production before.

Poor candidates:

• Every validation message on every form (unit tests do this faster).
• Visual details better covered by screenshot comparison.
• Third-party pages you do not control.

A useful exercise is to ask the planner agent for a plan of your critical flows, then cut it to the ten tests you would be most upset to lose. Those ten, kept green and stable, are worth more than a hundred generated tests nobody trusts.

If you are curious how the same browser-driving technology works outside testing, our guide to AI browser automation covers browser agents and computer use.

How to start with AI test automation this week

1. Update Playwright and run npx playwright init-agents --loop=claude (or your agent's loop).
2. Write a seed test that gets the app into a known starting state.
3. Add the E2E conventions above to CLAUDE.md or AGENTS.md.
4. Give the agent browser access through the Playwright CLI skills or Playwright MCP.
5. Ask for one test for your most important flow, and review the assertions first.
6. Run it with --repeat-each=10 before merging.
7. Add the suite to CI on every pull request, with traces on retry.

Expand one flow at a time. A small suite you trust beats a large one that cries wolf.

Frequently asked questions