Adaptive AI UI Testing

Overview

Use an adaptive AI test prompt when the goal is to validate a real user journey through the current UI, but the interface changes too frequently for a selector-heavy automated test to remain useful.

In this method, a test script is a Markdown execution prompt. It tells an AI agent what outcome to prove, which boundaries must remain intact, and what evidence counts as passing. It is not a permanent Playwright script and should not assume that labels, selectors, routes, or intermediate screens remain unchanged.

Run it one step at a time. Write a small script for the current step, run it, and look at the page. Then decide the next step from what the page shows. Do not write one script that runs the whole test. A step that did not do what you expected is a reason to change the next step, not to start over.

Store these prompts under:

tests/manual/<flow-name>.md

Author and run them with the two reusable skills below. Keep each product scenario as a Markdown prompt. Do not add a new skill for every flow.

Skills

Create a prompt

Type /create-automated-adaptive-ui-test in chat. The skill lives at .cursor/skills/create-automated-adaptive-ui-test/ and writes a self-contained prompt under tests/manual/.

Every generated prompt that uses MailSlurp must embed copy-pasteable snippets in a MailSlurp examples section. Pointing at MAILSLURP.md is not enough. Match the style in tests/manual/agent-email-conversation.md.

Run a prompt

Type /run-automated-adaptive-ui-test in chat and name the file. The skill lives at .cursor/skills/run-automated-adaptive-ui-test/.

The Markdown file is the test. The agent executes it with browser MCP and the embedded MailSlurp snippets. It must not rewrite the prompt into a permanent executable test program.

Inspect reads

When a prompt must prove internal state the UI does not show (for example account memory or a seeded room), add a helper endpoint (/create-helper-endpoint). Those reads need the session cookie and Bearer CRON_SECRET.

Login

If browser MCP needs a signed-in user, follow tests/manual/auth-login.md. The always-on rule .cursor/rules/browser-mcp-login.mdc points agents at that file so login details do not have to be repeated in every prompt.

Do not reuse a personal account, a previous test user, or an existing session. Create a fresh MailSlurp inbox and generated password, then sign up, verify, and sign in through the visible auth UI.

When the prompt under test is auth-login.md itself, run that file as written, including logout and the second sign-in. For any other prompt, reuse only the signup and first sign-in procedure.

Existing prompts

What this method tests

This approach is primarily for end-to-end UI behavior:

• Authentication and signup
• Navigation and user-visible state
• Interactions across two or more user roles
• Browser session and privacy boundaries
• External side effects such as received email
• The final state as rendered to each participant

Email can support the test without being its main subject. For example, MailSlurp can supply disposable signup addresses and verification codes while the assertions focus on the browser UI.

Use ordinary unit, integration, and deterministic browser tests for stable business rules. The adaptive prompt complements those tests; it does not replace them.

Write the Markdown test prompt

Use /create-automated-adaptive-ui-test so the required sections, isolation rules, MailSlurp snippets, and cleanup step are included.

1. Start with the product goal

Describe the user journey in plain language. State the final outcome, not the current implementation.

Good:

A brand submits an offer through a talent's public link, both sides can continue the negotiation, and each dashboard shows only the messages that participant is allowed to see.

Fragile:

Click the third button, wait for .message-row:nth-child(4), then open /dashboard/agent/conversations.

2. Define observable pass criteria

List every checkpoint that proves the journey worked:

• Which role performed each action
• What the next role received
• What appeared in each browser
• Which private information must not appear
• What final status or summary must agree with the transcript

Do not let an intermediate success stand in for the final assertion. If the goal is “the other side receives approval,” sending approval is not enough; the test must verify receipt.

3. Describe invariants, not selectors

Tell the agent to inspect the current accessibility tree, page text, network requests, and application code when needed. Name controls by purpose, such as “the action that opens Agent Links,” rather than freezing an exact selector.

Exact values are appropriate only when they are product contracts:

• Required environment
• Expected role separation
• Expected final amount or state created during this run
• A room-specific email reply address
• A privacy boundary

4. Make temporary state explicit

The prompt should tell the agent to record, for the current run, in a private temporary file outside the repository:

• Browser context names
• MailSlurp inbox IDs and addresses
• Account roles
• Public link slug
• Conversation or room ID
• Relevant email IDs and message headers

These values must be discovered again on every run. Never copy temporary IDs, verification codes, passwords, cookies, or inbox addresses from an older test into the prompt or the report.

5. Embed runnable MailSlurp snippets

If the test creates an inbox, waits for mail, replies, or deletes an inbox, include a MailSlurp examples section with the commands the executor should run. Use placeholders such as /path/to/zooly2 and inboxId.

A prompt that only links to .cursor/skills/create-automated-adaptive-ui-test/MAILSLURP.md is incomplete.

6. Define recovery behavior

The prompt should distinguish:

• A slow development compile from a failure
• A successful backend request with a stale UI from a successful user flow
• A same-side email acknowledgment from a cross-side notification
• A transient browser automation miss from a real product defect

Allow the agent to take a fresh page snapshot, inspect network responses, wait, or retry a visible interaction. Do not authorize bypassing authentication, writing directly to the database, or calling business functions to manufacture a passing state.

Browser setup

Use one isolated browser context per role. Pages in different contexts must not share cookies or browser storage.

For a two-party flow, use names such as:

talent
brand

Start every run from a new named context. If a selected context is already signed in, log out through the visible product UI and confirm it is anonymous before continuing.

Keep both browsers open while the test runs. This makes it possible to verify that updates appear for both roles and that private messages do not cross the role boundary.

If a browser tab is accidentally closed, reopen the exact known application or conversation URL in the same isolated context. Confirm authentication and conversation history before continuing.

MailSlurp lifecycle

Configure the API key

Keep the key in the repository root .env.local:

MAILSLURP_API_KEY="..."

Never commit the real key or paste it into documentation, test prompts, source files, or chat output.

One MailSlurp client, no fallback

Every test talks to MailSlurp through one helper in the repo, tests/manual/lib/mailslurp.mjs. It uses the MailSlurp package installed at the repo root, so any checkout that ran npm install has it, and it can be imported from a script anywhere on disk. It reads the key from the environment, then from the root .env.local.

There is no fallback to raw MailSlurp web requests. A fallback can turn a test setup mistake into a fake product bug: MailSlurp's plain "send email" request ignores the inbox you name and sends from a brand-new random address, so the agent correctly ignores the reply as coming from a stranger. The helper sends from the inbox itself and stops if the message left from any other address.

Start every run with the self-check. It creates two inboxes, sends one message between them, confirms the sender, and deletes both:

node /path/to/zooly2/tests/manual/lib/mailslurp.mjs check

When something is wrong, the helper stops with the reason: the package is not installed, the key is missing or rejected, nothing arrived before the timeout, or a message went out from the wrong address.

Create one inbox per role

node /path/to/zooly2/tests/manual/lib/mailslurp.mjs create-inbox

Create separate inboxes for separate users. Avoid a short expiry when the test may pause or continue later.

Keep a private cleanup list of only the inbox IDs created by this run. Do not put those IDs in the test report.

Read verification or notification email

Wait for a new email instead of sleeping for a fixed duration:

import * as mailslurp from "/path/to/zooly2/tests/manual/lib/mailslurp.mjs";

const email = await mailslurp.waitForLatestEmail(inboxId, 60_000);
const code = (email.body ?? "").match(/\b(\d{6})\b/)?.[1];
if (!code) throw new Error("Verification code not found");

When an inbox can receive multiple kinds of email, do not assume the newest message is the one under test. Compare the inbox state before and after an action, then identify new messages by timestamp, subject, sender, recipient, and content.

Do not print full bodies if they may contain private data. Extract only the facts needed for the assertion.

Delete inboxes when the run ends

Present the test report first. Then delete every inbox this run created, with no question to the user:

node /path/to/zooly2/tests/manual/lib/mailslurp.mjs delete-inbox <inboxId>

Delete only the inboxes this run created, including inboxes from a failed or partial try. If a delete fails, the helper says why. Report that reason and do not claim the cleanup worked. Then remove the private credential file and confirm the count only, without exposing IDs, addresses, codes, or passwords.

Do not call “delete all inboxes.” Other developers or active tests may own them.

Run locally

A common local topology is:

• Authentication: http://localhost:3003
• Zooly app: http://localhost:3004
• Talent dashboard: http://localhost:3004/dashboard

Discover the current ports rather than assuming they are unchanged.

The authentication identity and Zooly app account are separate concerns. After signup or login through the auth app, visit the local Zooly app so its database can create or resolve the application account.

Before testing:

1. Confirm the required apps respond.
2. Confirm .env.local points at the intended local dependencies.
3. Start with fresh isolated browser contexts.
4. Create fresh MailSlurp inboxes.
5. Sign in through tests/manual/auth-login.md when the scenario needs a user.
6. Run the flow through the visible UI.

Outbound email can still be tested locally when configured. A complete inbound email conversation additionally requires SES routing whose callback targets the same local environment and database. If a local notification uses SendGrid with Reply-To: support@zooly.ai, it validates outbound delivery only; do not send test replies to support.

Run on dev

Use:

• Authentication: https://auth.zooly.ai
• Zooly app: https://dev.zooly.ai

Wait for the intended deployment to finish before starting. Create all test accounts and conversations in dev; do not reuse local room IDs.

Recommended sequence:

1. Open the dev dashboard in the talent browser.
2. Follow the redirect to auth.zooly.ai.
3. Create and verify the talent account using tests/manual/auth-login.md.
4. Return to dev.zooly.ai so the dev app account is provisioned.
5. Discover the talent's public link from the current UI.
6. Open the public link in the isolated brand browser.
7. Perform the role-specific product flow.
8. Authenticate the brand through auth.zooly.ai with a second MailSlurp inbox.
9. Return the brand to dev.zooly.ai so the brand app account exists in the dev database.
10. Verify the final UI separately in both browser contexts.

For an email-reply flow, verify the first notification before replying:

• Sender uses the dev SES agent domain.
• Reply-To contains the current room ID and dev reply domain.
• The conversation link points to dev.zooly.ai and the current room.

Stop if any of these identify another environment.

Validate each role's UI

Do not validate only the side that initiated the action.

For each role:

1. Open its Conversations view.
2. Select the exact conversation created by the test.
3. Wait for the full transcript.
4. Match visible messages against the actions and external events.
5. Verify ordering and source labels such as “received by email.”
6. Verify summaries and status agree with the latest transcript.

For privacy-sensitive flows, explicitly list messages that must be absent from the other role's view. A complete transcript in an operator or talent view can be correct while the same transcript in a customer or brand view is a data leak.

Report the result

The final test report should state:

• Environment tested
• Roles and major checkpoints completed
• Final user-visible outcome
• Whether both role views were verified
• Whether privacy boundaries passed
• External delivery evidence without exposing private content
• Any stale summary or contradictory status
• Any blocker with concrete evidence
• Whether browser contexts were preserved or closed
• How many MailSlurp inboxes this run deleted (count only)

Never include API keys, passwords, verification codes, auth cookies, inbox IDs, inbox addresses, or full private messages in the report.

Reference

• Create skill: .cursor/skills/create-automated-adaptive-ui-test/
• Run skill: .cursor/skills/run-automated-adaptive-ui-test/
• Login prompt: tests/manual/auth-login.md
• Browser MCP login rule: .cursor/rules/browser-mcp-login.mdc
• Long journey example: tests/manual/agent-email-conversation.md
• Bug reproduction example: tests/manual/brand-login-required-error.md

Copy the structure of those prompts for new scenarios, but rewrite the product goal, invariants, pass criteria, and privacy assertions for the feature being tested.