Sync and Run Test - Skill

Sync one manual test with the code, run it in the cloud, and get pass or fail with video

Overview

The sync-and-run skill takes one manual test and does the whole loop: check the test against the current code, update it when a step is out of date, then run it as a UI test with a walkthrough video.

Use it as the base for the nightly QA dashboard: one run per test, all tests in parallel, one red / yellow / green row per test with its video.

How to use the skill

Type /sync-and-run-test in chat and name one file under tests/manual/. When the run is for an issue, also name the Jira key.

The local launcher lives at .cursor/skills/sync-and-run-test/. One command runs the whole sequence and prints the report when it ends. The run can take hours; the laptop must stay on until it exits.

The cloud path needs no laptop. Start a run against the dev deployment with the team's shared secret, and read the result back by run ID.

What happens

Sync stage

The agent compares every code-facing claim in the test with the code. When a claim is stale, it changes only the test file and pushes it to dev, retrying once when dev moved. When the push still fails, it opens a pull request and the UI test does not run.

UI stage

The agent runs the test in the browser with fresh accounts and inboxes, then records the walkthrough video and uploads it. Only the video survives; the HTML report is deleted with the box.

Report

Each run reports:

  • Whether the run finished and whether the test passed
  • Whether the sync step changed the test
  • The video link, when one was produced
  • One-sentence summaries for each stage
  • Both box IDs and whether each box was deleted

Self-test

The local launcher has a self-test with stub phases: pass, sync fail, and pull request. It fails when a run uses more than 12 steps.

Local vs cloud

  • Local. Runs on a QStash dev server on the laptop. Runs do not appear in the Upstash console. The laptop posts the Jira comment itself.
  • Cloud. Runs on cloud QStash against the dev deployment. Runs appear on the Workflow page of the Upstash console. The cloud run does not post to Jira yet.

The cloud workflow and trigger live in @zooly/ui-test-runner-srv, with thin routes in zooly-app. The boot scripts and agent prompts are ported from the local sync and UI skills.

Building the dashboard on top

The dashboard exists: the test console starts runs and reads their reports from the app database. One run per test file, all tests in a suite in parallel, one red / yellow / green row per test with its video.

  • Start one run per test file. Different tests run in parallel; they do not share files.
  • Track each run by its run ID. The final result carries the dashboard flags: test-updated and test-passed.
  • Map the result to a row: red when the test failed, yellow when the test was updated, green when it passed. A pull-request fallback means the UI test was skipped.
  • Keep the video link on every row.

Reference