Run adaptive UI tests and suites in the cloud from the admin, and read the red / yellow / green report
The test console is an admin page where a person starts cloud test runs and reads what happened. It is Test console in the admin menu, and it is open to admins only.
A run takes the manual test, syncs it with the latest code on dev, then runs
it on a fresh box and records a walkthrough video. That part is the same cloud
run described in Cloud Test Runs — the
console is the place where runs are started and read.
The Tests panel lists every manual test with its name. Filter to find one, and Run starts it at once.
On Tests / Suites there is one run option: Skip the walkthrough video when a test passes. A run started with it on still records a video for every test that fails — that video is how a person sees what went wrong — and for every test the run had to update so it matched the code. A passing test that changed files also keeps its video, because that change opens a pull request and the walkthrough is how a person watches the fix. A test that already matched the code, changed nothing, and then passes ends with no video at all. That is the point: a regression run of many tests mostly passes, and the videos of passing tests are watched by no one, while a test that just moved, or one that moved the product, still has a walkthrough to look at. Leave the box unchecked when a run should show its work either way, for example while a test itself is being written. The choice applies to every test in the run, single or suite.
A suite is a named list of tests, for example a Sanity test. Select tests in the list, give the suite a name, and create it. A suite can be edited later — rename it, add or remove tests — or deleted. Deleting a suite never deletes its past runs.
Running a suite starts every test in it together. Each test is its own cloud run: it syncs its own file and records its own video, so runs never share files or get in each other's way. The run option on Tests / Suites applies to the whole suite, including the rule that an updated test still records a video.
Every start — one test or a whole suite — is one run on the Recent runs list. Each row shows the run's name, when it started, whether it is still running, and a summary: how many tests are running, passed, failed, and were synced. The list is paged: 50 runs per page, newest first, with Prev and Next below the list, and the filter to one person's runs works on every page.
Opening a run shows the report: one row per test with its color, its summaries, and its video. The open run is part of the page address, so a reload or a shared link lands on the exact same report.
Every test row in the report carries a small review block with a status and a remark. The status starts at New and moves to Reviewed or In process as a person works through that test. The remark is a short free-text note — double-click it to write one, for example why a failure can be ignored or what a sync changed. What a person sets stays on that test row; running the same suite again creates a new run whose tests start at New.
Every test shows what its AI work cost, next to how long it took: the row shows the cost of its cloud agent, and the run header and the runs list show the cost of the whole run. The price comes from the cloud box itself: the agent prices every reply it generates, and the run collects that when the box reports back. Old runs from before cost tracking have no price shown.
A test that was synced and then passed shows yellow, not green: the test itself moved, so a person should look at what changed. A test that was synced and then failed shows red.
Every row carries its walkthrough video when one was produced, a one-sentence summary of the sync when it changed anything, and a one-sentence summary of how the run ended.
A test that failed carries one more thing: its OpenCode log — everything the cloud agent wrote while it ran, bundled into one downloadable file. The video shows what a person saw; the log shows what the agent was thinking. When a summary makes no sense or an agent seems to have stalled, download the log from the same row and read what actually happened on the box. Passing tests keep no log.