Ace map · plan · 2026-10-22

Map study

A long-running study that takes its work from the map. Each mission tests one flow's steps that no person or study has tried yet. A tester gets the goals, not the clicks. The lead grades the result against what the map says the person should see, writes the outcome back to the map, and sends findings into the fix train. A step only becomes tested this way.

What's left

The map has 446 steps. Of those, 315 have been seen working by an agent but not tested (visual). 99 have never been run or only partly (untested or partial). 32 have known findings (broken or missing). Nothing is tested yet. The study works through them in this order:

  1. Untested and partial (99). Run on the rig each one needs; see the table below.
  2. Broken and missing (32). Re-check once the fix train lands a fix, as a mission blocked on that fix.
  3. Visual → tested (315). A fresh-eyes pass, one flow per mission. Most of it runs on the web rig, so it is the overnight work that runs in parallel.

What each step needs

Every flow gets a rig: what a tester needs before they can try it. Most flows have one rig. A step that needs more, such as a teammate or a cloud account, overrides its flow's rig.

RigMachinesUntested + partialRuns in parallel?
webOne Mac. A source host from origin/main on its own port and home. The tester drives the app in a headless browser, plus the ace CLI.17 ^303 ^326 ^343 ^377 ^378 ^390 ^399 ^418 ^508 ^517 ^519 ^524 ^548 ^558 ^571 ^761 ^776yes many on Mac A, one host each. No screen use.
ghLike web, with the GitHub testbed (githubnext/ace-testbed) signed in.5 ^379 ^616 ^617 ^718 ^742yes one tester, so two testers don't race on the same PRs.
macOne Mac. Ace Canary driven with computer use. Covers the Keychain, notifications, menus, windows, updates, Ace Helper and Login Items.34 ^3 ^4 ^7 ^13 ^14 ^15 ^17 ^18 ^19 ^48 ^49 ^50 ^58 ^59 ^62 ^64 ^65 ^66 ^304 ^305 ^511 ^578 ^624 ^650 ^652 ^655–^658 ^663 ^664 ^878 ^879 ^897one per screen Mac B only, since computer use would take Mac A's screen from you.
pairTwo Macs, two people (iamnbutler and samechocolixir) on the tailnet, with a study directory. Covers teammates, sharing, offline hosts and invoking in another person's channel.27 ^103 ^104 ^109 ^114 ^115 ^116 ^119 ^331 ^332 ^333 ^349 ^352–^355 ^362 ^363 ^389 ^588 ^619 ^779 ^798 ^833 ^858 ^870 ^871 ^882one pair Both Macs' hosts, driven headless, so the screens stay free.
cloudpair, plus a Workers and compute account. Covers hosted channels, moving to a cloud workspace, the deployed directory and the deployed web app.11 ^124 ^125 ^126 ^380 ^391 ^425 ^426 ^427 ^793 ^862 ^863after pair It needs pair's directory and teammates.
phoneYour phone: camera, home screen, on-screen keyboard and dictation.5 ^782 ^787 ^800 ^809 ^810any time About 15 minutes by you. A person's pass counts as tested.

The rig for each flow lives in study.yaml in the map repository, with per-step overrides. The visual pass splits about the same way. Of its 315 steps, about 250 are web or gh. The rest are mac, mostly Settings, the first run and the workspace's windows and tabs.

The loop

mapsteps not tested spool: studya mission per flow rig runnerresumes its tester debriefper step lead gradesagainst sees rank.yamlqueue or needs fix trainspool: fix-queue study next -t rig worked → tested finding merged → re-check
One mission per flow. Steps that worked go back to the map as tested. Findings go to the rank page, where the fix train picks them up. A merged fix opens a re-check mission.
  1. Issue. A new study command in the map skill writes one spool task per flow that has steps not yet tested. Each task is tagged with its rig, and untested steps go ahead of visual ones. The task holds the flow's goals, which are the step titles phrased as things a person wants. It leaves out where and sees, because those give away the path.
  2. Run. Each rig has a runner, a shell loop with this cycle:
    • spool next --stream study -t <rig>
    • Write the mission prompt.
    • Resume that rig's tester session, which keeps one memory across missions as in the last study.
    • Save the debrief.
    • spool complete
    Testers are Claude Code sessions, with Desktop Tools on the mac rig and Codex as the escape hatch when computer use stalls. They follow the last study's tester.md. They report what they saw, quote errors exactly, rate confusion 0–3, and never read Ace's source.
  3. Grade. The lead reads each debrief and compares it with each step's sees. A step that worked becomes tested, and the tester's screenshot can replace the map's shot. Confusion is an outcome of its own: a 2 or 3 is filed as an issue, but it doesn't keep the step from tested. Failures become broken or missing, with the tester's quote.
  4. Triage. A clear bug or decided change goes into rank.yaml's queue, then bun run queue adds it to the fix train. A decision goes into needs for you. Blockers stop that rig's runner until the fix lands.
  5. Re-check. When a fix merges, the lead adds a re-check mission for its ids to study, blocked on the fix task (spool block --by). The web rig re-checks from origin/main right away. The mac, pair and cloud rigs wait for the next canary.
  6. Publish. After each batch the lead commits the map and deploys it to workers.dev and Nexthub. The rank page's counts show progress.

The lead is the only one who edits the map. Testers never see it, so their reports stay those of a person meeting Ace cold.

Machines and schedule

WhereWhat runsNotes
Mac A (natembp)Lead, two fix workers, web testers ×2, gh tester, pair's A side (your Ace Canary, driven headless)Everything here is headless, so your screen and your installed Ace are untouched. Each tester gets its own host port and home, avoiding 4140–4142, 5140 and 4400–4420.
Mac B (Nate16)mac tester (Canary 0.0.50, computer use), then pair's B sideReachable over SSH. Codex 0.162 is installed. It sleeps after 1 minute, so the runner wraps it in caffeinate.
Your phonephone rig15 minutes, whenever suits you.
  1. Setup, about 45 minutes. I can do all of it before you're back:
    • Write study.yaml and the study command, and make the spool stream.
    • Write the runner, the mission prompt template and an ace-study-2 folder for runs, debriefs and the lead's log.
    • Set up the Mac B projects and caffeinate.
    • Start the web hosts.
  2. Block 1, about 4 hours. These run in parallel:
    • web ×2 and gh: 22 untested steps, then the visual pass.
    • mac on Mac B: 34 steps. A canary ships mid-block, so Update Ace (^13 ^14 ^664) has a real update to install.
    • The fix train runs throughout.
  3. Block 2, about 3 hours. pair (27), then cloud (11). Mac B's Canary switches from the mac tester to pair's B side, because the update and Ace Helper missions would disrupt pair. The web visual pass keeps going.
  4. Overnight. The web visual pass finishes: about 100 flows across 3 testers at roughly 10 minutes each. mac's visual flows run on Mac B. Re-check missions run as fixes land. Canaries ship every 2–3 hours when there are fixes.

Decided

  • Tailnet. The butler home tailnet. Everything Ace-related on it can be reset for the study, including directories, shares and old hosts.
  • pair's A side. Your Ace Canary on Mac A, limited to the study projects.
  • Cloud. A throwaway account.
  • Testers. Claude Code with Desktop Tools, with Codex as the escape hatch.
  • Tested. Worked means tested. Confusion is logged as issues, and the bar can change later.