Worked examples

Each example lists what it needs, the steps, and what you should see. They use only what is in the repository: the launch flag, Playwright over the Chrome DevTools Protocol (CDP), the scripts in scripts/, and the fixture pages in test/.

Examples 2 and 3 need the main branch, because the read pipeline is not in v0.1.5. Run Troy from a source checkout for those (Installation). Every example works the same way: start Troy with the port, let the agent attach, detach without closing anything.

1. Check a page you are signed into

Goal: after a deploy, confirm that a page behind a login renders, and collect any console errors and failed requests.

Needs: Troy v0.1.5 or later, Node.js, and npm install playwright in a scratch folder.

1. Quit Troy, then start it with the port:

   open -a Troy --args --cdp-port=9333

2. Sign in to the site in Troy and open the page you want checked, for example https://staging.example.com/billing.

3. Save this as check.mjs. It finds the tab by address, reloads it with listeners attached, and reads what a person would see. Reloading changes the person’s tab, so ask before you run it.

   import { chromium } from 'playwright'
   
   const browser = await chromium.connectOverCDP('http://127.0.0.1:9333')
   const page = browser.contexts()[0].pages()
     .find((p) => p.url().startsWith('https://staging.example.com/billing'))
   if (!page) throw new Error('open the billing page in Troy first')
   
   const problems = []
   page.on('console', (m) => { if (m.type() === 'error') problems.push(`console: ${m.text()}`) })
   page.on('pageerror', (e) => problems.push(`pageerror: ${e.message}`))
   page.on('requestfailed', (r) => problems.push(`failed: ${r.url()} ${r.failure()?.errorText}`))
   
   await page.reload({ waitUntil: 'networkidle' })
   const heading = await page.getByRole('heading', { level: 1 }).first().innerText()
   console.log({ heading, problems })
   
   await browser.close()   // detaches; Troy and the tab stay open

4. Run node check.mjs.

You should see the page’s main heading and a list of problems, empty when the page is clean. The tab is still open and still signed in afterwards.

If the attach fails, see Troubleshooting.

2. Read a page as Markdown or JSON

Goal: give an agent the page’s visible text in reading order as Markdown, or as JavaScript Object Notation (JSON) with selectors it can act on. Main only.

1. From the checkout, start Troy with the port and open the page:

   npm run browser -- --cdp-port=9333 https://example.com/

   The first http or https address on the command line opens in the first tab.

2. Read it as Markdown:

   node scripts/read.mjs --url https://example.com/

   You should see the title as a heading, the page text in reading order with links written as [text](href), and a footer line starting > read: with the block count, the region count and the time.

3. Read it as JSON and list every link with its selector:

   node scripts/read.mjs --url https://example.com/ --json --out page.json
   node -e "const d = JSON.parse(require('fs').readFileSync('page.json', 'utf8'));
     for (const b of d.blocks) if (b.role === 'link') console.log(b.text, '->', b.selector)"

4. Act on one of those selectors from Playwright, attached as in example 1:

   await page.locator('html > body > div > p:nth-of-type(2) > a').click()

   Use a selector from your own page.json; the one above is only an example of the form.

--url must equal the tab’s address exactly, trailing slash included. On Linux, set TROY_ENDPOINT_FILE="$HOME/.config/Troy/agent-endpoint.json" first.

3. Read a canvas dashboard

Goal: see how a read handles text the Document Object Model (DOM) does not contain. Main only. This uses the repository’s own fixture, a dashboard whose numbers are drawn on a canvas.

1. Serve the fixtures and open the dashboard in Troy:

   python3 -m http.server 8200 --bind 127.0.0.1 --directory test/fixtures &
   npm run browser -- --cdp-port=9333 http://127.0.0.1:8200/canvas-dashboard.html

2. Read it:

   node scripts/read.mjs --url http://127.0.0.1:8200/canvas-dashboard.html

   You should see the heading “Operations Dashboard”, read from the DOM, then a fenced block labelled [ocr region, untranscribed] that gives the canvas’s size and position. The footer reports one region for optical character recognition (OCR) with engine: none. The numbers on the canvas are not in the output, because no OCR engine is connected.

3. To look at the region yourself, take its box from --json output and screenshot just that area from Playwright:

   const { x, y, w, h } = doc.regions[0].box   // doc: the parsed --json output
   await page.screenshot({ path: 'region.png', clip: { x, y, width: w, height: h } })

To transcribe it instead, on macOS with the Xcode command line tools, run the pipeline from your own code with appleVisionEngine(). How Troy reads a page has the full sketch.

4. Fill a form without submitting it

Goal: answer form questions with checks at every step, and confirm that the submit button is out of reach. This uses test/fixture/form.html, which has custom Yes and No controls repeated across several questions and a hidden field that appears after one answer.

1. Serve the fixture and open it in Troy:

   python3 -m http.server 8100 --bind 127.0.0.1 --directory test/fixture &
   npm run browser -- --cdp-port=9333 http://127.0.0.1:8100/form.html

2. Point the scripts at Troy and authorize the local host for this session. The generic target allows no host until you name one:

   export APPLY_FILL_CDP=http://127.0.0.1:9333
   A=(--target generic --allow-host 127.0.0.1)

   If another 127.0.0.1 tab is open (from example 3, say), the scripts refuse to guess between them. Close it, or add --url form.html.

3. Answer “Yes” to the question with id q-using, and require the hidden field to appear:

   node scripts/click.mjs "${A[@]}" --within "#q-using" --text "Yes" --expect-visible "#usercount"

   --text "Yes" without --within is refused, because “Yes” appears once per question.

4. List the fields, which now include the revealed text area:

   node scripts/detect.mjs "${A[@]}"

5. Preview, then write, an answer. The answer always arrives on standard input:

   printf '%s' "About 40 a week" | node scripts/fill.mjs "${A[@]}" --selector '#ta-users' --dry-run
   printf '%s' "About 40 a week" | node scripts/fill.mjs "${A[@]}" --selector '#ta-users'

   You should see before, after and ok: true, and exit code 0. A mismatch exits 2.

6. Try the two things the scripts refuse:

   node scripts/click.mjs "${A[@]}" --within "#q-noop" --text "Yes"   # control that does nothing
   node scripts/click.mjs "${A[@]}" --selector "#danger"              # "Submit application"

   The first reports NOT VERIFIED and exits 2, because nothing on the page changed. The second is refused before anything is clicked.

The same steps work on a real form with the ycombinator target or a target file of your own. See Action scripts.

5. Add an unpacked extension

Goal: run a Chrome extension you trust in Troy.

1. Choose View > Extensions Folder. Troy creates and opens <profile>/extensions/.
2. Copy the extension’s folder in, so that its manifest.json sits at extensions/<name>/manifest.json.
3. Choose View > Reload Extensions.

You should see a notice listing what loaded, with the error for anything that failed. Extensions load with file access off, and Electron implements only part of the Chrome extension programming interfaces, so some extensions load but do not work. There is no toolbar button for extensions yet.