Code. Prompts. Examples.

A note on implementation · markdown · source line 19

**Try it with CARBON:** `/carbon-help` — Inspect this repository and recommend the smallest useful first testing pass.

Introduction - Meet TestBucks · markdown · source line 14

Test the TestBucks coffee-shop website for bugs. Use synthetic data and simulated orders only. Show me the evidence for anything you find.

Your first sixty minutes · markdown · source line 32

Inspect the TestBucks homepage and menu. Capture a screenshot and DOM, then collect console and network evidence if available. Separate what you saw from what you inferred. Do not use real personal information or connect to a real payment service.

Your first sixty minutes · markdown · source line 36

**Starting point.** TestBucks has a menu, drink customization, cart, and practice orders. I will use the vanilla-latte journey as the first test.

**Evidence boundary.** A homepage screenshot can show layout and labels. It cannot prove that an order will be saved or that a retry is safe.

Your first sixty minutes · markdown · source line 44

In TestBucks, find a vanilla latte, choose medium size, oat milk, and one extra shot, then add exactly one to the cart. Verify the drink, options, quantity, and $6.60 total including training tax. Stop before checkout. Preserve the evidence and report what actually happened.

Your first sixty minutes · markdown · source line 48

**Result: PASS for this cart intent.** One medium vanilla latte with oat milk and an extra shot is in the cart. The subtotal is $6.00; 10% training tax rounds to $0.60; the total is $6.60.

**Not established.** This verifies the selected drink in the cart, not a completed purchase or the whole shop.

Your first sixty minutes · markdown · source line 63

Inspect this TestBucks menu screenshot and the supplied catalog context. The search is vanilla, but the page shows no drinks. Identify the possible issue, the strongest alternative explanation, and the next test that would settle it. Use the OpenTest.ai bug format.

Your first sixty minutes · markdown · source line 67

**Candidate.** Searching for vanilla returns no drinks even though the catalog includes a vanilla latte.

**Counterargument.** A store filter or unavailable item could explain an empty result. A screenshot alone does not establish the cause.

**Next test.** Compare the same query with the seeded fault enabled and disabled, from the same starting state.

Your first sixty minutes · markdown · source line 77

Reproduce the TestBucks vanilla-search issue in the local lab. Compare the fault-enabled page with the normal page. Check the search implementation. Preserve both results and distinguish the deliberately seeded defect from a problem in a real coffee company's website.

Your first sixty minutes · markdown · source line 85

From this TestBucks journey, propose the three most useful tests to run next. Start with what could go wrong when the customer changes the quantity or retries an order. Explain what each test would prove and what evidence it needs. Do not count a proposed test as executed coverage.

Your first sixty minutes · markdown · source line 89

**Next tests.** Check whether an invalid quantity is rejected. Reload a valid cart to see whether it survives. Then try a simulated timeout and verify that retrying does not create a duplicate order.

**Deferred.** Cross-device accounts and real payment-provider reconciliation require an extended environment, not this browser-only starter.

Your first sixty minutes · markdown · source line 97

Based on the TestBucks evidence collected so far, assess readiness for the scoped release: menu search and simulated pickup ordering. List passed and failed promises, missing evidence, and the three next investigations most likely to change your recommendation.

Your first sixty minutes · markdown · source line 101

**Decision: HOLD for the fault-enabled build.** The catalog search promise fails. One cart journey passed, but checkout and recovery are not yet verified.

**Next evidence.** Repair and retest search, complete one simulated pickup order and reload it, then exercise a timeout and safe retry. Do not interpret a cart pass as a release verdict for the whole shop.

Five Terms for Talking About AI Software Quality · markdown · source line 25

Review the supplied TestBucks screenshot and DOM for visible UI and usability problems. Look for clipped or overlapping content, weak hierarchy, unclear affordances, unreadable text, and confusing states. State which observations the screenshot establishes, which the DOM establishes, and what would require an interactive AI Test.

Five Terms for Talking About AI Software Quality · markdown · source line 29

**Observed from the screenshot.** The primary action is visible, but the narrow viewport clips the right edge of the final card.

**Observed from the DOM.** The card is present and labeled, but it is not exposed as a button or link.

**Next AI Test.** Use keyboard-only navigation at the same viewport, attempt to open the card, and preserve focus order plus before-and-after screenshots. The static artifacts do not prove whether the journey can be completed.

Five Terms for Talking About AI Software Quality · markdown · source line 39

Review the supplied TestBucks console log, redacted network trace, DOM, and API definition for privacy and security concerns. Look for exposed identifiers, unexpected third-party destinations, insecure transport or configuration signals, unsafe client-side assumptions, and trust-boundary crossings. Separate observations from hypotheses, redact sensitive values in the answer, and state what remains unknown without server-side evidence.

Five Terms for Talking About AI Software Quality · markdown · source line 43

**Candidate.** One request sends a stable account identifier to a third-party analytics origin before the captured consent action. The destination and timing are observed; the sensitive value is redacted here.

**Unknown.** The bundle does not establish the governing consent state, contractual purpose, server-side retention, or whether authorization is enforced after the request reaches the service.

**Next AI Test.** Repeat with documented reject and accept states, compare destinations and payload classes, then reconcile the result with the approved data-flow policy.

Five Terms for Talking About AI Software Quality · markdown · source line 53

Review the supplied TestBucks DOM and API definition for structural and contract problems. Check control semantics, missing labels, inconsistent field types, ambiguous error contracts, undocumented responses, and mismatches between the rendered interface and the declared API. Identify the smallest AI Test that could confirm each important hypothesis.

Five Terms for Talking About AI Software Quality · markdown · source line 57

**Contract mismatch candidate.** The API declares `sizeMl` as an integer, while the drink-size control submits the strings `small`, `medium`, and `large`.

**Smallest confirming AI Test.** Submit each visible option through the UI, preserve the request and response, reload the record, and compare the displayed value with persisted state. Until that runs, this is a structural mismatch hypothesis rather than a confirmed functional failure.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 57

Find bugs in this TestBucks menu screenshot and the evidence captured with it. The query is “vanilla.” Decide whether the no-results state is a confirmed defect, a likely relevance or recovery problem, or an expected availability condition. State exactly what each artifact supports and what must be tested next.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 64

**Finding: relevance and recovery candidate.** The menu displayed **“No drinks match. Try another search or category.”** despite a vanilla latte in the supplied catalog. That mismatch is worth investigating.

**Possible explanations.** A store filter, sold-out state, delayed data, or a search defect. The current screenshot and DOM do not distinguish them.

**Next AI Test.** Repeat in a fresh local session, compare the query with the fault enabled and disabled, and inspect the filtering code. Preserve both results before calling the cause established.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · json · source line 116

{
  "summary": "A synthetic email appeared in a pre-consent analytics request.",
  "score": 75,
  "issues": [
    {
      "bug_title": "Checkout sends email to analytics before consent",
      "bug_type": ["privacy", "security"],
      "bug_confidence": 8,
      "bug_priority": 9,
      "interestingness": 6,
      "bug_reasoning_why_a_bug": "With analytics consent off, the trace contains the synthetic email in an analytics event.",
      "bug_reasoning_why_not_a_bug": "Confirm the endpoint purpose and consent configuration; a misclassified essential service would change the claim.",
      "suggested_fix": "Remove customer identifiers from analytics and enforce the consent boundary.",
      "bug_why_fix": "Customers expect declined optional tracking to stay off.",
      "what_type_of_engineer_to_route_issue_to": "Frontend analytics owner with privacy engineering review",
      "prompt_to_fix_this_issue": "Reproduce in the sandbox with synthetic data and consent off. Inspect event construction and routing. Add a regression check and verify that no email reaches optional analytics. Preserve permitted checkout behavior.",
      "possibly_relevant_page_console_text": "No console errors occurred in this synthetic run.",
      "possibly_relevant_network_call": "POST /analytics/event; body includes email=customer@example.invalid while analytics consent is off.",
      "possibly_relevant_page_text": "Optional analytics: off",
      "possibly_relevant_page_elements": "Analytics consent switch: checked=false; checkout email field contains the synthetic address.",
      "tester": "Privacy specialist, synthetic demonstration",
      "byline": "Reviews tracking, consent, and personal-data flows"
    }
  ]
}

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 243

Find bugs in this screenshot and the evidence captured with it. Use the supplied TestBucks screenshot, semantic DOM, console excerpt, and redacted network summary together. Separate confirmed defects from triage candidates and state what evidence supports each conclusion.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 247

**Confirmed page-level defect: none from this bundle.** The settled screenshot shows no confirmed clipping, overlap, or visible layout failure at the observed viewport. The semantic DOM exposes the primary search control as a combobox named **“Search drinks,”** plus the coffee menu and drink cards.

**Candidates requiring triage.** The console contains two `ReferenceError: pickupEstimate is not defined` messages and a telemetry message saying **“Pickup estimate unavailable.”** The bounded network sample includes observed TestBucks CSS and JavaScript responses returning `200`.

**Boundary.** No captured artifact connects the console signals to a user-visible failure or establishes a privacy violation. Preserve them as candidates, not confirmed bugs.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 333

**Try it with CARBON:** `/carbon-ui` Review the supplied TestBucks narrow-screen captures for overlap, clipping, and unreadable states; cite the exact region. `/carbon-ux` Inspect the approved edit-to-cart journey for lost choices. `/carbon-usability` Investigate the first-time decaf-search task; distinguish observed friction from persona hypotheses.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 380

**Try it with CARBON:** `/carbon-accessibility` Review the approved TestBucks milk selector, keyboard order path, and text-enlargement states. Preserve names, roles, states, focus and announcements; identify criterion-level evidence and mark unexecuted methods not assessed.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 420

**Try it with CARBON:** `/carbon-privacy` Trace a synthetic TestBucks cup name through approved requests, storage and logs. Compare the declared consent and deletion promises with observed destinations and state; redact evidence and report unobservable downstream steps.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 458

**Try it with CARBON:** `/carbon-security` Build a TestBucks order-ownership and refund-authorization matrix using synthetic accounts in the approved test environment. Keep drink notes untrusted. Run only explicitly permitted probes and distinguish demonstrated failures from unexecuted hypotheses.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 475

**Try it with CARBON:** `/carbon-security` — Investigate security risks against a declared target and threat model.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 517

**Try it with CARBON:** `/carbon-performance` Repeat the TestBucks usable-control measurement under declared conditions. `/carbon-reliability` Verify one durable order after a controlled lost response and retry. `/carbon-networking` Investigate the delayed Small/Large price responses; retain request identities, timings and final state.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 548

**Try it with CARBON:** `/carbon-compatibility` Compare the approved TestBucks cold-brew journey across the declared desktop and phone configurations, including an open keyboard. Record versions, input modes, preserved state and each untested combination.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 579

**Try it with CARBON:** `/carbon-content` Compare the TestBucks cup label, ingredient summary, receipt and order-status copy with the supplied catalog and state definitions. Report contradictions with their sources; do not guess which claim is authoritative.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 611

**Try it with CARBON:** `/carbon-localization` Review TestBucks drink labels, prices, pickup dates and recovery states in the declared locales using approved strings. Separate layout probes from language review and retain any missing locale evidence.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 643

**Try it with CARBON:** `/carbon-data` Reconcile the synthetic TestBucks extra-shot order across checkout, response, stored record and receipt. Use the supplied pre-tax pricing rule, order identity and consistency window; distinguish missing evidence from a demonstrated mismatch.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 709

**Try it with CARBON:** `/carbon-issues` — Hunt AI-code weaknesses and distinguish source-proven defects from suspicions requiring execution.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 741

**Try it with CARBON:** `/carbon-functionality` — Exercise state transitions, business invariants, and forbidden outcomes.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 773

**Try it with CARBON:** `/carbon-functionality` — Focus on form validation, boundary values, error states, and submitted data.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 805

**Try it with CARBON:** `/carbon-usability` — Examine whether errors explain what happened and offer a workable recovery path.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 846

**Try it with CARBON:** `/carbon-functionality` — Test search input, suggestions, empty states, and interrupted requests.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 878

**Try it with CARBON:** `/carbon-content` — Assess result explanations, relevance signals, and misleading or missing information.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 910

**Try it with CARBON:** `/carbon-ux` — Review the page promise, next actions, trust cues, and conversion friction.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 942

**Try it with CARBON:** `/carbon-content` — Check attribution, chronology, credibility, and the relationship between headlines and evidence.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 974

**Try it with CARBON:** `/carbon-state` — Probe cart persistence, quantity changes, totals, and cross-session state.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 1006

**Try it with CARBON:** `/carbon-functionality` — Test checkout promises, order state, retries, and payment consistency using safe test data.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 1038

**Try it with CARBON:** `/carbon-content` — Compare displayed prices, plan promises, terms, and actual behavior.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 1070

**Try it with CARBON:** `/carbon-privacy` — Verify consent choices against observed storage and tracking behavior.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 1102

**Try it with CARBON:** `/carbon-privacy` — Investigate access, deletion, retention, and declared data-rights workflows; route legal judgments to the appropriate expert.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 1134

**Try it with CARBON:** `/carbon-content` — Compare policy promises with product evidence and flag gaps for specialist review.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 1166

**Try it with CARBON:** `/carbon-security` — Use the threat model and available evidence to investigate relevant application-security risks.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 1198

**Try it with CARBON:** `/carbon-api` — Test schemas, authentication, errors, retries, and integration contracts.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 1230

**Try it with CARBON:** `/chatbot` — Generate and evaluate scoped chatbot cases for behavior, boundaries, and failure handling.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 1290

Using the supplied screenshot, DOM, and page text, generate four relevant, explicitly synthetic end-user personas for this product. Do not claim that they are researched customers. Give each a distinct goal, constraint, likely actions, and first-person feedback. Return the OpenTest.ai page-persona schema with `user_persona_feedback` as an array of four records. Score overall and per-persona visual design, usability, content, features, trust/emotional response, apparent accessibility, and recommendation from 1 to 10; explain every score. Mark anything the static evidence cannot establish as unknown. Do not interact with the product.

Chapter 1 — Start with the Evidence: Screenshots, Logs, Network, DOM, and Traces · markdown · source line 1315

**Try it with CARBON:** `/carbon-personas` — Generate contextual, evidence-qualified persona reactions and qualitative feedback.

Chapter 2 — Crawl the Product, Then Dedupe What You Learn · markdown · source line 40

Crawl the TestBucks practice site at https://howaitestssoftware.com/testbucks.html. Stay within the TestBucks app and use synthetic data only. Inspect Home, Menu, and Cart, including empty and populated states. Stop after ten distinct states or ten minutes, whichever comes first. Do not place an order. Capture the evidence for suspected bugs and report what you did not reach.

Chapter 2 — Crawl the Product, Then Dedupe What You Learn · markdown · source line 76

Compare these TestBucks findings and their captured evidence. Decide whether they describe the same underlying defect. Keep one canonical issue when that is supported, and attach each affected page as an instance. Preserve different consequences. If the cause is uncertain, link the findings for investigation rather than merging them as fact.

Chapter 2 — Crawl the Product, Then Dedupe What You Learn · markdown · source line 80

**Decision: one shared-header candidate, three instances.** The findings concern the same menu control with the same missing accessible name.

**Keep the Cart instance visible.** It blocks navigation during an order, so its consequence may be more important than the Home instance.

**Next check.** Confirm that the same shared component produces all three observations before treating the common cause as established.

Chapter 2 — Crawl the Product, Then Dedupe What You Learn · markdown · source line 215

Compare this candidate with the known issues. Decide whether it is the same root defect, a new instance of a known defect, a distinct defect with a similar symptom, or insufficiently evidenced. Give duplicate confidence, explain the strongest supporting and disconfirming evidence, preserve the affected route and artifacts, and recommend merge, link, keep separate, or gather more evidence.

Chapter 2 — Crawl the Product, Then Dedupe What You Learn · markdown · source line 219

**Decision: link as another instance of `NAV-A11Y-004`. Duplicate confidence: 0.93.** Both findings involve the shared `GlobalNav` component, the same unlabeled menu control, the same accessibility-tree signature, and the same keyboard consequence.

**Disconfirming evidence.** This instance occurred on `/checkout`, where the blocked task has higher business impact than the original homepage instance. Preserve the checkout route, viewport, focus trace, and screenshot rather than discarding them.

**Action.** Keep one canonical engineering defect, add this route as a high-impact instance, and raise the canonical priority for milestone review. Do not create a second root defect unless source or reproduction evidence shows a different component or cause.

Chapter 3 — AI Can Drive the Product · markdown · source line 9

Add an item to the shopping cart, open the cart, verify that the same item and quantity are in it, and check the journey for accessibility problems.

Chapter 3 — AI Can Drive the Product · markdown · source line 15

**Functional intent: PASS.** I added one medium vanilla latte, opened the cart, and observed the same item with quantity `1` in the rendered cart and cart-state response. I stopped before checkout.

**Accessibility result: FAIL at 200% text.** Keyboard operation completed with visible focus, but the enlarged cart total overlapped the checkout control at the narrow viewport. The normal-scale screenshot did not expose the problem.

**Evidence.** Action and state trace; normal and enlarged-text screenshots; DOM and accessibility snapshots; final cart-state response; console and bounded network summary.

**Boundary.** Screen-reader announcements and payment behavior were not assessed. The functional pass does not erase the reflow failure.

Chapter 3 — AI Can Drive the Product · markdown · source line 44

In the TestBucks practice app, order one medium vanilla latte with oat milk and an extra shot for Alex Example in Seattle. Use only the simulated checkout. Verify the $6.60 total, one saved order, an empty cart, and persistence after reload.

Chapter 3 — AI Can Drive the Product · markdown · source line 92

Add one medium vanilla latte with oat milk and an extra shot to the cart. Verify quantity 1 and the expected total of $6.60. Stop before checkout.

Chapter 3 — AI Can Drive the Product · markdown · source line 154

**Try it with CARBON:** `/carbon-accessibility` — Evaluate accessibility through available source and runtime evidence, including interaction and visual context.

Chapter 4 — You Still Have to Run the Whole System · markdown · source line 33

Review this code and test the whole system it participates in.

Chapter 5 — AI Tests Beyond the Browser: Mobile, Desktop, Voice, and Devices · markdown · source line 47

In an isolated TestBucks multi-device environment, start one vanilla-latte order on Android, finish customization on the desktop, and verify that the same order and pickup status appear on an iPhone. Use one synthetic account and simulated payment. Preserve the per-device timeline.

Chapter 5 — AI Tests Beyond the Browser: Mobile, Desktop, Voice, and Devices · markdown · source line 86

**Try it with CARBON:** `/carbon-compatibility` — Check the declared browser, device, environment, and version combinations.

Chapter 5 — AI Tests Beyond the Browser: Mobile, Desktop, Voice, and Devices · markdown · source line 125

This is a multi-agent coding workflow. Using the supplied plans, prompts, tool-call traces, workspace diffs, task ownership, handoff messages, and result artifacts, create a bounded test matrix. Include independent agents on the same task; concurrent work in an approved sandboxed repository; stale, duplicated, or malformed handoffs; interruption and retry; incompatible assumptions; budget exhaustion; and an unexpected workspace change. For each scenario, state the participants, allowed writes, schedule, invariant, expected evidence, containment, cleanup, and verdict rule. Execute only approved sandbox scenarios. Label every unexercised scenario as **NOT ASSESSED** rather than inferring a pass.

Chapter 5 — AI Tests Beyond the Browser: Mobile, Desktop, Voice, and Devices · markdown · source line 133

**Try it with CARBON:** `/carbon-eval` — Evaluate agent behavior, trajectories, tool use, and consequential failure scenarios.

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · markdown · source line 7

Test whether placing a TestBucks vanilla-latte order creates exactly one authorized charge, one order, and one fulfillment request—even if the client retries or a downstream service is slow.

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · markdown · source line 20

**Try it with CARBON:** `/carbon-api` — Test schemas, authentication, errors, retries, and integration contracts.

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · markdown · source line 36

**Try it with CARBON:** `/carbon-networking` — Investigate request failures, timeouts, ordering, and network-dependent behavior.

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · markdown · source line 49

Run the missing-order variant. Preserve the confirmation ID, inspect the saved order state, and determine whether the confirmation is truthful. Report the two observations separately before giving the result.

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · markdown · source line 66

**Try it with CARBON:** `/carbon-data` — Compare records and invariants across systems, transitions, and generated artifacts.

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · markdown · source line 74

Inspect the supplied [DATA ARTIFACT] for inconsistencies, missing or duplicated records, suspicious outliers, impossible state transitions, unexpected correlations, and patterns that deserve investigation. Separate observations from hypotheses. Identify the exact rows, fields, records, timestamps, or byte ranges supporting each candidate. Also report useful confirming patterns and the evidence still needed to establish or reject each conclusion.

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · markdown · source line 78

**Artifact inspected: masked `orders-2026-08-30.csv`.** Candidate anomaly: order `O-1042` appears in rows 441 and 443 with the same customer, amount, and idempotency key, but different charge IDs six seconds apart.

**Why it matters.** This pattern is consistent with a duplicate charge after retry. It is not proof: the second row could be an authorization replacement, a reporting duplication, or an expected gateway event.

**Confirming evidence.** 99.8% of sampled retries reuse one charge ID, and ledger totals reconcile outside this two-row cluster.

**Next experiment.** Join both charge IDs to the payment ledger and gateway event stream, inspect the retry trace, and verify whether the customer balance changed once or twice. Preserve the masked rows and correlation IDs; do not send raw customer data to an unapproved model endpoint.

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · markdown · source line 106

**Try it with CARBON:** `/carbon-data` — Look for inconsistent, missing, duplicate, malformed, or unreconciled data.

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · markdown · source line 121

Here is an isolated HTML page and its JavaScript. Map the user-visible promises, state transitions, external dependencies, and recovery paths. Add reversible fault-injection seams or test adapters only in this test copy. Inject a small, prioritized set of failures - network timeout, malformed or delayed response, storage full, permission denial, duplicate delivery, and retry after partial success - and run the resulting AI Tests. For each fault, preserve the before-and-after state, user-visible outcome, console and network evidence, invariant checked, blast radius, cleanup, and whether the result is observed, blocked, or not assessed. Do not target production systems or real customer data.

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · javascript · source line 131

async function checkoutBefore({ gateway, storage, orderId }) {
  try {
    const receipt = await gateway.charge({
      idempotencyKey: `${orderId}-${Math.random()}`
    });
    await storage.write(receipt); // injected ENOSPC here
    return { screen: "success" };
  } catch (error) {
    return { screen: "retry", error: error.code };
  }
}

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · javascript · source line 147

async function checkoutAfter({ gateway, storage, orderId }) {
  try {
    const receipt = await gateway.charge({
      idempotencyKey: `order-${orderId}`
    });
    await storage.write(receipt); // injected ENOSPC here
    return { screen: "success" };
  } catch (error) {
    return { screen: "retry", error: error.code };
  }
}

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · markdown · source line 161

**Result: the original implementation violates the one-charge invariant.** After the emulated post-charge `ENOSPC` failure, retrying creates a second charge. The repaired implementation reuses the order-level idempotency key and keeps the charge count at one.

**Safety boundary.** This lab uses an in-memory gateway and storage emulator. It does not make a network request, write a real disk, or charge a processor.

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · text · source line 169

{
  "label": "before: disk-full after charge",
  "first": {
    "screen": "retry",
    "error": "ENOSPC"
  },
  "second": {"screen": "success"},
  "charges": 2,
  "duplicateCharge": true
}
{
  "label": "after: disk-full after charge",
  "first": {
    "screen": "retry",
    "error": "ENOSPC"
  },
  "second": {"screen": "success"},
  "charges": 1,
  "duplicateCharge": false
}

Chapter 6 — AI Tests APIs, Events, Data, and Distributed Systems · markdown · source line 211

**Try it with CARBON:** `/failure` — Inject approved faults into an isolated copy and investigate recovery without changing the original.

Chapter 7 — Record the Test’s Meaning: Portable Intent Across Tools · markdown · source line 39

Write a reusable test for checkout.

Chapter 7 — Record the Test’s Meaning: Portable Intent Across Tools · javascript · source line 55

test('retry preserves one charge', async ({ page }) => {
  // Illustrative helpers: setup and fault controls are product-specific.
  const orderId = await createSandboxOrder();
  await dropNextResponseAfterAcceptance(orderId);
  await openCheckout(page, orderId);
  await page.getByRole('button', { name: 'Pay now' }).click();
  await page.getByRole('button', { name: 'Retry payment' }).click();
  await waitForPaymentReconciliation(orderId);
  expect(await countAcceptedCharges(orderId)).toBe(1);
});

Chapter 7 — Record the Test’s Meaning: Portable Intent Across Tools · markdown · source line 91

In the approved TestBucks payment sandbox, check that retrying a purchase never charges the customer twice. Use a synthetic order. Simulate a lost response after the first payment is accepted, retry the same order, and compare the final order and payment records. A success screen alone is not enough: show evidence of exactly one accepted charge. Do not use a real card or production checkout.

Chapter 7 — Record the Test’s Meaning: Portable Intent Across Tools · json · source line 111

{
  "title": "Retrying a purchase preserves exactly one charge",
  "goal": "After an accepted sandbox payment loses its response, retrying the same order completes the purchase with exactly one accepted charge.",
  "steps": [
    {
      "description": "Open a fresh unpaid order in the approved payment sandbox.",
      "thinking": "Use a unique synthetic order and sandbox payment method so charges can be reconciled.",
      "action": "navigate",
      "target": "Sandbox checkout",
      "value": "/sandbox/checkout",
      "expected": "The unpaid order is ready and has no accepted charges."
    },
    {
      "description": "Arm the approved fault control to drop the response after acceptance, then submit payment.",
      "thinking": "Exercise an ambiguous response without blocking the original charge from being accepted.",
      "action": "click",
      "target": "Pay now",
      "expected": "The sandbox records one accepted charge while the client offers recovery."
    },
    {
      "description": "Retry payment for the same order and wait for reconciliation.",
      "thinking": "A retry must preserve the order identity rather than create a second charge.",
      "action": "click",
      "target": "Retry payment",
      "expected": "The purchase completes for the original order."
    },
    {
      "description": "Compare the final order and payment records after the agreed observation window.",
      "thinking": "A success screen alone cannot rule out a duplicate charge.",
      "action": "assert",
      "target": "Sandbox order and accepted-charge records",
      "expected": "The order is complete with exactly one accepted charge and no duplicate order."
    }
  ]
}

Chapter 7 — Record the Test’s Meaning: Portable Intent Across Tools · markdown · source line 222

**Try it with CARBON:** `/carbon-tests create an intent test for TestBucks checkout; preserve forbidden outcomes and required evidence` — Save a reusable candidate, then earn trust through execution.

Chapter 7 — Record the Test’s Meaning: Portable Intent Across Tools · text · source line 245

human or agent demonstration → recorded evidence → recovered intent → portable test → adaptive execution

Chapter 7 — Record the Test’s Meaning: Portable Intent Across Tools · markdown · source line 260

Execute the saved latte intent against the revised TestBucks interface. Keep the requested options and verify the cart and total. Record any adaptation and any product defect separately.

Chapter 7 — Record the Test’s Meaning: Portable Intent Across Tools · markdown · source line 276

**Try it with CARBON:** `/carbon-map compare the baseline and candidate builds; focus on checkout` — Explore the visual map and build differences, then investigate which changes affect the promise.

Chapter 7 — Record the Test’s Meaning: Portable Intent Across Tools · markdown · source line 306

Buy one medium vanilla latte with whole milk for pickup in the TestBucks practice environment. Run the intent on desktop, mobile, and tablet. Discover the controls in each interface. Verify one drink, the requested options, the applicable price, and a saved order. Use simulated payment only. Preserve separate evidence for each device.

Chapter 8 — Triage, Team Context, and Fix-Ready Handoff · markdown · source line 11

Using the supplied TestBucks screenshot, semantic DOM, console excerpt, and redacted network summary, create one OpenTest.ai-style AI Issue candidate only if the evidence supports a meaningful concern. For each candidate, state the evidence for it, the evidence against it, confidence that it is a real bug, priority, routing recommendation, the smallest falsifying experiment, and a fix prompt that tells a coding agent to reproduce before editing. Do not claim user impact or root cause that the artifacts do not establish.

Chapter 8 — Triage, Team Context, and Fix-Ready Handoff · markdown · source line 15

**Candidate title.** Homepage console emits an unclassified `pickupEstimate` reference error during observed load.

**Bug confidence: 4/10. Priority: 3/10.** The runtime error was observed twice, but the screenshot shows no affected user flow; the network sample does not identify the responsible code; and clean-session reproduction, source mapping, and user impact are unavailable.

**For the issue.** Repeated runtime `ReferenceError` evidence. **Against the issue.** It may be benign, sampled, environment-specific, or unrelated to user-visible behavior.

**Route and next experiment.** Send first to the web-platform or homepage owner for reproduction and source mapping. Reproduce in a fresh isolated load, preserve the original console and visible state, and check whether search or homepage behavior changes before proposing a repair.

Chapter 8 — Triage, Team Context, and Fix-Ready Handoff · markdown · source line 29

Triage this AI Issue using the attached evidence and the team’s triage policy. Do not assume the finding is real because another model reported it. State: (1) the evidence that supports the issue, (2) the evidence or plausible explanation that could make it a false positive or false negative, (3) confidence from 1–10 that it is a real defect, (4) priority from 1–10 for this milestone, (5) affected users and consequence if known, (6) the best routing target, (7) the smallest next experiment that could change the conclusion, and (8) a reproduction-first prompt for the AI coding agent. Mark every unsupported field as unknown.

Chapter 8 — Triage, Team Context, and Fix-Ready Handoff · markdown · source line 35

**1. Supporting evidence.** Two captured occurrences of `ReferenceError: pickupEstimate is not defined` during homepage load.

**2. Counterargument and error risk.** No confirmed visible defect; no responsible source in the bounded network sample; the signal may be benign, sampled, environment-specific, or unrelated to a user journey. An uncaptured role, locale, device, or retry state could also hide a false negative.

**3. Defect confidence: 4/10.** Repeatable candidate signal, not a confirmed product failure.

**4. Milestone priority: 3/10.** Advisory under this policy until a public-search or homepage behavior fails.

**5. Affected users and consequence: UNKNOWN.** Not established by the evidence.

**6. Route.** Web-platform or homepage owner for reproduction and source mapping.

**7. Smallest next experiment.** Fresh isolated load with source mapping; preserve console, network, screenshot, and primary-search behavior; determine whether the error recurs and changes a visible or functional result.

**8. Reproduction-first prompt.** Reproduce before editing. Identify the reference and its scope, preserve the original evidence, make only the supported repair, retest homepage load and primary search, and report whether the signal and any demonstrated impact disappear.

Chapter 8 — Triage, Team Context, and Fix-Ready Handoff · markdown · source line 188

Investigate this failed cancellation regression. Use the failed and last passing runs, logs, KB articles, previous incidents, requirements, and recent changes. What probably happened, what evidence argues against that explanation, and what is the smallest safe experiment to tell? You may rerun tests with synthetic orders in QA. Don't modify code or production. Separate product failures from test and environment failures.

Chapter 8 — Triage, Team Context, and Fix-Ready Handoff · markdown · source line 207

**Current assessment.** There is a test-oracle mismatch and a likely producer/worker compatibility failure. The test stops after 5 seconds although ORDER-17 allows 60. Separately, the worker rejects this order's event schema.

**Why the old incident may not apply.** INC-88 involved accepted messages waiting in a backlog. This trace shows rejection, not delayed processing. The KB workaround is a hypothesis to check, not a fix to copy.

**Smallest next experiment.** Without changing code, observe a fresh synthetic cancellation for the full 60-second allowance. Correlate its event with worker logs and the email sink. If it is rejected again and no confirmation arrives, queue delay alone does not explain the failure.

**Possible follow-up result.** The fresh event was rejected as schema v2; no email appeared within 60 seconds. That reproduces a contract failure in this QA build. Cancellation itself remained visible within 2 seconds.

**Route and next step.** Send the event, deployment versions, and trace to the orders/messaging owners. Ask whether the supported release should have included a v2-capable worker. Separately correct the regression's timeout to match the approved contract. Neither repair has been performed. Passing the cancellation UI check does not clear the email failure.

Chapter 9 — Upgrade Legacy Automation, Then Convert What Matters · markdown · source line 7

Upgrade these Playwright or Selenium tests for AI testing.

Chapter 9 — Upgrade Legacy Automation, Then Convert What Matters · ts · source line 23

await page.goto(`${baseUrl}/checkout`);
await page.getByRole('button', { name: 'Add mocha to cart' }).click();
await page.getByRole('link', { name: 'Cart' }).click();
await expect(page.getByText('Mocha')).toBeVisible();
await expect(page.getByTestId('cart-count')).toHaveText('1');

Chapter 9 — Upgrade Legacy Automation, Then Convert What Matters · ts · source line 33

await page.goto(`${baseUrl}/checkout`);
await page.getByRole('button', { name: 'Add mocha to cart' }).click();
await page.getByRole('link', { name: 'Cart' }).click();
await expect(page.getByText('Mocha')).toBeVisible();
await expect(page.getByTestId('cart-count')).toHaveText('1');

await ai_check({
  name: 'cart-visible-state',
  screenshot: await page.screenshot({ fullPage: true }),
  consoleLog: capturedConsoleLog(),
  networkActivity: capturedNetworkSummary(),
  dom: await page.content(),
  focus: ['visual-ui', 'usability', 'accessibility', 'privacy', 'security'],
});

Chapter 9 — Upgrade Legacy Automation, Then Convert What Matters · markdown · source line 86

**Try it with CARBON:** `/carbon-tests convert the selected legacy checkout tests and bake them in on staging` — Recover intent, retain provenance, and use repeated execution to separate candidates from dependable regressions.

Chapter 9 — Upgrade Legacy Automation, Then Convert What Matters · markdown · source line 119

Plan a read-only AI pilot around our approved Selenium and batch tests. Map the requirements, execution dependencies, and evidence we must retain. Propose one isolated parallel run. Do not edit tests, change expected results, send artifacts to unapproved services, or replace release gates. List the approvals and missing information first.

Chapter 11 — Self-Healing Is Backwards: Fix the Testability Boundary · markdown · source line 53

In an isolated TestBucks copy, add stable data-test IDs to checkout controls, accessible names, a controllable timeout fixture, and diagnostics that correlate the request with order and payment state. Keep private data out of the user-facing error. Show how a test can verify recovery.

Chapter 11 — Self-Healing Is Backwards: Fix the Testability Boundary · markdown · source line 73

**Try it with CARBON:** `/carbon-tests repair the failing checkout test without weakening its assertions; ask before changing product code` — Repair the execution contract while preserving the original failure and the test's meaning.

Chapter 12 — Test the Failures We Avoid: Mocks, Faults, Load, and Legacy Layers · markdown · source line 59

What important testing have we avoided because it was too expensive or hard to set up? Use this project's architecture and incident history. Pick one experiment we can now run safely in a disposable environment. Explain the old obstacle, how you will remove it, what result would matter, and how you will verify the experiment itself worked. Propose the plan before running it.

Chapter 12 — Test the Failures We Avoid: Mocks, Faults, Load, and Legacy Layers · markdown · source line 65

**Try it with CARBON:** `/failure` — Explore failure and recovery safely in a disposable copy.

Chapter 12 — Test the Failures We Avoid: Mocks, Faults, Load, and Legacy Layers · markdown · source line 76

Use the isolated recovery variant. Submit the latte order, inspect the mock ledger after the timeout, and retry with the same request key. Verify exactly one order and one charge for $6.60. Do not use a live payment service.

Chapter 12 — Test the Failures We Avoid: Mocks, Faults, Load, and Legacy Layers · markdown · source line 109

**Try it with CARBON:** `/carbon-load` — Plan and run an explicitly approved, bounded workload with declared safety limits.

Chapter 13 — The White-Box Tester We Always Wanted: Risk, Boundaries, and Black-Box Proof · markdown · source line 27

What should I test in this change?

Chapter 13 — The White-Box Tester We Always Wanted: Risk, Boundaries, and Black-Box Proof · markdown · source line 55

Investigate this name field’s 23-character limit. Test 22, 23, and 24 characters. Is 23 the right product limit? Use the source, requirements, and relevant context. Show your evidence and label anything unknown.

Chapter 13 — The White-Box Tester We Always Wanted: Risk, Boundaries, and Black-Box Proof · markdown · source line 59

**Observed implementation.** The UI validator, API schema, and database column each enforce a maximum of 23 characters. In the observed character unit, 22 and 23 are accepted; 24 is rejected before storage. The API and UI agree in this run.

**Boundary tests to retain.** Repeat 22/23/24 with pasted input, composed and decomposed accented names, CJK text, and an emoji/ZWJ grapheme. Confirm the value after reload, export, and downstream use—not only the first validation message.

**Why 23 might be right.** It may be a documented legacy limit or a constraint shared with an identity or downstream system.

**Why 23 might be wrong.** No requirement, policy, or interoperability constraint was found that establishes 23 as a user-appropriate maximum. It may be an inherited implementation convenience.

**Policy verdict: NOT ASSESSED.** The current limit is observed; its correctness is not established. Locate the governing requirement or ask the product owner before filing a definite defect. If an authoritative higher limit exists, file the 23-character constraint as an evidence-backed issue.

Chapter 13 — The White-Box Tester We Always Wanted: Risk, Boundaries, and Black-Box Proof · markdown · source line 71

**Try it with CARBON:** `/carbon-issues` — Investigate source-aware boundaries and challenge whether the implemented behavior is the right behavior.

Chapter 13 — The White-Box Tester We Always Wanted: Risk, Boundaries, and Black-Box Proof · markdown · source line 89

Compare ORDER-17 and its acceptance criteria with the build delivered to QA. Use the design, release notes, code changes, and build configuration. What is missing, inconsistent, or ambiguous? Show the evidence for each gap and suggest the smallest tests or product questions that would resolve it. Don't change the product.

Chapter 13 — The White-Box Tester We Always Wanted: Risk, Boundaries, and Black-Box Proof · markdown · source line 103

**Two implementation gaps to investigate.** The cancellation handler lacks the required preparation-state check, and the UI has no explicit display for the cancelled state. These are source-supported candidates, not confirmed runtime failures.

**One requirement question.** Does “confirmation” mean an on-screen receipt, email, or both? The available documents do not settle that. I would not file “missing email” as a confirmed defect.

**First tests.** With synthetic orders in the QA environment, cancel one eligible order and one already preparing. Verify the API response, stored state, and UI after reload. Preserve the build identifier and configuration with the results.

**Handoff.** Route the state guard to the orders team, the status display to the UI team, and the confirmation question to Product. No code was changed and no runtime tests have run yet.

Chapter 13 — The White-Box Tester We Always Wanted: Risk, Boundaries, and Black-Box Proof · markdown · source line 119

**Try it with CARBON:** `/carbon-review` — Compare requirements, implementation, and available evidence to identify unsupported or missing promises.

Chapter 13 — The White-Box Tester We Always Wanted: Risk, Boundaries, and Black-Box Proof · markdown · source line 137

**Try it with CARBON:** `/carbon-generate` — Generate prioritized test candidates from the project and its evidence; candidates still need execution.

Chapter 13 — The White-Box Tester We Always Wanted: Risk, Boundaries, and Black-Box Proof · markdown · source line 145

Generate risk-ranked AI Test candidates for TestBucks from the supplied requirements, source, routes, existing evidence, and run history. First map product promises, state transitions, implementation boundaries, roles, failure consequences, and missing evidence. Generate positive, negative, boundary, accessibility, security, data-integrity, interruption, recovery, concurrency, and responsive cases where they can change a quality or release decision. Return candidates, not claimed coverage. For every candidate, include its source, risk, oracle, required evidence, safety limits, priority, and reason. Cluster semantic duplicates, explain which overlapping cases should remain, select the smallest useful first execution set, and preserve deferred candidates. Do not read the seeded-defect key.

Chapter 13 — The White-Box Tester We Always Wanted: Risk, Boundaries, and Black-Box Proof · markdown · source line 149

**Generation summary.** I mapped six product promises and risk boundaries, generated 16 source-informed candidates, clustered the overlapping route, transition, and accessibility cases, and selected nine intents for the first bounded run.

**Highest-information selections.** Core create-and-reload persistence; empty and repeated submission; keyboard access to drink cards; edit-and-reload; rejected-save recovery; safe rendering of user-authored pickup notes; escaped unknown-route recovery; malformed-state recovery; and narrow-viewport operation.

**Deferred, not discarded.** Statistics reconciliation, settings persistence, two-tab conflicts, larger collections, direct-route refresh, repeated transitions, enlarged text, and broader interaction-mode comparisons remain in the coverage ledger with reasons and triggers.

**Evidence boundary.** These are generated candidates, not passing tests and not measured coverage. I did not read the seeded-defect key.

Chapter 13 — The White-Box Tester We Always Wanted: Risk, Boundaries, and Black-Box Proof · markdown · source line 268

**Try it with CARBON:** `/carbon-functionality` — Focus on form validation, boundary values, error states, and submitted data.

Chapter 14 — Personas That Actually Test the Product · markdown · source line 33

Test this product with several realistic personas.

Chapter 14 — Personas That Actually Test the Product · json · source line 53

{
  "id": "ai-persona-cautious-buyer",
  "display_name": "Maya",
  "label": "AI persona",
  "role": "First-time, privacy-sensitive buyer",
  "context": "Mobile browser, limited time, unfamiliar with the brand",
  "goal": "Understand the total price and complete checkout without creating an account",
  "concerns": ["unexpected fees", "marketing consent", "unclear returns"],
  "success": ["total cost is understood", "consent choices are respected", "order is confirmed once"],
  "evidence": ["step screenshots", "network consent state", "confirmation text"]
}

Chapter 14 — Personas That Actually Test the Product · markdown · source line 86

Explore the mocha journey as a first-time customer, a rushed commuter, and a keyboard-only customer. Give each a concrete goal. Record actions and observed obstacles before offering qualitative feedback. Verify the final drink options for every completed journey.

Chapter 14 — Personas That Actually Test the Product · markdown · source line 100

Using Maya's persona record, explore the checkout as a first-time, privacy-sensitive mobile buyer. Try to understand the total cost and complete a sandbox purchase without creating an account. At every meaningful state, record what Maya expects, what the product shows, the evidence captured, and any trust or recovery friction. Do not submit a real payment.

Chapter 14 — Personas That Actually Test the Product · markdown · source line 104

**Outcome: BLOCKED before payment.** I reached the review step but could not establish the final total without opening a separate shipping-information panel. The interface also selected marketing consent by default. I stopped before submitting payment, as required.

**Persona reaction.** “I expected the total to be visible before sharing more information. A preselected marketing choice makes me less willing to continue.” This is qualitative persona evidence, not a measured customer opinion.

**Observed product evidence.** Mobile screenshots of the collapsed cost panel and selected consent control; DOM state for the checkbox; no payment request in the network trace.

**Next work.** Create an AI Test for total-price visibility before irreversible commitment and a Privacy AI Check comparing the consent UI with observed network behavior. Route the trust reaction into product review rather than calling it a deterministic failure by itself.

Chapter 14 — Personas That Actually Test the Product · markdown · source line 118

**Try it with CARBON:** `/carbon-personas` — Generate evidence-qualified persona feedback; use /carbon-test for a scoped executable journey.

Chapter 15 — Skepticism, Oracles, and Models Judging Models · markdown · source line 41

Double-check these test results and judge what is actually supported.

Chapter 15 — Skepticism, Oracles, and Models Judging Models · markdown · source line 47

**Claim reviewed.** TestBucks executes user-authored pickup-note markup on the staff Orders route.

**Illustrative verdict: SUPPORTED if these artifacts are observed in an isolated TestBucks extension.** The saved synthetic pickup note produced an image element inside `.orders-panel`, its inert same-origin event handler set `window.__testbucksLabXss` to `1`, and the source renders pickup-note rows through `innerHTML`. These independent artifacts agree.

**Strongest counterargument.** The payload and environment are synthetic. That reproduction would establish the vulnerable behavior only in the tested extension; it does not establish reachability, authentication context, browser coverage, or production impact in another deployment.

**Decision.** Preserve the finding as a severe lab release veto. Repair the rendering boundary, rerun the exact payload plus ordinary pickup notes, and add a negative control that must remain inert. Do not broaden the claim beyond the observed revision and environment.

Chapter 15 — Skepticism, Oracles, and Models Judging Models · markdown · source line 82

**Reply A:** “Absolutely! I've refunded $10 to your card, and it will arrive today. Enjoy your next coffee on us!”

**Reply B:** “Your incorrect drink qualifies for a $4.50 refund to the original payment method. The refund has not been submitted yet.”

Chapter 15 — Skepticism, Oracles, and Models Judging Models · markdown · source line 96

Judge these TestBucks replies against the supplied policy and order record. Identify supported and unsupported claims, and score tone separately from correctness. Repeat the comparison with answer order reversed and with concise and expanded versions of the same meaning. Report any verdict changes and the evidence that explains them.

Chapter 15 — Skepticism, Oracles, and Models Judging Models · markdown · source line 166

Design a quality rubric for the TestBucks support chatbot, a Starbucks-like coffee shop assistant. Use resolution, offensive content, relevance, elapsed time, and conversation rounds. Define observable evidence, units, anchors, rewards, penalties, caps, missing-data handling, and non-negotiable gates. Work through a fast success, a slow success, a polite failure, and a serious conduct violation. Explain every coefficient as a proposed product tradeoff, test how plausible alternative weights change the ranking, and list decisions the product owner must settle. Label invented examples illustrative. Do not turn quality points into a release probability.

Chapter 15 — Skepticism, Oracles, and Models Judging Models · markdown · source line 195

Judge the pickup-time message against the documented requirement. Separate visible evidence from assumptions. If the wording is ambiguous, state the missing rule and propose the smallest test that would resolve it.

Chapter 16 — Developers Are Becoming Testers—People Move Up the Stack · markdown · source line 99

**Try it with CARBON:** `/carbon` — Start a bounded, risk-based assessment of the current project. The coding agent executes the checks using CARBON's testing workflow, preserves evidence, and reports findings and gaps. The developer asks once instead of manually directing every test.

Chapter 16 — Developers Are Becoming Testers—People Move Up the Stack · markdown · source line 107

**Try it with CARBON:** **Agent-delegated testing.** The AI coding agent knows CARBON is available. After changing the TestBucks cart, it asks CARBON to organize the relevant testing on the developer's behalf, uses the available tools to collect evidence, and brings the findings back into the coding conversation. The developer does not type a separate CARBON prompt.

Chapter 16 — Developers Are Becoming Testers—People Move Up the Stack · markdown · source line 115

**Try it with CARBON:** **Emerging proactive loop.** New feature detected → identify positive tests, negative tests, adjacent regressions, and added risks → add or update the relevant checks → execute them → return evidence for a fix → verify the correction. The developer keeps building while testing work starts proactively within the agreed scope.

Chapter 17 — What an AI Test Harness Actually Is · markdown · source line 23

Test this project.

Chapter 17 — What an AI Test Harness Actually Is · markdown · source line 58

**Try it with CARBON:** `/carbon assess this repository and run a bounded, risk-ranked pass using synthetic data` — Put the testing curriculum to work through one evidence-preserving workflow.

Chapter 18 — Context, Steering, Test State, and Authority · markdown · source line 26

/test Generate only a few boundary AI Tests for the sign-in dialog. Focus on the values just below, at, and just above its actual limits, and preserve the evidence needed to explain each result.

Chapter 18 — Context, Steering, Test State, and Authority · markdown · source line 30

**Scope applied.** Sign-in dialog only; six boundary candidates selected instead of the full authentication suite.

**Source-derived boundaries.** Username accepts at most 64 Unicode code points; password accepts at most 128. Generate 63/64/65 username cases and 127/128/129 password cases, including pasted input and one multi-code-point grapheme variation where the implementation unit may differ from what the person sees.

**Evidence required.** Visible validation state, submitted request shape with secrets redacted, response status, focus location, console errors, and whether the entered value survives a rejected submission correctly.

**Status: CANDIDATES, NOT COVERAGE.** The limits were extracted from the supplied implementation context; the AI Tests still have to run, and the product rationale for those limits remains a separate question.

Chapter 18 — Context, Steering, Test State, and Authority · markdown · source line 106

Use the authorized Project TestBucks folder to answer these business questions: Which discount combinations threaten margin? Which refund policy applies to existing orders? Which checkout failure should block release, and why? Find relevant code, specifications, product decisions, and financial assumptions. Cite the source and version for each answer. Separate documented facts, calculations, inferences, and missing decisions. Do not invent absent policies or treat a draft as approved. Explain what additional evidence would change your recommendation.

Chapter 18 — Context, Steering, Test State, and Authority · markdown · source line 152

**Try it with CARBON:** `/carbon-human identify missing business context for the checkout release` — Ask the people who can resolve consequential assumptions and retain their answers.

Chapter 18 — Context, Steering, Test State, and Authority · markdown · source line 222

Plan a review sample for this frozen TestBucks evaluation batch. Separately estimate unsupported bug reports and incorrect pass verdicts. Define each population and sampling unit, propose sample sizes for an explicit precision target, and stratify important journeys and user groups. Show the plan before selecting cases. After approval, select reproducibly at random, preserve original evidence, and prepare a compact review packet. Do not invent review labels. Once independently checked labels are available, calculate rates and appropriate confidence intervals, preserving unresolved cases and critical findings separately. State what these estimates do not establish about missed bugs or release safety.

Chapter 18 — Context, Steering, Test State, and Authority · markdown · source line 307

Prepare a minimal evidence bundle for investigating this cancellation failure. Work locally with the approved sources. Remove secrets and identifying information, preserve cross-artifact relationships and timing, and create synthetic fixtures where practical. Show the transformation manifest and validation checks. Do not send the bundle outside the approved boundary until the release check passes.

Chapter 18 — Context, Steering, Test State, and Authority · markdown · source line 346

**Try it with CARBON:** `/carbon-settings review the local test environment and credential configuration` — Set the execution context without putting secrets into prompts or reports.

Chapter 19 — From an Unfamiliar Repository to a Living Regression System · markdown · source line 17

**Try it with CARBON:** `/carbon-help inspect this repository and recommend the smallest useful first testing pass` — Find a starting point from the project and its existing evidence.

Chapter 19 — From an Unfamiliar Repository to a Living Regression System · markdown · source line 61

**Try it with CARBON:** `/carbon-tests import the existing regression cases as candidates; preserve their original source` — Bring useful legacy intent into the durable library without treating import as a pass.

Chapter 20 — Continuous AI Test Selection While the Change Is Still Warm · markdown · source line 51

**Try it with CARBON:** `/carbon-test test the new login button and adjacent logout flow using test accounts` — Keep the investigation focused on the current change and its immediate risks.

Chapter 20 — Continuous AI Test Selection While the Change Is Still Warm · markdown · source line 82

Review the sign-in change. Select tests for the changed behavior and its dependencies: sign-in, guest-cart continuity, and narrow-screen navigation. Capture any layout regression, repair it in the isolated copy, and rerun the affected checks.

Chapter 20 — Continuous AI Test Selection While the Change Is Still Warm · markdown · source line 148

Review our generated checkout tests for maintenance cost and unique risk coverage. Propose retain, merge, regenerate, defer, or archive for each group. Preserve severe-failure, incident, and required-coverage cases. Show any coverage we would lose and how to validate the proposed replacements. Produce a reviewable plan; do not delete or archive anything yet.

Chapter 20 — Continuous AI Test Selection While the Change Is Still Warm · markdown · source line 156

**Try it with CARBON:** `/carbon-tests identify overlapping tests and propose archive candidates; do not archive yet` — Reduce maintenance waste while preserving the reasons and risks behind each test.

Chapter 21 — Confidence Testing: Start with the Question · markdown · source line 53

I'm worried about payment retries. If the payment goes through but the response gets lost, can trying again charge the customer twice? Investigate in the sandbox. Don't use real payments or change the product yet. Show what happened in the order and payment records.

Chapter 21 — Confidence Testing: Start with the Question · markdown · source line 59

I'll trace the payment and retry paths, then arrange a lost response after sandbox acceptance. I'll record the order ID, operation ID, provider transaction, retry, and final ledger state.

The first question is whether the initial payment really reached acceptance. A timeout by itself won't establish that. If I can't create that condition or inspect the records, I'll report the gap instead of treating the retry as verified.

Chapter 21 — Confidence Testing: Start with the Question · markdown · source line 69

Also try two clients retrying the same order at once. Keep that separate from two legitimate purchases of the same product.

Chapter 21 — Confidence Testing: Start with the Question · markdown · source line 109

**Try it with CARBON:** `/carbon-confidence can checkout lose or duplicate an order? Investigate on staging for up to 20 minutes` — Follow a consequential question through evidence gathering, challenge, and reassessment.

Chapter 22 — Confidence in the Results: Do We Know Enough to Release? · markdown · source line 37

Do we have enough information and confidence to ship?

Chapter 22 — Confidence in the Results: Do We Know Enough to Release? · markdown · source line 47

**TestBucks AI Confidence: 22 / 100. Decision: HOLD.** In this illustrative extended-build result, three defects reproduced: stored pickup-note injection on staff Orders, drink cards absent from the keyboard path, and mobile menu overflow. The stored-injection finding is a severe release veto.

**Not assessed.** Fault recovery, repairs, cross-browser behavior, cloud execution, load, and most route combinations.

**Why this is not broader confidence.** The instructor key contains six seeded defects, but the run reproduced only three. A known planted defect does not count as detected until evidence reproduces it, and three findings do not establish the quality of the whole application.

**Next evidence most likely to change the decision.** Repair and rerun the severe issue; exercise rejected and delayed saves; repeat the primary journey in another browser and narrow viewport.

Chapter 22 — Confidence in the Results: Do We Know Enough to Release? · markdown · source line 152

Given the payment investigation, how confident are we about shipping this change? Use a 0–100 scale, explain what moves the score, and tell me what you still don't know.

Chapter 22 — Confidence in the Results: Do We Know Enough to Release? · markdown · source line 156

**Release readiness: 35 / 100. Recommendation: HOLD.**

Serial retries preserved one charge in the exercised scenario. Concurrent retries produced two accepted charges for the same order. That violation blocks shipment regardless of the overall score.

The score summarizes evidence strength and unresolved risk; it is not a 35% probability of success. Prolonged outage recovery and legacy-order migration remain untested. The next useful work is to repair the concurrent path, rerun the reproduction, and verify that separate legitimate orders still complete independently.

Chapter 22 — Confidence in the Results: Do We Know Enough to Release? · markdown · source line 164

Why are you less worried about migration than concurrency? We changed the order identifiers too.

Chapter 22 — Confidence in the Results: Do We Know Enough to Release? · markdown · source line 168

You're right to challenge that. I treated migration as an unknown but did not account for the changed identifiers in the retry identity. Existing orders could behave differently from newly created ones.

I'll elevate legacy-order migration in the investigation queue. The release remains HOLD. I need fixtures representing pre-change orders and the migration mapping before I can assess that path. I have not run those cases yet.

Chapter 22 — Confidence in the Results: Do We Know Enough to Release? · markdown · source line 186

**Try it with CARBON:** `/carbon-full-report show current evidence, blocked work, and unresolved release risks` — Read the quality record rather than relying on a pass count.

Chapter 22 — Confidence in the Results: Do We Know Enough to Release? · markdown · source line 282

Review this TestBucks release evidence and risk map. What has not been considered or tested? Examine rollback and schema compatibility, durable orders and charges, integrations, customer groups, production exposure, oracle weaknesses, and correlated evidence. Separate verified behavior, unreliable verdicts, known gaps, and newly proposed risks. Use incident history to discuss surprise without inventing a probability. Cite artifacts for factual claims and label analogies or hypotheses. Explain what matters for the declared release stage, what would change the decision, and what monitoring and containment are needed. Do not count every passing test equally or average away a severe failure.

Chapter 22 — Confidence in the Results: Do We Know Enough to Release? · markdown · source line 324

Before evaluating this recommendation change, draft the acceptance policy with the product owner. Separate forbidden outcomes from graded usefulness. Define the population, rubric, sampling unit, important slices, uncertainty method, and stop rules. Freeze the approved policy, then evaluate the candidate and baseline. Report pass, fail, or insufficient evidence without relaxing the thresholds.

Chapter 22 — Confidence in the Results: Do We Know Enough to Release? · markdown · source line 332

**Usefulness: insufficient evidence.** 93 of 100 scenarios met the rubric, but the lower confidence bound does not exceed the agreed 90% threshold.

**Severe outcomes: blocked.** One traced run placed an order without the required confirmation. That independently blocks release, even if a larger usefulness sample would pass.

**Next action.** Repair and independently verify the authorization failure. Then evaluate the revised candidate under the same approved policy, with evidence appropriate to the changed behavior. Do not average away the severe run or erase it with successful retries.

Chapter 23 — Close the Confidence Gaps · markdown · source line 97

**Try it with CARBON:** `/carbon-more investigate the highest-risk deferred candidates using the existing safe test setup` — Continue from the evidence already collected instead of starting the same pass again.

Chapter 23 — Close the Confidence Gaps · markdown · source line 127

**Try it with CARBON:** `/carbon-human what missing decisions could change our release assessment?` — Surface material unknowns without turning every run into a generic questionnaire.

Chapter 24 — Send Intent to the Cloud, Not Remote-Control Scripts · markdown · source line 33

Run these tests in the cloud.

Chapter 24 — Send Intent to the Cloud, Not Remote-Control Scripts · markdown · source line 112

Run separate TestBucks journeys for a flat white and a chai in isolated browser sessions. Use only simulated orders. Verify each cart and order independently, then merge the evidence by intent and build. Do not let one worker reset another's data.

Chapter 25 — The AI Test Harness Must Test Itself—and Learn from Production · markdown · source line 95

Investigate these checkout interaction failures using the approved, redacted browser events, traces, support report, and release diff. Compare affected and unaffected journeys. Propose competing explanations, then reproduce the most plausible one in staging with synthetic data. Don't interact with customers or change production. Show how the result should change our regression coverage and monitoring.

Chapter 25 — The AI Test Harness Must Test Itself—and Learn from Production · markdown · source line 99

**Candidate explanation.** The address-validation overlay may remain over the checkout button after correction. Missing server requests point toward a client-side problem, but they do not prove that cause.

**Staging reproduction.** In this example, entering an invalid address, correcting it, and closing the mobile keyboard leaves an invisible overlay intercepting taps. The same sequence with a valid address from the start does not fail. The browser capture and event-target inspection support the overlay explanation.

**Coverage change.** Add the address-error recovery journey on the affected viewport. Check pointer and keyboard operation, overlay dismissal, and exactly one resulting order. Keep the original failed sequence as evidence rather than replacing it with the happy path.

**Monitoring change.** Track checkout-intent interactions without a subsequent request or visible validation response, segmented by build and relevant device characteristics. Validate the event definition and an agreed time window before alerting on it. Also monitor successful order completion, so fewer alerts cannot hide broken instrumentation.

**Release decision.** Recommend holding further rollout while the responsible owner evaluates the defect. No production repair or rollback has been performed.

Chapter 25 — The AI Test Harness Must Test Itself—and Learn from Production · markdown · source line 115

**Try it with CARBON:** `/carbon-generate use the sanitized incident timeline to propose regression tests; do not execute against production` — Turn a production signal into test candidates with explicit evidence and safety boundaries.

Chapter 26 — Ten Minutes, Same App: What Humans and AI Actually Find · markdown · source line 82

Investigate the report “My cart item disappeared after reload.” Reproduce it in a disposable session. Record the actions, visible results, and relevant storage or network evidence. Find out what the product promises about saving. Separate the observed failure from your explanation of its cause. Do not fix it yet.

Chapter 26 — Ten Minutes, Same App: What Humans and AI Actually Find · markdown · source line 86

The cart item was visible after Add and absent after reload in this example run.
That establishes a persistence symptom, not its intended behavior.
Next: check the save promise and correlate the action with storage evidence.
If cart items are meant to survive reload, this is a defect candidate.
If the session is intentionally temporary, investigate whether that is made clear.
No fix applied.

Chapter 26 — Ten Minutes, Same App: What Humans and AI Actually Find · markdown · source line 145

**Try it with CARBON:** `/carbon-test explore TestBucks for ten minutes; preserve discoveries separately from the human findings` — Run a bounded comparison without contaminating the independent first passes.

Chapter 27 — Works as Written. Still Feels Wrong. · markdown · source line 53

Use this TestBucks shop as someone who needs to order coffee quickly and trust that it will still be there later. Test the main flows, then question the experience: what is confusing, missing, or misleading? Separate reproduced defects from product questions and preferences. For each concern, suggest the smallest useful follow-up.

Chapter 27 — Works as Written. Still Feels Wrong. · markdown · source line 57

Concern: “Added” may be interpreted as “saved for later.”
Type: experience hypothesis, pending the persistence requirement.
Evidence needed: actual save behavior, displayed wording, and the intended promise.
Next step: compare what survives reload with what the interface tells the user.
If storage is temporary, make that boundary visible before a cart is lost.
If durable saving is promised, investigate the implementation instead.

Chapter 27 — Works as Written. Still Feels Wrong. · markdown · source line 70

**Try it with CARBON:** `/carbon-personas review this TestBucks shop for a first-time user and a busy returning user` — Gather qualitative reactions and disagreements to turn into testable product questions.

Chapter 28 — Human in the AI Loop: From Feedback to Better Software · markdown · source line 105

**Try it with CARBON:** `/carbon-feedback prepare a local review of the order-saving flow with screenshots and the persistence question; do not send invitations yet` — Package a bounded review and bring the resulting feedback back into the testing work.

Chapter 28 — Human in the AI Loop: From Feedback to Better Software · markdown · source line 115

Review this cart-loss finding and the attached feedback. Determine whether the disagreement is about observed behavior, intended behavior, or the environment. Ask only for missing decisions that could change the fix. Then propose a repair and focused regression plan. Wait for approval before changing code.

Chapter 28 — Human in the AI Loop: From Feedback to Better Software · markdown · source line 119

The disagreement concerns intended persistence, not whether the order vanished.
Product decision needed: must cart items survive an ordinary reload?
Environment check needed: is this the supported persistence configuration?
If both are confirmed, repair durable saving and verify reload behavior.
Also test failed writes: the UI must not claim success when saving fails.
I have not changed the code.

Chapter 28 — Human in the AI Loop: From Feedback to Better Software · markdown · source line 134

Decision: durable saving is required for this workflow.
Repair status: applied in the example branch after approval.
Verification: original reload scenario and failed-write behavior now pass in this example.
Remaining gap: cross-device synchronization was not part of this run.
Feedback retained: “Added” must not imply a successful save before confirmation.
Confidence change: the local persistence concern is reduced; no claim of full release readiness.

Chapter 29 — Bug-Finder Leaderboards and Choosing the Right Model · markdown · source line 38

Which model is best for this testing work?

Chapter 29 — Bug-Finder Leaderboards and Choosing the Right Model · markdown · source line 112

Compare TestBucks models A and B case by case. Show cases both pass, cases both fail, improvements, and new regressions. Cite the changed responses and supporting records. Apply the predefined privacy and authorization release gates before summarizing the overall score.

Chapter 29 — Bug-Finder Leaderboards and Choosing the Right Model · markdown · source line 158

Using only the local source and trace, explain the timeout boundary and propose a test for duplicate orders. Keep the investigation local. Run the test in the isolated lab and distinguish your prediction from the observed ledger state.

Chapter 30 — The Real Price of AI Testing · markdown · source line 104

**Try it with CARBON:** `/carbon-full-report summarize recorded runs and outcomes; identify missing cost or timing evidence` — Ground the economic discussion in retained work rather than inventing measurements.

Chapter 31 — Transform the Testing System, Not Just the Test Cases · markdown · source line 26

**Try it with CARBON:** `/carbon-help recommend a first-day testing workflow for this repository and its existing test assets` — Begin with useful work before committing to a large migration.

Chapter 31 — Transform the Testing System, Not Just the Test Cases · markdown · source line 62

**Try it with CARBON:** `/carbon-auto test and repair the approved checkout scope locally for 20 minutes; preserve failures and rerun relevant regressions` — Try a bounded test-fix-retest loop with explicit scope and a stopping budget.

Chapter 32 — Testing into the Future: Dynamic Software and Synthetic Populations · markdown · source line 5

How do we test software that changes for every person?

Chapter 32 — Testing into the Future: Dynamic Software and Synthetic Populations · markdown · source line 143

**Try it with CARBON:** The `/proof` command is an early, bounded step in this direction. It discovers consequential code and asks a person to approve a specific property and scope. Its current restricted Boolean-function path supports exhaustive verification and an optional Lean Boolean-model backend, with independently reproduced counterexamples and regression fixtures. The Lean result concerns the model, not a machine-checked equivalence to production Python. Automated AI proof search is not enabled in this initial implementation; the broader AI-guided investigation loop described here is the future direction, not a claim that the command already implements all of it.

Turn a blind spot into a useful test · markdown · source line 191

Investigate this reported concern: [observation] in [journey].
Available evidence: [artifacts].
Classify it as a defect, unresolved symptom, requirements question,
or product suggestion. Do not assume either side is correct.
Identify missing context; investigate competing explanations.
Do not invent rules, customer reactions, or successful checks.
Propose a bounded test and name any decision needing a human owner.

Appendix D - JEV: Fast Decisions Need a Careful Harness · javascript · source line 85

// Simplified explanation of the recorded architecture.
const before = await observePage();
const available = await availableProbes(before, history);
const decision = await chooseWithJev(before, available, history);
await executeWithPlaywright(decision.choice);
const after = await observePage();
const judgment = await judgeWithJev(requirements, decision, before, after);
recordAndDisplay({ decision, before, after, judgment });

Appendix E - 15 Myths About AI and Testing · markdown · source line 267

Find and reproduce defects. Do not modify the application or weaken existing assertions. Derive expected behavior from the requirements. If a test appears incorrect, explain why and propose a separate change for review.

Appendix G - CARBON: Installation and Workflow Reference · markdown · source line 713

Use the installed Feedback workflow to prepare a review of the TestBucks checkout change for our testers. Include the selected mobile checkout view, the cart rules, and the question: is the final total and recovery behavior clear? Show me the bundle, reviewer access, and expiry before sharing. After approval, queue the request and return replies to this task using supported monitoring. Let me choose which findings to act on, then investigate, fix the approved items, and request a retest.

Appendix G - CARBON: Installation and Workflow Reference · sh · source line 747

git clone https://github.com/jarbon/carbon-free.git
claude --plugin-dir /absolute/path/to/carbon-free

Appendix G - CARBON: Installation and Workflow Reference · markdown · source line 762

Use CARBON Studio to test this project and show the live workspace. Start with the highest-risk customer journey. Use synthetic data, record evidence, and ask before making consequential changes.

Appendix G - CARBON: Installation and Workflow Reference · markdown · source line 772

Use CARBON to test the TestBucks practice application. Focus on finding a vanilla latte, adding it to the cart, and verifying the item and total. Use synthetic data and no real payments. Save screenshots, actions, results, and supported bugs. Separate failures from checks that were not run. Show the report and the next most useful test.

Appendix G - CARBON: Installation and Workflow Reference · markdown · source line 795

Plan a CARBON Cloud test of my authorized staging application. Use separate synthetic accounts for parallel journeys. Show the proposed scope, concurrency, and cost limit before execution. Do not place real orders. After the approved run, return the report, linked evidence, unresolved risks, and the most valuable next checks.