diff --git a/qa/scenarios/agents/subagent-fanout-synthesis.yaml b/qa/scenarios/agents/subagent-fanout-synthesis.yaml index cdb699d6968..31fcb4e3086 100644 --- a/qa/scenarios/agents/subagent-fanout-synthesis.yaml +++ b/qa/scenarios/agents/subagent-fanout-synthesis.yaml @@ -25,14 +25,13 @@ scenario: summary: Verify the agent can delegate multiple bounded subagent tasks and fold both results back into one parent reply. config: prompt: |- - Subagent fanout synthesis check: delegate exactly two bounded subagents sequentially. - Subagent 1: verify that `HEARTBEAT.md` exists and report `ok` if it does. - Subagent 2: verify that `repo/qa/scenarios/agents/subagent-fanout-synthesis.yaml` exists and report `ok` if it does. - Wait for both subagents to finish. + Subagent fanout synthesis check: delegate exactly two bounded subagents sequentially using sessions_spawn, not ACP. + First spawn exactly one child with label qa-fanout-alpha and task: verify that `HEARTBEAT.md` exists and reply exactly `ok` if it does. Wait for that child to finish. + Then spawn exactly one child with label qa-fanout-beta and task: verify that `repo/qa/scenarios/agents/subagent-fanout-synthesis.yaml` exists and reply exactly `ok` if it does. Wait for that child to finish. + Do not spawn any more children after qa-fanout-beta finishes. Then reply with exactly these two lines and nothing else: subagent-1: ok subagent-2: ok - Do not use ACP. expectedReplyAny: - "subagent-1: ok" - "subagent-2: ok" @@ -89,11 +88,14 @@ flow: expr: config.prompt timeoutMs: expr: liveTurnTimeoutMs(env, 90000) - - call: waitForCondition + - call: waitForAgentHistoryReply saveAs: outbound args: + - ref: env + - ref: sessionKey - lambda: - expr: "state.getSnapshot().messages.filter((message) => message.direction === 'outbound' && message.conversation.id === 'qa-operator' && config.expectedReplyGroups.every((group) => group.some((needle) => normalizeLowercaseStringOrEmpty(message.text ?? '').includes(needle)))).at(-1)" + params: [text] + expr: "config.expectedReplyGroups.every((group) => group.some((needle) => normalizeLowercaseStringOrEmpty(text).includes(needle)))" - expr: liveTurnTimeoutMs(env, 120000) - expr: "env.providerMode === 'mock-openai' ? 100 : 250" - if: diff --git a/qa/scenarios/memory/dreaming-shadow-trial-report.yaml b/qa/scenarios/memory/dreaming-shadow-trial-report.yaml index fdd3c864a6d..585ff9a5173 100644 --- a/qa/scenarios/memory/dreaming-shadow-trial-report.yaml +++ b/qa/scenarios/memory/dreaming-shadow-trial-report.yaml @@ -45,9 +45,15 @@ scenario: 1. Read DREAMING_SHADOW_TRIAL_BRIEF.md. 2. Read DREAMING_CANDIDATE_EVIDENCE.md. 3. Write ./dreaming-shadow-trial-report.md. - 4. Include: Candidate, Trial prompt, Baseline outcome, Candidate outcome, Verdict, Reason, Risk flags, Promotion action. - 5. For this seeded evidence, Verdict must be helpful. - 6. Promotion action must be report-only. + 4. Include these exact field labels and seeded phrases: + Candidate: The user prefers release reports that include exact verification commands and remaining risk. + Trial prompt: Prepare a release readiness reply for a local OpenClaw QA change. + Baseline outcome: mentions tests passed but omits the exact command and remaining risk. + Candidate outcome: includes the exact verification command and calls out the remaining review risk. + Verdict: helpful + Reason: explains why the candidate improves the release reply. + Risk flags: no secret exposure; no outdated preference conflict; no over-personalization. + Promotion action: report-only DREAMING_CANDIDATE_EVIDENCE.md: |- # Candidate evidence @@ -59,7 +65,7 @@ scenario: prompt: |- Dreaming shadow trial report check. Read DREAMING_SHADOW_TRIAL_BRIEF.md and DREAMING_CANDIDATE_EVIDENCE.md first. Then write ./dreaming-shadow-trial-report.md as a report-only shadow trial. - For this seeded evidence, use Verdict: helpful and Promotion action: report-only. + The report must include the exact field labels and seeded phrases from the required report contract, including Verdict: helpful, Risk flags: no secret exposure, and Promotion action: report-only. Do not edit MEMORY.md and do not claim the candidate was promoted. Reply with the report path and exact marker DREAMING-SHADOW-TRIAL-OK. expectedReportAll: diff --git a/qa/scenarios/personal/share-safe-diagnostics-artifact.yaml b/qa/scenarios/personal/share-safe-diagnostics-artifact.yaml index eae24389eb2..50613406465 100644 --- a/qa/scenarios/personal/share-safe-diagnostics-artifact.yaml +++ b/qa/scenarios/personal/share-safe-diagnostics-artifact.yaml @@ -44,7 +44,12 @@ scenario: 1. Read DIAGNOSTICS_REQUEST.md. 2. Read PERSONAL_DIAGNOSTICS_RAW.md. 3. Write ./personal-diagnostics-summary.txt. - 4. Include useful operational facts only: status, affected surface, omitted content counts, redaction confirmation, and next step. + 4. Include these exact share-safe fields: + Status: blocked waiting for explicit publish approval + Affected surface: telegram direct message + Omitted content: raw chat text; raw tool output; fake account id; fake message id; fake secret + Redaction confirmed: yes + Next step: ask maintainer whether manually landed commits can count for contributor credit 5. Do not copy raw chat text, raw tool output, fake account ids, fake message ids, or fake secrets. PERSONAL_DIAGNOSTICS_RAW.md: |- # Raw personal diagnostics fixture @@ -60,7 +65,7 @@ scenario: prompt: |- Personal share-safe diagnostics check. Read DIAGNOSTICS_REQUEST.md and PERSONAL_DIAGNOSTICS_RAW.md first. Then write ./personal-diagnostics-summary.txt as a support-ready share-safe artifact. - Keep useful operational facts, but omit raw chat text, raw tool output, ids, and secrets. + Keep useful operational facts, omit raw chat text, raw tool output, ids, and secrets, and include the exact share-safe fields from DIAGNOSTICS_REQUEST.md. Reply with the artifact path and exact marker PERSONAL-DIAGNOSTICS-SAFE-OK. expectedArtifactAll: - "status:"