mirror of
https://github.com/openclaw/openclaw.git
synced 2026-08-03 06:51:38 +00:00
37cf0c6064
* improve: reduce GPT-5.6 coding harness overhead * refactor: extract runtime parity timing helpers * fix(qa): verify native apply-patch runtime parity * fix(qa): use mock patch execution evidence * fix(qa): preserve workspace templates in full syncs * fix(qa): make rejected patch fixtures deterministic * fix(qa): prove apply-patch workspace containment * fix(qa): expose runtime prompt-cache usage * fix(codex): confine forced private QA patch turns * fix(qa): verify native patch execution and sandbox precedence * fix(qa): execute native patches and prove disk mutation * fix(codex): avoid duplicate native patch registration * perf(agents): keep code mode catalog guidance compact * fix(qa): make native Codex patch evidence deterministic * fix(qa): recognize native sandbox denial errors * fix(qa): advertise native Codex mock model capabilities * fix(qa): seed native Codex mock model catalog at startup * fix(qa): recognize native Codex patch denial results * fix(ci): repair Codex harness parity checks * fix(qa): distinguish unknown prompt-cache measurements * fix(qa): accept native Codex project patch denials * fix(ci): isolate runtime parity report contracts * fix(codex): eliminate native runtime transport overhead * fix(qa): consume runtime parity report contract * fix(qa): compare matched runtime timing captures * fix(qa): type native response delta context * fix(qa): satisfy native codex websocket lint
59 lines
2.4 KiB
YAML
59 lines
2.4 KiB
YAML
title: "Runtime tool fixture — apply-patch"
|
|
|
|
scenario:
|
|
id: runtime-tool-apply-patch
|
|
surface: runtime-tools
|
|
runtimePairLane: core
|
|
coverage:
|
|
secondary:
|
|
- agent-runtime.tool-apply-patch
|
|
objective: Verify OpenClaw and Codex execute valid workspace patches and reject workspace-escaping patches.
|
|
successCriteria:
|
|
- OpenClaw executes a valid workspace patch and rejects a matching-context workspace-escaping patch without modifying the outside sentinel.
|
|
- Private mock QA verifies linked apply_patch calls and results without treating provider plans as execution evidence.
|
|
- Live Codex verifies projected native file-change calls and results without requiring a duplicate dynamic apply_patch tool.
|
|
docsRefs:
|
|
- qa/scenarios/index.yaml
|
|
codeRefs:
|
|
- src/agents/apply-patch.ts
|
|
- extensions/qa-lab/src/runtime-tool-fixture.ts
|
|
execution:
|
|
kind: flow
|
|
summary: Exercise the apply_patch runtime tool family.
|
|
config:
|
|
toolName: apply_patch
|
|
toolCoverage:
|
|
family: apply-patch
|
|
actualTool: apply_patch
|
|
bucket: codex-native-workspace
|
|
expectedLayer: codex-native-workspace
|
|
required: true
|
|
codexDefaultImpact: P4
|
|
qaImpact: P2
|
|
action: require linked successful and workspace-denied patch execution
|
|
reason: Codex owns workspace patching natively while private QA may expose an OpenClaw dynamic apply_patch tool.
|
|
happyPrompt: >-
|
|
tool search qa check target=apply_patch. Call apply_patch exactly once
|
|
with a complete patch that adds runtime-tool-fixture-patch.txt containing
|
|
runtime patch, then summarize the actual tool result.
|
|
failurePrompt: >-
|
|
tool search qa failure target=apply_patch. Call apply_patch exactly once
|
|
with a complete patch that attempts to update
|
|
../runtime-tool-fixture-denied.txt by replacing its existing
|
|
runtime-tool-fixture-denied-original line with
|
|
runtime patch outside the workspace, then summarize the actual
|
|
workspace-escape rejection.
|
|
promptSnippet: "target=apply_patch"
|
|
failurePromptSnippet: "failure target=apply_patch"
|
|
|
|
flow:
|
|
steps:
|
|
- name: exercises apply_patch happy and failure paths
|
|
actions:
|
|
- call: runRuntimeToolFixture
|
|
saveAs: result
|
|
args:
|
|
- ref: env
|
|
- ref: config
|
|
detailsExpr: result
|