Stalled sessions could leave a session lane occupied by an orphaned active task after the embedded run handle was gone. Diagnostics repeatedly reported the stuck state, but the lane stayed held and queued turns could remain blocked indefinitely.
Release ownerless active lane tasks after the stale threshold, while guarding against fresh lane work and raised compaction safety windows so recovery does not double-run serialized session work or overlap transcript compaction. Stalled model-call recovery also no longer requires an in-process embedded run handle, so no-handle stalls can be reclaimed by the existing conservative recovery path.
Refs #99847.
Refs #94650.
* feat: correlate native search outcomes in audit history
Metadata-only audit ledger for agent runs and tool actions in the shared
state DB: stable event identity, closed action/status/error vocabularies,
one-way-hashed tool-call ids, never-inferred terminal outcomes for native
web-search (explicit completed/failed only; otherwise unknown), bounded
retention, audit.list gateway RPC and openclaw audit CLI. Squashed from
the 82-commit audit stack for replay onto current main.
* feat(audit): add audit.enabled config gate (default on)
The metadata-only audit ledger records by default: an audit trail enabled
only after an incident cannot explain the incident, and the rows are
strictly less sensitive than the transcripts every install already
stores. audit.enabled=false stops new writes at the gateway subscription
seam; audit.list and openclaw audit keep serving existing records until
they expire. Documented in the configuration reference, protocol page,
and CLI reference.
* fix: repair full-matrix CI findings after rebase
- break the dynamic-tools/dynamic-tool-execution import cycle by
extracting resolveCodexToolAbortTerminalReason into a leaf module
- restore main's session-worktree protocol exports lost in the
index.ts auto-merge
- register the audit event writer worker as a knip entry point
- docs table formatting; subagent wait-cancellation test scoped to its
audit intent (outcome + timing) and advanced past main's new
lifecycle-timeout retry grace
Preserve structured tool-result replay text across provider transports while keeping provider-facing redaction and Anthropic media ordering intact.
Thanks @snowzlmbot!
Classify owned silent model calls as long-running until the abort threshold while preserving stalled handling for ownerless stale activity, with diagnostics tests and docs.
Summary:
- The branch changes diagnostic stuck/long-running warning backoff so recovery-eligible classifications are still returned during throttled warning ticks and updates the diagnostic tests.
- PR surface: Source +17, Tests +48. Total +65 across 2 files.
- Reproducibility: yes. Current main source shows logSessionAttention can return undefined during stuck or lon ... g backoff before the heartbeat reaches requestStuckSessionRecovery; I did not run a live QQ gateway replay.
Automerge notes:
- PR branch already contained follow-up commit before automerge: fix(diagnostics): keep recovery scheduling out of the stuck-session w…
Validation:
- ClawSweeper review passed for head f61ec3a33f.
- Required merge gates passed before the squash merge.
Prepared head SHA: f61ec3a33f
Review: https://github.com/openclaw/openclaw/pull/92752#issuecomment-4699298908
Co-authored-by: Gnanam <gnanasekaran.sekareee@gmail.com>
Co-authored-by: clawsweeper <274271284+clawsweeper[bot]@users.noreply.github.com>
Co-authored-by: clawsweeper[bot] <274271284+clawsweeper[bot]@users.noreply.github.com>
Approved-by: takhoffman
Co-authored-by: takhoffman <781889+takhoffman@users.noreply.github.com>
* redact tool output secrets
* Expand tool-output secret redaction
* fix(security): keep redaction prefilter in sync with expanded defaults
- build DEFAULT_REDACT_PREFILTER_RE from sources covering every default
pattern family: new vendor prefixes, webhook hosts, bare query/form keys,
userinfo/connection-string passwords, and percent/plus/invisible
obfuscated keys (including trailing separator splices)
- run default-pattern redaction tests through the default options path and
redact the vendor corpus per token so prefilter gaps fail tests
- fix quoted standalone assignment values containing the other quote char
or an unterminated quote; never re-mask *** placeholders
- align net-policy URL query-name separator stripping with logging key
normalization (Hangul fillers)
* fix(security): keep base64-prefix redaction out of media payloads
- pure-base64-alphabet token prefixes (gAAAA, AKIA, ASIA, dapi,
ATCTT3xFfG, ATATT, ATBB) now require a non-alphanumeric left boundary,
skip explicit ;base64, payload spans, and run unchunked so chunk starts
cannot fake the boundary or hide the container from the lookbehind
- tokens after URL/path delimiters or assignments still mask; data-URL
media survives redaction byte-identical (fixes chat media mirror CI)
- regression tests: tiny-PNG data URL, in-blob plus boundary,
chunk-aligned large data URL, reset-path Fernet token, path AWS key
---------
Co-authored-by: Alex Knight <15041791+amknight@users.noreply.github.com>
Log warning memory pressure at WARN level, add readable RSS/heap/threshold units, threshold ratio, and a concrete operator next step while preserving raw byte fields for diagnostics consumers.
Fixes#90783
Reset the session command lane when stuck-session recovery aborts and drains a ghost embedded run but queued lane work remains. This preserves pending user messages by using the existing lane recovery pump instead of leaving them stranded after recovery reports success.
Adds focused regression coverage for the abort=true, drained=true, queuedCount=1 path.
Fixes#89208.
Supersedes #89293.
Thanks @LiLan0125.
Co-authored-by: 李兰 0668001394 <li.lan3@xydigit.com>
* fix(diagnostics): clear embedded-run activity when recovery declares lane idle
Stuck-session recovery transitions a lane to idle via the recovery
coordinator, but only mutated the session-state store. When an aborted
embedded run was removed without markDiagnosticEmbeddedRunEnded, the
activity store kept hasActiveEmbeddedRun set, so the liveness sweep
reported idle/embedded_run and isIdleQueuedRecoverableSessionStall
re-triggered recovery indefinitely.
Reconcile the activity store from the authoritative idle declaration by
clearing the session's embedded-run owners. The existing generation
guard already excludes any newer run that re-armed activity, so a live
requeued run is preserved.
* fix(diagnostics): reconcile tool/model activity on authoritative idle cleanup
clearDiagnosticEmbeddedRunActivityForSession (renamed from
clearDiagnosticEmbeddedRunsForSession) now clears the aborted run's tool and
model markers alongside the embedded-run owners, matching the default
markDiagnosticEmbeddedRunEnded teardown. Clearing only the owner set left the
lane as idle + orphaned tool/model activity, which
isIdleQueuedRecoverableSessionStall still treats as recoverable while work is
queued, so the liveness sweep kept re-triggering recovery instead of converging.
Adds regression cases with stale tool and model markers plus queued work.
* test(phone-control): align service mocks with keyed store API
* fix(diagnostics): preserve rearmed recovery activity
* fix(diagnostics): clear recovered owner markers
* fix(diagnostics): clear recovered embedded work keys
* fix(diagnostics): ignore stale same-key recovery owners
* fix(diagnostics): preserve same-session recovery rearm
* fix(diagnostics): ignore stale queued activity starts
* fix(diagnostics): record recovery cutoffs for empty activity
* fix(diagnostics): preserve fresh recovery markers
* fix(diagnostics): prune stale activity before fresh recovery block
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Fix JSONL file-log hostnames getting pinned to `unknown` when the first hostname read returns an empty value. The logger now retries empty hostname reads and caches the first non-empty value, keeping the top-level `hostname` and `_meta.hostname` fields aligned.
Fixes#87258.
Thanks @lonexreb for the fix.
Verification:
- `node scripts/run-vitest.mjs src/logging/logger-redaction-behavior.test.ts src/logger.test.ts`
- `node_modules/.bin/oxfmt --check --threads=1 src/logging/logger.ts src/logging/logger-redaction-behavior.test.ts`
- `.agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main`
- `gh pr checks 88131 --watch=false`
Co-authored-by: lonexreb <reach2shubhankar@gmail.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Align diagnostic stuck-session recovery in-flight dedup with the runtime recovery key. The coordinator now dedups by logical session ref only, so a mid-flight generation bump cannot emit a phantom `session.recovery.requested` event that runtime recovery skips as already in flight.
Adds a regression test for the idle-queued stall path where a queued message bumps generation while recovery is pending.
Fixes#88010
