Commit Graph

7143 Commits

Author SHA1 Message Date
Peter Steinberger
c5d6dcfc84 fix(scripts): run changed checks locally when Blacksmith never ran them (#114225)
* fix(scripts): run changed checks locally when Blacksmith never ran them

AGENTS.md already says trusted-source work falls back to local execution when
the remote backend is unavailable, but the tooling did not implement it: a lease,
broker, DNS, or network failure surfaced as a plain exit 1, so the lanes were
reported red without ever having been evaluated. That is worse than slow — a run
that never happened looked the same as a run that failed.

Tee the wrapper output and use its run summary as the discriminator. The summary
only appears once the command reached the box, so a failure carrying
`command-exit` is a real verdict and propagates unchanged; anything else never
produced one and re-runs locally, with a loud note so the proof summary records
which machine produced it.

Deliberately a positive test for `command-exit` rather than a blocklist of
infrastructure errors. Guessing wrong toward "infrastructure" only re-runs the
checks locally; guessing wrong toward "real failure" would block on an outage.
It must never widen to "fall back on any non-zero exit" — prompt snapshots are
Linux-only truth and would pass locally on macOS, turning a red gate green.

The sparse-checkout path keeps no fallback: it exists precisely because the
checkout cannot resolve the diff refs, so there is nothing local to run.

* fix(scripts): require positive pre-dispatch evidence before falling back

A missing run summary does not prove the remote never started: a wrapper that
crashes or loses its output transport after dispatch looks identical. Reading
that absence as "never ran" would rerun locally and could turn an unknown or
failing Linux-only lane green, which is the exact masking this guard exists to
prevent.

Require a positive pre-dispatch signature instead, and keep the command-exit
veto. Also reapply backpressure on the tee: inherited stdio got it from the OS,
piping does not, so a verbose delegated run could buffer its whole output here.
2026-07-26 22:02:36 -04:00
Peter Steinberger
4f34c60d45 build(deps): drop dead glob/markdown-it ownership-manifest entries (#114218)
* build(deps): drop stale root partial-json and dead ownership-manifest entries

partial-json is owned by packages/ai (which declares its own copy); glob and
markdown-it no longer exist as root dependencies. Audit follow-up to #114006.

* build(deps): keep root partial-json (openclaw/ai bundling contract); drop only dead manifest entries
2026-07-26 21:33:27 -04:00
Jesse Merhi
d7627d6f4c refactor(prompt): use plain inbound context labels and drop system-tag sanitizer (#112000)
* refactor(prompt): plain inbound context labels with a provenance marker

Replaces trust-worded inbound context labels ("(untrusted metadata)",
"(untrusted, for context)") with plain labels plus a fixed provenance
marker suffix appended to every OpenClaw-injected context header.

Detection keys on the marker, not label text, so strippers stay correct
across UI, TUI, replay, /trace segmentation, memory recall, and the Swift
chat preprocessor. Drops sanitizeInboundSystemTags in favor of the marker
boundary plus trusted system-prompt narration.

Renames the untrusted-named plugin SDK context identifiers to
channel-provenance names, keeping deprecated aliases registered for
removal after 2026-09-08.

Adds `openclaw doctor --fix` migrations that rewrite legacy inbound
labels in stored SQLite transcripts and purge legacy envelope-
contaminated LanceDB recall rows.

* fix(ci): resolve gate failures for plain inbound context labels

- doctor sqlite readers: open read-only connections via openNodeSqliteDatabase
  so the Kysely connection-boundary guardrail holds; unexport the now-internal
  transcript snapshot type (Knip unused-export gate).
- compat registry: split the record table into registry-records.ts and
  plugin-sdk-subpath-records.ts. The new compat record pushed registry.ts past
  the 700-line oxlint cap; suppressions are disallowed, so follow the existing
  sibling record-module pattern. Public exports and PluginCompatCode literals
  unchanged.
- acp-runtime test: assert current finalization behavior (newline normalization
  only). The bracket de-fang and System: rewrite it expected were removed with
  sanitizeInboundSystemTags; forged system lines are neutralized at the
  system-event queue, the single chokepoint feeding the System:-per-line render.
- regenerate docs_map and the plugin SDK API baseline manifest.

* fix(prompt): harden inbound context label migration and drop in-band sanitizer

Review follow-ups on the plain-label + provenance-marker change:

- Remove src/security/system-tags.ts. Rewriting inbound text to neutralize
  look-alike `System:`/`[System]` markers corrupted legitimate user text and is
  not a real injection boundary; role separation plus external-content wrapping
  is. Explicit product decision, recorded at the system-event queue.
- Narrow the LanceDB legacy-row purge so it cannot delete benign memories. It
  now requires a complete known legacy sentinel line, a legacy label followed by
  a fenced JSON body, or the complete legacy external-content header. The prior
  predicates matched ordinary prose such as `Notes (untrusted metadata):`, and
  deletion is irreversible.
- Make explicit-empty canonical ChannelStructuredContext win over the deprecated
  alias via a present/absent result instead of collapsing `[]` to undefined.
- Keep `\r?` in the active-memory doctor rule. It is the only rule spanning the
  header's line break, migrated assistant rows skip newline normalization, and
  without it the marked-header replace wins and the body strips to empty. Added
  a CRLF regression test.
- Fix stale comments that described removed behavior, and cover the Swift
  prose-block strip path.

Claude-Session: https://claude.ai/code/session_01WNzsPddQmxy9Y7jKD4wAxH
2026-07-27 11:27:53 +10:00
Peter Steinberger
f9721b9809 fix(e2e): use the newest Windows smoke snapshot (#114157)
* fix(e2e): select newest Windows smoke snapshot

* test(e2e): preserve undated snapshot ordering
2026-07-26 15:34:04 -04:00
Peter Steinberger
96aa6be670 fix(e2e): harden Windows prerequisite downloads (#114153) 2026-07-26 15:01:59 -04:00
Peter Steinberger
c16c0dbef4 fix(release): allow macOS snapshot selection in candidate smoke (#114144) 2026-07-26 14:08:31 -04:00
Peter Steinberger
2880c82fcf fix(release): handle conflicting npm lock forks (#114122) 2026-07-26 11:08:50 -04:00
Peter Steinberger
2b7b0f2440 fix(release): preserve npm lock version forks (#114115) 2026-07-26 10:12:35 -04:00
Dallin Romney
354c6db731 fix(release): retain workflow ref after failed validation (#114104) 2026-07-26 06:13:26 -07:00
Patrick Erichsen
aa0d3192d2 feat(channels): add Buzz channel plugin 2026-07-26 14:09:23 +01:00
Peter Steinberger
19a98c873c refactor(auth): finish SQLite-only auth profile cutover (#114033)
* refactor(auth): finish SQLite-only profile cutover

* test(auth): isolate SQLite shadow fixtures

* test(auth): satisfy OAuth refresh lint

* test(auth): infer legacy sidecar fixtures

* fix(auth): fail closed on unreadable stores

* fix(auth): recheck migration readiness on lookup

* fix(auth): preserve compatibility owner checks

* fix(auth): align SQLite cutover proof

* style(qa): format SQLite auth helpers

* fix(auth): recover interrupted SQLite migrations

* fix(auth): bind materialized SQLite refs

* style(auth): clarify receipt hash value

* fix(auth): preserve state-only OAuth routing

* fix(auth): snapshot receipted migration sources

* fix(auth): fail closed across recovery races

* fix(auth): close legacy recovery gaps

* fix(auth): serialize SQLite refresh recovery
2026-07-26 07:27:54 -04:00
Peter Steinberger
9eae43bd37 refactor(infra): move exec approvals into the shared SQLite state DB (#114063)
* refactor(infra): move exec approvals into the shared SQLite state DB

Delete the file-runtime exec-approvals store (exec-approvals.json + .lock
sidecar machinery) on both runtimes and make the reserved
exec_approvals_config singleton row canonical. Doctor owns the one-time
import with claim/verify/receipt discipline; runtime fails closed with a
doctor instruction while un-migrated legacy state exists. The wire CAS
contract, socket semantics, and gateway auth-token derivations are
unchanged. Kills the #113929 lock-contention bug class structurally and
nets around -2.9k lines.

* fix(infra): green CI gates and retire file-era exec approvals tests

Break the migration-type import cycle with a leaf contract, regenerate the
plugin-SDK API and native i18n baselines for the intentional surface change,
drop unused exports, and replace the macOS file-era approvals test suite with
SQLite-backed behavior coverage per the obsolete-internals test policy.

* chore: green max-lines ratchet, native i18n baseline, and unused-export scan
2026-07-26 06:39:23 -04:00
Peter Steinberger
5347285d6b improve(models): source pricing from hosted catalog (#114060)
* feat(model-catalog): serve hosted fallback pricing

* refactor(config): retire client pricing bootstrap settings

* refactor(gateway): delete client pricing refresh runtime

* docs(models): explain hosted catalog pricing

* fix(model-catalog): preserve pricing privacy and aliases

* fix(model-catalog): fingerprint pricing eligibility

* fix(model-catalog): harden pricing endpoint checks

* fix(model-catalog): materialize source-safe pricing aliases

* fix(model-catalog): keep unknown pricing fallbacks safe

* fix(model-catalog): reject zero-only hosted prices

* fix(model-catalog): fail closed without pricing policy metadata

* refactor(utils): extract usage pricing normalization

* fix(model-catalog): rebuild policy-owned pricing namespaces

* test(model-catalog): type publisher cost fixtures

* chore(config): regenerate schema baselines

* fix(utils): keep raw pricing tiers private
2026-07-26 03:48:25 -04:00
Peter Steinberger
f6131a4fbf build(deps): remove npm shrinkwrap; mirror pnpm lock into transient package locks (#114006)
* build(deps): remove npm shrinkwrap; mirror pnpm lock into transient package locks

npm 12 removed shrinkwrap (command + tarball/root loading). Delete all 82
committed npm-shrinkwrap.json files and stop publishing lockfiles; keep
pnpm-lock.yaml as the single reviewed dependency boundary. The generator
becomes scripts/generate-npm-package-lock.mjs and feeds plugin bundling via
a transient package-lock.json + npm ci (works on npm 11 and 12). Tarball
validation treats the published 2026.7.2 beta train as a shrinkwrap
transition; self-update npm detection now uses install topology instead of
the shipped shrinkwrap.

* fix(deps): repair lint, deadcode, and test-type lanes for the npm 12 migration

- sort integrity comparisons with an explicit comparator (oxlint)
- keep resolveBunGlobalNodeModules module-local (knip unused-export gate)
- model npm pack --json as npm<=11 array / npm 12 name-keyed object
- default calver destructuring in the tarball test fixture
2026-07-26 01:29:55 -04:00
Peter Steinberger
fc3476b116 fix(policy): drop retired config coverage entry (#114014) 2026-07-25 21:29:50 -07:00
Peter Steinberger
7b6a29ca11 fix(release): support npm 12 in ClawHub materializer (#114008) 2026-07-25 21:19:28 -07:00
Peter Steinberger
b2f6ecc2a9 chore(compat): date the annotated deprecation families and expose removal-pending debt (#114002)
* chore(compat): date the annotated deprecation families and expose removal-pending debt

Adds 10 dated compat-registry records for shipped deprecated surfaces that
previously had @deprecated annotations only (removeAfter 2026-10-01 per the
approved 60-day window), extends the beta5 session-store record with the
package-root aliases, surfaces removal-pending records with due dates,
blockers, and readers in the plugin boundary report, and converts undated
cleanup obligations (CI timing job, zizmor excessive-permissions, staged
Codex CI auth, memory-host legacy state dir, agent-DB schema-ladder floors)
into dated markers visible to the weekly dated-TODO sweep. Metadata,
diagnostics, and docs only - no runtime behavior change.

* fix(compat): keep extension src test paths out of core string literals

The core import guardrail forbids extensions/*/src/ path shapes in core
production string literals; cite the package-root google-meet test and the
registry test for official-plugin-export-aliases instead.
2026-07-25 21:01:20 -07:00
Peter Steinberger
08138cabe0 refactor(policy): drop unreachable sensitive-logging redaction check (#113993)
The policy/data-handling-redaction-disabled check could never emit a finding:
scanPolicyDataHandling records the sensitiveLoggingRedaction evidence with a
hardcoded value true, and the finding builder only fired on value !== true.
Redaction is unconditional in src/logging/redact.ts, which hardcodes tools mode
and reads only redactPatterns, so no config can turn it off.

Delete the check, its finding builder, check id, and the validateOnly fix class
that existed solely for it. Keep the public
dataHandling.sensitiveLogging.requireRedaction policy key: it is a policy.jsonc
contract, it still drives openclaw policy compare baseline strictness, and its
shape stays validated.

Make the key's satisfied status explicit instead of silent: the policy rule
declares satisfiedByInvariant pointing at the evidence source policy state
records (oc://openclaw.invariant/logging/redaction), which openclaw policy check
emits in dataHandling evidence and the attestation. A metadata test asserts every
rule names either its checks or its invariant, never both and never neither, and
that policy state actually emits the declared source.

Also drop the stale logging.redactSensitive entry from the policy config coverage
manifest; that config key is retired.
2026-07-25 20:00:25 -07:00
Peter Steinberger
3807591ff4 build(lint): adopt stylelint for Control UI css templates and stylesheets (#113971) 2026-07-25 19:13:46 -07:00
Peter Steinberger
f90cef67c8 feat(ui): drag custom sidebar groups between built-in session zones (#113948)
* feat(ui): drag custom sidebar groups between built-in session zones

* test(ui): remove stale split ratio fixture

* perf(ui): raise startup budget baseline for session-section ordering pref
2026-07-25 19:11:25 -07:00
Peter Steinberger
4859d524ed refactor(tasks): classify harness-owned subagent rows without plugin ids (#113964)
* refactor(tasks): classify harness-owned subagent rows without plugin ids

* refactor(plugin-sdk): require taskKind for harness-owned subagent runtimes

* docs(tasks): record harness-ownership invariant at the predicate

* test(plugin-sdk): declare harness task kind in scope fixtures
2026-07-25 18:41:13 -07:00
Peter Steinberger
4596fd2dd2 feat(ci): weekly Codex sweep for date-carrying to-dos (#113914)
* feat(ci): weekly Codex sweep for date-carrying to-dos

Mondays 06:23 UTC (plus default-branch-only manual dispatch with
dry_run): a deterministic prefilter collects candidate lines where
to-do markers co-occur with date tokens, plus every deprecated compat
record from the plugin registry; a pinned openai/codex-action step
judges each candidate in context (genuine dated commitment vs
historical date or fixture) and writes an OVERDUE / DUE-30-days /
FUTURE report; a separate privileged job validates the report (tracked
file:line locations, date shape, inert text) and upserts one
marker-tagged tracking issue via the Barnacle app token, commenting
only when items newly become due.

The app token is minted on a fresh runner and checkout — never beside
Codex or its child processes; only the validated report artifact
crosses jobs. No permission-* subsets on token minting (installations
reject explicit subsets; see pr-ci-sweeper).

* chore(ci): localize upsert helpers with no external consumers
2026-07-25 18:18:06 -07:00
Peter Steinberger
ce5ab8adb8 refactor(gateway): remove dead sessions.observer.ask rpc (#113791)
* refactor(gateway): remove dead sessions.observer.ask rpc

* docs: record btw and companion contract split

* fix(gateway): unexport observer model sanitizer after ask removal
2026-07-25 14:44:02 -07:00
Peter Steinberger
d5a3740707 fix(package): prevent installs from deleting required runtime modules (#113821)
* fix(package): preserve imported runtime chunks after install

* fix(package): isolate public build from private QA

* test(package): prove installed postinstall dependency graph
2026-07-25 14:05:41 -07:00
Peter Steinberger
7490560c67 fix(package): prevent installs from deleting the Control UI (#113856) 2026-07-25 14:03:32 -07:00
Peter Steinberger
a18c314377 fix(install): preserve shared Node compile caches (#113837) 2026-07-25 13:17:58 -07:00
Peter Steinberger
6e2cde0fba fix(e2e): assert canonical agent entries after delete (#113827) 2026-07-25 12:55:46 -07:00
Peter Steinberger
bde7033ac4 feat(models): dynamic model catalogs via hosted refresh overlay (#113660)
* feat(models): add remote catalog core

* feat(config): add model catalog refresh settings

* feat(models): wire remote catalog refresh

* ci(models): publish hosted catalog

* docs(models): document catalog refresh

* test(models): allow catalog count growth

* refactor(models): preserve safe catalog merge

* ci(models): publish catalog through GitHub

* docs(models): link public catalog history

* fix(model-catalog): enforce Kysely store access

* fix(model-catalog): trim unused catalog exports

* docs(models): refresh generated docs map
2026-07-25 12:15:05 -07:00
Peter Steinberger
be95b78ae6 fix(release): handle large preparation diffs (#113760) 2026-07-25 10:15:47 -07:00
Peter Steinberger
f1538c6d6c fix(release): validate complete package sets in candidate smoke (#113600)
* fix(release): pass only root dependencies to candidate smoke

* fix(release): stage plugin candidates for Parallels

* fix(release): bind plugin preflight package identity

* fix(test): stream upstream npm tarballs

* fix(test): retry post-update gateway startup

* fix(test): install fresh candidates through registry

* fix(test): route fresh candidates through registry

* test(release): update Parallels source contracts

* fix(test): split host and guest registry URLs
2026-07-25 07:59:06 -07:00
Vincent Koc
a4d367e38c fix(crabbox): keep untrusted bootstrap pnpm pin current (#113728) 2026-07-25 22:46:10 +08:00
Peter Steinberger
367ff3cea1 feat(sqlite): migrate persisted media to canonical facts and stop legacy writes (#113695)
* feat(sqlite): migrate persisted media to canonical facts and stop legacy writes

PR 3 of the media legacy retirement program — the operator-approved
canonical cutover.

- openclaw doctor --fix owns one idempotent migration: active
  transcript_events rows canonicalize to __openclaw.media (facts-first
  gap-fill, bare legacy kinds to fact.kind, transcribed indexes and
  workspace dirs onto per-fact fields) via the transcript replacement
  owner; cold plain/.zst archives rewrite through temp-file + codec
  readback + event/id verification + atomic replace; trajectory
  runtime snapshots canonicalize IN PLACE (telemetry preserved, never
  row deletion). Invalid JSON, genuinely ambiguous legacy-only sparse
  alignment, or a changed source aborts that owner without partial work;
  reruns are no-ops.
- Per-agent schema advances to v16 as a pure downgrade guard (main
  independently took v15 for board/session-sharing tables; no
  columns/tables/indexes change here, shared-state DB untouched). v15
  databases repair canonical indexes before the version assertion so
  repairable installations never strand.
- The user-turn builder stops writing top-level legacy Media* fields;
  shouldPersistStructuredMediaEntries and the aligned projection mode
  are deleted; the generic transcript append boundary canonicalizes
  every message role so SDK/mirror writers cannot mint new legacy rows.
- Internal persisted-reader legacy fallbacks are removed; the public
  SDK projection stays until retirement PR 4's window expires.

Hardening from three adversarial review rounds, each with fixture
regressions: in-place trajectory canonicalization instead of row
deletion; repair-before-assert on the v15 path; all-roles append
canonicalization; duplicate-preserving exact row rewrites; v0-v15
reopen guards; complete canonical facts bypass compact legacy
projections (PR-1 dual-write rows migrate cleanly); SQLite LIKE
underscore escaped so populated foreign databases are never claimed.

* fix(sqlite): align schema-support metadata and gates with the v16 cutover

package.json agent schema support advances to 16; verifier and board
parity fixtures run doctor migration before steady-state access (the
production guards were correct); two test-only exports removed; the
migration module registered in the doctor raw-SQLite allowlist.
2026-07-25 07:15:44 -07:00
sunlit-deng
7d7bfb2047 fix(scripts): bound release metadata git lookups (#112556)
* fix(scripts): bound release metadata git lookups

* fix(scripts): keep fractional git timeouts bounded

* test(scripts): gate fake git shim to Unix

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-25 07:12:27 -07:00
Vincent Koc
687923189e fix(sqlite): make reliability proofs cross-platform safe (#113652)
* test(sqlite): canonicalize reliability crash barriers

* fix(sqlite): normalize reliability database paths

* test(sqlite): allow Windows reliability proof to finish
2026-07-25 21:50:27 +08:00
Peter Steinberger
1dcb35b204 refactor(agents): split main session recovery (#113687) 2026-07-25 13:39:15 +00:00
Peter Steinberger
a2acc8a755 fix(ui): keep subagent details in task panel (#113671) 2026-07-25 06:13:06 -07:00
shawnduggan
d30edbca53 docs: map shawnduggan clawtributor email (#112251) 2026-07-25 06:12:53 -07:00
ooiuuii
ceeb861418 fix(install): report source UI build failures correctly (#112053) 2026-07-25 05:42:23 -07:00
Peter Steinberger
b06f40a821 refactor(plugin-sdk): share ingress lifecycle fan-in (#113648)
* refactor(channels): share ingress lifecycle fan-in

* chore(plugin-sdk): refresh ingress API baseline
2026-07-25 05:34:35 -07:00
Peter Steinberger
3d1369ff4f test(telegram): consolidate bot test harness (#113571)
* test(telegram): consolidate bot test harness

* test(plugin-sdk): expose isolated test state

Promote the isolated OpenClaw test-state lifecycle through a narrow published Plugin SDK subpath so extension tests no longer import private core helpers. This intentional SDK surface addition is maintainer-approved.

* test(telegram): use public test-state seam

* test(plugin-sdk): keep test state local-only

Match the existing channel test-helper boundary: bundled extension tests can import the focused SDK source entrypoint, while ordinary builds and the published npm package exclude it. The earlier public classification existed only on this unmerged PR branch and was never a shipped contract.

* test(plugins): map test-state in package boundaries
2026-07-25 05:28:26 -07:00
Peter Steinberger
eddd5224ad fix(e2e): expose redacted packaged plugin failures (#113635)
Co-authored-by: Peter Steinberger <steipete@golden-gate.local>
2026-07-25 04:48:08 -07:00
Peter Steinberger
421e287c45 fix(ui): show Swarm progress in Chat (#113619)
* fix(ui): show Swarm progress in chat

* style(ui): format rebased Swarm integration

* fix(ui): bound inline Swarm progress height
2026-07-25 04:36:50 -07:00
Peter Steinberger
825d6f7f1f fix(openai): align Codex discovery client version (#113615)
* fix(openai): align Codex discovery client version

* chore: leave release notes to release workflow
2026-07-25 04:33:12 -07:00
Vincent Koc
d4794c90a0 test(sqlite): cover interrupted canonical index repair (#113624) 2026-07-25 19:27:17 +08:00
Peter Steinberger
57e2f220de fix(ci): restore packaged OpenWebUI and update validation (#113614)
Co-authored-by: Peter Steinberger <steipete@golden-gate.local>
2026-07-25 04:14:41 -07:00
joshavant
f153858045 fix(onepassword): make SecretRef setup production-safe 2026-07-25 06:03:30 -05:00
Vincent Koc
2b19ae1f00 fix(snapshot): recover complete pending sqlite snapshots (#113607) 2026-07-25 18:50:50 +08:00
Peter Steinberger
55d66fbf98 chore(scripts): remove resolved investigation tools and orphans (#113532)
* chore(scripts): remove orphaned wrappers

* chore(scripts): remove resolved investigation tools

* refactor(code-mode): remove orphaned plugin namespaces

* test(code-mode): remove stale namespace import
2026-07-25 02:47:40 -07:00
Vitor Cepeda Lopes
2296e898e1 fix(release): stop Windows package descendants after timeout (#111956)
* fix(release): terminate Windows package command trees

* fix(release): type package runner test output

* fix(release): scope package runner normalization to Windows

* test(release): run package timeout proof on Windows CI

---------

Co-authored-by: TheAngryPit <16145902+TheAngryPit@users.noreply.github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-25 02:20:27 -07:00
Vincent Koc
3af03aee6c refactor(approvals): share native target resolvers (#113568) 2026-07-25 16:23:14 +08:00