Commit Graph

51 Commits

Author SHA1 Message Date
Peter Steinberger
694ca50e97 Revert "refactor: move runtime state to SQLite"
This reverts commit f91de52f0d.
2026-05-13 13:33:38 +01:00
Peter Steinberger
f91de52f0d refactor: move runtime state to SQLite
* refactor: remove stale file-backed shims

* fix: harden sqlite state ci boundaries

* refactor: store matrix idb snapshots in sqlite

* fix: satisfy rebased CI guardrails

* refactor: store current conversation bindings in sqlite table

* refactor: store tui last sessions in sqlite table

* refactor: reset sqlite schema history

* refactor: drop unshipped sqlite table migration

* refactor: remove plugin index file rollback

* refactor: drop unshipped sqlite sidecar migrations

* refactor: remove runtime commitments kv migration

* refactor: preserve kysely sync result types

* refactor: drop unshipped sqlite schema migration table

* test: keep session usage coverage sqlite-backed

* refactor: keep sqlite migration doctor-only

* refactor: isolate device legacy imports

* refactor: isolate push voicewake legacy imports

* refactor: isolate remaining runtime legacy imports

* refactor: tighten sqlite migration guardrails

* test: cover sqlite persisted enum parsing

* refactor: isolate legacy update and tui imports

* refactor: tighten sqlite state ownership

* refactor: move legacy imports behind doctor

* refactor: remove legacy session row lookup

* refactor: canonicalize memory transcript locators

* refactor: drop transcript path scope fallbacks

* refactor: drop runtime legacy session delivery pruning

* refactor: store tts prefs only in sqlite

* refactor: remove cron store path runtime

* refactor: use cron sqlite store keys

* refactor: rename telegram message cache scope

* refactor: read memory dreaming status from sqlite

* refactor: rename cron status store key

* refactor: stop remembering transcript file paths

* test: use sqlite locators in agent fixtures

* refactor: remove file-shaped commitments and cron store surfaces

* refactor: keep compaction transcript handles out of session rows

* refactor: derive transcript handles from session identity

* refactor: derive runtime transcript handles

* refactor: remove gateway session locator reads

* refactor: remove transcript locator from session rows

* refactor: store raw stream diagnostics in sqlite

* refactor: remove file-shaped transcript rotation

* refactor: hide legacy trajectory paths from runtime

* refactor: remove runtime transcript file bridges

* refactor: repair database-first rebase fallout

* refactor: align tests with database-first state

* refactor: remove transcript file handoffs

* refactor: sync post-compaction memory by transcript scope

* refactor: run codex app-server sessions by id

* refactor: bind codex runtime state by session id

* refactor: pass memory transcripts by sqlite scope

* refactor: remove transcript locator cleanup leftovers

* test: remove stale transcript file fixtures

* refactor: remove transcript locator test helper

* test: make cron sqlite keys explicit

* test: remove cron runtime store paths

* test: remove stale session file fixtures

* test: use sqlite cron keys in diagnostics

* refactor: remove runtime delivery queue backfill

* test: drop fake export session file mocks

* refactor: rename acp session read failure flag

* refactor: rename acp row session key

* refactor: remove session store test seams

* refactor: move legacy session parser tests to doctor

* refactor: reindex managed memory in place

* refactor: drop stale session store wording

* refactor: rename session row helpers

* refactor: rename sqlite session entry modules

* refactor: remove transcript locator leftovers

* refactor: trim file-era audit wording

* refactor: clean managed media through sqlite

* fix: prefer explicit agent for exports

* fix: use prepared agent for session resets

* fix: canonicalize legacy codex binding import

* test: rename state cleanup helper

* docs: align backup docs with sqlite state

* refactor: drop legacy Pi usage auth fallback

* refactor: move legacy auth profile imports to doctor

* refactor: keep Pi model discovery auth in memory

* refactor: remove MSTeams legacy learning key fallback

* refactor: store model catalog config in sqlite

* refactor: use sqlite model catalog at runtime

* refactor: remove model json compatibility aliases

* refactor: store auth profiles in sqlite

* refactor: seed copied auth profiles in sqlite

* refactor: make auth profile runtime sqlite-addressed

* refactor: migrate hermes secrets into sqlite auth store

* refactor: move plugin install config migration to doctor

* refactor: rename plugin index audit checks

* test: drop auth file assumptions

* test: remove legacy transcript file assertions

* refactor: drop legacy cli session aliases

* refactor: store skill uploads in sqlite

* refactor: keep subagent attachments in sqlite vfs

* refactor: drop subagent attachment cleanup state

* refactor: move legacy session aliases to doctor

* refactor: require node 24 for sqlite state runtime

* refactor: move provider caches into sqlite state

* fix: harden virtual agent filesystem

* refactor: enforce database-first runtime state

* refactor: rename compaction transcript rotation setting

* test: clean sqlite refactor test types

* refactor: consolidate sqlite runtime state

* refactor: model session conversations in sqlite

* refactor: stop deriving cron delivery from session keys

* refactor: stop classifying sessions from key shape

* refactor: hydrate announce targets from typed delivery

* refactor: route heartbeat delivery from typed sqlite context

* refactor: tighten typed sqlite session routing

* refactor: remove session origin routing shadow

* refactor: drop session origin shadow fixtures

* perf: query sqlite vfs paths by prefix

* refactor: use typed conversation metadata for sessions

* refactor: prefer typed session routing metadata

* refactor: require typed session routing metadata

* refactor: resolve group tool policy from typed sessions

* refactor: delete dead session thread info bridge

* Show Codex subscription reset times in channel errors (#80456)

* feat(plugin-sdk): consolidate session workflow APIs

* fix(agents): allow read-only agent mount reads

* [codex] refresh plugin regression fixtures

* fix(agents): restore compaction gateway logs

* test: tighten gateway startup assertions

* Redact persisted secret-shaped payloads [AI] (#79006)

* test: tighten device pair notify assertions

* test: tighten hermes secret assertions

* test: assert matrix client error shapes

* test: assert config compat warnings

* fix(heartbeat): remap cron-run exec events to session keys (#80214)

* fix(codex): route btw through native side threads

* fix(auth): accept friendly OpenAI order for Codex profiles

* fix(codex): rotate auth profiles inside harness

* fix: keep browser status page probe within timeout

* test: assert agents add outputs

* test: pin cron read status

* fix(agents): avoid Pi resource discovery stalls

Co-authored-by: dataCenter430 <titan032000@gmail.com>

* fix: retire timed-out codex app-server clients

* test: tighten qa lab runtime assertions

* test: check security fix outputs

* test: verify extension runtime messages

* feat(wake): expose typed sessionKey on wake protocol + system event CLI

* fix(gateway): await session_end during shutdown drain and track channel + compaction lifecycle paths (#57790)

* test: guard talk consult call helper

* fix(codex): scale context engine projection (#80761)

* fix(codex): scale context engine projection

* fix: document Codex context projection scaling

* fix: document Codex context projection scaling

* fix: document Codex context projection scaling

* fix: document Codex context projection scaling

* chore: align Codex projection changelog

* chore: realign Codex projection changelog

* fix: isolate Codex projection patch

---------

Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org>
Co-authored-by: Josh Lehman <josh@martian.engineering>

* refactor: move agent runtime state toward piless

* refactor: remove cron session reaper

* refactor: move session management to sqlite

* refactor: finish database-first state migration

* chore: refresh generated sqlite db types

* refactor: remove stale file-backed shims

* test: harden kysely type coverage

# Conflicts:
#	.agents/skills/kysely-database-access/SKILL.md
#	src/infra/kysely-sync.types.test.ts
#	src/proxy-capture/store.sqlite.test.ts
#	src/state/openclaw-agent-db.test.ts
#	src/state/openclaw-state-db.test.ts

* refactor: remove cron store path runtime

* refactor: keep compaction transcript handles out of session rows

* refactor: derive embedded transcripts from sqlite identity

* refactor: remove embedded transcript locator handoff

* refactor: remove runtime transcript file bridges

* refactor: remove transcript file handoffs

* refactor: remove MSTeams legacy learning key fallback

* refactor: store model catalog config in sqlite

* refactor: use sqlite model catalog at runtime

# Conflicts:
#	docs/cli/secrets.md
#	docs/gateway/authentication.md
#	docs/gateway/secrets.md

* fix: keep oauth sibling sync sqlite-local

# Conflicts:
#	src/commands/onboard-auth.test.ts

* refactor: remove task session store maintenance

# Conflicts:
#	src/commands/tasks.ts

* refactor: keep diagnostics in state sqlite

* refactor: enforce database-first runtime state

* refactor: consolidate sqlite runtime state

* Show Codex subscription reset times in channel errors (#80456)

* fix(codex): refresh subscription limit resets

* fix(codex): format reset times for channels

* Update CHANGELOG with latest changes and fixes

Updated CHANGELOG with recent fixes and improvements.

* fix(codex): keep command load failures on codex surface

* fix(codex): format account rate limits as rows

* fix(codex): summarize account limits as usage status

* fix(codex): simplify account limit status

* test: tighten subagent announce queue assertion

* test: tighten session delete lifecycle assertions

* test: tighten cron ops assertions

* fix: track cron execution milestones

* test: tighten hermes secret assertions

* test: assert matrix sync store payloads

* test: assert config compat warnings

* fix(codex): align btw side thread semantics

* fix(codex): honor codex fallback blocking

* fix(agents): avoid Pi resource discovery stalls

* test: tighten codex event assertions

* test: tighten cron assertions

* Fix Codex app-server OAuth harness auth

* refactor: move agent runtime state toward piless

* refactor: move device and push state to sqlite

* refactor: move runtime json state imports to doctor

* refactor: finish database-first state migration

* chore: refresh generated sqlite db types

* refactor: clarify cron sqlite store keys

* refactor: remove stale file-backed shims

* refactor: bind codex runtime state by session id

* test: expect sqlite trajectory branch export

* refactor: rename session row helpers

* fix: keep legacy device identity import in doctor

* refactor: enforce database-first runtime state

* refactor: consolidate sqlite runtime state

* build: align pi contract wrappers

* chore: repair database-first rebase

* refactor: remove session file test contracts

* test: update gateway session expectations

* refactor: stop routing from session compatibility shadows

* refactor: stop persisting session route shadows

* refactor: use typed delivery context in clients

* refactor: stop echoing session route shadows

* refactor: repair embedded runner rebase imports

# Conflicts:
#	src/agents/pi-embedded-runner/run/attempt.tool-call-argument-repair.ts

* refactor: align pi contract imports

* refactor: satisfy kysely sync helper guard

* refactor: remove file transcript bridge remnants

* refactor: remove session locator compatibility

* refactor: remove session file test contracts

* refactor: keep rebase database-first clean

* refactor: remove session file assumptions from e2e

* docs: clarify database-first goal state

* test: remove legacy store markers from sqlite runtime tests

* refactor: remove legacy store assumptions from runtime seams

* refactor: align sqlite runtime helper seams

* test: update memory recall sqlite audit mock

* refactor: align database-first runtime type seams

* test: clarify doctor cron legacy store names

* fix: preserve sqlite session route projections

* test: fix copilot token cache test syntax

* docs: update database-first proof status

* test: align database-first test fixtures

* docs: update database-first proof status

* refactor: clean extension database-first drift

* test: align agent session route proof

* test: clarify doctor legacy path fixtures

* chore: clean database-first changed checks

* chore: repair database-first rebase markers

* build: allow baileys git subdependency

* chore: repair exp-vfs rebase drift

* chore: finish exp-vfs rebase cleanup

* chore: satisfy rebase lint drift

* chore: fix qqbot rebase type seam

* chore: fix rebase drift leftovers

* fix: keep auth profile oauth secrets out of sqlite

* fix: repair rebase drift tests

* test: stabilize pairing request ordering

* test: use source manifests in plugin contract checks

* fix: restore gateway session metadata after rebase

* fix: repair database-first rebase drift

* fix: clean up database-first rebase fallout

* test: stabilize line quick reply receipt time

* fix: repair extension rebase drift

* test: keep transcript redaction tests sqlite-backed

* fix: carry injected transcript redaction through sqlite

* chore: clean database branch rebase residue

* fix: repair database branch CI drift

* fix: repair database branch CI guard drift

* fix: stabilize oauth tls preflight test

* test: align database branch fast guards

* test: repair build artifact boundary guards

* chore: clean changelog rebase markers

---------

Co-authored-by: pashpashpash <nik@vault77.ai>
Co-authored-by: Eva <eva@100yen.org>
Co-authored-by: stainlu <stainlu@newtype-ai.org>
Co-authored-by: Jason Zhou <jason.zhou.design@gmail.com>
Co-authored-by: Ruben Cuevas <hi@rubencu.com>
Co-authored-by: Pavan Kumar Gondhi <pavangondhi@gmail.com>
Co-authored-by: Shakker <shakkerdroid@gmail.com>
Co-authored-by: Kaspre <36520309+Kaspre@users.noreply.github.com>
Co-authored-by: dataCenter430 <titan032000@gmail.com>
Co-authored-by: Kaspre <kaspre@gmail.com>
Co-authored-by: pandadev66 <nova.full.stack@outlook.com>
Co-authored-by: Eva <admin@100yen.org>
Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org>
Co-authored-by: Josh Lehman <josh@martian.engineering>
Co-authored-by: jeffjhunter <support@aipersonamethod.com>
2026-05-13 13:15:12 +01:00
Peter Steinberger
b9185703bc chore: add model transport debugging 2026-05-11 10:11:03 +01:00
Vincent Koc
1fe15f2306 docs(logging): document bounded Talk log records 2026-05-06 03:02:02 -07:00
Vincent Koc
41736de923 docs: audit and fix 4 pages (pi version bump + 3 typography/H1) 2026-05-05 21:14:55 -07:00
stain lu
84920fad4e security(logging): redact payment credential fields (#75230)
Summary:
- The PR adds payment-credential redaction patterns and a key-aware structured field redaction helper, wires it into tool payload sanitization, and updates focused tests, logging docs, and the changelog.

ClawSweeper fixups:
- No separate fixup commits were needed after automerge opt-in.

Validation:
- ClawSweeper review passed for head 5f5f1fadbb.
- Required merge gates passed before the squash merge.

Prepared head SHA: 5f5f1fadbb
Review: https://github.com/openclaw/openclaw/pull/75230#issuecomment-4355538755

Co-authored-by: stainlu <stainlu@newtype-ai.org>
2026-05-01 05:45:28 +00:00
Peter Steinberger
e25b542100 fix(cli): fall back to file logs when local logs rpc closes 2026-04-29 08:10:15 +01:00
Peter Steinberger
ccc9dd5eef fix: keep session history redaction forced 2026-04-27 23:59:47 +01:00
Vincent Koc
94f5827c6e docs(logging): note Control UI tool payload redaction
Document the redaction surface added in f3e8c50df3: custom logging.redactPatterns now apply to Control UI tool start args, partial/final result payloads, derived exec output, and patch summaries on top of the built-in defaults.
2026-04-27 15:39:49 -07:00
Vincent Koc
fc0e6e4650 docs(logging): document structured file fields 2026-04-26 15:01:19 -07:00
Vincent Koc
2194a8c64c docs(logging): document request trace scopes 2026-04-26 14:13:15 -07:00
Vincent Koc
f0566e410a docs(diagnostics): document model call size timing 2026-04-26 13:43:22 -07:00
Vincent Koc
df542f75a9 fix(logging): expose trace fields in file logs 2026-04-26 12:52:04 -07:00
Vincent Koc
406ae72fd2 fix(logging): redact persisted transcript text 2026-04-26 12:12:44 -07:00
Vincent Koc
cd89adf0ac fix(logging): rotate file logs instead of suppressing 2026-04-25 22:55:33 -07:00
Vincent Koc
4cba24a4c3 fix(logging): redact console and file sinks 2026-04-25 21:50:00 -07:00
Vincent Koc
7741dbb759 docs: split OpenTelemetry export into its own page under gateway
Logging.md had grown to 487 lines with ~300 lines dedicated to
OpenTelemetry export — wire protocol, full metric/span catalog, env
vars, captureContent shape, sampling, the diagnostic event catalog,
and protocol notes — leaving the genuine logging overview buried
behind exporter reference material.

Move the OTEL surface to a dedicated page and slim logging.md to a
focused logs overview:

- Add docs/gateway/opentelemetry.md (OpenTelemetry export). Same
  content reorganized: how it fits together, quick start, signals,
  configuration reference + env vars table, privacy/captureContent,
  sampling/flushing, full metric and span catalog, diagnostic event
  catalog, no-exporter mode, diagnostics flags pointer, disable.
- docs/logging.md: drop the OTEL section in favor of a short
  'Diagnostics and OpenTelemetry' summary that cross-links the new
  page and the diagnostics-flags page. Drops 273 lines net. Also
  drops the redundant body H1, retitles to 'Logging' (was 'Logging
  overview' which mismatched sidebar usage), and refreshes the
  Related list.
- docs/docs.json: insert gateway/opentelemetry into the
  'Health and diagnostics' sidebar group, reorder pages so the user-
  facing health/run pages come before exporter/internals pages, and
  put logging next to opentelemetry where readers naturally
  associate them.
- docs/gateway/diagnostics.md, docs/gateway/logging.md,
  docs/gateway/configuration-reference.md: cross-link the new page
  and sentence-case stale Title-Cased Related entries on
  diagnostics.md.
2026-04-25 16:46:53 -07:00
Vincent Koc
64582bb3a7 docs(diagnostics-otel): clarify genai semconv exports 2026-04-25 12:30:14 -07:00
Peter Steinberger
30325f567c fix: use prompt snapshots for live context diagnostics 2026-04-25 20:25:44 +01:00
Vincent Koc
f204f0c999 docs(logging): document new OTEL metrics and spans from recent diagnostics-otel feats
Five recent diagnostics-otel feat commits added user-facing OpenTelemetry
surfaces but did not update docs/logging.md, so the listed metrics and
spans drifted out of sync with what the plugin actually exports:

- 7bbd47349e adds gen_ai.client.token.usage histogram (GenAI semconv)
- b8a41739d5 adds memory heap/rss histograms, pressure counter and span
- d6ef1fcf24 adds openclaw.tool.loop counters and span
- ff172f46a5 adds openclaw.context.assembled span
- 44114328b4 adds openclaw.provider.request_id_hash attr on
  openclaw.model.call spans

Append the new metrics under existing model-usage and exec sections,
add a 'Diagnostics internals' subsection for memory + tool-loop
metrics, and add the three new spans (context.assembled, tool.loop,
memory.pressure) plus the request-id-hash attribute to the spans
listing.
2026-04-25 11:35:20 -07:00
Vincent Koc
bd32b1a906 feat(diagnostics): add outbound delivery lifecycle events
Add bounded outbound message delivery lifecycle diagnostics and OTEL export without message body, recipient, room, media path, or raw channel result data.
2026-04-25 01:26:34 -07:00
Vincent Koc
3e3bba4f30 feat(diagnostics): emit exec process telemetry (#71451) 2026-04-25 00:12:58 -07:00
Vincent Koc
56eb1ffabf fix(diagnostics-otel): support preloaded sdk mode (#71450) 2026-04-24 23:55:34 -07:00
Vincent Koc
d4d4a8c14e feat(diagnostics-otel): add content capture controls
Add opt-in diagnostics OTEL content capture controls, keep raw content export default-off, and guard the content-capture tests against magic truncation bounds.
2026-04-24 16:41:28 -07:00
Vincent Koc
2777b089b5 docs: normalize frontmatter titles to sentence case 2026-04-23 13:15:17 -07:00
Peter Steinberger
cfcdf002c8 docs: refresh legacy tts and logging docs 2026-04-04 10:19:38 +01:00
Vincent Koc
637f15375b docs: fix Gateway & Ops audit findings (7 pages)
- cli-backends.md: remove duplicate modelAliases key
- discovery.md: add missing transport=gateway and displayName TXT keys
- authentication.md: retitle to "Authentication (Model Providers)", add
  disambiguation Note pointing to gateway connection auth docs
- health.md: expand frontmatter scope, add --probe flag and response shape docs
- gateway-lock.md: remove stale hardcoded date, add Related section
- troubleshooting.md: fix wrong auth cross-link (model auth -> gateway config)
- logging.md: add Related section linking to gateway logging internals
2026-03-31 14:24:19 +09:00
Vincent Koc
ca3165a3e7 fix(docs): deduplicate titles across 9 pages
Differentiate titles that were identical across multiple pages:
- Platform pages: add '(Platform)' suffix to distinguish from install guides
- Legacy root-level pages: add '(legacy path)' suffix for files that
  redirect to canonical tools/ paths
- Logging: 'Logging' -> 'Logging Overview' (root) and 'Gateway Logging'
- building-extensions: add '(redirect)' suffix
2026-03-22 16:02:28 -07:00
maweibin
98a03c490b Feat/logger support log level validation0222 (#23436)
* 1、环境变量**:新增 `OPENCLAW_LOG_LEVEL`,可取值 `silent|fatal|error|warn|info|debug|trace`。设置后同时覆盖**文件日志**与**控制台**的级别,优先级高于配置文件。
2、启动参数**:在 `openclaw gateway run` 上新增 `--log-level <level>`,对该次进程同时生效于文件与控制台;未传时仍使用环境变量或配置文件。

* fix(logging): make log-level override global and precedence-safe

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-22 11:15:13 +01:00
Seb Slight
abcaa8c7a9 Docs: add nav titles across docs (#5689) 2026-01-31 15:04:03 -06:00
cpojer
8cab78abbc chore: Run pnpm format:fix. 2026-01-31 21:13:13 +09:00
Peter Steinberger
9a7160786a refactor: rename to openclaw 2026-01-30 03:16:21 +01:00
Peter Steinberger
6d16a658e5 refactor: rename clawdbot to moltbot with legacy compat 2026-01-27 12:21:02 +00:00
Peter Steinberger
612a27f3dd feat: add diagnostics flags 2026-01-25 10:40:27 +00:00
Peter Steinberger
d19a0249f8 fix: align rolling logs to local time 2026-01-22 07:02:52 +00:00
Peter Steinberger
6734f2d71c fix: wire OTLP logs for diagnostics 2026-01-20 22:51:47 +00:00
Peter Steinberger
5c4079f66c feat: add diagnostics events and otel exporter 2026-01-20 18:56:15 +00:00
Peter Steinberger
3eb48cbea7 docs: complete channels rename sweep 2026-01-13 08:40:39 +00:00
Peter Steinberger
90342a4f3a refactor!: rename chat providers to channels 2026-01-13 08:40:39 +00:00
Peter Steinberger
efa5f0bfe0 feat: improve logs output and docs 2026-01-09 02:51:17 +01:00
Peter Steinberger
db4d0b8e75 docs: reorganize documentation structure 2026-01-07 00:45:46 +01:00
Peter Steinberger
4845c615cb docs: link source references to GitHub 2026-01-06 20:25:08 +00:00
Peter Steinberger
8be168b180 fix: redact sensitive tokens in tool summaries 2026-01-06 00:41:12 +01:00
Peter Steinberger
246adaa119 chore: rename project to clawdbot 2026-01-04 14:38:51 +00:00
Peter Steinberger
bb54e60179 fix(logging): decouple file logs from console verbose 2026-01-03 12:32:14 +00:00
Peter Steinberger
e3c3d108fe refactor(logging): shorten subsystem prefixes 2025-12-22 19:42:22 +00:00
Peter Steinberger
b06d1ed072 docs(logging): clarify console color behavior 2025-12-21 17:36:30 +00:00
Peter Steinberger
a8629e1855 fix(logging): simplify tty color detection 2025-12-21 13:34:13 +00:00
Peter Steinberger
0146784e18 feat(logging): add console color modes 2025-12-21 13:26:50 +00:00
Peter Steinberger
5b2e7d4464 refactor(logging): add subsystem console formatting 2025-12-21 13:24:15 +00:00