Commit Graph

349 Commits

Author SHA1 Message Date
Peter Steinberger
218dcd815a feat(tooling): enforce noUncheckedIndexedAccess in the extensions lane (NUIA phase 4) (#105132)
* fix(extensions): make indexed access explicit across channel plugins

Transport-payload-safe burn-down: malformed Telegram/Discord/QQ/LINE
and sibling channel input keeps existing skip paths; no synthesized
fields, no new throws in delivery loops. Zalo escape sentinels preserve
literal matches instead of undefined replacements.

* fix(extensions): make indexed access explicit across provider and memory plugins

Stream and model iteration, tool-block guards, capture guards, and
sparse accumulators; singleton model reads carry named invariants.

* fix(extensions): make indexed access explicit across tooling plugins, flip the extensions lane

Remaining plugins (oc-path, qa-lab, browser, logbook, and siblings) plus
the tsconfig.extensions.json flag flip. Cleanup: logbook sampleFrames
NaN index at max=1, QA retry clamp at non-positive attempts, dead Canvas
probe and OpenShell no-op slice removed, twitch test setup leak excluded
from the prod lane.

* refactor(plugin-sdk): expose expectDefined via a focused SDK subpath

Extensions imported @openclaw/normalization-core directly, crossing the
external-plugin packaging boundary (it only worked because the runtime
builder bundles undeclared workspace helpers). expect-runtime joins the
canonical entrypoints JSON, generated exports, API baseline, docs, and
subpath contract test; all 78 extension imports now use the SDK seam.
Two scanner-shaped locals renamed for review-bundle hygiene.

* chore(plugin-sdk): raise surface budgets for the expect-runtime subpath

One new entrypoint with one callable export, added intentionally as the
packaging-honest seam for extension invariant helpers.
2026-07-12 09:17:31 +01:00
Josh Lehman
0a8e3604ba refactor: flip sessions and transcripts to sqlite storage (#98236)
* refactor(sessions): migrate runtime storage to sqlite

* test(sessions): fix sqlite CI regressions

* test(sessions): align remaining sqlite fixtures

* fix(codex): require sqlite trajectory recorder

* test(sessions): align orphan recovery sqlite fixture

* test(sessions): align sqlite rebase fixtures

* fix(sessions): finish current-main integration of the sqlite flip

Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.

* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs

Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.

* feat(sessions): keep archived transcripts by default with zstd cold storage

Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.

* fix(state): converge agent DB migration lineages and bound database growth

Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.

Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.

* fix(codex): resolve the store path for thread-history import via the SDK

The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.

* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors

A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.

* fix(sessions): complete sqlite conflict translations

* test(sqlite): align hardening checks with maintenance

* test(sessions): inspect compressed transcript archives

* fix(tests): await session seeds and drop unused helpers flagged by CI lint

The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.

* test(sessions): type legacy proof events directly

* test(sessions): align hardening contracts

* perf(sessions): read usage transcript sizes from SQL aggregates

Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.

* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir

Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.

* test(agents): seed harness admission through sqlite

* fix(sqlite): close agent db on pragma setup failure

* fix(doctor): compact and retrofit incremental auto-vacuum after session import

The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-11 14:50:37 -07:00
Peter Steinberger
8fe6ac7fb9 refactor: remove pre-2026.4 compatibility shims and legacy migrations (#104650)
* refactor(matrix): retire pre-2026.4 legacy crypto and flat-storage migrations

* refactor(cli): drop pre-tsdown daemon-cli dist compat shim from the build

* refactor(diffs): remove deprecated image*/format tool params and output aliases

* refactor(ui): drop pre-gateway-scoped localStorage readers and legacy theme map

* refactor: remove assorted pre-2026.4 compat shims and deprecated aliases

* fix(discord): map legacy thread-binding fields in the doctor JSON import

* fix(msteams): keep bot read fallback for legacy imported conversation rows

* test(msteams): cover legacy bot-only imported conversation references

* fix(ui): keep channel-prefixed session key display fallback

* chore: refresh native i18n inventory and build-docker step count

* chore(matrix): drop now-unused migration-config module

* chore: re-pin plugin SDK public export budget after compat removals

* chore: refresh native i18n inventory after rebase

* chore: re-pin plugin SDK callable budget and refresh i18n inventory after rebase
2026-07-11 13:22:27 -07:00
Peter Steinberger
4271b82fd1 test: align plugin prerelease expectations (#104542)
* test: align plugin prerelease expectations

* style: format browser doctor test
2026-07-11 09:00:56 -07:00
Vincent Koc
63dbb336ca fix(ollama): preserve embedding readiness base path 2026-07-11 16:22:22 +08:00
Vincent Koc
d2ac533c6c fix(embeddings): skip local services for remote overrides 2026-07-11 16:22:22 +08:00
Vincent Koc
5988420f89 fix(ollama): drop stale embedding config alias 2026-07-11 16:22:22 +08:00
Vincent Koc
1b3438d7ec fix(plugin-runtime): keep local service config host-owned 2026-07-11 16:22:22 +08:00
Vincent Koc
ccab708c16 fix(plugin-sdk): align local service release contract 2026-07-11 16:22:22 +08:00
Vincent Koc
e62c216d75 refactor(embeddings): inject local service acquisition 2026-07-11 16:22:22 +08:00
Vincent Koc
fa5f24b021 fix(ollama): lease embedding local services 2026-07-11 16:22:22 +08:00
Peter Steinberger
7bf80dc2c6 chore(tooling): enforce formatting and refresh TypeScript checks (#104239)
* chore(tooling): enforce current formatter and refresh checks

* chore(tooling): keep release changelog formatter-owned

* chore(tooling): retain compatible Node type surface

* ci: enforce formatting for docs-only changes

* ci: isolate docs formatter check

* chore(tooling): apply updated lint and format rules

* chore(tooling): satisfy updated switch lint

* style(ui): apply Linux formatter layout

* test(doctor): match quiet local audio contribution

* test(doctor): assert quiet output only

* test(doctor): follow restored information contract
2026-07-11 01:09:51 -07:00
Peter Steinberger
dba64d574f chore(release): set version to 2026.7.2 2026-07-11 04:00:49 +01:00
pengxuewu-lab
fb52a989b2 fix(plugin-sdk): update ssrf-runtime import path in cdp-proxy-bypass (#96994)
* fix(plugin-sdk): update ssrf-runtime import path in cdp-proxy-bypass.ts

The import path `openclaw/plugin-sdk/ssrf-runtime-internal` was renamed
to `openclaw/plugin-sdk/ssrf-runtime` in a previous version, but
cdp-proxy-bypass.ts still referenced the old path. The neighbouring
cdp.helpers.ts already uses the correct path. This caused the browser
plugin to fail to load with "Package subpath is not defined by exports".

Fixes #96639

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(plugin-sdk): update ssrf-runtime import path in cdp-proxy-bypass

The import path `openclaw/plugin-sdk/ssrf-runtime-internal` was renamed
to `openclaw/plugin-sdk/ssrf-runtime` in a previous version, but
cdp-proxy-bypass.ts still referenced the old path. The neighbouring
cdp.helpers.ts already uses the correct path. This caused the browser
plugin to fail to load with "Package subpath is not defined by exports".

Update the corresponding vi.mock path in the test file to match.

Fixes #96639

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(plugin-sdk): promote managed-proxy bypass exports to public ssrf-runtime

Export registerManagedProxyBrowserCdpBypass and fetchConfiguredLocalOriginWithSsrFGuard
from the public ssrf-runtime path so that the browser plugin and ollama embedding
provider can lazy-load without hitting the private ssrf-runtime-internal subpath
that is excluded from package.json exports.

Fixes #96639

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(build): bundle private SSRF runtime for packaged plugins

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-09 12:20:18 +01:00
huangjianxiong
29d2a1ed42 fix(ollama): use truncateUtf16Safe for malformed NDJSON log truncation (#102593)
* fix(ollama): use truncateUtf16Safe for malformed NDJSON log warning

The parseNdjsonStream function uses naive .slice(0, 120) on malformed
NDJSON data in log warnings which can split surrogate pairs. Replace
with truncateUtf16Safe().

* test(ollama): cover UTF-16-safe NDJSON warnings

* test(ollama): satisfy lint in NDJSON coverage

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-09 11:38:54 +01:00
Vincent Koc
b88f80940a refactor: localize internal implementation symbols (#101831) 2026-07-07 10:23:36 -07:00
Vincent Koc
3d206140f3 refactor: localize internal implementation constants (#101758) 2026-07-07 08:34:13 -07:00
Peter Steinberger
80537c1ba4 feat(macos): load provider catalog during AI onboarding (#101132)
* feat(macos): load onboarding providers from gateway

* test(crestodian): widen setup config mock

* fix(crestodian): satisfy onboarding lint gate

* chore(macos): refresh onboarding localization inventory

* test(plugins): cover guided Copilot secret metadata
2026-07-06 20:57:56 +01:00
Andy Ye
9d9661e473 fix(ollama): fall back when native streams end early (#100482)
* Classify Ollama incomplete stream errors for fallback

* fix(ollama): own incomplete-stream failover classification

* test(ollama): pin incomplete-stream error contract

* docs(changelog): note Ollama fallback routing

Co-authored-by: Andy Ye <35905412+TurboTheTurtle@users.noreply.github.com>

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-06 03:12:49 +01:00
Peter Steinberger
614e87cce1 chore: update dependencies (#100027) 2026-07-04 14:56:50 -04:00
Peter Steinberger
1fef99962e feat(nodes): add auto-discovered Ollama inference (#99234)
* feat(nodes): add local Ollama inference

* fix(gateway): preserve plugin node runtime for agent turns

* feat(ollama): add node inference opt-out

* test(security): preserve plugin runtime exports

* test(security): preserve plugin runtime exports

* test(security): preserve plugin runtime exports

* fix(ci): raise artifact build heap
2026-07-03 01:14:30 -07:00
Vincent Koc
c0511692d8 test(plugins): repair prerelease validation fixtures 2026-07-01 18:02:09 -07:00
Vincent Koc
66e676d29b chore(release): close out 2026.6.11 on main 2026-06-30 11:31:08 -07:00
Yzx
3047e6c43d fix: Ollama Cloud tool calls fail on second turn (#96474)
Tested and it works.
2026-06-29 16:48:46 -06:00
NIO
a82cff8892 fix(ollama): bound cloud-auth 401 JSON response reads 2026-06-28 18:53:28 -07:00
Peter Steinberger
56259606d1 fix(agent-core): ignore truncated tool calls (#97140)
* fix(agent-core): ignore truncated tool calls

Co-authored-by: Galin Iliev <5711535+galiniliev@users.noreply.github.com>

* fix(agent-core): require explicit tool-call terminals

---------

Co-authored-by: Galin Iliev <5711535+galiniliev@users.noreply.github.com>
2026-06-27 03:31:42 +01:00
joshavant
0a14444924 Bound successful provider response reads 2026-06-24 19:08:22 -05:00
Alix-007
d1c2934d0d fix(ollama): bound model-discovery JSON response reads (#96027)
* fix(ollama): bound model-discovery JSON response reads

The /api/tags and /api/show discovery reads in extensions/ollama/src/provider-models.ts
parsed their HTTP responses with an unbounded await response.json(). Ollama base URLs
are user-supplied and can point at remote/cloud endpoints, so a hostile or buggy server
(or one reachable via SSRF) could stream an unbounded or never-ending JSON body and drive
model discovery into OOM.

Route both reads through the shared @openclaw/media-core byte-bounded reader
(readResponseWithLimit, re-exported via openclaw/plugin-sdk/response-limit-runtime) under
a single 16 MiB cap before JSON.parse, cancelling the stream on overflow. Overflow throws a
bounded error that the existing fail-soft handlers swallow, so a capped endpoint degrades
gracefully: /api/tags returns { reachable: false, models: [] } and /api/show returns {}.

Symmetric counterpart to the #95103/#95108 response-limit campaign.

AI-assisted.

* fix(ollama): reuse shared bounded JSON reader for model discovery

Replace the local readOllamaDiscoveryJson helper with the shared
readProviderJsonResponse (from openclaw/plugin-sdk/provider-http), which
already enforces the 16 MiB cap, cancels the stream on overflow, and wraps
malformed JSON with the caller label. The /api/tags and /api/show discovery
reads now go through it directly while keeping the existing fail-soft
handlers ({ reachable: false, models: [] } and {}).

Add a focused regression test: when a discovery stream exceeds the JSON byte
cap, fetchOllamaModels returns { reachable: false, models: [] },
queryOllamaModelShowInfo returns {}, and the bounded reader cancels the body
mid-flight so less than the full advertised stream is read.
2026-06-24 10:58:13 -04:00
Vincent Koc
0671c08900 chore(release): close out 2026.6.10 on main (#96271)
* chore(release): close out 2026.6.10 on main

* chore(release): align native app metadata for 2026.6.10

* chore(release): sync Android 2026.6.10 notes

* docs(changelog): preserve 2026.6.9 history

* docs(changelog): preserve 2026.6.9 history
2026-06-24 11:51:14 +08:00
Patrick Erichsen
0feffda3fc fix(plugins): remove simpleicons icon color paths (#95987) 2026-06-23 12:16:02 -07:00
Patrick Erichsen
0a97f73402 feat: add bundled plugin icon manifest URLs (#95845) 2026-06-22 22:14:18 -07:00
Jason O'Neal
92264fbb8f fix(ollama): skip auto-discovery for remote/cloud base URLs (#93956)
* fix(ollama): skip auto-discovery for remote/cloud base URLs

When the Ollama provider base URL points to a remote/cloud instance
(e.g. ollama.com), the plugin should not auto-discover all available
models via /api/tags. Cloud instances are shared tenants where the
provider manages the model catalog; users should only get models they
explicitly configure.

- Add remote-baseUrl guard in resolveOllamaDiscoveryResult
- Local/loopback URLs still auto-discover as before
- Remote URLs with explicit models return only those models
- Remote URLs without explicit models return null (skip discovery)
- Add tests covering remote guard, explicit models, and local fallback

* fix ollama cloud discovery ci

* fix(ollama): narrow discovery guard to hosted Ollama Cloud only

The previous guard blocked auto-discovery for ALL remote base URLs
without explicit models. This was too broad — it also blocked
self-hosted Ollama instances at custom domains (e.g.,
https://ollama.mycompany.com).

Replace the !isLocalOllamaBaseUrl() check with a targeted
isHostedOllamaCloud() check that only matches *.ollama.com
hostnames. Remote self-hosted Ollama endpoints now correctly
auto-discover as before.

Add isHostedOllamaCloud() helper with unit tests and a
regression test confirming remote self-hosted URLs still
auto-discover.

* fix(ollama): ensure models array in explicit-models return path

* fix(ollama): replace deprecated config-types import with local type

The openclaw/plugin-sdk/config-types subpath is deprecated and flagged
by the CI architecture check. Replace it with a local OllamaProviderConfigInput
type alias defined from non-deprecated provider-model-shared exports.

- discovery-shared.ts: define OllamaProviderConfigInput locally
- provider-base-url.ts: define OllamaProviderConfigInput locally
- Both files: remove import from openclaw/plugin-sdk/config-types

* chore(ollama): drop unrelated formatting churn
2026-06-22 18:08:05 +00:00
wood fish
1fc4342a02 fix(ollama): honor memory embedding output dimensionality (#94811) 2026-06-22 17:49:43 +00:00
Peter Steinberger
11484f8a14 fix(ollama): support GLM-5.2 cloud discovery 2026-06-21 19:00:15 -04:00
Vincent Koc
b039e949b6 chore(release): close out 2026.6.9 2026-06-21 12:24:15 +08:00
Vincent Koc
6037d1a85c fix(ollama): bound stream error bodies 2026-06-19 19:18:38 +02:00
Vincent Koc
2c8d19d73e fix(ollama): bound embedding error bodies 2026-06-19 19:18:38 +02:00
Vincent Koc
6774e7f259 chore(release): sync main to 2026.6.8 2026-06-17 07:25:30 +08:00
zhang-guiping
f0b5d78ff9 fix(ollama): preserve configured API during discovery (#93729)
* fix(ollama): preserve configured API during discovery

* fix(ollama): keep compatible discovery base URL

* fix(ollama): route compatible APIs through configured transport

---------

Co-authored-by: Vincent Koc <25068+vincentkoc@users.noreply.github.com>
2026-06-17 03:29:46 +08:00
Shakker
920e6a8eec chore: set version 2026.6.9 2026-06-16 19:54:07 +01:00
zhang-guiping
bb164384c2 [Bug]: ollama-cloud runtime fails DNS lookup for ai.ollama.com, while ollama/<model>:cloud works (#92594)
* fix(ollama): repair retired cloud provider endpoint

Route configured Ollama Cloud provider ids through plugin doctor compatibility migrations so doctor --fix can rewrite the retired ai.ollama.com endpoint before runtime reads persisted config.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(doctor): align provider fixture with typed config

Ensure the doctor registry provider-scoped migration test uses a fully typed provider fixture so the test type-check shard validates the intended behavior.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(ollama): align doctor fixture with typed config

Use fully typed provider and model fixtures in the Ollama doctor contract tests so the extension test type-check shard validates the migration behavior.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(ollama): preserve custom cloud provider base url

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(ollama): avoid logging retired endpoint secrets

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Vincent Koc <25068+vincentkoc@users.noreply.github.com>
2026-06-16 14:17:57 +08:00
OfflynAI
81924cfd5e fix(ollama): preserve length-limited responses (#89160)
Preserve Ollama length termination and surface incomplete token-limited embedded-agent turns without discarding durable tool output.

Fixes #89051.

Co-authored-by: joelnishanth <140015627+joelnishanth@users.noreply.github.com>
2026-06-14 10:59:37 -07:00
Vincent Koc
6fcc945702 fix(agents): trim dense text delta snapshots
Trim dense plain text-delta stream snapshots for OpenAI-compatible, Responses, and Ollama providers while preserving full snapshots on stream checkpoints and terminal events.

Reconstruct partial-less text deltas in the agent loop so live message updates continue to advance for immutable snapshot providers, and document the optional text_delta.partial contract.

Fixes #86599.
2026-06-09 13:21:23 +09:00
Peter Steinberger
af79cd6a9d fix: preserve live Ollama catalog metadata 2026-06-07 14:00:09 -07:00
Peter Steinberger
96e5812426 docs: document medium extension sources 2026-06-04 21:33:54 -04:00
Peter Steinberger
1878ca0820 chore(release): prepare 2026.6.2 beta 2026-06-04 00:06:52 +01:00
Peter Steinberger
e254346bc2 chore(release): prepare 2026.6.3 beta 2026-06-03 23:42:34 +01:00
Vincent Koc
6c7644268f fix(test): stabilize ARM extension timer tests 2026-06-02 07:53:25 -07:00
Peter Steinberger
a14eacf372 chore(release): set version 2026.6.2 2026-06-01 23:06:55 +01:00
Peter Steinberger
8e28c773fe chore(release): prepare 2026.6.1 2026-06-01 10:30:15 +01:00