diff --git a/docs/docs.json b/docs/docs.json index b0d739f5200..60e4900f21a 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -1554,6 +1554,7 @@ "gateway/security/index", "gateway/security/exposure-runbook", "gateway/security/secure-file-operations", + "gateway/security/shrinkwrap", "gateway/security/audit-checks", "gateway/operator-scopes", "gateway/sandboxing", @@ -1815,6 +1816,7 @@ "pages": [ "reference/RELEASING", "reference/full-release-validation", + "reference/release-performance-sweep", "reference/test", "ci", "help/scripts" diff --git a/docs/gateway/security/index.md b/docs/gateway/security/index.md index 5424840bf4c..cb20c5c31e8 100644 --- a/docs/gateway/security/index.md +++ b/docs/gateway/security/index.md @@ -63,67 +63,11 @@ OpenClaw source checkouts use `pnpm-lock.yaml`. The published `openclaw` npm package and OpenClaw-owned npm plugin packages include `npm-shrinkwrap.json`, npm's publishable dependency lockfile, so package installs use the reviewed transitive dependency graph from the release instead of resolving a fresh graph -at install time. Suitable OpenClaw-owned npm plugin packages can also publish -with explicit `bundledDependencies`, so their runtime dependency files are -carried in the plugin tarball instead of depending only on install-time -resolution. +at install time. -This is a supply-chain hardening measure: - -- release installs are more reproducible; -- transitive dependency updates become visible review surfaces; -- the package tarball contains the dependency graph that release validators - checked; -- suitable OpenClaw-owned plugin tarballs contain the dependency files from - that graph; -- `package-lock.json` stays out of the published package, because npm does not - treat it as the publishable lock contract. - -Shrinkwrap is not a sandbox and does not make every dependency trustworthy. It -does not replace `openclaw security audit`, host isolation, npm provenance, -signature/audit checks, or `--ignore-scripts` install smoke tests when those are -appropriate. Treat it as a release reproducibility and review-control boundary. - -Maintainers should update and verify shrinkwrap whenever the root package or an -OpenClaw-owned published plugin package changes its published dependency graph: - -```bash -pnpm deps:shrinkwrap:generate -pnpm deps:shrinkwrap:check -``` - -The generator resolves npm's publishable lock format but rejects generated -package versions that are not already present in `pnpm-lock.yaml`, preserving -the pnpm dependency age, override, and patch review boundary. - -Use `pnpm deps:shrinkwrap:root:generate` and -`pnpm deps:shrinkwrap:root:check` only when you intentionally want to refresh -the root `openclaw` package without touching plugin packages. - -Review `pnpm-lock.yaml`, `npm-shrinkwrap.json`, bundled plugin dependency -payloads, and any `package-lock.json` diff as security-sensitive. The package -validators require shrinkwrap in new root package tarballs and the plugin npm -publish path checks plugin-local shrinkwrap, installs package-local bundled -dependencies, and then packs or publishes. Package validators reject -`package-lock.json`. - -To inspect a published package: - -```bash -npm pack openclaw@ --json --pack-destination /tmp/openclaw-pack -tar -tf /tmp/openclaw-pack/openclaw-.tgz | grep '^package/npm-shrinkwrap.json$' -``` - -To inspect an OpenClaw-owned plugin package, replace the package spec and check -the same tar entry: - -```bash -npm pack @openclaw/discord@ --json --pack-destination /tmp/openclaw-plugin-pack -tar -tf /tmp/openclaw-plugin-pack/openclaw-discord-.tgz | grep '^package/npm-shrinkwrap.json$' -tar -tf /tmp/openclaw-plugin-pack/openclaw-discord-.tgz | grep '^package/node_modules/' -``` - -Background: [npm-shrinkwrap.json](https://docs.npmjs.com/cli/v11/configuring-npm/npm-shrinkwrap-json). +Shrinkwrap is a supply-chain hardening and release reproducibility boundary, +not a sandbox. For the plain-English model, maintainer commands, and package +inspection checks, see [npm shrinkwrap](/gateway/security/shrinkwrap). ### Deployment and host trust diff --git a/docs/gateway/security/shrinkwrap.md b/docs/gateway/security/shrinkwrap.md new file mode 100644 index 00000000000..46aa2d1f228 --- /dev/null +++ b/docs/gateway/security/shrinkwrap.md @@ -0,0 +1,111 @@ +--- +summary: "Plain-English and technical explanation of npm shrinkwrap in OpenClaw releases" +read_when: + - You want to know what npm shrinkwrap means in an OpenClaw release + - You are reviewing package lockfiles, dependency changes, or supply-chain risk + - You are validating root or plugin npm packages before publishing +title: "npm shrinkwrap" +--- + +OpenClaw source checkouts use `pnpm-lock.yaml`. Published OpenClaw npm +packages use `npm-shrinkwrap.json`, npm's publishable dependency lockfile, so +package installs use the dependency graph reviewed during release. + +## The easy version + +Shrinkwrap is a receipt for the dependency tree that ships with an npm package. +It tells npm which exact transitive package versions to install. + +For OpenClaw releases, that means: + +- the published package does not ask npm to invent a fresh dependency graph at + install time; +- dependency changes become easier to review because they appear in a lockfile; +- release validation can test the same graph users will install; +- package-size or native-dependency surprises are easier to spot before + publishing. + +Shrinkwrap is not a sandbox. It does not make a dependency safe by itself, and +it does not replace host isolation, `openclaw security audit`, package +provenance, or install smoke tests. + +The short mental model: + +| File | Where it matters | What it means | +| --------------------- | ------------------------ | --------------------------------- | +| `pnpm-lock.yaml` | OpenClaw source checkout | Maintainer dependency graph | +| `npm-shrinkwrap.json` | Published npm package | npm install graph for users | +| `package-lock.json` | Local npm apps | Not the OpenClaw publish contract | + +## Why OpenClaw uses it + +OpenClaw is a gateway, plugin host, model router, and agent runtime. A default +install can affect startup time, disk use, native package downloads, and +supply-chain exposure. + +Shrinkwrap gives release review a stable boundary: + +- reviewers can see transitive dependency movement; +- package validators can reject unexpected lockfile drift; +- package acceptance can test installs with the graph that will ship; +- plugin packages can carry their own locked dependency graph instead of + relying on the root package to own plugin-only dependencies. + +The goal is not "more lockfiles." The goal is reproducible release installs +with clear ownership. + +## Technical details + +The root `openclaw` npm package and OpenClaw-owned npm plugin packages include +`npm-shrinkwrap.json` when they publish. Suitable OpenClaw-owned plugin +packages can also publish with explicit `bundledDependencies`, so their runtime +dependency files are carried in the plugin tarball instead of depending only on +install-time resolution. + +Maintain the boundary like this: + +```bash +pnpm deps:shrinkwrap:generate +pnpm deps:shrinkwrap:check +``` + +The generator resolves npm's publishable lock format but rejects generated +package versions that are not already present in `pnpm-lock.yaml`. That keeps +the pnpm dependency age, override, and patch-review boundary intact. + +Use root-only commands only when intentionally refreshing the root package +without touching plugin packages: + +```bash +pnpm deps:shrinkwrap:root:generate +pnpm deps:shrinkwrap:root:check +``` + +Review these files as security-sensitive: + +- `pnpm-lock.yaml` +- `npm-shrinkwrap.json` +- bundled plugin dependency payloads +- any `package-lock.json` diff + +OpenClaw package validators require shrinkwrap in new root package tarballs. +The plugin npm publish path checks plugin-local shrinkwrap, installs +package-local bundled dependencies, and then packs or publishes. Package +validators reject `package-lock.json` for published OpenClaw packages. + +To inspect a published root package: + +```bash +npm pack openclaw@ --json --pack-destination /tmp/openclaw-pack +tar -tf /tmp/openclaw-pack/openclaw-.tgz | grep '^package/npm-shrinkwrap.json$' +``` + +To inspect an OpenClaw-owned plugin package: + +```bash +npm pack @openclaw/discord@ --json --pack-destination /tmp/openclaw-plugin-pack +tar -tf /tmp/openclaw-plugin-pack/openclaw-discord-.tgz | grep '^package/npm-shrinkwrap.json$' +tar -tf /tmp/openclaw-plugin-pack/openclaw-discord-.tgz | grep '^package/node_modules/' +``` + +Background: [npm-shrinkwrap.json](https://docs.npmjs.com/cli/v11/configuring-npm/npm-shrinkwrap-json). diff --git a/docs/reference/release-performance-sweep.md b/docs/reference/release-performance-sweep.md new file mode 100644 index 00000000000..a921298efb6 --- /dev/null +++ b/docs/reference/release-performance-sweep.md @@ -0,0 +1,272 @@ +--- +summary: "Release performance, install footprint, dependency count, and shrinkwrap audit for the April to May 2026 cleanup sweep" +read_when: + - You are validating the May 2026 performance and package-size cleanup + - You need the numbers behind the OpenClaw performance and dependency blog post + - You are changing release gates, package shrinkwrap, or plugin dependency boundaries +title: "Release performance sweep" +--- + +This page captures the technical evidence behind the May 2026 OpenClaw +performance and dependency cleanup. + +Two audits are combined here: + +- **Release performance sweep:** GitHub Releases from `v2026.5.27` back through + stable `v2026.4.23`, using the `OpenClaw Performance` workflow, + `profile=smoke`, `repeat=1`, mock-provider lane. +- **Earlier April context:** published `clawgrit-reports` mock-provider + baselines from `v2026.4.1` through `v2026.5.2`, used only to avoid treating + the broken late-April releases as the public performance baseline. +- **Install footprint sweep:** fresh `npm install --ignore-scripts` installs + into temporary packages, with `du -sk node_modules` for size and a + `node_modules` walk for package-instance counts. +- **npm package size sweep:** `npm pack openclaw@ --dry-run --json` + for published releases, recording compressed tarball size, unpacked size, and + file count. + + +The main performance sweep uses one smoke sample per tag. Earlier April context +uses published repeat-3 medians from `clawgrit-reports`. Use both as trend +evidence and regression-hunting signal, not as statistically significant +release-gate data. + + +## Quick read + +Performance sweep: + +- Requested releases: **76** +- Artifact-backed points: **73** +- CI unavailable: **3** +- Best cold agent turn: `v2026.5.27-beta.1` at **2,575ms** +- Best warm agent turn: `v2026.5.27-beta.1` at **2,217ms** +- Best source gateway default `readyz` p50: `v2026.5.27-beta.1` at **1,462ms** +- Latest stable measured point: `v2026.5.27` +- Earlier April narrative baseline: `v2026.4.14` published median at + **9,819ms cold**, **7,458ms warm**, and **686.2MB peak RSS**. + +Install footprint sweep: + +- Root `npm-shrinkwrap.json` is absent in `2026.5.20` and present in + `2026.5.22`. +- `2026.5.22` installed a **911.8 MB** nested `openclaw/node_modules` tree. +- `2026.5.27` still installed a **675.9 MB** nested tree. +- Current `main` keeps shrinkwrap but installs **0 MB** nested + `openclaw/node_modules`. +- Current `main` installs at **407.4 MB** and **314 installed dependencies**. +- Published npm package size peaked at `2026.3.31`: **43.3 MB** compressed, + **182.6 MB** unpacked, and **21,037 files**. +- Latest stable `2026.5.27` npm package size is **17.8 MB** compressed, + **79.0 MB** unpacked, and **12,509 files**. + +## Headline interpretation + +Do not use the late-April broken rows as public performance baselines. +`v2026.4.23` and `v2026.4.29` are useful regression evidence, but the large +`14x`-style deltas mostly describe the recovery from a bad release line. + +For the blog narrative, use the earlier April published baseline as scale: + +| Metric | Earlier April baseline | `v2026.5.27` | Delta | +| --------------- | ---------------------: | -----------: | -----------------------: | +| Cold agent turn | 9,819ms | 3,378ms | 65.6% lower, 2.9x faster | +| Warm agent turn | 7,458ms | 2,973ms | 60.1% lower, 2.5x faster | +| Agent peak RSS | 686.2MB | 635.5MB | 7.4% lower | + +The earlier April baseline is `v2026.4.14` from the published +`clawgrit-reports` mock-provider run. That run used repeat 3 and failed only +because the diagnostic timeline was not emitted; the cold, warm, and RSS +medians are still useful as rough scale. Treat this as narrative context, not a +release-gate statistic. + +Within the single-sample stable May sweep, the line moved more modestly: + +| Metric | `v2026.5.2` | `v2026.5.27` | Delta | +| --------------- | ----------: | -----------: | ----------: | +| Cold agent turn | 3,897ms | 3,378ms | 13.3% lower | +| Warm agent turn | 3,610ms | 2,973ms | 17.6% lower | +| Agent peak RSS | 613.7MB | 635.5MB | 3.6% higher | + +Best prerelease point in the single-sample sweep: + +| Metric | `v2026.5.27` | `v2026.5.27-beta.1` | Delta | +| --------------- | -----------: | ------------------: | ----------: | +| Cold agent turn | 3,378ms | 2,575ms | 23.8% lower | +| Warm agent turn | 2,973ms | 2,217ms | 25.4% lower | +| Agent peak RSS | 635.5MB | 635.3MB | flat | + +Install footprint: + +| Metric | Baseline | Current main | Delta | +| ----------------------------------------------- | --------: | -----------: | ----------: | +| Install size from `2026.5.22` peak | 1,020.6MB | 407.4MB | 60.1% lower | +| Install size from latest release `2026.5.27` | 786.9MB | 407.4MB | 48.2% lower | +| Dependencies from monthly high `2026.2.26` | 645 | 314 | 51.3% lower | +| Dependencies from latest release `2026.5.27` | 371 | 314 | 15.4% lower | +| Nested `openclaw/node_modules` from `2026.5.22` | 911.8MB | 0MB | removed | +| Nested `openclaw/node_modules` from `2026.5.27` | 675.9MB | 0MB | removed | + +npm package size: + +| Version | Compressed tarball | Unpacked package | Files | Notes | +| ----------- | -----------------: | ---------------: | -----: | --------------------------------- | +| `2026.1.30` | 12.8MB | 33.5MB | 4,607 | early rebranded package | +| `2026.2.26` | 23.6MB | 82.9MB | 10,125 | feature growth | +| `2026.3.31` | 43.3MB | 182.6MB | 21,037 | package-size high point | +| `2026.4.29` | 22.9MB | 74.6MB | 9,309 | package pruning visible | +| `2026.5.12` | 23.4MB | 80.1MB | 12,035 | major external-plugin split | +| `2026.5.22` | 17.2MB | 76.9MB | 12,386 | docs/assets excluded from package | +| `2026.5.27` | 17.8MB | 79.0MB | 12,509 | latest stable package | + +`2026.5.12` is the visible plugin-extraction milestone in the changelog: +Amazon Bedrock, Bedrock Mantle, Slack, OpenShell sandbox, Anthropic Vertex, +Matrix, and WhatsApp moved out of the core dependency path so their dependency +cones install with those plugins instead of every core install. + +## Kova agent turn summary + +The April stable line contains two different stories. Earlier April was slow +but recognizable. Late April became a regression cliff. `v2026.5.2` is where +the mock-provider lane first drops into the 3-5s range and starts passing +consistently in the supplied sweep. + +Earlier published context: + +| Release | Kova | Cold turn | Warm turn | Agent peak RSS | +| ------------ | ---- | --------: | --------: | -------------: | +| `v2026.4.10` | FAIL | 11,031ms | 7,962ms | 679.0MB | +| `v2026.4.12` | FAIL | 11,965ms | 8,289ms | 713.5MB | +| `v2026.4.14` | FAIL | 9,819ms | 7,458ms | 686.2MB | +| `v2026.4.20` | FAIL | 22,314ms | 18,811ms | 810.8MB | +| `v2026.4.22` | FAIL | 9,630ms | 7,459ms | 743.0MB | + +Supplied single-sample sweep: + +| Release | Kova | Cold turn | Warm turn | Agent peak RSS | +| ------------------- | ---- | --------: | --------: | -------------: | +| `v2026.4.23` | FAIL | 47,847ms | 8,010ms | 1,082.7MB | +| `v2026.4.24` | FAIL | 48,264ms | 25,483ms | 996.0MB | +| `v2026.4.25` | FAIL | 81,080ms | 59,172ms | 1,113.9MB | +| `v2026.4.26` | FAIL | 76,771ms | 54,941ms | 1,140.8MB | +| `v2026.4.27` | FAIL | 60,902ms | 33,699ms | 1,156.0MB | +| `v2026.4.29` | FAIL | 94,031ms | 57,334ms | 3,613.7MB | +| `v2026.5.2` | PASS | 3,897ms | 3,610ms | 613.7MB | +| `v2026.5.7` | PASS | 3,923ms | 3,693ms | 654.1MB | +| `v2026.5.12` | PASS | 7,248ms | 6,629ms | 834.8MB | +| `v2026.5.18` | PASS | 3,301ms | 2,913ms | 630.3MB | +| `v2026.5.20` | PASS | 3,413ms | 2,952ms | 643.2MB | +| `v2026.5.22` | PASS | 4,494ms | 4,093ms | 654.3MB | +| `v2026.5.26` | PASS | 2,626ms | 2,282ms | 660.4MB | +| `v2026.5.27-beta.1` | PASS | 2,575ms | 2,217ms | 635.3MB | +| `v2026.5.27` | PASS | 3,378ms | 2,973ms | 635.5MB | + +## Source probes + +Source probes were skipped for 17 successful older refs because those source +trees did not yet have the required probe entry points. Agent-turn metrics still +exist for those refs. + +Representative source-probe points: + +| Release | Default `readyz` p50 | 50 plugins `readyz` p50 | CLI health p50 | Plugin max RSS | +| ------------------- | -------------------: | ----------------------: | -------------: | -------------: | +| `v2026.4.29` | 2,819ms | 2,618ms | 1,679ms | 389.0MB | +| `v2026.5.2` | 2,324ms | 2,013ms | 1,384ms | 377.2MB | +| `v2026.5.7` | 1,649ms | 1,540ms | 1,175ms | 387.6MB | +| `v2026.5.18` | 1,942ms | 1,927ms | 607ms | 426.5MB | +| `v2026.5.20` | 1,966ms | 1,987ms | 621ms | 455.0MB | +| `v2026.5.22` | 2,081ms | 1,884ms | 5,095ms | 444.2MB | +| `v2026.5.26` | 1,546ms | 1,634ms | 656ms | 400.4MB | +| `v2026.5.27-beta.1` | 1,462ms | 1,548ms | 548ms | 394.0MB | +| `v2026.5.27` | 1,874ms | 1,925ms | 660ms | 398.0MB | + +The `v2026.5.22` CLI health spike is visible in this table even though the +agent-turn lane still passed. Keep the source probes when investigating +targeted CLI or gateway regressions. + +## Install footprint audit + +Dependency samples use one stable release per month, plus the +`2026.5.22` shrinkwrap-introduction event, latest `2026.5.27`, and current +`main`. + +| Point | Installed deps | Fresh install | OpenClaw package | Nested `openclaw/node_modules` | Root shrinkwrap | Canvas install behavior | +| ------------------ | -------------: | ------------: | ---------------: | -----------------------------: | --------------- | ----------------------------------------- | +| Jan `2026.1.30` | 605 | 438.4MB | 45.8MB | 2.4MB | no | top-level wrapper + `darwin-arm64` | +| Feb `2026.2.26` | 645 | 575.7MB | 110.1MB | 3.5MB | no | top-level wrapper + `darwin-arm64` | +| Mar `2026.3.31` | 438 | 584.1MB | 234.8MB | 0MB | no | top-level wrapper + `darwin-arm64` | +| Apr `2026.4.29` | 392 | 335.0MB | 97.4MB | 0MB | no | none installed | +| `2026.5.22` | 401 | 1,020.6MB | 1,020.4MB | 911.8MB | yes | nested: all 12 `@napi-rs/canvas` packages | +| May `2026.5.26` | 371 | 767.5MB | 767.4MB | 656.4MB | yes | nested: all 12 `@napi-rs/canvas` packages | +| Latest `2026.5.27` | 371 | 786.9MB | 786.7MB | 675.9MB | yes | nested: all 12 `@napi-rs/canvas` packages | +| Current `main` | 314 | 407.4MB | 101.0MB | 0MB | yes | top-level wrapper + `darwin-arm64` | + +### Shrinkwrap boundary + +Published tarball inspection verifies the boundary: + +| Version | Published stable? | Root `npm-shrinkwrap.json` | Notes | +| ----------- | ----------------- | -------------------------- | ------------------------------------- | +| `2026.5.20` | yes | no | last stable release before shrinkwrap | +| `2026.5.21` | no | n/a | no stable npm release | +| `2026.5.22` | yes | yes | shrinkwrap introduced | +| `2026.5.23` | no | n/a | no stable npm release | +| `2026.5.24` | no | n/a | no stable npm release | +| `2026.5.25` | no | n/a | no stable npm release | +| `2026.5.26` | yes | yes | nested dependency tree still present | +| `2026.5.27` | yes | yes | nested dependency tree still present | +| `main` | n/a | yes | nested dependency tree removed | + +The important distinction: **shrinkwrap itself is not the problem**. Current +`main` still ships root shrinkwrap. The problem was the package shape that made +npm materialize a large nested OpenClaw dependency tree and all 12 +`@napi-rs/canvas` platform packages. + +For a plain-English explanation of shrinkwrap and the maintainer-level package +checks, see [npm shrinkwrap](/gateway/security/shrinkwrap). + +## Supply-chain interpretation + +Dependency count is an operational security metric, not only an install-size +metric. Every package expands the set of maintainers, tarballs, transitive +updates, optional native binaries, and install-time behaviors that operators +must trust. + +The cleanup direction is: + +- keep heavy and optional capabilities outside the default core install +- make plugin packages own their runtime dependency graph +- avoid runtime package-manager repair during Gateway startup +- preserve deterministic installs without causing all-platform native package + materialization +- keep install scripts disabled in package acceptance and measurement paths +- catch nested dependency trees and native optional dependency explosions before + publishing + +Related docs: + +- [Plugin dependency resolution](/plugins/dependency-resolution) +- [Plugin inventory](/plugins/plugin-inventory) +- [Full release validation](/reference/full-release-validation) + +## Unavailable performance runs + +| Release | Run | Result | Reason | +| ------------------- | ---------------------------------------------------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------- | +| `v2026.5.3-1` | [26561664645](https://github.com/openclaw/openclaw/actions/runs/26561664645) | failure | mock-provider job failed: CLI startup timed out waiting for qa-channel ready; no qa-channel accounts reported | +| `v2026.5.3` | [26561666722](https://github.com/openclaw/openclaw/actions/runs/26561666722) | failure | mock-provider job failed: CLI startup timed out waiting for qa-channel ready; no qa-channel accounts reported | +| `v2026.4.29-beta.2` | [26561683635](https://github.com/openclaw/openclaw/actions/runs/26561683635) | cancelled | optional baseline fetch hung before artifact upload | + +## Follow-up gates + +Recommended release checks from this sweep: + +1. Run the mock-provider performance smoke for release candidates and retain + artifacts. +2. Track cold turn, warm turn, agent RSS, Gateway `readyz`, and CLI health. +3. Fresh-install the packed tarball with scripts disabled. +4. Record installed dependency count, install size, package size, nested + `openclaw/node_modules` size, and native optional package shape. +5. Fail or hold release review when nested dependency trees or all-platform + native packages appear unexpectedly.