mirror of
https://github.com/openclaw/openclaw.git
synced 2026-05-06 05:30:42 +00:00
fix(plugins): simplify bundled runtime deps staging
* fix(plugins): simplify bundled runtime deps staging * refactor(plugins): declare bundled root runtime deps * fix(plugins): isolate pnpm runtime dependency installs * test(gateway): wait for deferred agent routing calls in server suite * test(ci): follow extracted update-channel assertions * fix(plugins): bypass pnpm age gate for bundled runtime deps * test: drop stale rebase leftovers * test: preserve mirrored root dependency drift guard * test: stage mirrored deps in facade fixtures * fix(plugin-sdk): expose provider setup metadata * test(plugin-sdk): satisfy spread lint in facade deps fixture * refactor(plugins): share bundled runtime deps install flow * fix(plugins): finish runtime deps rebase cleanup * fix(plugins): remove stale mirror import * refactor(plugins): centralize bundled runtime root preparation * fix(plugins): skip Windows pnpm cmd shims * refactor(plugins): let package managers own runtime deps staging * fix(plugins): validate staged runtime deps * fix(plugins): preserve lazy runtime deps fallback
This commit is contained in:
committed by
GitHub
parent
86f473d8b9
commit
8cf724a381
@@ -107,7 +107,10 @@ Docs: https://docs.openclaw.ai
|
||||
- Plugins: cache unchanged plugin manifest loads by file signature, reducing repeated JSON/JSON5 parsing and manifest normalization in bursty startup and runtime registry paths. Refs #73532 and #73647; carries forward #73678. Thanks @TheDutchRuler.
|
||||
- Plugins/runtime-deps: cache unchanged bundled runtime mirror dist-file materialization decisions and close file-lock handles on owner-write failures, reducing repeated startup chunk scans and avoiding FileHandle-GC recovery stalls. Refs #73532. Thanks @oadiazp and @bstanbury.
|
||||
- Plugins/runtime-deps: retry and defer transient cleanup failures for owned runtime staging directories so CLI startup no longer aborts after a successful bundled dependency swap. Refs #73903. Thanks @bobfreeman1989.
|
||||
- Plugins/runtime-deps: cache bundled runtime-deps JSON/package files and root chunk import scans by file signature, reducing repeated staged-runtime scanning during bundled channel startup. Refs #73647 and #73705. Thanks @mattmcintyre and @bmilne1981.
|
||||
- Plugins/runtime-deps: cache bundled runtime-deps JSON/package files by file signature, reducing repeated staged-runtime metadata reads during bundled channel startup. Refs #73647 and #73705. Thanks @mattmcintyre and @bmilne1981.
|
||||
- Plugins/runtime-deps: delegate bundled plugin dependency staging to complete npm/pnpm install plans with durable runtime state, removing retained-manifest and source-checkout cache reconciliation from Gateway startup. Refs #73532. Thanks @oadiazp, @bstanbury, and @jmfraga.
|
||||
- Plugins/runtime-deps: replace Gateway-start root chunk dependency inference with explicit mirrored-root dependency metadata, reducing staged runtime scans while preserving lazy per-plugin installs. Refs #73532. Thanks @oadiazp and @bstanbury.
|
||||
- Plugins/runtime-deps: run pnpm staged installs outside the repository workspace and disable pnpm release-age gates for exact bundled runtime dependency materialization, so bundled plugin dependency repair writes packages into the generated stage without blocking fresh packaged dependencies. Refs #73532. Thanks @oadiazp and @bstanbury.
|
||||
- CLI/TUI: keep `chat.history` off model-catalog discovery so initial Gateway-backed TUI history loads cannot block behind slow provider/plugin model scans on low-core hosts. Refs #73524. Thanks @harshcatsystems-collab.
|
||||
- Channels/WhatsApp: flag recently reconnected linked accounts in channel status even when the socket is currently healthy, so flapping WhatsApp Web sessions no longer look clean after a brief reconnect. Refs #73602. Thanks @Vksh07.
|
||||
- Channels/WhatsApp: log shared dispatcher delivery failures with reply kind, message id, chat id, and connection id, so typing-without-send reports can identify whether the WhatsApp send path rejected a generated reply. Refs #74269. Thanks @tomcosta-git.
|
||||
|
||||
@@ -520,6 +520,8 @@ For npm-sourced installs, `openclaw plugins install` runs project-local `npm ins
|
||||
Bundled OpenClaw-owned plugins are the only startup repair exception: when a packaged install sees one enabled by plugin config, legacy channel config, or its bundled default-enabled manifest, startup installs that plugin's missing runtime dependencies before import. Third-party plugins should not rely on startup installs; keep using the explicit plugin installer.
|
||||
</Note>
|
||||
|
||||
Bundled package-level runtime deps are explicit metadata, not inferred from built JavaScript at gateway startup. If a shared OpenClaw root dependency must be available inside the external bundled-plugin runtime mirror, declare it in `openclaw.bundle.mirroredRootRuntimeDependencies` in the root package manifest.
|
||||
|
||||
## Related
|
||||
|
||||
- [Building plugins](/plugins/building-plugins) — step-by-step getting started guide
|
||||
|
||||
18
package.json
18
package.json
@@ -1728,5 +1728,23 @@
|
||||
"@whiskeysockets/baileys@7.0.0-rc.9": "patches/@whiskeysockets__baileys@7.0.0-rc.9.patch",
|
||||
"@agentclientprotocol/claude-agent-acp@0.31.0": "patches/@agentclientprotocol__claude-agent-acp@0.31.0.patch"
|
||||
}
|
||||
},
|
||||
"openclaw": {
|
||||
"bundle": {
|
||||
"mirroredRootRuntimeDependencies": [
|
||||
"@agentclientprotocol/sdk",
|
||||
"@lydell/node-pty",
|
||||
"croner",
|
||||
"dotenv",
|
||||
"jiti",
|
||||
"json5",
|
||||
"jszip",
|
||||
"markdown-it",
|
||||
"semver",
|
||||
"tar",
|
||||
"tslog",
|
||||
"web-push"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -213,3 +213,28 @@ export function collectBundledPluginRootRuntimeMirrorErrors(params) {
|
||||
|
||||
return errors.toSorted((left, right) => left.localeCompare(right));
|
||||
}
|
||||
|
||||
export function collectDeclaredRootRuntimeDependencyMetadataErrors(rootPackageJson) {
|
||||
const declaredRootRuntimeDeps = collectRuntimeDependencySpecs(rootPackageJson);
|
||||
const declaredMirrorDeps =
|
||||
rootPackageJson?.openclaw?.bundle?.mirroredRootRuntimeDependencies ?? [];
|
||||
if (!Array.isArray(declaredMirrorDeps)) {
|
||||
return ["package.json openclaw.bundle.mirroredRootRuntimeDependencies must be an array."];
|
||||
}
|
||||
|
||||
const errors = [];
|
||||
for (const dependencyName of declaredMirrorDeps) {
|
||||
if (typeof dependencyName !== "string" || dependencyName.trim().length === 0) {
|
||||
errors.push(
|
||||
"package.json openclaw.bundle.mirroredRootRuntimeDependencies entries must be non-empty strings.",
|
||||
);
|
||||
continue;
|
||||
}
|
||||
if (!declaredRootRuntimeDeps.has(dependencyName)) {
|
||||
errors.push(
|
||||
`package.json openclaw.bundle.mirroredRootRuntimeDependencies declares '${dependencyName}' but package.json dependencies/optionalDependencies do not include it.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return errors.toSorted((left, right) => left.localeCompare(right));
|
||||
}
|
||||
|
||||
@@ -35,6 +35,7 @@ import {
|
||||
collectBuiltBundledPluginStagedRuntimeDependencyErrors,
|
||||
collectBundledPluginRootRuntimeMirrorErrors,
|
||||
collectBundledPluginRuntimeDependencySpecs,
|
||||
collectDeclaredRootRuntimeDependencyMetadataErrors,
|
||||
collectRootDistBundledRuntimeMirrors,
|
||||
} from "./lib/bundled-plugin-root-runtime-mirrors.mjs";
|
||||
import { collectPackUnpackedSizeErrors as collectNpmPackUnpackedSizeErrors } from "./lib/npm-pack-budget.mjs";
|
||||
@@ -52,6 +53,7 @@ export { collectBundledExtensionManifestErrors } from "./lib/bundled-extension-m
|
||||
export {
|
||||
collectBuiltBundledPluginStagedRuntimeDependencyErrors,
|
||||
collectBundledPluginRootRuntimeMirrorErrors,
|
||||
collectDeclaredRootRuntimeDependencyMetadataErrors,
|
||||
collectRootDistBundledRuntimeMirrors,
|
||||
packageNameFromSpecifier,
|
||||
} from "./lib/bundled-plugin-root-runtime-mirrors.mjs";
|
||||
@@ -162,10 +164,16 @@ function checkBundledExtensionMetadata() {
|
||||
requiredRootMirrors,
|
||||
rootPackageJson: rootPackage,
|
||||
});
|
||||
const rootMirrorMetadataErrors = collectDeclaredRootRuntimeDependencyMetadataErrors(rootPackage);
|
||||
const builtArtifactErrors = collectBuiltBundledPluginStagedRuntimeDependencyErrors({
|
||||
bundledPluginsDir: resolve("dist/extensions"),
|
||||
});
|
||||
const errors = [...manifestErrors, ...rootMirrorErrors, ...builtArtifactErrors];
|
||||
const errors = [
|
||||
...manifestErrors,
|
||||
...rootMirrorErrors,
|
||||
...rootMirrorMetadataErrors,
|
||||
...builtArtifactErrors,
|
||||
];
|
||||
if (errors.length > 0) {
|
||||
console.error("release-check: bundled extension manifest validation failed:");
|
||||
for (const error of errors) {
|
||||
|
||||
@@ -8,6 +8,7 @@ import {
|
||||
collectBuiltBundledPluginStagedRuntimeDependencyErrors,
|
||||
collectBundledPluginRootRuntimeMirrorErrors,
|
||||
collectBundledPluginRuntimeDependencySpecs,
|
||||
collectDeclaredRootRuntimeDependencyMetadataErrors,
|
||||
collectRootDistBundledRuntimeMirrors,
|
||||
} from "./lib/bundled-plugin-root-runtime-mirrors.mjs";
|
||||
import { parsePackageRootArg } from "./lib/package-root-args.mjs";
|
||||
@@ -36,6 +37,7 @@ const errors = [
|
||||
requiredRootMirrors,
|
||||
rootPackageJson,
|
||||
}),
|
||||
...collectDeclaredRootRuntimeDependencyMetadataErrors(rootPackageJson),
|
||||
...collectBuiltBundledPluginStagedRuntimeDependencyErrors({
|
||||
bundledPluginsDir: builtPluginsDir,
|
||||
}),
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { importFreshModule } from "openclaw/plugin-sdk/test-fixtures";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { loadPluginManifestRegistry } from "../../plugins/manifest-registry.js";
|
||||
@@ -651,6 +652,102 @@ describe("bundled channel entry shape guards", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("does not load bundled runtime entries through external staged runtime deps during discovery", async () => {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "openclaw-bundled-runtime-deps-"));
|
||||
const stageRoot = fs.mkdtempSync(path.join(os.tmpdir(), "openclaw-bundled-stage-"));
|
||||
const previousBundledPluginsDir = process.env.OPENCLAW_BUNDLED_PLUGINS_DIR;
|
||||
const previousPluginStageDir = process.env.OPENCLAW_PLUGIN_STAGE_DIR;
|
||||
const pluginDir = path.join(root, "dist", "extensions", "alpha");
|
||||
const testGlobal = globalThis as typeof globalThis & {
|
||||
__bundledRuntimeDepMarker?: string;
|
||||
};
|
||||
fs.mkdirSync(pluginDir, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(root, "package.json"),
|
||||
JSON.stringify({ name: "openclaw", version: "2026.4.21" }),
|
||||
"utf8",
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(pluginDir, "package.json"),
|
||||
JSON.stringify({
|
||||
name: "@openclaw/alpha",
|
||||
version: "2026.4.21",
|
||||
type: "module",
|
||||
dependencies: {
|
||||
"alpha-runtime-dep": "1.0.0",
|
||||
},
|
||||
}),
|
||||
"utf8",
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(pluginDir, "plugin.js"),
|
||||
[
|
||||
"import { marker } from 'alpha-runtime-dep';",
|
||||
"globalThis.__bundledRuntimeDepMarker = marker;",
|
||||
"export default { id: 'alpha', meta: { label: marker }, config: {} };",
|
||||
"",
|
||||
].join("\n"),
|
||||
"utf8",
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(pluginDir, "index.js"),
|
||||
[
|
||||
`import { defineBundledChannelEntry } from ${JSON.stringify(pathToFileURL(path.resolve("src/plugin-sdk/channel-entry-contract.ts")).href)};`,
|
||||
"export default defineBundledChannelEntry({",
|
||||
" id: 'alpha',",
|
||||
" name: 'Alpha',",
|
||||
" description: 'Alpha',",
|
||||
" importMetaUrl: import.meta.url,",
|
||||
" plugin: { specifier: './plugin.js' },",
|
||||
"});",
|
||||
"",
|
||||
].join("\n"),
|
||||
"utf8",
|
||||
);
|
||||
|
||||
process.env.OPENCLAW_PLUGIN_STAGE_DIR = stageRoot;
|
||||
const { resolveBundledRuntimeDependencyInstallRoot } =
|
||||
await import("../../plugins/bundled-runtime-deps.js");
|
||||
const installRoot = resolveBundledRuntimeDependencyInstallRoot(pluginDir);
|
||||
const depRoot = path.join(installRoot, "node_modules", "alpha-runtime-dep");
|
||||
fs.mkdirSync(depRoot, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(depRoot, "package.json"),
|
||||
JSON.stringify({
|
||||
name: "alpha-runtime-dep",
|
||||
version: "1.0.0",
|
||||
type: "module",
|
||||
main: "index.js",
|
||||
}),
|
||||
"utf8",
|
||||
);
|
||||
fs.writeFileSync(path.join(depRoot, "index.js"), "export const marker = 'staged-alpha';\n");
|
||||
|
||||
mockAlphaDistExtensionRuntime();
|
||||
|
||||
try {
|
||||
process.env.OPENCLAW_BUNDLED_PLUGINS_DIR = path.join(root, "dist", "extensions");
|
||||
|
||||
const bundled = await importFreshModule<typeof import("./bundled.js")>(
|
||||
import.meta.url,
|
||||
"./bundled.js?scope=bundled-runtime-deps",
|
||||
);
|
||||
|
||||
expect(bundled.getBundledChannelPlugin("alpha")).toBeUndefined();
|
||||
expect(testGlobal.__bundledRuntimeDepMarker).toBeUndefined();
|
||||
} finally {
|
||||
restoreBundledPluginsDir(previousBundledPluginsDir);
|
||||
if (previousPluginStageDir === undefined) {
|
||||
delete process.env.OPENCLAW_PLUGIN_STAGE_DIR;
|
||||
} else {
|
||||
process.env.OPENCLAW_PLUGIN_STAGE_DIR = previousPluginStageDir;
|
||||
}
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
fs.rmSync(stageRoot, { recursive: true, force: true });
|
||||
delete testGlobal.__bundledRuntimeDepMarker;
|
||||
}
|
||||
});
|
||||
|
||||
it("swallows and caches bundled plugin and setup load failures", async () => {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "openclaw-bundled-load-failure-"));
|
||||
const previousBundledPluginsDir = process.env.OPENCLAW_BUNDLED_PLUGINS_DIR;
|
||||
|
||||
@@ -36,9 +36,13 @@ type BundledChannelEntryRuntimeContract = {
|
||||
accountInspect?: boolean;
|
||||
};
|
||||
register: (api: unknown) => void;
|
||||
loadChannelPlugin: () => ChannelPlugin;
|
||||
loadChannelSecrets?: () => ChannelPlugin["secrets"] | undefined;
|
||||
loadChannelAccountInspector?: () => NonNullable<ChannelPlugin["config"]["inspectAccount"]>;
|
||||
loadChannelPlugin: (options?: BundledEntryModuleLoadOptions) => ChannelPlugin;
|
||||
loadChannelSecrets?: (
|
||||
options?: BundledEntryModuleLoadOptions,
|
||||
) => ChannelPlugin["secrets"] | undefined;
|
||||
loadChannelAccountInspector?: (
|
||||
options?: BundledEntryModuleLoadOptions,
|
||||
) => NonNullable<ChannelPlugin["config"]["inspectAccount"]>;
|
||||
setChannelRuntime?: (runtime: PluginRuntime) => void;
|
||||
};
|
||||
|
||||
@@ -239,7 +243,7 @@ function loadGeneratedBundledChannelEntry(params: {
|
||||
rootScope: params.rootScope,
|
||||
metadata: params.metadata,
|
||||
entry: params.metadata.source,
|
||||
installRuntimeDeps: true,
|
||||
installRuntimeDeps: false,
|
||||
}),
|
||||
);
|
||||
if (!entry) {
|
||||
@@ -586,7 +590,7 @@ function getBundledChannelSecretsForRoot(
|
||||
}
|
||||
try {
|
||||
const secrets =
|
||||
entry.loadChannelSecrets?.() ??
|
||||
entry.loadChannelSecrets?.({ installRuntimeDeps: false }) ??
|
||||
getBundledChannelPluginForRoot(id, rootScope, loadContext)?.secrets;
|
||||
loadContext.lazySecretsById.set(id, secrets ?? null);
|
||||
return secrets;
|
||||
@@ -612,7 +616,7 @@ function getBundledChannelAccountInspectorForRoot(
|
||||
return undefined;
|
||||
}
|
||||
try {
|
||||
const inspector = entry.loadChannelAccountInspector();
|
||||
const inspector = entry.loadChannelAccountInspector({ installRuntimeDeps: false });
|
||||
loadContext.lazyAccountInspectorsById.set(id, inspector);
|
||||
return inspector;
|
||||
} catch (error) {
|
||||
|
||||
@@ -56,12 +56,54 @@ function createInstalledRuntimeDeps(): InstalledRuntimeDeps {
|
||||
return [];
|
||||
}
|
||||
|
||||
function readRetainedRuntimeDepsManifest(installRoot: string): string[] {
|
||||
const manifestPath = path.join(installRoot, ".openclaw-runtime-deps.json");
|
||||
const parsed = JSON.parse(fs.readFileSync(manifestPath, "utf8")) as { specs?: unknown };
|
||||
return Array.isArray(parsed.specs)
|
||||
? parsed.specs.filter((entry): entry is string => typeof entry === "string")
|
||||
: [];
|
||||
function parseInstallSpec(spec: string): { name: string; version: string } {
|
||||
const versionSeparator = spec.startsWith("@") ? spec.indexOf("@", 1) : spec.lastIndexOf("@");
|
||||
if (versionSeparator <= 0) {
|
||||
throw new Error(`Invalid install spec ${spec}`);
|
||||
}
|
||||
return {
|
||||
name: spec.slice(0, versionSeparator),
|
||||
version: spec.slice(versionSeparator + 1),
|
||||
};
|
||||
}
|
||||
|
||||
function materializeRuntimeDeps(params: BundledRuntimeDepsInstallParams): void {
|
||||
for (const spec of params.installSpecs ?? params.missingSpecs) {
|
||||
const { name, version } = parseInstallSpec(spec);
|
||||
writeJson(path.join(params.installRoot, "node_modules", ...name.split("/"), "package.json"), {
|
||||
name,
|
||||
version: version.replace(/^[~^]/u, ""),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function readMaterializedRuntimeDepSpecs(
|
||||
installRoot: string,
|
||||
expectedSpecs: readonly string[],
|
||||
): string[] {
|
||||
return expectedSpecs.flatMap((spec) => {
|
||||
const { name } = parseInstallSpec(spec);
|
||||
const packageJsonPath = path.join(
|
||||
installRoot,
|
||||
"node_modules",
|
||||
...name.split("/"),
|
||||
"package.json",
|
||||
);
|
||||
if (!fs.existsSync(packageJsonPath)) {
|
||||
return [];
|
||||
}
|
||||
const parsed = JSON.parse(fs.readFileSync(packageJsonPath, "utf8")) as {
|
||||
name?: unknown;
|
||||
version?: unknown;
|
||||
};
|
||||
return typeof parsed.name === "string" && typeof parsed.version === "string"
|
||||
? [`${parsed.name}@${parsed.version}`]
|
||||
: [];
|
||||
});
|
||||
}
|
||||
|
||||
function expectNoLegacyRuntimeDepsManifest(installRoot: string): void {
|
||||
expect(fs.existsSync(path.join(installRoot, ".openclaw-runtime-deps.json"))).toBe(false);
|
||||
}
|
||||
|
||||
function createNonInteractivePrompter(
|
||||
@@ -437,6 +479,7 @@ describe("doctor bundled plugin runtime deps", () => {
|
||||
},
|
||||
installDeps: (params) => {
|
||||
installed.push(params);
|
||||
materializeRuntimeDeps(params);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -472,6 +515,7 @@ describe("doctor bundled plugin runtime deps", () => {
|
||||
},
|
||||
installDeps: (params) => {
|
||||
installed.push(params);
|
||||
materializeRuntimeDeps(params);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -500,6 +544,7 @@ describe("doctor bundled plugin runtime deps", () => {
|
||||
},
|
||||
installDeps: (params) => {
|
||||
installed.push(params);
|
||||
materializeRuntimeDeps(params);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -512,7 +557,10 @@ describe("doctor bundled plugin runtime deps", () => {
|
||||
},
|
||||
]);
|
||||
expect(installRoot).not.toBe(root);
|
||||
expect(readRetainedRuntimeDepsManifest(installRoot)).toEqual(["grammy@1.37.0"]);
|
||||
expect(readMaterializedRuntimeDepSpecs(installRoot, ["grammy@1.37.0"])).toEqual([
|
||||
"grammy@1.37.0",
|
||||
]);
|
||||
expectNoLegacyRuntimeDepsManifest(installRoot);
|
||||
});
|
||||
|
||||
it("logs runtime dependency repair progress before and after install", async () => {
|
||||
@@ -534,9 +582,7 @@ describe("doctor bundled plugin runtime deps", () => {
|
||||
|
||||
expect(logs).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.stringContaining(
|
||||
"Installing bundled plugin runtime deps (1 missing, 1 install specs): grammy@1.37.0",
|
||||
),
|
||||
expect.stringContaining("Installing bundled plugin runtime deps (1 specs): grammy@1.37.0"),
|
||||
expect.stringContaining("Installed bundled plugin runtime deps in"),
|
||||
]),
|
||||
);
|
||||
@@ -622,6 +668,7 @@ describe("doctor bundled plugin runtime deps", () => {
|
||||
},
|
||||
installDeps: (params) => {
|
||||
installed.push(params);
|
||||
materializeRuntimeDeps(params);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -658,6 +705,7 @@ describe("doctor bundled plugin runtime deps", () => {
|
||||
},
|
||||
installDeps: (params) => {
|
||||
installed.push(params);
|
||||
materializeRuntimeDeps(params);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -740,6 +788,7 @@ describe("doctor bundled plugin runtime deps", () => {
|
||||
},
|
||||
installDeps: (params) => {
|
||||
installed.push(params);
|
||||
materializeRuntimeDeps(params);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -752,10 +801,13 @@ describe("doctor bundled plugin runtime deps", () => {
|
||||
},
|
||||
]);
|
||||
expect(installRoot).toContain(stageDir);
|
||||
expect(readRetainedRuntimeDepsManifest(installRoot)).toEqual(["@slack/web-api@7.15.1"]);
|
||||
expect(readMaterializedRuntimeDepSpecs(installRoot, ["@slack/web-api@7.15.1"])).toEqual([
|
||||
"@slack/web-api@7.15.1",
|
||||
]);
|
||||
expectNoLegacyRuntimeDepsManifest(installRoot);
|
||||
});
|
||||
|
||||
it("repairs only missing deps into the final layered stage dir", async () => {
|
||||
it("repairs the complete dependency plan into the final layered stage dir", async () => {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "openclaw-doctor-bundled-"));
|
||||
const baselineStageDir = fs.mkdtempSync(
|
||||
path.join(os.tmpdir(), "openclaw-doctor-bundled-baseline-"),
|
||||
@@ -797,14 +849,14 @@ describe("doctor bundled plugin runtime deps", () => {
|
||||
expect(installed).toEqual([
|
||||
{
|
||||
installRoot,
|
||||
missingSpecs: ["grammy@1.37.0"],
|
||||
installSpecs: ["grammy@1.37.0"],
|
||||
missingSpecs: ["@slack/web-api@7.15.1", "grammy@1.37.0"],
|
||||
installSpecs: ["@slack/web-api@7.15.1", "grammy@1.37.0"],
|
||||
},
|
||||
]);
|
||||
expect(readRetainedRuntimeDepsManifest(installRoot)).toEqual(["grammy@1.37.0"]);
|
||||
expectNoLegacyRuntimeDepsManifest(installRoot);
|
||||
});
|
||||
|
||||
it("drops stale retained bundled deps when repairing a subset", async () => {
|
||||
it("drops stale legacy bundled deps manifests when repairing a subset", async () => {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "openclaw-doctor-bundled-"));
|
||||
writeJson(path.join(root, "package.json"), { name: "openclaw" });
|
||||
writeBundledChannelPlugin(root, "telegram", { grammy: "1.37.0" });
|
||||
@@ -829,6 +881,7 @@ describe("doctor bundled plugin runtime deps", () => {
|
||||
},
|
||||
installDeps: (params) => {
|
||||
installed.push(params);
|
||||
materializeRuntimeDeps(params);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -840,6 +893,9 @@ describe("doctor bundled plugin runtime deps", () => {
|
||||
},
|
||||
]);
|
||||
expect(installRoot).not.toBe(root);
|
||||
expect(readRetainedRuntimeDepsManifest(installRoot)).toEqual(["grammy@1.37.0"]);
|
||||
expect(readMaterializedRuntimeDepSpecs(installRoot, ["grammy@1.37.0"])).toEqual([
|
||||
"grammy@1.37.0",
|
||||
]);
|
||||
expectNoLegacyRuntimeDepsManifest(installRoot);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -4,7 +4,7 @@ import { formatCliCommand } from "../cli/command-format.js";
|
||||
import type { OpenClawConfig } from "../config/types.openclaw.js";
|
||||
import { resolveOpenClawPackageRootSync } from "../infra/openclaw-root.js";
|
||||
import {
|
||||
createBundledRuntimeDepsWritableInstallSpecs,
|
||||
createBundledRuntimeDepsInstallSpecs,
|
||||
repairBundledRuntimeDepsInstallRootAsync,
|
||||
resolveBundledRuntimeDependencyPackageInstallRootPlan,
|
||||
scanBundledPluginRuntimeDeps,
|
||||
@@ -164,18 +164,15 @@ export async function maybeRepairBundledPluginRuntimeDeps(params: {
|
||||
return;
|
||||
}
|
||||
|
||||
const missingSpecs = missing.map((dep) => `${dep.name}@${dep.version}`);
|
||||
const installRootPlan = resolveBundledRuntimeDependencyPackageInstallRootPlan(packageRoot, {
|
||||
env,
|
||||
});
|
||||
const installSpecs = createBundledRuntimeDepsWritableInstallSpecs({
|
||||
const installSpecs = createBundledRuntimeDepsInstallSpecs({
|
||||
deps,
|
||||
searchRoots: installRootPlan.searchRoots,
|
||||
installRoot: installRootPlan.installRoot,
|
||||
});
|
||||
note(
|
||||
[
|
||||
"Bundled plugin runtime deps are missing.",
|
||||
"Bundled plugin runtime deps need staging.",
|
||||
...missing.map((dep) => `- ${dep.name}@${dep.version} (used by ${dep.pluginIds.join(", ")})`),
|
||||
`Fix: run ${formatCliCommand("openclaw doctor --fix")} to install them.`,
|
||||
].join("\n"),
|
||||
@@ -198,14 +195,14 @@ export async function maybeRepairBundledPluginRuntimeDeps(params: {
|
||||
try {
|
||||
const { createCliProgress } = await import("../cli/progress.js");
|
||||
progress = createCliProgress({
|
||||
label: `Installing bundled plugin runtime deps (${missingSpecs.length})`,
|
||||
label: `Installing bundled plugin runtime deps (${installSpecs.length})`,
|
||||
indeterminate: true,
|
||||
enabled: process.env.VITEST !== "true" || process.env.OPENCLAW_TEST_RUNTIME_LOG === "1",
|
||||
});
|
||||
const installStartedAt = Date.now();
|
||||
logRuntimeDepsInstallProgress(
|
||||
params.runtime,
|
||||
`Installing bundled plugin runtime deps (${missingSpecs.length} missing, ${installSpecs.length} install specs): ${missingSpecs.join(", ")}`,
|
||||
`Installing bundled plugin runtime deps (${installSpecs.length} specs): ${installSpecs.join(", ")}`,
|
||||
);
|
||||
heartbeat = setInterval(() => {
|
||||
logRuntimeDepsInstallProgress(
|
||||
@@ -216,7 +213,7 @@ export async function maybeRepairBundledPluginRuntimeDeps(params: {
|
||||
heartbeat.unref?.();
|
||||
const result = await repairBundledRuntimeDepsInstallRootAsync({
|
||||
installRoot: installRootPlan.installRoot,
|
||||
missingSpecs,
|
||||
missingSpecs: installSpecs,
|
||||
installSpecs,
|
||||
env: params.env ?? process.env,
|
||||
installDeps: params.installDeps
|
||||
|
||||
@@ -245,7 +245,7 @@ describe("prepareGatewayPluginBootstrap runtime-deps staging", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("pre-stages only missing runtime deps while retaining the full startup dependency set", async () => {
|
||||
it("pre-stages the full startup dependency set", async () => {
|
||||
scanBundledPluginRuntimeDeps.mockReturnValueOnce({
|
||||
deps: [
|
||||
{ name: "alpha-runtime", version: "1.0.0", pluginIds: ["telegram"] },
|
||||
@@ -267,7 +267,7 @@ describe("prepareGatewayPluginBootstrap runtime-deps staging", () => {
|
||||
expect(repairBundledRuntimeDepsInstallRootAsync).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
installRoot: "/runtime",
|
||||
missingSpecs: ["grammy@1.37.0"],
|
||||
missingSpecs: ["alpha-runtime@1.0.0", "grammy@1.37.0"],
|
||||
installSpecs: ["alpha-runtime@1.0.0", "grammy@1.37.0"],
|
||||
}),
|
||||
);
|
||||
|
||||
@@ -87,19 +87,18 @@ async function prestageGatewayBundledRuntimeDeps(params: {
|
||||
if (missing.length === 0) {
|
||||
return;
|
||||
}
|
||||
const missingSpecs = missing.map((dep) => `${dep.name}@${dep.version}`);
|
||||
const installSpecs = deps.map((dep) => `${dep.name}@${dep.version}`);
|
||||
const installRoot = resolveBundledRuntimeDependencyPackageInstallRoot(packageRoot, {
|
||||
env: process.env,
|
||||
});
|
||||
const startedAt = Date.now();
|
||||
params.log.info(
|
||||
`[plugins] staging bundled runtime deps before gateway startup (${missingSpecs.length} missing, ${installSpecs.length} install specs): ${missingSpecs.join(", ")}`,
|
||||
`[plugins] staging bundled runtime deps before gateway startup (${installSpecs.length} specs): ${installSpecs.join(", ")}`,
|
||||
);
|
||||
try {
|
||||
await repairBundledRuntimeDepsInstallRootAsync({
|
||||
installRoot,
|
||||
missingSpecs,
|
||||
missingSpecs: installSpecs,
|
||||
installSpecs,
|
||||
env: process.env,
|
||||
warn: (message) => params.log.warn(`[plugins] ${message}`),
|
||||
@@ -111,7 +110,7 @@ async function prestageGatewayBundledRuntimeDeps(params: {
|
||||
return;
|
||||
}
|
||||
params.log.info(
|
||||
`[plugins] installed bundled runtime deps before gateway startup in ${Date.now() - startedAt}ms: ${missingSpecs.join(", ")}`,
|
||||
`[plugins] installed bundled runtime deps before gateway startup in ${Date.now() - startedAt}ms: ${installSpecs.join(", ")}`,
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -100,9 +100,13 @@ export type BundledChannelEntryContract<TPlugin = ChannelPlugin> = {
|
||||
configSchema: ChannelEntryConfigSchema<TPlugin>;
|
||||
features?: BundledChannelEntryFeatures;
|
||||
register: (api: OpenClawPluginApi) => void;
|
||||
loadChannelPlugin: () => TPlugin;
|
||||
loadChannelSecrets?: () => ChannelPlugin["secrets"] | undefined;
|
||||
loadChannelAccountInspector?: () => NonNullable<ChannelPlugin["config"]["inspectAccount"]>;
|
||||
loadChannelPlugin: (options?: BundledEntryModuleLoadOptions) => TPlugin;
|
||||
loadChannelSecrets?: (
|
||||
options?: BundledEntryModuleLoadOptions,
|
||||
) => ChannelPlugin["secrets"] | undefined;
|
||||
loadChannelAccountInspector?: (
|
||||
options?: BundledEntryModuleLoadOptions,
|
||||
) => NonNullable<ChannelPlugin["config"]["inspectAccount"]>;
|
||||
setChannelRuntime?: (runtime: PluginRuntime) => void;
|
||||
};
|
||||
|
||||
@@ -448,15 +452,22 @@ export function defineBundledChannelEntry<TPlugin = ChannelPlugin>({
|
||||
typeof configSchema === "function"
|
||||
? configSchema()
|
||||
: ((configSchema ?? emptyChannelConfigSchema()) as ChannelEntryConfigSchema<TPlugin>);
|
||||
const loadChannelPlugin = () => loadBundledEntryExportSync<TPlugin>(importMetaUrl, plugin);
|
||||
const loadChannelPlugin = (options?: BundledEntryModuleLoadOptions) =>
|
||||
loadBundledEntryExportSync<TPlugin>(importMetaUrl, plugin, options);
|
||||
const loadChannelSecrets = secrets
|
||||
? () => loadBundledEntryExportSync<ChannelPlugin["secrets"] | undefined>(importMetaUrl, secrets)
|
||||
? (options?: BundledEntryModuleLoadOptions) =>
|
||||
loadBundledEntryExportSync<ChannelPlugin["secrets"] | undefined>(
|
||||
importMetaUrl,
|
||||
secrets,
|
||||
options,
|
||||
)
|
||||
: undefined;
|
||||
const loadChannelAccountInspector = accountInspect
|
||||
? () =>
|
||||
? (options?: BundledEntryModuleLoadOptions) =>
|
||||
loadBundledEntryExportSync<NonNullable<ChannelPlugin["config"]["inspectAccount"]>>(
|
||||
importMetaUrl,
|
||||
accountInspect,
|
||||
options,
|
||||
)
|
||||
: undefined;
|
||||
const setChannelRuntime = runtime
|
||||
|
||||
@@ -4,6 +4,7 @@ import { fileURLToPath, pathToFileURL } from "node:url";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
clearBundledRuntimeDependencyNodePaths,
|
||||
ensureBundledPluginRuntimeDeps,
|
||||
resolveBundledRuntimeDependencyInstallRoot,
|
||||
} from "../plugins/bundled-runtime-deps.js";
|
||||
import { shouldExpectNativeJitiForJavaScriptTestRuntime } from "../test-utils/jiti-runtime.js";
|
||||
@@ -180,6 +181,18 @@ function writeStagedRuntimeDepPackage(params: {
|
||||
fs.writeFileSync(path.join(depRoot, "index.js"), params.source ?? "export {};\n", "utf8");
|
||||
}
|
||||
|
||||
function concreteRuntimeDepVersionForTest(version: string): string {
|
||||
return version.startsWith("^") || version.startsWith("~") ? version.slice(1) : version;
|
||||
}
|
||||
|
||||
function parseRuntimeDepSpecForTest(spec: string): { name: string; version: string } {
|
||||
const atIndex = spec.lastIndexOf("@");
|
||||
return {
|
||||
name: spec.slice(0, atIndex),
|
||||
version: spec.slice(atIndex + 1),
|
||||
};
|
||||
}
|
||||
|
||||
function createPackagedBundledPluginDirWithStagedRuntimeDep(params: {
|
||||
marker: string;
|
||||
prefix: string;
|
||||
@@ -227,14 +240,24 @@ function createPackagedBundledPluginDirWithStagedRuntimeDep(params: {
|
||||
const installRoot = resolveBundledRuntimeDependencyInstallRoot(pluginRoot, {
|
||||
env,
|
||||
});
|
||||
writeStagedRuntimeDepPackage({
|
||||
installRoot,
|
||||
name: STAGED_RUNTIME_DEP_NAME,
|
||||
version: "1.0.0",
|
||||
source: `export const marker = ${JSON.stringify(params.marker)};\n`,
|
||||
ensureBundledPluginRuntimeDeps({
|
||||
env,
|
||||
pluginId,
|
||||
pluginRoot,
|
||||
installDeps: ({ installRoot: runtimeInstallRoot, installSpecs = [] }) => {
|
||||
for (const spec of installSpecs) {
|
||||
const dep = parseRuntimeDepSpecForTest(spec);
|
||||
writeStagedRuntimeDepPackage({
|
||||
installRoot: runtimeInstallRoot,
|
||||
name: dep.name,
|
||||
version: concreteRuntimeDepVersionForTest(dep.version),
|
||||
...(dep.name === STAGED_RUNTIME_DEP_NAME
|
||||
? { source: `export const marker = ${JSON.stringify(params.marker)};\n` }
|
||||
: {}),
|
||||
});
|
||||
}
|
||||
},
|
||||
});
|
||||
writeStagedRuntimeDepPackage({ installRoot, name: "semver", version: "7.7.4" });
|
||||
writeStagedRuntimeDepPackage({ installRoot, name: "tslog", version: "4.10.2" });
|
||||
|
||||
return {
|
||||
bundledPluginsDir,
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -71,16 +71,45 @@ export function copyBundledPluginRuntimeRoot(sourceRoot: string, targetRoot: str
|
||||
}
|
||||
removeBundledRuntimeMirrorPathIfTypeChanged(targetPath, "file");
|
||||
copyBundledRuntimeMirrorFileAtomic(sourcePath, targetPath);
|
||||
try {
|
||||
const sourceMode = fs.statSync(sourcePath).mode;
|
||||
fs.chmodSync(targetPath, sourceMode | 0o600);
|
||||
} catch {
|
||||
// Readable copied files are enough for plugin loading.
|
||||
}
|
||||
chmodBundledRuntimeMirrorFileReadable(sourcePath, targetPath);
|
||||
}
|
||||
pruneStaleBundledRuntimeMirrorEntries(targetRoot, mirroredNames);
|
||||
}
|
||||
|
||||
export function materializeBundledRuntimeMirrorFile(sourcePath: string, targetPath: string): void {
|
||||
if (path.resolve(sourcePath) === path.resolve(targetPath)) {
|
||||
return;
|
||||
}
|
||||
try {
|
||||
if (
|
||||
fs.realpathSync(sourcePath) === fs.realpathSync(targetPath) &&
|
||||
!fs.lstatSync(targetPath).isSymbolicLink()
|
||||
) {
|
||||
return;
|
||||
}
|
||||
} catch {
|
||||
// Missing targets are expected before the mirror file is materialized.
|
||||
}
|
||||
fs.mkdirSync(path.dirname(targetPath), { recursive: true, mode: 0o755 });
|
||||
fs.rmSync(targetPath, { recursive: true, force: true });
|
||||
try {
|
||||
fs.linkSync(sourcePath, targetPath);
|
||||
return;
|
||||
} catch {
|
||||
fs.copyFileSync(sourcePath, targetPath);
|
||||
}
|
||||
chmodBundledRuntimeMirrorFileReadable(sourcePath, targetPath);
|
||||
}
|
||||
|
||||
function chmodBundledRuntimeMirrorFileReadable(sourcePath: string, targetPath: string): void {
|
||||
try {
|
||||
const sourceMode = fs.statSync(sourcePath).mode;
|
||||
fs.chmodSync(targetPath, sourceMode | 0o600);
|
||||
} catch {
|
||||
// Readable mirrored files are enough for plugin loading.
|
||||
}
|
||||
}
|
||||
|
||||
function pruneStaleBundledRuntimeMirrorEntries(
|
||||
targetRoot: string,
|
||||
mirroredNames: Set<string>,
|
||||
|
||||
@@ -35,6 +35,21 @@ function isBigIntStatOptions(options: unknown): boolean {
|
||||
);
|
||||
}
|
||||
|
||||
function writeGeneratedRuntimeDepsManifest(rootDir: string, specs: readonly string[]): void {
|
||||
const dependencies = Object.fromEntries(
|
||||
specs.map((spec) => {
|
||||
const atIndex = spec.lastIndexOf("@");
|
||||
return [spec.slice(0, atIndex), spec.slice(atIndex + 1)];
|
||||
}),
|
||||
);
|
||||
fs.mkdirSync(rootDir, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(rootDir, "package.json"),
|
||||
JSON.stringify({ name: "openclaw-runtime-deps-install", private: true, dependencies }),
|
||||
"utf8",
|
||||
);
|
||||
}
|
||||
|
||||
describe("prepareBundledPluginRuntimeRoot", () => {
|
||||
it("materializes root JavaScript chunks in external mirrors", () => {
|
||||
const packageRoot = makeTempRoot();
|
||||
@@ -110,6 +125,7 @@ describe("prepareBundledPluginRuntimeRoot", () => {
|
||||
"utf8",
|
||||
);
|
||||
fs.writeFileSync(path.join(depRoot, "index.js"), "export const marker = 'stage-ok';\n", "utf8");
|
||||
writeGeneratedRuntimeDepsManifest(installRoot, ["playwright-core@1.0.0"]);
|
||||
|
||||
const staleMirrorChunk = path.join(installRoot, "dist", "pw-ai.js");
|
||||
fs.mkdirSync(path.dirname(staleMirrorChunk), { recursive: true });
|
||||
@@ -141,14 +157,14 @@ describe("prepareBundledPluginRuntimeRoot", () => {
|
||||
false,
|
||||
);
|
||||
expect(fs.lstatSync(path.join(installRoot, "dist", "config-runtime.js")).isSymbolicLink()).toBe(
|
||||
true,
|
||||
false,
|
||||
);
|
||||
expect(fs.lstatSync(path.join(installRoot, "dist", "string-runtime.js")).isSymbolicLink()).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
it("reuses root chunk materialization decisions across bundled plugin mirrors", () => {
|
||||
it("reuses prepared root mirrors across bundled plugins", () => {
|
||||
const packageRoot = makeTempRoot();
|
||||
const stageDir = makeTempRoot();
|
||||
const env = { ...process.env, OPENCLAW_PLUGIN_STAGE_DIR: stageDir };
|
||||
@@ -162,6 +178,10 @@ describe("prepareBundledPluginRuntimeRoot", () => {
|
||||
);
|
||||
fs.writeFileSync(rootChunk, "export const shared = 'root';\n", "utf8");
|
||||
fs.writeFileSync(externalChunk, "import zod from 'zod'; export const schema = zod;\n", "utf8");
|
||||
const installRoot = resolveBundledRuntimeDependencyInstallRoot(
|
||||
path.join(packageRoot, "dist", "extensions", "alpha"),
|
||||
{ env },
|
||||
);
|
||||
|
||||
for (const pluginId of ["alpha", "beta"]) {
|
||||
const pluginRoot = path.join(packageRoot, "dist", "extensions", pluginId);
|
||||
@@ -186,28 +206,20 @@ describe("prepareBundledPluginRuntimeRoot", () => {
|
||||
),
|
||||
"utf8",
|
||||
);
|
||||
const installRoot = resolveBundledRuntimeDependencyInstallRoot(pluginRoot, { env });
|
||||
fs.mkdirSync(path.join(installRoot, "node_modules", `${pluginId}-runtime`), {
|
||||
const pluginInstallRoot = resolveBundledRuntimeDependencyInstallRoot(pluginRoot, { env });
|
||||
fs.mkdirSync(path.join(pluginInstallRoot, "node_modules", `${pluginId}-runtime`), {
|
||||
recursive: true,
|
||||
});
|
||||
fs.writeFileSync(
|
||||
path.join(installRoot, "node_modules", `${pluginId}-runtime`, "package.json"),
|
||||
path.join(pluginInstallRoot, "node_modules", `${pluginId}-runtime`, "package.json"),
|
||||
JSON.stringify({ name: `${pluginId}-runtime`, version: "1.0.0", type: "module" }),
|
||||
"utf8",
|
||||
);
|
||||
}
|
||||
writeGeneratedRuntimeDepsManifest(installRoot, ["alpha-runtime@1.0.0", "beta-runtime@1.0.0"]);
|
||||
|
||||
const realReadFileSync = fs.readFileSync.bind(fs);
|
||||
const realReaddirSync = fs.readdirSync.bind(fs);
|
||||
const readPaths: string[] = [];
|
||||
const readdirPaths: string[] = [];
|
||||
vi.spyOn(fs, "readFileSync").mockImplementation(((target, options) => {
|
||||
const targetPath = target.toString();
|
||||
if (targetPath === rootChunk || targetPath === externalChunk) {
|
||||
readPaths.push(targetPath);
|
||||
}
|
||||
return realReadFileSync(target, options as never);
|
||||
}) as typeof fs.readFileSync);
|
||||
vi.spyOn(fs, "readdirSync").mockImplementation(((target, options) => {
|
||||
const targetPath = target.toString();
|
||||
if (
|
||||
@@ -229,8 +241,12 @@ describe("prepareBundledPluginRuntimeRoot", () => {
|
||||
});
|
||||
}
|
||||
|
||||
expect(readPaths.filter((entry) => entry === rootChunk)).toHaveLength(1);
|
||||
expect(readPaths.filter((entry) => entry === externalChunk)).toHaveLength(1);
|
||||
expect(fs.lstatSync(path.join(installRoot, "dist", "shared-runtime.js")).isSymbolicLink()).toBe(
|
||||
false,
|
||||
);
|
||||
expect(
|
||||
fs.lstatSync(path.join(installRoot, "dist", "external-runtime.js")).isSymbolicLink(),
|
||||
).toBe(false);
|
||||
expect(readdirPaths).toHaveLength(1);
|
||||
});
|
||||
|
||||
@@ -276,6 +292,7 @@ describe("prepareBundledPluginRuntimeRoot", () => {
|
||||
JSON.stringify({ name: "alpha-runtime", version: "1.0.0", type: "module" }),
|
||||
"utf8",
|
||||
);
|
||||
writeGeneratedRuntimeDepsManifest(installRoot, ["alpha-runtime@1.0.0"]);
|
||||
|
||||
const realReaddirSync = fs.readdirSync.bind(fs);
|
||||
const readdirPaths: string[] = [];
|
||||
@@ -341,6 +358,7 @@ describe("prepareBundledPluginRuntimeRoot", () => {
|
||||
JSON.stringify({ name: "qqbot-runtime", version: "1.0.0", type: "module" }),
|
||||
"utf8",
|
||||
);
|
||||
writeGeneratedRuntimeDepsManifest(installRoot, ["qqbot-runtime@1.0.0"]);
|
||||
|
||||
const prepared = prepareBundledPluginRuntimeRoot({
|
||||
pluginId: "qqbot",
|
||||
@@ -424,6 +442,7 @@ describe("prepareBundledPluginRuntimeRoot", () => {
|
||||
JSON.stringify({ name: "qqbot-runtime", version: "1.0.0", type: "module" }),
|
||||
"utf8",
|
||||
);
|
||||
writeGeneratedRuntimeDepsManifest(installRoot, ["qqbot-runtime@1.0.0"]);
|
||||
|
||||
const prepared = prepareBundledPluginRuntimeRoot({
|
||||
pluginId: "qqbot",
|
||||
@@ -489,6 +508,7 @@ describe("prepareBundledPluginRuntimeRoot", () => {
|
||||
JSON.stringify({ name: "qqbot-runtime", version: "1.0.0", type: "module" }),
|
||||
"utf8",
|
||||
);
|
||||
writeGeneratedRuntimeDepsManifest(installRoot, ["qqbot-runtime@1.0.0"]);
|
||||
|
||||
const lockPath = path.join(installRoot, ".openclaw-runtime-mirror.lock");
|
||||
const fingerprintLockStates: Array<{ source: "runtime" | "canonical"; locked: boolean }> = [];
|
||||
@@ -551,6 +571,7 @@ describe("prepareBundledPluginRuntimeRoot", () => {
|
||||
JSON.stringify({ name: "whatsapp-runtime", version: "1.0.0", type: "module" }),
|
||||
"utf8",
|
||||
);
|
||||
writeGeneratedRuntimeDepsManifest(installRoot, ["whatsapp-runtime@1.0.0"]);
|
||||
|
||||
const prepared = prepareBundledPluginRuntimeRoot({
|
||||
pluginId: "whatsapp",
|
||||
@@ -610,6 +631,7 @@ describe("prepareBundledPluginRuntimeRoot", () => {
|
||||
JSON.stringify({ name: "whatsapp-runtime", version: "1.0.0", type: "module" }),
|
||||
"utf8",
|
||||
);
|
||||
writeGeneratedRuntimeDepsManifest(installRoot, ["whatsapp-runtime@1.0.0"]);
|
||||
|
||||
const prepared = prepareBundledPluginRuntimeRoot({
|
||||
pluginId: "whatsapp",
|
||||
|
||||
@@ -1,28 +1,33 @@
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import type { OpenClawConfig } from "../config/types.openclaw.js";
|
||||
import {
|
||||
ensureBundledPluginRuntimeDeps,
|
||||
materializeBundledRuntimeMirrorDistFile,
|
||||
resolveBundledRuntimeDependencyInstallRootPlan,
|
||||
resolveBundledRuntimeDependencyPackageRoot,
|
||||
registerBundledRuntimeDependencyNodePath,
|
||||
shouldMaterializeBundledRuntimeMirrorDistFile,
|
||||
withBundledRuntimeDepsFilesystemLock,
|
||||
type BundledRuntimeDepsInstallParams,
|
||||
} from "./bundled-runtime-deps.js";
|
||||
import {
|
||||
markBundledRuntimeDistMirrorPrepared,
|
||||
shouldReusePreparedBundledRuntimeDistMirror,
|
||||
} from "./bundled-runtime-dist-mirror-cache.js";
|
||||
import {
|
||||
copyBundledPluginRuntimeRoot,
|
||||
materializeBundledRuntimeMirrorFile,
|
||||
precomputeBundledRuntimeMirrorMetadata,
|
||||
refreshBundledPluginRuntimeMirrorRoot,
|
||||
type PrecomputedBundledRuntimeMirrorMetadata,
|
||||
} from "./bundled-runtime-mirror.js";
|
||||
|
||||
const bundledRuntimeDepsRetainSpecsByInstallRoot = new Map<string, readonly string[]>();
|
||||
const BUNDLED_RUNTIME_MIRROR_LOCK_DIR = ".openclaw-runtime-mirror.lock";
|
||||
|
||||
export type PreparedBundledPluginRuntimeLoadRoot = {
|
||||
pluginRoot: string;
|
||||
modulePath: string;
|
||||
setupModulePath?: string;
|
||||
};
|
||||
|
||||
export function isBuiltBundledPluginRuntimeRoot(pluginRoot: string): boolean {
|
||||
const extensionsDir = path.dirname(pluginRoot);
|
||||
const buildDir = path.dirname(extensionsDir);
|
||||
@@ -39,36 +44,51 @@ export function prepareBundledPluginRuntimeRoot(params: {
|
||||
env?: NodeJS.ProcessEnv;
|
||||
logInstalled?: (installedSpecs: readonly string[]) => void;
|
||||
}): { pluginRoot: string; modulePath: string } {
|
||||
return prepareBundledPluginRuntimeLoadRoot(params);
|
||||
}
|
||||
|
||||
export function prepareBundledPluginRuntimeLoadRoot(params: {
|
||||
pluginId: string;
|
||||
pluginRoot: string;
|
||||
modulePath: string;
|
||||
setupModulePath?: string;
|
||||
env?: NodeJS.ProcessEnv;
|
||||
config?: OpenClawConfig;
|
||||
installDeps?: (params: BundledRuntimeDepsInstallParams) => void;
|
||||
registerRuntimeAliasRoot?: (rootDir: string) => void;
|
||||
logInstalled?: (installedSpecs: readonly string[]) => void;
|
||||
}): PreparedBundledPluginRuntimeLoadRoot {
|
||||
const env = params.env ?? process.env;
|
||||
const installRootPlan = resolveBundledRuntimeDependencyInstallRootPlan(params.pluginRoot, {
|
||||
env,
|
||||
});
|
||||
const installRoot = installRootPlan.installRoot;
|
||||
const retainSpecs = bundledRuntimeDepsRetainSpecsByInstallRoot.get(installRoot) ?? [];
|
||||
const depsInstallResult = ensureBundledPluginRuntimeDeps({
|
||||
pluginId: params.pluginId,
|
||||
pluginRoot: params.pluginRoot,
|
||||
env,
|
||||
retainSpecs,
|
||||
config: params.config,
|
||||
installDeps: params.installDeps,
|
||||
});
|
||||
if (depsInstallResult.installedSpecs.length > 0) {
|
||||
bundledRuntimeDepsRetainSpecsByInstallRoot.set(
|
||||
installRoot,
|
||||
[...new Set([...retainSpecs, ...depsInstallResult.retainSpecs])].toSorted((left, right) =>
|
||||
left.localeCompare(right),
|
||||
),
|
||||
);
|
||||
params.logInstalled?.(depsInstallResult.installedSpecs);
|
||||
}
|
||||
if (path.resolve(installRoot) === path.resolve(params.pluginRoot)) {
|
||||
return { pluginRoot: params.pluginRoot, modulePath: params.modulePath };
|
||||
ensureOpenClawPluginSdkAlias(path.dirname(path.dirname(params.pluginRoot)));
|
||||
return {
|
||||
pluginRoot: params.pluginRoot,
|
||||
modulePath: params.modulePath,
|
||||
...(params.setupModulePath ? { setupModulePath: params.setupModulePath } : {}),
|
||||
};
|
||||
}
|
||||
const packageRoot = resolveBundledRuntimeDependencyPackageRoot(params.pluginRoot);
|
||||
if (packageRoot) {
|
||||
registerBundledRuntimeDependencyNodePath(packageRoot);
|
||||
params.registerRuntimeAliasRoot?.(packageRoot);
|
||||
}
|
||||
for (const searchRoot of installRootPlan.searchRoots) {
|
||||
registerBundledRuntimeDependencyNodePath(searchRoot);
|
||||
params.registerRuntimeAliasRoot?.(searchRoot);
|
||||
}
|
||||
const mirrorRoot = mirrorBundledPluginRuntimeRoot({
|
||||
pluginId: params.pluginId,
|
||||
@@ -82,6 +102,15 @@ export function prepareBundledPluginRuntimeRoot(params: {
|
||||
pluginRoot: params.pluginRoot,
|
||||
mirroredRoot: mirrorRoot,
|
||||
}),
|
||||
...(params.setupModulePath
|
||||
? {
|
||||
setupModulePath: remapBundledPluginRuntimePath({
|
||||
source: params.setupModulePath,
|
||||
pluginRoot: params.pluginRoot,
|
||||
mirroredRoot: mirrorRoot,
|
||||
}),
|
||||
}
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -200,10 +229,18 @@ function ensureBundledRuntimeMirrorDirectory(targetRoot: string): void {
|
||||
fs.mkdirSync(targetRoot, { recursive: true, mode: 0o755 });
|
||||
}
|
||||
|
||||
function isPathInsideDirectory(childPath: string, parentPath: string): boolean {
|
||||
const relative = path.relative(path.resolve(parentPath), path.resolve(childPath));
|
||||
return relative !== "" && !relative.startsWith("..") && !path.isAbsolute(relative);
|
||||
}
|
||||
|
||||
function mirrorBundledRuntimeDistRootEntries(params: {
|
||||
sourceDistRoot: string;
|
||||
mirrorDistRoot: string;
|
||||
}): void {
|
||||
const mirrorRootDirectories =
|
||||
path.basename(params.sourceDistRoot) === "dist" ||
|
||||
path.basename(params.sourceDistRoot) === "dist-runtime";
|
||||
for (const entry of fs.readdirSync(params.sourceDistRoot, { withFileTypes: true })) {
|
||||
if (entry.name === "extensions") {
|
||||
continue;
|
||||
@@ -213,24 +250,25 @@ function mirrorBundledRuntimeDistRootEntries(params: {
|
||||
if (path.resolve(sourcePath) === path.resolve(targetPath)) {
|
||||
continue;
|
||||
}
|
||||
if (entry.isFile() && shouldMaterializeBundledRuntimeMirrorDistFile(sourcePath)) {
|
||||
materializeBundledRuntimeMirrorDistFile(sourcePath, targetPath);
|
||||
if (entry.isDirectory() && isPathInsideDirectory(targetPath, sourcePath)) {
|
||||
continue;
|
||||
}
|
||||
if (fs.existsSync(targetPath)) {
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
fs.symlinkSync(sourcePath, targetPath, entry.isDirectory() ? "junction" : "file");
|
||||
} catch {
|
||||
if (fs.existsSync(targetPath)) {
|
||||
const sourceStat = fs.statSync(sourcePath);
|
||||
if (sourceStat.isDirectory()) {
|
||||
if (!mirrorRootDirectories) {
|
||||
continue;
|
||||
}
|
||||
if (entry.isDirectory()) {
|
||||
copyBundledPluginRuntimeRoot(sourcePath, targetPath);
|
||||
} else if (entry.isFile()) {
|
||||
fs.copyFileSync(sourcePath, targetPath);
|
||||
}
|
||||
refreshBundledPluginRuntimeMirrorRoot({
|
||||
pluginId: `openclaw-dist:${entry.name}`,
|
||||
sourceRoot: sourcePath,
|
||||
targetRoot: targetPath,
|
||||
tempDirParent: params.mirrorDistRoot,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
if (sourceStat.isFile()) {
|
||||
materializeBundledRuntimeMirrorFile(sourcePath, targetPath);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -354,7 +392,7 @@ function writeRuntimeModuleWrapper(sourcePath: string, targetPath: string): void
|
||||
fs.writeFileSync(targetPath, content, "utf8");
|
||||
}
|
||||
|
||||
function ensureOpenClawPluginSdkAlias(distRoot: string): void {
|
||||
export function ensureOpenClawPluginSdkAlias(distRoot: string): void {
|
||||
const pluginSdkDir = path.join(distRoot, "plugin-sdk");
|
||||
if (!fs.existsSync(pluginSdkDir)) {
|
||||
return;
|
||||
|
||||
@@ -22,7 +22,11 @@ import {
|
||||
type DetachedTaskLifecycleRuntime,
|
||||
} from "../tasks/detached-task-runtime-state.js";
|
||||
import { withEnv } from "../test-utils/env.js";
|
||||
import { resolveBundledRuntimeDependencyInstallRootPlan } from "./bundled-runtime-deps.js";
|
||||
import {
|
||||
resolveBundledRuntimeDependencyInstallRootPlan,
|
||||
type BundledRuntimeDepsInstallParams,
|
||||
} from "./bundled-runtime-deps.js";
|
||||
import { ensureOpenClawPluginSdkAlias } from "./bundled-runtime-root.js";
|
||||
import { clearPluginCommands } from "./command-registry-state.js";
|
||||
import { getPluginCommandSpecs } from "./command-specs.js";
|
||||
import { listCompactionProviderIds } from "./compaction-provider.js";
|
||||
@@ -947,9 +951,9 @@ describe("loadOpenClawPlugins", () => {
|
||||
fs.writeFileSync(path.join(pluginSdkDir, "core.js"), "export const core = 1;\n", "utf8");
|
||||
fs.writeFileSync(path.join(aliasDir, "sentinel.txt"), "keep\n", "utf8");
|
||||
|
||||
__testing.ensureOpenClawPluginSdkAlias(distRoot);
|
||||
ensureOpenClawPluginSdkAlias(distRoot);
|
||||
fs.writeFileSync(path.join(pluginSdkDir, "core.js"), "export const core = 2;\n", "utf8");
|
||||
__testing.ensureOpenClawPluginSdkAlias(distRoot);
|
||||
ensureOpenClawPluginSdkAlias(distRoot);
|
||||
|
||||
expect(fs.existsSync(path.join(aliasDir, "sentinel.txt"))).toBe(true);
|
||||
expect(fs.readFileSync(path.join(aliasDir, "core.js"), "utf8")).toContain("core.js");
|
||||
@@ -1047,7 +1051,7 @@ module.exports = {
|
||||
},
|
||||
bundledRuntimeDepsInstaller: ({ installRoot, missingSpecs }) => {
|
||||
expect(logger.info).toHaveBeenCalledWith(
|
||||
"[plugins] discord staging bundled runtime deps (1 missing, 1 install specs): discord-runtime@1.0.0",
|
||||
"[plugins] discord staging bundled runtime deps (1 specs): discord-runtime@1.0.0",
|
||||
);
|
||||
installedSpecs.push(...missingSpecs);
|
||||
expect(fs.realpathSync(installRoot)).toBe(fs.realpathSync(plugin.dir));
|
||||
@@ -1142,7 +1146,7 @@ module.exports = {
|
||||
"[plugins] discord installed bundled runtime deps: discord-runtime@1.0.0",
|
||||
);
|
||||
expect(logger.info).not.toHaveBeenCalledWith(
|
||||
"[plugins] discord staging bundled runtime deps (1 missing, 1 install specs): discord-runtime@1.0.0",
|
||||
"[plugins] discord staging bundled runtime deps (1 specs): discord-runtime@1.0.0",
|
||||
);
|
||||
});
|
||||
|
||||
@@ -1926,7 +1930,7 @@ module.exports = {
|
||||
).toBe(false);
|
||||
expect(
|
||||
fs.lstatSync(path.join(actualInstallRoot, "dist", "config-runtime.js")).isSymbolicLink(),
|
||||
).toBe(true);
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("loads bundled plugins with plugin-sdk imports from an external stage dir", () => {
|
||||
@@ -2224,6 +2228,26 @@ module.exports = {
|
||||
|
||||
try {
|
||||
let actualInstallRoot = "";
|
||||
const installExternalRuntime = ({ installRoot }: BundledRuntimeDepsInstallParams) => {
|
||||
actualInstallRoot = installRoot;
|
||||
const depRoot = path.join(installRoot, "node_modules", "external-runtime");
|
||||
fs.mkdirSync(depRoot, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(depRoot, "package.json"),
|
||||
JSON.stringify({
|
||||
name: "external-runtime",
|
||||
version: "1.0.0",
|
||||
type: "module",
|
||||
exports: "./index.js",
|
||||
}),
|
||||
"utf-8",
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(depRoot, "index.js"),
|
||||
"export default { marker: 'dist-runtime-ok' };\n",
|
||||
"utf-8",
|
||||
);
|
||||
};
|
||||
const registry = loadOpenClawPlugins({
|
||||
cache: false,
|
||||
config: {
|
||||
@@ -2231,26 +2255,7 @@ module.exports = {
|
||||
enabled: true,
|
||||
},
|
||||
},
|
||||
bundledRuntimeDepsInstaller: ({ installRoot }) => {
|
||||
actualInstallRoot = installRoot;
|
||||
const depRoot = path.join(installRoot, "node_modules", "external-runtime");
|
||||
fs.mkdirSync(depRoot, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(depRoot, "package.json"),
|
||||
JSON.stringify({
|
||||
name: "external-runtime",
|
||||
version: "1.0.0",
|
||||
type: "module",
|
||||
exports: "./index.js",
|
||||
}),
|
||||
"utf-8",
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(depRoot, "index.js"),
|
||||
"export default { marker: 'dist-runtime-ok' };\n",
|
||||
"utf-8",
|
||||
);
|
||||
},
|
||||
bundledRuntimeDepsInstaller: installExternalRuntime,
|
||||
});
|
||||
|
||||
const record = registry.plugins.find((entry) => entry.id === "acpx");
|
||||
@@ -2277,6 +2282,7 @@ module.exports = {
|
||||
enabled: true,
|
||||
},
|
||||
},
|
||||
bundledRuntimeDepsInstaller: installExternalRuntime,
|
||||
});
|
||||
|
||||
const reloadedRecord = reloadedRegistry.plugins.find((entry) => entry.id === "acpx");
|
||||
@@ -2294,7 +2300,7 @@ module.exports = {
|
||||
}
|
||||
});
|
||||
|
||||
it("loads native ESM deps from a layered baseline stage dir", () => {
|
||||
it("loads native ESM deps from the writable stage dir without reusing a layered baseline", () => {
|
||||
const packageRoot = makeTempDir();
|
||||
const baselineStageDir = makeTempDir();
|
||||
const writableStageDir = makeTempDir();
|
||||
@@ -2417,17 +2423,39 @@ module.exports = {
|
||||
enabled: true,
|
||||
},
|
||||
},
|
||||
bundledRuntimeDepsInstaller: () => {
|
||||
throw new Error("baseline deps should not reinstall");
|
||||
bundledRuntimeDepsInstaller: ({ installRoot }) => {
|
||||
const depRoot = path.join(installRoot, "node_modules", "external-runtime");
|
||||
fs.mkdirSync(depRoot, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(depRoot, "package.json"),
|
||||
JSON.stringify({
|
||||
name: "external-runtime",
|
||||
version: "1.0.0",
|
||||
type: "module",
|
||||
exports: "./index.js",
|
||||
}),
|
||||
"utf-8",
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(depRoot, "index.js"),
|
||||
"export default { marker: 'writable-ok' };\n",
|
||||
"utf-8",
|
||||
);
|
||||
},
|
||||
});
|
||||
|
||||
const layeredRecord = registry.plugins.find((entry) => entry.id === "acpx");
|
||||
expect(layeredRecord?.error).toBeUndefined();
|
||||
expect(layeredRecord?.status).toBe("loaded");
|
||||
expect(fs.readFileSync(path.join(baselineDepRoot, "index.js"), "utf-8")).toContain(
|
||||
"baseline-ok",
|
||||
);
|
||||
expect(
|
||||
fs.realpathSync(path.join(installRootPlan.installRoot, "node_modules", "external-runtime")),
|
||||
).toBe(fs.realpathSync(baselineDepRoot));
|
||||
fs.readFileSync(
|
||||
path.join(installRootPlan.installRoot, "node_modules", "external-runtime", "index.js"),
|
||||
"utf-8",
|
||||
),
|
||||
).toContain("writable-ok");
|
||||
});
|
||||
|
||||
it("loads source-checkout bundled runtime deps without mirroring the repo tree", () => {
|
||||
|
||||
@@ -33,27 +33,14 @@ import { buildPluginApi } from "./api-builder.js";
|
||||
import { inspectBundleMcpRuntimeSupport } from "./bundle-mcp.js";
|
||||
import {
|
||||
clearBundledRuntimeDependencyNodePaths,
|
||||
ensureBundledPluginRuntimeDeps,
|
||||
installBundledRuntimeDeps,
|
||||
materializeBundledRuntimeMirrorDistFile,
|
||||
resolveBundledRuntimeDependencyInstallRootPlan,
|
||||
resolveBundledRuntimeDependencyPackageRoot,
|
||||
registerBundledRuntimeDependencyNodePath,
|
||||
shouldMaterializeBundledRuntimeMirrorDistFile,
|
||||
withBundledRuntimeDepsFilesystemLock,
|
||||
type BundledRuntimeDepsInstallParams,
|
||||
} from "./bundled-runtime-deps.js";
|
||||
import { clearBundledRuntimeDistMirrorPreparationCache } from "./bundled-runtime-dist-mirror-cache.js";
|
||||
import {
|
||||
clearBundledRuntimeDistMirrorPreparationCache,
|
||||
markBundledRuntimeDistMirrorPrepared,
|
||||
shouldReusePreparedBundledRuntimeDistMirror,
|
||||
} from "./bundled-runtime-dist-mirror-cache.js";
|
||||
import {
|
||||
copyBundledPluginRuntimeRoot,
|
||||
precomputeBundledRuntimeMirrorMetadata,
|
||||
refreshBundledPluginRuntimeMirrorRoot,
|
||||
type PrecomputedBundledRuntimeMirrorMetadata,
|
||||
} from "./bundled-runtime-mirror.js";
|
||||
ensureOpenClawPluginSdkAlias,
|
||||
prepareBundledPluginRuntimeLoadRoot,
|
||||
} from "./bundled-runtime-root.js";
|
||||
import {
|
||||
clearPluginCommands,
|
||||
listRegisteredPluginCommands,
|
||||
@@ -297,7 +284,6 @@ export function clearPluginLoaderCache(): void {
|
||||
}
|
||||
|
||||
const defaultLogger = () => createSubsystemLogger("plugins");
|
||||
const BUNDLED_RUNTIME_MIRROR_LOCK_DIR = ".openclaw-runtime-mirror.lock";
|
||||
|
||||
function isPromiseLike(value: unknown): value is PromiseLike<unknown> {
|
||||
return (
|
||||
@@ -707,314 +693,6 @@ function resolveCanonicalDistRuntimeSource(source: string): string {
|
||||
return fs.existsSync(candidate) ? candidate : source;
|
||||
}
|
||||
|
||||
function mirrorBundledPluginRuntimeRoot(params: {
|
||||
pluginId: string;
|
||||
pluginRoot: string;
|
||||
installRoot: string;
|
||||
}): string {
|
||||
const sourceDistRoot = path.dirname(path.dirname(params.pluginRoot));
|
||||
const mirrorParent = path.join(params.installRoot, path.basename(sourceDistRoot), "extensions");
|
||||
const mirrorRoot = path.join(mirrorParent, params.pluginId);
|
||||
const precomputedPluginRootMetadata =
|
||||
path.resolve(mirrorRoot) === path.resolve(params.pluginRoot)
|
||||
? undefined
|
||||
: precomputeBundledRuntimeMirrorMetadata({ sourceRoot: params.pluginRoot });
|
||||
const precomputedCanonicalPluginRootMetadata =
|
||||
precomputeCanonicalBundledRuntimeDistPluginMetadata({
|
||||
pluginRoot: params.pluginRoot,
|
||||
sourceDistRoot,
|
||||
});
|
||||
|
||||
return withBundledRuntimeDepsFilesystemLock(
|
||||
params.installRoot,
|
||||
BUNDLED_RUNTIME_MIRROR_LOCK_DIR,
|
||||
() => {
|
||||
const preparedMirrorParent = prepareBundledPluginRuntimeDistMirror({
|
||||
installRoot: params.installRoot,
|
||||
pluginRoot: params.pluginRoot,
|
||||
precomputedCanonicalPluginRootMetadata,
|
||||
});
|
||||
const preparedMirrorRoot = path.join(preparedMirrorParent, params.pluginId);
|
||||
fs.mkdirSync(params.installRoot, { recursive: true });
|
||||
try {
|
||||
fs.chmodSync(params.installRoot, 0o755);
|
||||
} catch {
|
||||
// Best-effort only: staged roots may live on filesystems that reject chmod.
|
||||
}
|
||||
fs.mkdirSync(preparedMirrorParent, { recursive: true });
|
||||
try {
|
||||
fs.chmodSync(preparedMirrorParent, 0o755);
|
||||
} catch {
|
||||
// Best-effort only: the access check below will surface non-writable dirs.
|
||||
}
|
||||
fs.accessSync(preparedMirrorParent, fs.constants.W_OK);
|
||||
if (path.resolve(preparedMirrorRoot) === path.resolve(params.pluginRoot)) {
|
||||
return preparedMirrorRoot;
|
||||
}
|
||||
refreshBundledPluginRuntimeMirrorRoot({
|
||||
pluginId: params.pluginId,
|
||||
sourceRoot: params.pluginRoot,
|
||||
targetRoot: preparedMirrorRoot,
|
||||
tempDirParent: preparedMirrorParent,
|
||||
precomputedSourceMetadata: precomputedPluginRootMetadata,
|
||||
});
|
||||
return preparedMirrorRoot;
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
function prepareBundledPluginRuntimeDistMirror(params: {
|
||||
installRoot: string;
|
||||
pluginRoot: string;
|
||||
precomputedCanonicalPluginRootMetadata?: PrecomputedBundledRuntimeMirrorMetadata;
|
||||
}): string {
|
||||
const sourceExtensionsRoot = path.dirname(params.pluginRoot);
|
||||
const sourceDistRoot = path.dirname(sourceExtensionsRoot);
|
||||
const sourceDistRootName = path.basename(sourceDistRoot);
|
||||
const mirrorDistRoot = path.join(params.installRoot, sourceDistRootName);
|
||||
const mirrorExtensionsRoot = path.join(mirrorDistRoot, "extensions");
|
||||
ensureBundledRuntimeMirrorDirectory(mirrorDistRoot);
|
||||
fs.mkdirSync(mirrorExtensionsRoot, { recursive: true, mode: 0o755 });
|
||||
ensureBundledRuntimeDistPackageJson(mirrorDistRoot);
|
||||
if (!shouldReusePreparedBundledRuntimeDistMirror({ sourceDistRoot, mirrorDistRoot })) {
|
||||
mirrorBundledRuntimeDistRootEntries({
|
||||
sourceDistRoot,
|
||||
mirrorDistRoot,
|
||||
});
|
||||
markBundledRuntimeDistMirrorPrepared({ sourceDistRoot, mirrorDistRoot });
|
||||
}
|
||||
if (sourceDistRootName === "dist-runtime") {
|
||||
mirrorCanonicalBundledRuntimeDistRoot({
|
||||
installRoot: params.installRoot,
|
||||
pluginRoot: params.pluginRoot,
|
||||
sourceRuntimeDistRoot: sourceDistRoot,
|
||||
precomputedSourceMetadata: params.precomputedCanonicalPluginRootMetadata,
|
||||
});
|
||||
}
|
||||
ensureOpenClawPluginSdkAlias(mirrorDistRoot);
|
||||
return mirrorExtensionsRoot;
|
||||
}
|
||||
|
||||
function ensureBundledRuntimeMirrorDirectory(targetRoot: string): void {
|
||||
try {
|
||||
const stat = fs.lstatSync(targetRoot);
|
||||
if (stat.isDirectory() && !stat.isSymbolicLink()) {
|
||||
return;
|
||||
}
|
||||
fs.rmSync(targetRoot, { recursive: true, force: true });
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
fs.mkdirSync(targetRoot, { recursive: true, mode: 0o755 });
|
||||
}
|
||||
|
||||
function mirrorBundledRuntimeDistRootEntries(params: {
|
||||
sourceDistRoot: string;
|
||||
mirrorDistRoot: string;
|
||||
}): void {
|
||||
for (const entry of fs.readdirSync(params.sourceDistRoot, { withFileTypes: true })) {
|
||||
if (entry.name === "extensions") {
|
||||
continue;
|
||||
}
|
||||
const sourcePath = path.join(params.sourceDistRoot, entry.name);
|
||||
const targetPath = path.join(params.mirrorDistRoot, entry.name);
|
||||
if (path.resolve(sourcePath) === path.resolve(targetPath)) {
|
||||
continue;
|
||||
}
|
||||
if (entry.isFile() && shouldMaterializeBundledRuntimeMirrorDistFile(sourcePath)) {
|
||||
materializeBundledRuntimeMirrorDistFile(sourcePath, targetPath);
|
||||
continue;
|
||||
}
|
||||
if (fs.existsSync(targetPath)) {
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
fs.symlinkSync(sourcePath, targetPath, entry.isDirectory() ? "junction" : "file");
|
||||
} catch {
|
||||
if (fs.existsSync(targetPath)) {
|
||||
continue;
|
||||
}
|
||||
if (entry.isDirectory()) {
|
||||
copyBundledPluginRuntimeRoot(sourcePath, targetPath);
|
||||
} else if (entry.isFile()) {
|
||||
fs.copyFileSync(sourcePath, targetPath);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function mirrorCanonicalBundledRuntimeDistRoot(params: {
|
||||
installRoot: string;
|
||||
pluginRoot: string;
|
||||
sourceRuntimeDistRoot: string;
|
||||
precomputedSourceMetadata?: PrecomputedBundledRuntimeMirrorMetadata;
|
||||
}): void {
|
||||
const sourceCanonicalDistRoot = path.join(path.dirname(params.sourceRuntimeDistRoot), "dist");
|
||||
if (!fs.existsSync(sourceCanonicalDistRoot)) {
|
||||
return;
|
||||
}
|
||||
const targetCanonicalDistRoot = path.join(params.installRoot, "dist");
|
||||
ensureBundledRuntimeMirrorDirectory(targetCanonicalDistRoot);
|
||||
fs.mkdirSync(path.join(targetCanonicalDistRoot, "extensions"), { recursive: true, mode: 0o755 });
|
||||
ensureBundledRuntimeDistPackageJson(targetCanonicalDistRoot);
|
||||
if (
|
||||
!shouldReusePreparedBundledRuntimeDistMirror({
|
||||
sourceDistRoot: sourceCanonicalDistRoot,
|
||||
mirrorDistRoot: targetCanonicalDistRoot,
|
||||
})
|
||||
) {
|
||||
mirrorBundledRuntimeDistRootEntries({
|
||||
sourceDistRoot: sourceCanonicalDistRoot,
|
||||
mirrorDistRoot: targetCanonicalDistRoot,
|
||||
});
|
||||
markBundledRuntimeDistMirrorPrepared({
|
||||
sourceDistRoot: sourceCanonicalDistRoot,
|
||||
mirrorDistRoot: targetCanonicalDistRoot,
|
||||
});
|
||||
}
|
||||
ensureOpenClawPluginSdkAlias(targetCanonicalDistRoot);
|
||||
|
||||
const pluginId = path.basename(params.pluginRoot);
|
||||
const sourceCanonicalPluginRoot = path.join(sourceCanonicalDistRoot, "extensions", pluginId);
|
||||
if (!fs.existsSync(sourceCanonicalPluginRoot)) {
|
||||
return;
|
||||
}
|
||||
const targetCanonicalPluginRoot = path.join(targetCanonicalDistRoot, "extensions", pluginId);
|
||||
refreshBundledPluginRuntimeMirrorRoot({
|
||||
pluginId,
|
||||
sourceRoot: sourceCanonicalPluginRoot,
|
||||
targetRoot: targetCanonicalPluginRoot,
|
||||
tempDirParent: path.dirname(targetCanonicalPluginRoot),
|
||||
precomputedSourceMetadata: params.precomputedSourceMetadata,
|
||||
});
|
||||
}
|
||||
|
||||
function precomputeCanonicalBundledRuntimeDistPluginMetadata(params: {
|
||||
pluginRoot: string;
|
||||
sourceDistRoot: string;
|
||||
}): PrecomputedBundledRuntimeMirrorMetadata | undefined {
|
||||
if (path.basename(params.sourceDistRoot) !== "dist-runtime") {
|
||||
return undefined;
|
||||
}
|
||||
const pluginId = path.basename(params.pluginRoot);
|
||||
const sourceCanonicalPluginRoot = path.join(
|
||||
path.dirname(params.sourceDistRoot),
|
||||
"dist",
|
||||
"extensions",
|
||||
pluginId,
|
||||
);
|
||||
if (!fs.existsSync(sourceCanonicalPluginRoot)) {
|
||||
return undefined;
|
||||
}
|
||||
return precomputeBundledRuntimeMirrorMetadata({ sourceRoot: sourceCanonicalPluginRoot });
|
||||
}
|
||||
|
||||
function ensureBundledRuntimeDistPackageJson(mirrorDistRoot: string): void {
|
||||
const packageJsonPath = path.join(mirrorDistRoot, "package.json");
|
||||
if (fs.existsSync(packageJsonPath)) {
|
||||
return;
|
||||
}
|
||||
writeRuntimeJsonFile(packageJsonPath, { type: "module" });
|
||||
}
|
||||
|
||||
function writeRuntimeJsonFile(targetPath: string, value: unknown): void {
|
||||
fs.mkdirSync(path.dirname(targetPath), { recursive: true });
|
||||
fs.writeFileSync(targetPath, `${JSON.stringify(value, null, 2)}\n`, "utf8");
|
||||
}
|
||||
|
||||
function hasRuntimeDefaultExport(sourcePath: string): boolean {
|
||||
const text = fs.readFileSync(sourcePath, "utf8");
|
||||
return /\bexport\s+default\b/u.test(text) || /\bas\s+default\b/u.test(text);
|
||||
}
|
||||
|
||||
function writeRuntimeModuleWrapper(sourcePath: string, targetPath: string): void {
|
||||
const specifier = path.relative(path.dirname(targetPath), sourcePath).replaceAll(path.sep, "/");
|
||||
const normalizedSpecifier = specifier.startsWith(".") ? specifier : `./${specifier}`;
|
||||
const defaultForwarder = hasRuntimeDefaultExport(sourcePath)
|
||||
? [
|
||||
`import defaultModule from ${JSON.stringify(normalizedSpecifier)};`,
|
||||
`let defaultExport = defaultModule;`,
|
||||
`for (let index = 0; index < 4 && defaultExport && typeof defaultExport === "object" && "default" in defaultExport; index += 1) {`,
|
||||
` defaultExport = defaultExport.default;`,
|
||||
`}`,
|
||||
]
|
||||
: [
|
||||
`import * as module from ${JSON.stringify(normalizedSpecifier)};`,
|
||||
`let defaultExport = "default" in module ? module.default : module;`,
|
||||
`for (let index = 0; index < 4 && defaultExport && typeof defaultExport === "object" && "default" in defaultExport; index += 1) {`,
|
||||
` defaultExport = defaultExport.default;`,
|
||||
`}`,
|
||||
];
|
||||
const content = [
|
||||
`export * from ${JSON.stringify(normalizedSpecifier)};`,
|
||||
...defaultForwarder,
|
||||
"export { defaultExport as default };",
|
||||
"",
|
||||
].join("\n");
|
||||
try {
|
||||
if (fs.readFileSync(targetPath, "utf8") === content) {
|
||||
return;
|
||||
}
|
||||
} catch {
|
||||
// Missing or unreadable wrapper; rewrite below.
|
||||
}
|
||||
fs.mkdirSync(path.dirname(targetPath), { recursive: true });
|
||||
fs.writeFileSync(targetPath, content, "utf8");
|
||||
}
|
||||
|
||||
function ensureOpenClawPluginSdkAlias(distRoot: string): void {
|
||||
const pluginSdkDir = path.join(distRoot, "plugin-sdk");
|
||||
if (!fs.existsSync(pluginSdkDir)) {
|
||||
return;
|
||||
}
|
||||
|
||||
const aliasDir = path.join(distRoot, "extensions", "node_modules", "openclaw");
|
||||
const pluginSdkAliasDir = path.join(aliasDir, "plugin-sdk");
|
||||
writeRuntimeJsonFile(path.join(aliasDir, "package.json"), {
|
||||
name: "openclaw",
|
||||
type: "module",
|
||||
exports: {
|
||||
"./plugin-sdk": "./plugin-sdk/index.js",
|
||||
"./plugin-sdk/*": "./plugin-sdk/*.js",
|
||||
},
|
||||
});
|
||||
try {
|
||||
if (fs.existsSync(pluginSdkAliasDir) && !fs.lstatSync(pluginSdkAliasDir).isDirectory()) {
|
||||
fs.rmSync(pluginSdkAliasDir, { recursive: true, force: true });
|
||||
}
|
||||
} catch {
|
||||
// Another process may be creating the alias at the same time; mkdir/write
|
||||
// below will either converge or surface the real filesystem error.
|
||||
}
|
||||
fs.mkdirSync(pluginSdkAliasDir, { recursive: true });
|
||||
for (const entry of fs.readdirSync(pluginSdkDir, { withFileTypes: true })) {
|
||||
if (!entry.isFile() || path.extname(entry.name) !== ".js") {
|
||||
continue;
|
||||
}
|
||||
writeRuntimeModuleWrapper(
|
||||
path.join(pluginSdkDir, entry.name),
|
||||
path.join(pluginSdkAliasDir, entry.name),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function remapBundledPluginRuntimePath(params: {
|
||||
source: string | undefined;
|
||||
pluginRoot: string;
|
||||
mirroredRoot: string;
|
||||
}): string | undefined {
|
||||
if (!params.source) {
|
||||
return undefined;
|
||||
}
|
||||
const relative = path.relative(params.pluginRoot, params.source);
|
||||
if (relative.startsWith("..") || path.isAbsolute(relative)) {
|
||||
return params.source;
|
||||
}
|
||||
return path.join(params.mirroredRoot, relative);
|
||||
}
|
||||
|
||||
export const __testing = {
|
||||
buildPluginLoaderJitiOptions,
|
||||
buildPluginLoaderAliasMap,
|
||||
@@ -2468,7 +2146,6 @@ export function loadOpenClawPlugins(options: PluginLoadOptions = {}): PluginRegi
|
||||
});
|
||||
|
||||
const seenIds = new Map<string, PluginRecord["origin"]>();
|
||||
const bundledRuntimeDepsRetainSpecsByInstallRoot = new Map<string, string[]>();
|
||||
const memorySlot = normalized.slots.memory;
|
||||
let selectedMemoryPluginId: string | null = null;
|
||||
let memorySlotMatched = false;
|
||||
@@ -2627,24 +2304,21 @@ export function loadOpenClawPlugins(options: PluginLoadOptions = {}): PluginRegi
|
||||
let runtimeDepsInstallStartedAt: number | null = null;
|
||||
let runtimeDepsInstallSpecs: string[] = [];
|
||||
try {
|
||||
const installRootPlan = resolveBundledRuntimeDependencyInstallRootPlan(pluginRoot, {
|
||||
env,
|
||||
});
|
||||
const installRoot = installRootPlan.installRoot;
|
||||
const retainSpecs = bundledRuntimeDepsRetainSpecsByInstallRoot.get(installRoot) ?? [];
|
||||
const depsInstallResult = ensureBundledPluginRuntimeDeps({
|
||||
const preparedRuntimeRoot = prepareBundledPluginRuntimeLoadRoot({
|
||||
pluginId: record.id,
|
||||
pluginRoot,
|
||||
modulePath: runtimeCandidateSource,
|
||||
...(runtimeSetupSource ? { setupModulePath: runtimeSetupSource } : {}),
|
||||
env,
|
||||
config: cfg,
|
||||
retainSpecs,
|
||||
registerRuntimeAliasRoot: registerBundledRuntimeDependencyJitiAliases,
|
||||
installDeps: (installParams) => {
|
||||
const installSpecs = installParams.installSpecs ?? installParams.missingSpecs;
|
||||
runtimeDepsInstallStartedAt = Date.now();
|
||||
runtimeDepsInstallSpecs = installParams.missingSpecs;
|
||||
runtimeDepsInstallSpecs = installSpecs;
|
||||
if (shouldActivate) {
|
||||
logger.info(
|
||||
`[plugins] ${record.id} staging bundled runtime deps (${installParams.missingSpecs.length} missing, ${installSpecs.length} install specs): ${installParams.missingSpecs.join(", ")}`,
|
||||
`[plugins] ${record.id} staging bundled runtime deps (${installSpecs.length} specs): ${installSpecs.join(", ")}`,
|
||||
);
|
||||
}
|
||||
const installer =
|
||||
@@ -2654,58 +2328,27 @@ export function loadOpenClawPlugins(options: PluginLoadOptions = {}): PluginRegi
|
||||
installRoot: params.installRoot,
|
||||
installExecutionRoot: params.installExecutionRoot,
|
||||
missingSpecs: params.installSpecs ?? params.missingSpecs,
|
||||
installSpecs: params.installSpecs,
|
||||
env,
|
||||
warn: (message) => logger.warn(`[plugins] ${record.id}: ${message}`),
|
||||
}));
|
||||
installer(installParams);
|
||||
},
|
||||
logInstalled: (installedSpecs) => {
|
||||
if (shouldActivate) {
|
||||
const elapsed =
|
||||
runtimeDepsInstallStartedAt === null
|
||||
? ""
|
||||
: ` in ${Date.now() - runtimeDepsInstallStartedAt}ms`;
|
||||
logger.info(
|
||||
`[plugins] ${record.id} installed bundled runtime deps${elapsed}: ${installedSpecs.join(", ")}`,
|
||||
);
|
||||
}
|
||||
},
|
||||
});
|
||||
if (depsInstallResult.installedSpecs.length > 0) {
|
||||
bundledRuntimeDepsRetainSpecsByInstallRoot.set(
|
||||
installRoot,
|
||||
[...new Set([...retainSpecs, ...depsInstallResult.retainSpecs])].toSorted(
|
||||
(left, right) => left.localeCompare(right),
|
||||
),
|
||||
);
|
||||
if (shouldActivate) {
|
||||
const elapsed =
|
||||
runtimeDepsInstallStartedAt === null
|
||||
? ""
|
||||
: ` in ${Date.now() - runtimeDepsInstallStartedAt}ms`;
|
||||
logger.info(
|
||||
`[plugins] ${record.id} installed bundled runtime deps${elapsed}: ${depsInstallResult.installedSpecs.join(", ")}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
if (path.resolve(installRoot) !== path.resolve(pluginRoot)) {
|
||||
const packageRoot = resolveBundledRuntimeDependencyPackageRoot(pluginRoot);
|
||||
if (packageRoot) {
|
||||
registerBundledRuntimeDependencyNodePath(packageRoot);
|
||||
registerBundledRuntimeDependencyJitiAliases(packageRoot);
|
||||
}
|
||||
for (const searchRoot of installRootPlan.searchRoots) {
|
||||
registerBundledRuntimeDependencyNodePath(searchRoot);
|
||||
registerBundledRuntimeDependencyJitiAliases(searchRoot);
|
||||
}
|
||||
runtimePluginRoot = mirrorBundledPluginRuntimeRoot({
|
||||
pluginId: record.id,
|
||||
pluginRoot,
|
||||
installRoot,
|
||||
});
|
||||
runtimeCandidateSource =
|
||||
remapBundledPluginRuntimePath({
|
||||
source: runtimeCandidateSource,
|
||||
pluginRoot,
|
||||
mirroredRoot: runtimePluginRoot,
|
||||
}) ?? runtimeCandidateSource;
|
||||
runtimeSetupSource = remapBundledPluginRuntimePath({
|
||||
source: runtimeSetupSource,
|
||||
pluginRoot,
|
||||
mirroredRoot: runtimePluginRoot,
|
||||
});
|
||||
} else {
|
||||
ensureOpenClawPluginSdkAlias(path.dirname(path.dirname(pluginRoot)));
|
||||
}
|
||||
runtimePluginRoot = preparedRuntimeRoot.pluginRoot;
|
||||
runtimeCandidateSource = preparedRuntimeRoot.modulePath;
|
||||
runtimeSetupSource = preparedRuntimeRoot.setupModulePath;
|
||||
} catch (error) {
|
||||
if (shouldActivate && runtimeDepsInstallStartedAt !== null) {
|
||||
logger.error(
|
||||
|
||||
@@ -101,6 +101,7 @@ describe("normalizeRegisteredProvider", () => {
|
||||
modelAllowlist: {
|
||||
allowedKeys: [" demo/model ", "demo/model"],
|
||||
initialSelections: [" demo/model "],
|
||||
loadCatalog: true,
|
||||
message: " Demo models ",
|
||||
},
|
||||
},
|
||||
@@ -140,6 +141,7 @@ describe("normalizeRegisteredProvider", () => {
|
||||
modelAllowlist: {
|
||||
allowedKeys: ["demo/model"],
|
||||
initialSelections: ["demo/model"],
|
||||
loadCatalog: true,
|
||||
message: "Demo models",
|
||||
},
|
||||
},
|
||||
|
||||
@@ -82,13 +82,15 @@ function buildNormalizedModelAllowlist(
|
||||
}
|
||||
const allowedKeys = normalizeTextList(modelAllowlist.allowedKeys);
|
||||
const initialSelections = normalizeTextList(modelAllowlist.initialSelections);
|
||||
const loadCatalog = modelAllowlist.loadCatalog === true;
|
||||
const message = normalizeOptionalString(modelAllowlist.message);
|
||||
if (!allowedKeys && !initialSelections && !message) {
|
||||
if (!allowedKeys && !initialSelections && !loadCatalog && !message) {
|
||||
return undefined;
|
||||
}
|
||||
return {
|
||||
...(allowedKeys ? { allowedKeys } : {}),
|
||||
...(initialSelections ? { initialSelections } : {}),
|
||||
...(loadCatalog ? { loadCatalog } : {}),
|
||||
...(message ? { message } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1101,6 +1101,7 @@ export type ProviderPluginWizardSetup = {
|
||||
modelAllowlist?: {
|
||||
allowedKeys?: string[];
|
||||
initialSelections?: string[];
|
||||
loadCatalog?: boolean;
|
||||
message?: string;
|
||||
};
|
||||
/**
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
collectAppcastSparkleVersionErrors,
|
||||
collectBundledExtensionManifestErrors,
|
||||
collectBundledPluginRootRuntimeMirrorErrors,
|
||||
collectDeclaredRootRuntimeDependencyMetadataErrors,
|
||||
collectForbiddenPackContentPaths,
|
||||
collectInstalledBundledPluginRuntimeDepErrors,
|
||||
bundledRuntimeDependencySentinelCandidates,
|
||||
@@ -262,6 +263,34 @@ describe("bundled plugin root runtime mirrors", () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it("flags mirrored root runtime metadata without root deps", () => {
|
||||
expect(
|
||||
collectDeclaredRootRuntimeDependencyMetadataErrors({
|
||||
dependencies: { semver: "7.7.4" },
|
||||
openclaw: {
|
||||
bundle: {
|
||||
mirroredRootRuntimeDependencies: ["json5", "semver"],
|
||||
},
|
||||
},
|
||||
}),
|
||||
).toEqual([
|
||||
"package.json openclaw.bundle.mirroredRootRuntimeDependencies declares 'json5' but package.json dependencies/optionalDependencies do not include it.",
|
||||
]);
|
||||
});
|
||||
|
||||
it("accepts mirrored root runtime metadata backed by root deps", () => {
|
||||
expect(
|
||||
collectDeclaredRootRuntimeDependencyMetadataErrors({
|
||||
dependencies: { json5: "^2.2.3", semver: "7.7.4" },
|
||||
openclaw: {
|
||||
bundle: {
|
||||
mirroredRootRuntimeDependencies: ["json5", "semver"],
|
||||
},
|
||||
},
|
||||
}),
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it("does not derive root mirrors for root chunks sourced from the owning plugin", () => {
|
||||
const tempRoot = mkdtempSync(join(tmpdir(), "openclaw-root-mirror-owned-"));
|
||||
|
||||
|
||||
Reference in New Issue
Block a user