From f4682742d9d16a58058492d5cb6d2d6e372b9cef Mon Sep 17 00:00:00 2001 From: Hunter Miller Date: Mon, 2 Mar 2026 18:23:42 -0600 Subject: [PATCH] feat: update tlon channel/plugin to be more fully featured (#21208) * feat(tlon): sync with openclaw-tlon master - Add tlon CLI tool registration with binary lookup - Add approval, media, settings, foreigns, story, upload modules - Add http-api wrapper for Urbit connection patching - Update types for defaultAuthorizedShips support - Fix type compatibility with core plugin SDK - Stub uploadFile (API not yet available in @tloncorp/api-beta) - Remove incompatible test files (security, sse-client, upload) * chore(tlon): remove dead code Remove unused Urbit channel client files: - channel-client.ts - channel-ops.ts - context.ts These were not imported anywhere in the extension. * feat(tlon): add image upload support via @tloncorp/api - Import configureClient and uploadFile from @tloncorp/api - Implement uploadImageFromUrl using uploadFile - Configure API client before media uploads - Update dependency to github:tloncorp/api-beta#main * fix(tlon): restore SSRF protection with event ack tracking - Restore context.ts and channel-ops.ts for SSRF support - Restore sse-client.ts with urbitFetch for SSRF-protected requests - Add event ack tracking from openclaw-tlon (acks every 20 events) - Pass ssrfPolicy through authenticate() and UrbitSSEClient - Fixes security regression from sync with openclaw-tlon * fix(tlon): restore buildTlonAccountFields for allowPrivateNetwork The inlined payload building was missing allowPrivateNetwork field, which would prevent the setting from being persisted to config. * fix(tlon): restore SSRF protection in probeAccount - Restore channel-client.ts for UrbitChannelClient - Use UrbitChannelClient with ssrfPolicy in probeAccount - Ensures account probe respects allowPrivateNetwork setting * feat(tlon): add ownerShip to setup flow ownerShip should always be set as it controls who receives approval requests and can approve/deny actions. * chore(tlon): remove unused http-api.ts After restoring SSRF protection, probeAccount uses UrbitChannelClient instead of @urbit/http-api. The http-api.ts wrapper is no longer needed. * refactor(tlon): simplify probeAccount to direct /~/name request No channel needed - just authenticate and GET /~/name. Removes UrbitChannelClient, keeping only UrbitSSEClient for monitor. * chore(tlon): add logging for event acks * chore(tlon): lower ack threshold to 5 for testing * fix(tlon): address security review issues - Fix SSRF in upload.ts: use urbitFetch with SSRF protection - Fix SSRF in media.ts: use urbitFetch with SSRF protection - Add command whitelist to tlon tool to prevent command injection - Add getDefaultSsrFPolicy() helper for uploads/downloads * fix(tlon): restore auth retry and add reauth on SSE reconnect - Add authenticateWithRetry() helper with exponential backoff (restores lost logic from #39) - Add onReconnect callback to re-authenticate when SSE stream reconnects - Add UrbitSSEClient.updateCookie() method for proper cookie normalization on reauth * fix(tlon): add infinite reconnect with reset after max attempts Instead of giving up after maxReconnectAttempts, wait 10 seconds then reset the counter and keep trying. This ensures the monitor never permanently disconnects due to temporary network issues. * test(tlon): restore security, sse-client, and upload tests - security.test.ts: DM allowlist, group invite, bot mention detection, ship normalization - sse-client.test.ts: subscription handling, cookie updates, reconnection params - upload.test.ts: image upload with SSRF protection, error handling * fix(tlon): restore DM partner ship extraction for proper routing - Add extractDmPartnerShip() to extract partner from 'whom' field - Use partner ship for routing (more reliable than essay.author) - Explicitly ignore bot's own outbound DM events - Log mismatch between author and partner for debugging * chore(tlon): restore ack threshold to 20 * chore(tlon): sync slash commands support from upstream - Add stripBotMention for proper CommandBody parsing - Add command authorization logic for owner-only slash commands - Add CommandAuthorized and CommandSource to context payload * fix(tlon): resolve TypeScript errors in tests and monitor - Store validated account url/code before closure to fix type narrowing - Fix test type annotations for mode rules - Add proper Response type cast in sse-client mock - Use optional chaining for init properties * docs(tlon): update docs for new config options and capabilities - Document ownerShip for approval system - Document autoAcceptDmInvites and autoAcceptGroupInvites - Update status to reflect rich text and image support - Add bundled skill section - Update notes with formatting and image details - Fix pnpm-lock.yaml conflict * docs(tlon): fix dmAllowlist description and improve allowPrivateNetwork docs - Correct dmAllowlist: empty means no DMs allowed (not allow all) - Promote allowPrivateNetwork to its own section with examples - Add warning about SSRF protection implications * docs(tlon): clarify ownerShip is auto-authorized everywhere - Add ownerShip to minimal config example (recommended) - Document that owner is automatically allowed for DMs and channels - No need to add owner to dmAllowlist or defaultAuthorizedShips * docs(tlon): add capabilities table, troubleshooting, and config reference Align with Matrix docs format: - Capabilities table for quick feature reference - Troubleshooting section with common failures - Configuration reference with all options * docs(tlon): fix reactions status and expand bundled skill section - Reactions ARE supported via bundled skill (not missing) - Add link to skill GitHub repo - List skill capabilities: contacts, channels, groups, DMs, reactions, settings * fix(tlon): use crypto.randomUUID instead of Math.random for channel ID Fixes security test failure - Math.random is flagged as weak randomness. * docs: fix markdown lint - add blank line before * fix: address PR review issues for tlon plugin - upload.ts: Use fetchWithSsrFGuard directly instead of urbitFetch to preserve full URL path when fetching external images; add release() call - media.ts: Same fix - use fetchWithSsrFGuard for external media downloads; add release() call to clean up resources - channel.ts: Use urbitFetch for poke API to maintain consistent SSRF protection (DNS pinning + redirect handling) - upload.test.ts: Update mocks to use fetchWithSsrFGuard instead of urbitFetch Addresses blocking issues from jalehman's review: 1. Fixed incorrect URL being fetched (validateUrbitBaseUrl was stripping path) 2. Fixed missing release() calls that could leak resources 3. Restored guarded fetch semantics for poke operations * docs: add tlon changelog fragment * style: format tlon monitor * fix: align tlon lockfile and sse id generation * docs: fix onboarding markdown list spacing --------- Co-authored-by: Josh Lehman --- changelog/fragments/pr-21208.md | 1 + docs/channels/tlon.md | 142 +- extensions/tlon/index.ts | 173 ++ extensions/tlon/openclaw.plugin.json | 1 + extensions/tlon/package.json | 5 +- extensions/tlon/src/account-fields.ts | 2 + extensions/tlon/src/channel.ts | 245 ++- extensions/tlon/src/config-schema.ts | 5 + extensions/tlon/src/monitor/approval.ts | 278 +++ extensions/tlon/src/monitor/discovery.ts | 70 +- extensions/tlon/src/monitor/history.ts | 111 +- extensions/tlon/src/monitor/index.ts | 1915 +++++++++++++++--- extensions/tlon/src/monitor/media.ts | 166 ++ extensions/tlon/src/monitor/utils.ts | 308 ++- extensions/tlon/src/security.test.ts | 438 ++++ extensions/tlon/src/settings.ts | 391 ++++ extensions/tlon/src/targets.ts | 6 +- extensions/tlon/src/types.ts | 34 + extensions/tlon/src/urbit/channel-client.ts | 158 -- extensions/tlon/src/urbit/context.ts | 9 + extensions/tlon/src/urbit/foreigns.ts | 49 + extensions/tlon/src/urbit/send.ts | 61 +- extensions/tlon/src/urbit/sse-client.test.ts | 213 +- extensions/tlon/src/urbit/sse-client.ts | 79 +- extensions/tlon/src/urbit/story.ts | 347 ++++ extensions/tlon/src/urbit/upload.test.ts | 188 ++ extensions/tlon/src/urbit/upload.ts | 60 + pnpm-lock.yaml | 430 ++++ 28 files changed, 5264 insertions(+), 621 deletions(-) create mode 100644 changelog/fragments/pr-21208.md create mode 100644 extensions/tlon/src/monitor/approval.ts create mode 100644 extensions/tlon/src/monitor/media.ts create mode 100644 extensions/tlon/src/security.test.ts create mode 100644 extensions/tlon/src/settings.ts delete mode 100644 extensions/tlon/src/urbit/channel-client.ts create mode 100644 extensions/tlon/src/urbit/foreigns.ts create mode 100644 extensions/tlon/src/urbit/story.ts create mode 100644 extensions/tlon/src/urbit/upload.test.ts create mode 100644 extensions/tlon/src/urbit/upload.ts diff --git a/changelog/fragments/pr-21208.md b/changelog/fragments/pr-21208.md new file mode 100644 index 00000000000..594b15c313b --- /dev/null +++ b/changelog/fragments/pr-21208.md @@ -0,0 +1 @@ +- Tlon plugin: sync upstream account/settings workflows, restore SSRF-safe media + SSE fetch paths, and improve invite/approval handling reliability. (#21208) (thanks @arthyn) diff --git a/docs/channels/tlon.md b/docs/channels/tlon.md index dbd2015c4ef..f3e70c7152a 100644 --- a/docs/channels/tlon.md +++ b/docs/channels/tlon.md @@ -11,8 +11,8 @@ Tlon is a decentralized messenger built on Urbit. OpenClaw connects to your Urbi respond to DMs and group chat messages. Group replies require an @ mention by default and can be further restricted via allowlists. -Status: supported via plugin. DMs, group mentions, thread replies, and text-only media fallback -(URL appended to caption). Reactions, polls, and native media uploads are not supported. +Status: supported via plugin. DMs, group mentions, thread replies, rich text formatting, and +image uploads are supported. Reactions and polls are not yet supported. ## Plugin required @@ -50,27 +50,38 @@ Minimal config (single account): ship: "~sampel-palnet", url: "https://your-ship-host", code: "lidlut-tabwed-pillex-ridrup", + ownerShip: "~your-main-ship", // recommended: your ship, always allowed }, }, } ``` -Private/LAN ship URLs (advanced): +## Private/LAN ships -By default, OpenClaw blocks private/internal hostnames and IP ranges for this plugin (SSRF hardening). -If your ship URL is on a private network (for example `http://192.168.1.50:8080` or `http://localhost:8080`), +By default, OpenClaw blocks private/internal hostnames and IP ranges for SSRF protection. +If your ship is running on a private network (localhost, LAN IP, or internal hostname), you must explicitly opt in: ```json5 { channels: { tlon: { + url: "http://localhost:8080", allowPrivateNetwork: true, }, }, } ``` +This applies to URLs like: + +- `http://localhost:8080` +- `http://192.168.x.x:8080` +- `http://my-ship.local:8080` + +⚠️ Only enable this if you trust your local network. This setting disables SSRF protections +for requests to your ship URL. + ## Group channels Auto-discovery is enabled by default. You can also pin channels manually: @@ -99,7 +110,7 @@ Disable auto-discovery: ## Access control -DM allowlist (empty = allow all): +DM allowlist (empty = no DMs allowed, use `ownerShip` for approval flow): ```json5 { @@ -134,6 +145,56 @@ Group authorization (restricted by default): } ``` +## Owner and approval system + +Set an owner ship to receive approval requests when unauthorized users try to interact: + +```json5 +{ + channels: { + tlon: { + ownerShip: "~your-main-ship", + }, + }, +} +``` + +The owner ship is **automatically authorized everywhere** — DM invites are auto-accepted and +channel messages are always allowed. You don't need to add the owner to `dmAllowlist` or +`defaultAuthorizedShips`. + +When set, the owner receives DM notifications for: + +- DM requests from ships not in the allowlist +- Mentions in channels without authorization +- Group invite requests + +## Auto-accept settings + +Auto-accept DM invites (for ships in dmAllowlist): + +```json5 +{ + channels: { + tlon: { + autoAcceptDmInvites: true, + }, + }, +} +``` + +Auto-accept group invites: + +```json5 +{ + channels: { + tlon: { + autoAcceptGroupInvites: true, + }, + }, +} +``` + ## Delivery targets (CLI/cron) Use these with `openclaw message send` or cron delivery: @@ -141,8 +202,75 @@ Use these with `openclaw message send` or cron delivery: - DM: `~sampel-palnet` or `dm/~sampel-palnet` - Group: `chat/~host-ship/channel` or `group:~host-ship/channel` +## Bundled skill + +The Tlon plugin includes a bundled skill ([`@tloncorp/tlon-skill`](https://github.com/tloncorp/tlon-skill)) +that provides CLI access to Tlon operations: + +- **Contacts**: get/update profiles, list contacts +- **Channels**: list, create, post messages, fetch history +- **Groups**: list, create, manage members +- **DMs**: send messages, react to messages +- **Reactions**: add/remove emoji reactions to posts and DMs +- **Settings**: manage plugin permissions via slash commands + +The skill is automatically available when the plugin is installed. + +## Capabilities + +| Feature | Status | +| --------------- | --------------------------------------- | +| Direct messages | ✅ Supported | +| Groups/channels | ✅ Supported (mention-gated by default) | +| Threads | ✅ Supported (auto-replies in thread) | +| Rich text | ✅ Markdown converted to Tlon format | +| Images | ✅ Uploaded to Tlon storage | +| Reactions | ✅ Via [bundled skill](#bundled-skill) | +| Polls | ❌ Not yet supported | +| Native commands | ✅ Supported (owner-only by default) | + +## Troubleshooting + +Run this ladder first: + +```bash +openclaw status +openclaw gateway status +openclaw logs --follow +openclaw doctor +``` + +Common failures: + +- **DMs ignored**: sender not in `dmAllowlist` and no `ownerShip` configured for approval flow. +- **Group messages ignored**: channel not discovered or sender not authorized. +- **Connection errors**: check ship URL is reachable; enable `allowPrivateNetwork` for local ships. +- **Auth errors**: verify login code is current (codes rotate). + +## Configuration reference + +Full configuration: [Configuration](/gateway/configuration) + +Provider options: + +- `channels.tlon.enabled`: enable/disable channel startup. +- `channels.tlon.ship`: bot's Urbit ship name (e.g. `~sampel-palnet`). +- `channels.tlon.url`: ship URL (e.g. `https://sampel-palnet.tlon.network`). +- `channels.tlon.code`: ship login code. +- `channels.tlon.allowPrivateNetwork`: allow localhost/LAN URLs (SSRF bypass). +- `channels.tlon.ownerShip`: owner ship for approval system (always authorized). +- `channels.tlon.dmAllowlist`: ships allowed to DM (empty = none). +- `channels.tlon.autoAcceptDmInvites`: auto-accept DMs from allowlisted ships. +- `channels.tlon.autoAcceptGroupInvites`: auto-accept all group invites. +- `channels.tlon.autoDiscoverChannels`: auto-discover group channels (default: true). +- `channels.tlon.groupChannels`: manually pinned channel nests. +- `channels.tlon.defaultAuthorizedShips`: ships authorized for all channels. +- `channels.tlon.authorization.channelRules`: per-channel auth rules. +- `channels.tlon.showModelSignature`: append model name to messages. + ## Notes - Group replies require a mention (e.g. `~your-bot-ship`) to respond. - Thread replies: if the inbound message is in a thread, OpenClaw replies in-thread. -- Media: `sendMedia` falls back to text + URL (no native upload). +- Rich text: Markdown formatting (bold, italic, code, headers, lists) is converted to Tlon's native format. +- Images: URLs are uploaded to Tlon storage and embedded as image blocks. diff --git a/extensions/tlon/index.ts b/extensions/tlon/index.ts index 2a31956dd39..1cbcd35bc4c 100644 --- a/extensions/tlon/index.ts +++ b/extensions/tlon/index.ts @@ -1,8 +1,128 @@ +import { spawn } from "node:child_process"; +import { existsSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; import type { OpenClawPluginApi } from "openclaw/plugin-sdk"; import { emptyPluginConfigSchema } from "openclaw/plugin-sdk"; import { tlonPlugin } from "./src/channel.js"; import { setTlonRuntime } from "./src/runtime.js"; +const __dirname = dirname(fileURLToPath(import.meta.url)); + +// Whitelist of allowed tlon subcommands +const ALLOWED_TLON_COMMANDS = new Set([ + "activity", + "channels", + "contacts", + "groups", + "messages", + "dms", + "posts", + "notebook", + "settings", + "help", + "version", +]); + +/** + * Find the tlon binary from the skill package + */ +function findTlonBinary(): string { + // Check in node_modules/.bin + const skillBin = join(__dirname, "node_modules", ".bin", "tlon"); + console.log(`[tlon] Checking for binary at: ${skillBin}, exists: ${existsSync(skillBin)}`); + if (existsSync(skillBin)) return skillBin; + + // Check for platform-specific binary directly + const platform = process.platform; + const arch = process.arch; + const platformPkg = `@tloncorp/tlon-skill-${platform}-${arch}`; + const platformBin = join(__dirname, "node_modules", platformPkg, "tlon"); + console.log( + `[tlon] Checking for platform binary at: ${platformBin}, exists: ${existsSync(platformBin)}`, + ); + if (existsSync(platformBin)) return platformBin; + + // Fallback to PATH + console.log(`[tlon] Falling back to PATH lookup for 'tlon'`); + return "tlon"; +} + +/** + * Shell-like argument splitter that respects quotes + */ +function shellSplit(str: string): string[] { + const args: string[] = []; + let cur = ""; + let inDouble = false; + let inSingle = false; + let escape = false; + + for (const ch of str) { + if (escape) { + cur += ch; + escape = false; + continue; + } + if (ch === "\\" && !inSingle) { + escape = true; + continue; + } + if (ch === '"' && !inSingle) { + inDouble = !inDouble; + continue; + } + if (ch === "'" && !inDouble) { + inSingle = !inSingle; + continue; + } + if (/\s/.test(ch) && !inDouble && !inSingle) { + if (cur) { + args.push(cur); + cur = ""; + } + continue; + } + cur += ch; + } + if (cur) args.push(cur); + return args; +} + +/** + * Run the tlon command and return the result + */ +function runTlonCommand(binary: string, args: string[]): Promise { + return new Promise((resolve, reject) => { + const child = spawn(binary, args, { + env: process.env, + }); + + let stdout = ""; + let stderr = ""; + + child.stdout.on("data", (data) => { + stdout += data.toString(); + }); + + child.stderr.on("data", (data) => { + stderr += data.toString(); + }); + + child.on("error", (err) => { + reject(new Error(`Failed to run tlon: ${err.message}`)); + }); + + child.on("close", (code) => { + if (code !== 0) { + reject(new Error(stderr || `tlon exited with code ${code}`)); + } else { + resolve(stdout); + } + }); + }); +} + const plugin = { id: "tlon", name: "Tlon", @@ -11,6 +131,59 @@ const plugin = { register(api: OpenClawPluginApi) { setTlonRuntime(api.runtime); api.registerChannel({ plugin: tlonPlugin }); + + // Register the tlon tool + const tlonBinary = findTlonBinary(); + api.logger.info(`[tlon] Registering tlon tool, binary: ${tlonBinary}`); + api.registerTool({ + name: "tlon", + label: "Tlon CLI", + description: + "Tlon/Urbit API operations: activity, channels, contacts, groups, messages, dms, posts, notebook, settings. " + + "Examples: 'activity mentions --limit 10', 'channels groups', 'contacts self', 'groups list'", + parameters: { + type: "object", + properties: { + command: { + type: "string", + description: + "The tlon command and arguments. " + + "Examples: 'activity mentions --limit 10', 'contacts get ~sampel-palnet', 'groups list'", + }, + }, + required: ["command"], + }, + async execute(_id: string, params: { command: string }) { + try { + const args = shellSplit(params.command); + + // Validate first argument is a whitelisted tlon subcommand + const subcommand = args[0]; + if (!ALLOWED_TLON_COMMANDS.has(subcommand)) { + return { + content: [ + { + type: "text" as const, + text: `Error: Unknown tlon subcommand '${subcommand}'. Allowed: ${[...ALLOWED_TLON_COMMANDS].join(", ")}`, + }, + ], + details: { error: true }, + }; + } + + const output = await runTlonCommand(tlonBinary, args); + return { + content: [{ type: "text" as const, text: output }], + details: undefined, + }; + } catch (error: any) { + return { + content: [{ type: "text" as const, text: `Error: ${error.message}` }], + details: { error: true }, + }; + } + }, + }); }, }; diff --git a/extensions/tlon/openclaw.plugin.json b/extensions/tlon/openclaw.plugin.json index aa4e78dfbb2..799cc0b184c 100644 --- a/extensions/tlon/openclaw.plugin.json +++ b/extensions/tlon/openclaw.plugin.json @@ -1,6 +1,7 @@ { "id": "tlon", "channels": ["tlon"], + "skills": ["node_modules/@tloncorp/tlon-skill"], "configSchema": { "type": "object", "additionalProperties": false, diff --git a/extensions/tlon/package.json b/extensions/tlon/package.json index 99c952536c9..fabadbb8b7c 100644 --- a/extensions/tlon/package.json +++ b/extensions/tlon/package.json @@ -4,7 +4,10 @@ "description": "OpenClaw Tlon/Urbit channel plugin", "type": "module", "dependencies": { - "@urbit/aura": "^3.0.0" + "@tloncorp/api": "github:tloncorp/api-beta#main", + "@tloncorp/tlon-skill": "0.1.9", + "@urbit/aura": "^3.0.0", + "@urbit/http-api": "^3.0.0" }, "openclaw": { "extensions": [ diff --git a/extensions/tlon/src/account-fields.ts b/extensions/tlon/src/account-fields.ts index 6eea0c58af1..cbddd1d37b3 100644 --- a/extensions/tlon/src/account-fields.ts +++ b/extensions/tlon/src/account-fields.ts @@ -6,6 +6,7 @@ export type TlonAccountFieldsInput = { groupChannels?: string[]; dmAllowlist?: string[]; autoDiscoverChannels?: boolean; + ownerShip?: string; }; export function buildTlonAccountFields(input: TlonAccountFieldsInput) { @@ -21,5 +22,6 @@ export function buildTlonAccountFields(input: TlonAccountFieldsInput) { ...(typeof input.autoDiscoverChannels === "boolean" ? { autoDiscoverChannels: input.autoDiscoverChannels } : {}), + ...(input.ownerShip ? { ownerShip: input.ownerShip } : {}), }; } diff --git a/extensions/tlon/src/channel.ts b/extensions/tlon/src/channel.ts index cc7f14ea3e5..3b2dd73f388 100644 --- a/extensions/tlon/src/channel.ts +++ b/extensions/tlon/src/channel.ts @@ -1,5 +1,6 @@ +import crypto from "node:crypto"; +import { configureClient } from "@tloncorp/api"; import type { - ChannelAccountSnapshot, ChannelOutboundAdapter, ChannelPlugin, ChannelSetupInput, @@ -17,9 +18,74 @@ import { tlonOnboardingAdapter } from "./onboarding.js"; import { formatTargetHint, normalizeShip, parseTlonTarget } from "./targets.js"; import { resolveTlonAccount, listTlonAccountIds } from "./types.js"; import { authenticate } from "./urbit/auth.js"; -import { UrbitChannelClient } from "./urbit/channel-client.js"; import { ssrfPolicyFromAllowPrivateNetwork } from "./urbit/context.js"; -import { buildMediaText, sendDm, sendGroupMessage } from "./urbit/send.js"; +import { urbitFetch } from "./urbit/fetch.js"; +import { + buildMediaStory, + sendDm, + sendGroupMessage, + sendDmWithStory, + sendGroupMessageWithStory, +} from "./urbit/send.js"; +import { uploadImageFromUrl } from "./urbit/upload.js"; + +// Simple HTTP-only poke that doesn't open an EventSource (avoids conflict with monitor's SSE) +async function createHttpPokeApi(params: { + url: string; + code: string; + ship: string; + allowPrivateNetwork?: boolean; +}) { + const ssrfPolicy = ssrfPolicyFromAllowPrivateNetwork(params.allowPrivateNetwork); + const cookie = await authenticate(params.url, params.code, { ssrfPolicy }); + const channelId = `${Math.floor(Date.now() / 1000)}-${crypto.randomUUID()}`; + const channelPath = `/~/channel/${channelId}`; + const shipName = params.ship.replace(/^~/, ""); + + return { + poke: async (pokeParams: { app: string; mark: string; json: unknown }) => { + const pokeId = Date.now(); + const pokeData = { + id: pokeId, + action: "poke", + ship: shipName, + app: pokeParams.app, + mark: pokeParams.mark, + json: pokeParams.json, + }; + + // Use urbitFetch for consistent SSRF protection (DNS pinning + redirect handling) + const { response, release } = await urbitFetch({ + baseUrl: params.url, + path: channelPath, + init: { + method: "PUT", + headers: { + "Content-Type": "application/json", + Cookie: cookie.split(";")[0], + }, + body: JSON.stringify([pokeData]), + }, + ssrfPolicy, + auditContext: "tlon-poke", + }); + + try { + if (!response.ok && response.status !== 204) { + const errorText = await response.text(); + throw new Error(`Poke failed: ${response.status} - ${errorText}`); + } + + return pokeId; + } finally { + await release(); + } + }, + delete: async () => { + // No-op for HTTP-only client + }, + }; +} const TLON_CHANNEL_ID = "tlon" as const; @@ -31,6 +97,7 @@ type TlonSetupInput = ChannelSetupInput & { groupChannels?: string[]; dmAllowlist?: string[]; autoDiscoverChannels?: boolean; + ownerShip?: string; }; function applyTlonSetupConfig(params: { @@ -97,7 +164,7 @@ const tlonOutbound: ChannelOutboundAdapter = { error: new Error(`Invalid Tlon target. Use ${formatTargetHint()}`), }; } - if (parsed.kind === "direct") { + if (parsed.kind === "dm") { return { ok: true, to: parsed.ship }; } return { ok: true, to: parsed.nest }; @@ -113,16 +180,17 @@ const tlonOutbound: ChannelOutboundAdapter = { throw new Error(`Invalid Tlon target. Use ${formatTargetHint()}`); } - const ssrfPolicy = ssrfPolicyFromAllowPrivateNetwork(account.allowPrivateNetwork); - const cookie = await authenticate(account.url, account.code, { ssrfPolicy }); - const api = new UrbitChannelClient(account.url, cookie, { - ship: account.ship.replace(/^~/, ""), - ssrfPolicy, + // Use HTTP-only poke (no EventSource) to avoid conflicts with monitor's SSE connection + const api = await createHttpPokeApi({ + url: account.url, + ship: account.ship, + code: account.code, + allowPrivateNetwork: account.allowPrivateNetwork ?? undefined, }); try { const fromShip = normalizeShip(account.ship); - if (parsed.kind === "direct") { + if (parsed.kind === "dm") { return await sendDm({ api, fromShip, @@ -140,19 +208,69 @@ const tlonOutbound: ChannelOutboundAdapter = { replyToId: replyId, }); } finally { - await api.close(); + try { + await api.delete(); + } catch { + // ignore cleanup errors + } } }, sendMedia: async ({ cfg, to, text, mediaUrl, accountId, replyToId, threadId }) => { - const mergedText = buildMediaText(text, mediaUrl); - return await tlonOutbound.sendText!({ - cfg, - to, - text: mergedText, - accountId, - replyToId, - threadId, + const account = resolveTlonAccount(cfg, accountId ?? undefined); + if (!account.configured || !account.ship || !account.url || !account.code) { + throw new Error("Tlon account not configured"); + } + + const parsed = parseTlonTarget(to); + if (!parsed) { + throw new Error(`Invalid Tlon target. Use ${formatTargetHint()}`); + } + + // Configure the API client for uploads + configureClient({ + shipUrl: account.url, + shipName: account.ship.replace(/^~/, ""), + verbose: false, + getCode: async () => account.code!, }); + + const uploadedUrl = mediaUrl ? await uploadImageFromUrl(mediaUrl) : undefined; + + const api = await createHttpPokeApi({ + url: account.url, + ship: account.ship, + code: account.code, + allowPrivateNetwork: account.allowPrivateNetwork ?? undefined, + }); + + try { + const fromShip = normalizeShip(account.ship); + const story = buildMediaStory(text, uploadedUrl); + + if (parsed.kind === "dm") { + return await sendDmWithStory({ + api, + fromShip, + toShip: parsed.ship, + story, + }); + } + const replyId = (replyToId ?? threadId) ? String(replyToId ?? threadId) : undefined; + return await sendGroupMessageWithStory({ + api, + fromShip, + hostShip: parsed.hostShip, + channelName: parsed.channelName, + story, + replyToId: replyId, + }); + } finally { + try { + await api.delete(); + } catch { + // ignore cleanup errors + } + } }, }; @@ -170,7 +288,7 @@ export const tlonPlugin: ChannelPlugin = { }, capabilities: { chatTypes: ["direct", "group", "thread"], - media: false, + media: true, reply: true, threads: true, }, @@ -189,7 +307,7 @@ export const tlonPlugin: ChannelPlugin = { channels: { ...cfg.channels, tlon: { - ...(cfg.channels?.tlon as Record), + ...cfg.channels?.tlon, enabled, }, }, @@ -200,7 +318,7 @@ export const tlonPlugin: ChannelPlugin = { channels: { ...cfg.channels, tlon: { - ...(cfg.channels?.tlon as Record), + ...cfg.channels?.tlon, accounts: { ...cfg.channels?.tlon?.accounts, [accountId]: { @@ -215,11 +333,13 @@ export const tlonPlugin: ChannelPlugin = { deleteAccount: ({ cfg, accountId }) => { const useDefault = !accountId || accountId === "default"; if (useDefault) { - // oxlint-disable-next-line no-unused-vars - const { ship, code, url, name, ...rest } = (cfg.channels?.tlon ?? {}) as Record< - string, - unknown - >; + const { + ship: _ship, + code: _code, + url: _url, + name: _name, + ...rest + } = cfg.channels?.tlon ?? {}; return { ...cfg, channels: { @@ -228,15 +348,13 @@ export const tlonPlugin: ChannelPlugin = { }, } as OpenClawConfig; } - // oxlint-disable-next-line no-unused-vars - const { [accountId]: removed, ...remainingAccounts } = (cfg.channels?.tlon?.accounts ?? - {}) as Record; + const { [accountId]: _removed, ...remainingAccounts } = cfg.channels?.tlon?.accounts ?? {}; return { ...cfg, channels: { ...cfg.channels, tlon: { - ...(cfg.channels?.tlon as Record), + ...cfg.channels?.tlon, accounts: remainingAccounts, }, }, @@ -291,7 +409,7 @@ export const tlonPlugin: ChannelPlugin = { if (!parsed) { return target.trim(); } - if (parsed.kind === "direct") { + if (parsed.kind === "dm") { return parsed.ship; } return parsed.nest; @@ -325,11 +443,14 @@ export const tlonPlugin: ChannelPlugin = { return []; }); }, - buildChannelSummary: ({ snapshot }) => ({ - configured: snapshot.configured ?? false, - ship: (snapshot as { ship?: string | null }).ship ?? null, - url: (snapshot as { url?: string | null }).url ?? null, - }), + buildChannelSummary: ({ snapshot }) => { + const s = snapshot as { configured?: boolean; ship?: string; url?: string }; + return { + configured: s.configured ?? false, + ship: s.ship ?? null, + url: s.url ?? null, + }; + }, probeAccount: async ({ account }) => { if (!account.configured || !account.ship || !account.url || !account.code) { return { ok: false, error: "Not configured" }; @@ -337,33 +458,47 @@ export const tlonPlugin: ChannelPlugin = { try { const ssrfPolicy = ssrfPolicyFromAllowPrivateNetwork(account.allowPrivateNetwork); const cookie = await authenticate(account.url, account.code, { ssrfPolicy }); - const api = new UrbitChannelClient(account.url, cookie, { - ship: account.ship.replace(/^~/, ""), + // Simple probe - just verify we can reach /~/name + const { response, release } = await urbitFetch({ + baseUrl: account.url, + path: "/~/name", + init: { + method: "GET", + headers: { Cookie: cookie }, + }, ssrfPolicy, + timeoutMs: 30_000, + auditContext: "tlon-probe-account", }); try { - await api.getOurName(); + if (!response.ok) { + return { ok: false, error: `Name request failed: ${response.status}` }; + } return { ok: true }; } finally { - await api.close(); + await release(); } } catch (error) { return { ok: false, error: (error as { message?: string })?.message ?? String(error) }; } }, - buildAccountSnapshot: ({ account, runtime, probe }) => ({ - accountId: account.accountId, - name: account.name, - enabled: account.enabled, - configured: account.configured, - ship: account.ship, - url: account.url, - running: runtime?.running ?? false, - lastStartAt: runtime?.lastStartAt ?? null, - lastStopAt: runtime?.lastStopAt ?? null, - lastError: runtime?.lastError ?? null, - probe, - }), + buildAccountSnapshot: ({ account, runtime, probe }) => { + // Tlon-specific snapshot with ship/url for status display + const snapshot = { + accountId: account.accountId, + name: account.name, + enabled: account.enabled, + configured: account.configured, + ship: account.ship, + url: account.url, + running: runtime?.running ?? false, + lastStartAt: runtime?.lastStartAt ?? null, + lastStopAt: runtime?.lastStopAt ?? null, + lastError: runtime?.lastError ?? null, + probe, + }; + return snapshot as import("openclaw/plugin-sdk").ChannelAccountSnapshot; + }, }, gateway: { startAccount: async (ctx) => { @@ -372,7 +507,7 @@ export const tlonPlugin: ChannelPlugin = { accountId: account.accountId, ship: account.ship, url: account.url, - } as ChannelAccountSnapshot); + } as import("openclaw/plugin-sdk").ChannelAccountSnapshot); ctx.log?.info(`[${account.accountId}] starting Tlon provider for ${account.ship ?? "tlon"}`); return monitorTlonProvider({ runtime: ctx.runtime, diff --git a/extensions/tlon/src/config-schema.ts b/extensions/tlon/src/config-schema.ts index ea80212088d..4a091c8f650 100644 --- a/extensions/tlon/src/config-schema.ts +++ b/extensions/tlon/src/config-schema.ts @@ -25,6 +25,11 @@ const tlonCommonConfigFields = { autoDiscoverChannels: z.boolean().optional(), showModelSignature: z.boolean().optional(), responsePrefix: z.string().optional(), + // Auto-accept settings + autoAcceptDmInvites: z.boolean().optional(), // Auto-accept DMs from ships in dmAllowlist + autoAcceptGroupInvites: z.boolean().optional(), // Auto-accept all group invites + // Owner ship for approval system + ownerShip: ShipSchema.optional(), // Ship that receives approval requests and can approve/deny } satisfies z.ZodRawShape; export const TlonAccountSchema = z.object({ diff --git a/extensions/tlon/src/monitor/approval.ts b/extensions/tlon/src/monitor/approval.ts new file mode 100644 index 00000000000..549be04a88a --- /dev/null +++ b/extensions/tlon/src/monitor/approval.ts @@ -0,0 +1,278 @@ +/** + * Approval system for managing DM, channel mention, and group invite approvals. + * + * When an unknown ship tries to interact with the bot, the owner receives + * a notification and can approve or deny the request. + */ + +import type { PendingApproval } from "../settings.js"; + +export type { PendingApproval }; + +export type ApprovalType = "dm" | "channel" | "group"; + +export type CreateApprovalParams = { + type: ApprovalType; + requestingShip: string; + channelNest?: string; + groupFlag?: string; + messagePreview?: string; + originalMessage?: { + messageId: string; + messageText: string; + messageContent: unknown; + timestamp: number; + parentId?: string; + isThreadReply?: boolean; + }; +}; + +/** + * Generate a unique approval ID in the format: {type}-{timestamp}-{shortHash} + */ +export function generateApprovalId(type: ApprovalType): string { + const timestamp = Date.now(); + const randomPart = Math.random().toString(36).substring(2, 6); + return `${type}-${timestamp}-${randomPart}`; +} + +/** + * Create a pending approval object. + */ +export function createPendingApproval(params: CreateApprovalParams): PendingApproval { + return { + id: generateApprovalId(params.type), + type: params.type, + requestingShip: params.requestingShip, + channelNest: params.channelNest, + groupFlag: params.groupFlag, + messagePreview: params.messagePreview, + originalMessage: params.originalMessage, + timestamp: Date.now(), + }; +} + +/** + * Truncate text to a maximum length with ellipsis. + */ +function truncate(text: string, maxLength: number): string { + if (text.length <= maxLength) { + return text; + } + return text.substring(0, maxLength - 3) + "..."; +} + +/** + * Format a notification message for the owner about a pending approval. + */ +export function formatApprovalRequest(approval: PendingApproval): string { + const preview = approval.messagePreview ? `\n"${truncate(approval.messagePreview, 100)}"` : ""; + + switch (approval.type) { + case "dm": + return ( + `New DM request from ${approval.requestingShip}:${preview}\n\n` + + `Reply "approve", "deny", or "block" (ID: ${approval.id})` + ); + + case "channel": + return ( + `${approval.requestingShip} mentioned you in ${approval.channelNest}:${preview}\n\n` + + `Reply "approve", "deny", or "block"\n` + + `(ID: ${approval.id})` + ); + + case "group": + return ( + `Group invite from ${approval.requestingShip} to join ${approval.groupFlag}\n\n` + + `Reply "approve", "deny", or "block"\n` + + `(ID: ${approval.id})` + ); + } +} + +export type ApprovalResponse = { + action: "approve" | "deny" | "block"; + id?: string; +}; + +/** + * Parse an owner's response to an approval request. + * Supports formats: + * - "approve" / "deny" / "block" (applies to most recent pending) + * - "approve dm-1234567890-abc" / "deny dm-1234567890-abc" (specific ID) + * - "block" permanently blocks the ship via Tlon's native blocking + */ +export function parseApprovalResponse(text: string): ApprovalResponse | null { + const trimmed = text.trim().toLowerCase(); + + // Match "approve", "deny", or "block" optionally followed by an ID + const match = trimmed.match(/^(approve|deny|block)(?:\s+(.+))?$/); + if (!match) { + return null; + } + + const action = match[1] as "approve" | "deny" | "block"; + const id = match[2]?.trim(); + + return { action, id }; +} + +/** + * Check if a message text looks like an approval response. + * Used to determine if we should intercept the message before normal processing. + */ +export function isApprovalResponse(text: string): boolean { + const trimmed = text.trim().toLowerCase(); + return trimmed.startsWith("approve") || trimmed.startsWith("deny") || trimmed.startsWith("block"); +} + +/** + * Find a pending approval by ID, or return the most recent if no ID specified. + */ +export function findPendingApproval( + pendingApprovals: PendingApproval[], + id?: string, +): PendingApproval | undefined { + if (id) { + return pendingApprovals.find((a) => a.id === id); + } + // Return most recent + return pendingApprovals[pendingApprovals.length - 1]; +} + +/** + * Check if there's already a pending approval for the same ship/channel/group combo. + * Used to avoid sending duplicate notifications. + */ +export function hasDuplicatePending( + pendingApprovals: PendingApproval[], + type: ApprovalType, + requestingShip: string, + channelNest?: string, + groupFlag?: string, +): boolean { + return pendingApprovals.some((approval) => { + if (approval.type !== type || approval.requestingShip !== requestingShip) { + return false; + } + if (type === "channel" && approval.channelNest !== channelNest) { + return false; + } + if (type === "group" && approval.groupFlag !== groupFlag) { + return false; + } + return true; + }); +} + +/** + * Remove a pending approval from the list by ID. + */ +export function removePendingApproval( + pendingApprovals: PendingApproval[], + id: string, +): PendingApproval[] { + return pendingApprovals.filter((a) => a.id !== id); +} + +/** + * Format a confirmation message after an approval action. + */ +export function formatApprovalConfirmation( + approval: PendingApproval, + action: "approve" | "deny" | "block", +): string { + if (action === "block") { + return `Blocked ${approval.requestingShip}. They will no longer be able to contact the bot.`; + } + + const actionText = action === "approve" ? "Approved" : "Denied"; + + switch (approval.type) { + case "dm": + if (action === "approve") { + return `${actionText} DM access for ${approval.requestingShip}. They can now message the bot.`; + } + return `${actionText} DM request from ${approval.requestingShip}.`; + + case "channel": + if (action === "approve") { + return `${actionText} ${approval.requestingShip} for ${approval.channelNest}. They can now interact in this channel.`; + } + return `${actionText} ${approval.requestingShip} for ${approval.channelNest}.`; + + case "group": + if (action === "approve") { + return `${actionText} group invite from ${approval.requestingShip} to ${approval.groupFlag}. Joining group...`; + } + return `${actionText} group invite from ${approval.requestingShip} to ${approval.groupFlag}.`; + } +} + +// ============================================================================ +// Admin Commands +// ============================================================================ + +export type AdminCommand = + | { type: "unblock"; ship: string } + | { type: "blocked" } + | { type: "pending" }; + +/** + * Parse an admin command from owner message. + * Supports: + * - "unblock ~ship" - unblock a specific ship + * - "blocked" - list all blocked ships + * - "pending" - list all pending approvals + */ +export function parseAdminCommand(text: string): AdminCommand | null { + const trimmed = text.trim().toLowerCase(); + + // "blocked" - list blocked ships + if (trimmed === "blocked") { + return { type: "blocked" }; + } + + // "pending" - list pending approvals + if (trimmed === "pending") { + return { type: "pending" }; + } + + // "unblock ~ship" - unblock a specific ship + const unblockMatch = trimmed.match(/^unblock\s+(~[\w-]+)$/); + if (unblockMatch) { + return { type: "unblock", ship: unblockMatch[1] }; + } + + return null; +} + +/** + * Check if a message text looks like an admin command. + */ +export function isAdminCommand(text: string): boolean { + return parseAdminCommand(text) !== null; +} + +/** + * Format the list of blocked ships for display to owner. + */ +export function formatBlockedList(ships: string[]): string { + if (ships.length === 0) { + return "No ships are currently blocked."; + } + return `Blocked ships (${ships.length}):\n${ships.map((s) => `• ${s}`).join("\n")}`; +} + +/** + * Format the list of pending approvals for display to owner. + */ +export function formatPendingList(approvals: PendingApproval[]): string { + if (approvals.length === 0) { + return "No pending approval requests."; + } + return `Pending approvals (${approvals.length}):\n${approvals + .map((a) => `• ${a.id}: ${a.type} from ${a.requestingShip}`) + .join("\n")}`; +} diff --git a/extensions/tlon/src/monitor/discovery.ts b/extensions/tlon/src/monitor/discovery.ts index cc7f5d6b213..cce767ea4db 100644 --- a/extensions/tlon/src/monitor/discovery.ts +++ b/extensions/tlon/src/monitor/discovery.ts @@ -1,4 +1,5 @@ import type { RuntimeEnv } from "openclaw/plugin-sdk"; +import type { Foreigns } from "../urbit/foreigns.js"; import { formatChangesDate } from "./utils.js"; export async function fetchGroupChanges( @@ -15,34 +16,33 @@ export async function fetchGroupChanges( return changes; } return null; - } catch (error) { + } catch (error: any) { runtime.log?.( - `[tlon] Failed to fetch changes (falling back to full init): ${(error as { message?: string })?.message ?? String(error)}`, + `[tlon] Failed to fetch changes (falling back to full init): ${error?.message ?? String(error)}`, ); return null; } } -export async function fetchAllChannels( +export interface InitData { + channels: string[]; + foreigns: Foreigns | null; +} + +/** + * Fetch groups-ui init data, returning channels and foreigns. + * This is a single scry that provides both channel discovery and pending invites. + */ +export async function fetchInitData( api: { scry: (path: string) => Promise }, runtime: RuntimeEnv, -): Promise { +): Promise { try { - runtime.log?.("[tlon] Attempting auto-discovery of group channels..."); - const changes = await fetchGroupChanges(api, runtime, 5); - - // oxlint-disable-next-line typescript/no-explicit-any - let initData: any; - if (changes) { - runtime.log?.("[tlon] Changes data received, using full init for channel extraction"); - initData = await api.scry("/groups-ui/v6/init.json"); - } else { - initData = await api.scry("/groups-ui/v6/init.json"); - } + runtime.log?.("[tlon] Fetching groups-ui init data..."); + const initData = (await api.scry("/groups-ui/v6/init.json")) as any; const channels: string[] = []; - if (initData && initData.groups) { - // oxlint-disable-next-line typescript/no-explicit-any + if (initData?.groups) { for (const groupData of Object.values(initData.groups as Record)) { if (groupData && typeof groupData === "object" && groupData.channels) { for (const channelNest of Object.keys(groupData.channels)) { @@ -56,23 +56,31 @@ export async function fetchAllChannels( if (channels.length > 0) { runtime.log?.(`[tlon] Auto-discovered ${channels.length} chat channel(s)`); - runtime.log?.( - `[tlon] Channels: ${channels.slice(0, 5).join(", ")}${channels.length > 5 ? "..." : ""}`, - ); } else { runtime.log?.("[tlon] No chat channels found via auto-discovery"); - runtime.log?.("[tlon] Add channels manually to config: channels.tlon.groupChannels"); } - return channels; - } catch (error) { - runtime.log?.( - `[tlon] Auto-discovery failed: ${(error as { message?: string })?.message ?? String(error)}`, - ); - runtime.log?.( - "[tlon] To monitor group channels, add them to config: channels.tlon.groupChannels", - ); - runtime.log?.('[tlon] Example: ["chat/~host-ship/channel-name"]'); - return []; + const foreigns = (initData?.foreigns as Foreigns) || null; + if (foreigns) { + const pendingCount = Object.values(foreigns).filter((f) => + f.invites?.some((i) => i.valid), + ).length; + if (pendingCount > 0) { + runtime.log?.(`[tlon] Found ${pendingCount} pending group invite(s)`); + } + } + + return { channels, foreigns }; + } catch (error: any) { + runtime.log?.(`[tlon] Init data fetch failed: ${error?.message ?? String(error)}`); + return { channels: [], foreigns: null }; } } + +export async function fetchAllChannels( + api: { scry: (path: string) => Promise }, + runtime: RuntimeEnv, +): Promise { + const { channels } = await fetchInitData(api, runtime); + return channels; +} diff --git a/extensions/tlon/src/monitor/history.ts b/extensions/tlon/src/monitor/history.ts index 03360a12a6d..3674b175b3c 100644 --- a/extensions/tlon/src/monitor/history.ts +++ b/extensions/tlon/src/monitor/history.ts @@ -1,6 +1,25 @@ import type { RuntimeEnv } from "openclaw/plugin-sdk"; import { extractMessageText } from "./utils.js"; +/** + * Format a number as @ud (with dots every 3 digits from the right) + * e.g., 170141184507799509469114119040828178432 -> 170.141.184.507.799.509.469.114.119.040.828.178.432 + */ +function formatUd(id: string | number): string { + const str = String(id).replace(/\./g, ""); // Remove any existing dots + const reversed = str.split("").toReversed(); + const chunks: string[] = []; + for (let i = 0; i < reversed.length; i += 3) { + chunks.push( + reversed + .slice(i, i + 3) + .toReversed() + .join(""), + ); + } + return chunks.toReversed().join("."); +} + export type TlonHistoryEntry = { author: string; content: string; @@ -35,13 +54,11 @@ export async function fetchChannelHistory( const scryPath = `/channels/v4/${channelNest}/posts/newest/${count}/outline.json`; runtime?.log?.(`[tlon] Fetching history: ${scryPath}`); - // oxlint-disable-next-line typescript/no-explicit-any const data: any = await api.scry(scryPath); if (!data) { return []; } - // oxlint-disable-next-line typescript/no-explicit-any let posts: any[] = []; if (Array.isArray(data)) { posts = data; @@ -67,10 +84,8 @@ export async function fetchChannelHistory( runtime?.log?.(`[tlon] Extracted ${messages.length} messages from history`); return messages; - } catch (error) { - runtime?.log?.( - `[tlon] Error fetching channel history: ${(error as { message?: string })?.message ?? String(error)}`, - ); + } catch (error: any) { + runtime?.log?.(`[tlon] Error fetching channel history: ${error?.message ?? String(error)}`); return []; } } @@ -90,3 +105,87 @@ export async function getChannelHistory( runtime?.log?.(`[tlon] Cache has ${cache.length} messages, need ${count}, fetching from scry...`); return await fetchChannelHistory(api, channelNest, count, runtime); } + +/** + * Fetch thread/reply history for a specific parent post. + * Used to get context when entering a thread conversation. + */ +export async function fetchThreadHistory( + api: { scry: (path: string) => Promise }, + channelNest: string, + parentId: string, + count = 50, + runtime?: RuntimeEnv, +): Promise { + try { + // Tlon API: fetch replies to a specific post + // Format: /channels/v4/{nest}/posts/post/{parentId}/replies/newest/{count}.json + // parentId needs @ud formatting (dots every 3 digits) + const formattedParentId = formatUd(parentId); + runtime?.log?.( + `[tlon] Thread history - parentId: ${parentId} -> formatted: ${formattedParentId}`, + ); + + const scryPath = `/channels/v4/${channelNest}/posts/post/id/${formattedParentId}/replies/newest/${count}.json`; + runtime?.log?.(`[tlon] Fetching thread history: ${scryPath}`); + + const data: any = await api.scry(scryPath); + if (!data) { + runtime?.log?.(`[tlon] No thread history data returned`); + return []; + } + + let replies: any[] = []; + if (Array.isArray(data)) { + replies = data; + } else if (data.replies && Array.isArray(data.replies)) { + replies = data.replies; + } else if (typeof data === "object") { + replies = Object.values(data); + } + + const messages = replies + .map((item) => { + // Thread replies use 'memo' structure + const memo = item.memo || item["r-reply"]?.set?.memo || item; + const seal = item.seal || item["r-reply"]?.set?.seal; + + return { + author: memo?.author || "unknown", + content: extractMessageText(memo?.content || []), + timestamp: memo?.sent || Date.now(), + id: seal?.id || item.id, + } as TlonHistoryEntry; + }) + .filter((msg) => msg.content); + + runtime?.log?.(`[tlon] Extracted ${messages.length} thread replies from history`); + return messages; + } catch (error: any) { + runtime?.log?.(`[tlon] Error fetching thread history: ${error?.message ?? String(error)}`); + // Fall back to trying alternate path structure + try { + const altPath = `/channels/v4/${channelNest}/posts/post/id/${formatUd(parentId)}.json`; + runtime?.log?.(`[tlon] Trying alternate path: ${altPath}`); + const data: any = await api.scry(altPath); + + if (data?.seal?.meta?.replyCount > 0 && data?.replies) { + const replies = Array.isArray(data.replies) ? data.replies : Object.values(data.replies); + const messages = replies + .map((reply: any) => ({ + author: reply.memo?.author || "unknown", + content: extractMessageText(reply.memo?.content || []), + timestamp: reply.memo?.sent || Date.now(), + id: reply.seal?.id, + })) + .filter((msg: TlonHistoryEntry) => msg.content); + + runtime?.log?.(`[tlon] Extracted ${messages.length} replies from post data`); + return messages; + } + } catch (altError: any) { + runtime?.log?.(`[tlon] Alternate path also failed: ${altError?.message ?? String(altError)}`); + } + return []; + } +} diff --git a/extensions/tlon/src/monitor/index.ts b/extensions/tlon/src/monitor/index.ts index 7d2e8dbd31f..b3a0e092970 100644 --- a/extensions/tlon/src/monitor/index.ts +++ b/extensions/tlon/src/monitor/index.ts @@ -1,28 +1,44 @@ import type { RuntimeEnv, ReplyPayload, OpenClawConfig } from "openclaw/plugin-sdk"; import { createLoggerBackedRuntime, createReplyPrefixOptions } from "openclaw/plugin-sdk"; import { getTlonRuntime } from "../runtime.js"; +import { createSettingsManager, type TlonSettingsStore } from "../settings.js"; import { normalizeShip, parseChannelNest } from "../targets.js"; import { resolveTlonAccount } from "../types.js"; import { authenticate } from "../urbit/auth.js"; import { ssrfPolicyFromAllowPrivateNetwork } from "../urbit/context.js"; +import type { Foreigns, DmInvite } from "../urbit/foreigns.js"; import { sendDm, sendGroupMessage } from "../urbit/send.js"; import { UrbitSSEClient } from "../urbit/sse-client.js"; -import { fetchAllChannels } from "./discovery.js"; -import { cacheMessage, getChannelHistory } from "./history.js"; +import { + type PendingApproval, + type AdminCommand, + createPendingApproval, + formatApprovalRequest, + formatApprovalConfirmation, + parseApprovalResponse, + isApprovalResponse, + findPendingApproval, + removePendingApproval, + parseAdminCommand, + isAdminCommand, + formatBlockedList, + formatPendingList, +} from "./approval.js"; +import { fetchAllChannels, fetchInitData } from "./discovery.js"; +import { cacheMessage, getChannelHistory, fetchThreadHistory } from "./history.js"; +import { downloadMessageImages } from "./media.js"; import { createProcessedMessageTracker } from "./processed-messages.js"; import { extractMessageText, + extractCites, formatModelName, isBotMentioned, + stripBotMention, isDmAllowed, isSummarizationRequest, + type ParsedCite, } from "./utils.js"; -function formatError(err: unknown): string { - if (err instanceof Error) return err.message; - return String(err); -} - export type MonitorTlonOpts = { runtime?: RuntimeEnv; abortSignal?: AbortSignal; @@ -34,37 +50,14 @@ type ChannelAuthorization = { allowedShips?: string[]; }; -type UrbitMemo = { - author?: string; - content?: unknown; - sent?: number; -}; - -type UrbitSeal = { - "parent-id"?: string; - parent?: string; -}; - -type UrbitUpdate = { - id?: string | number; - response?: { - add?: { memo?: UrbitMemo }; - post?: { - id?: string | number; - "r-post"?: { - set?: { essay?: UrbitMemo; seal?: UrbitSeal }; - reply?: { - id?: string | number; - "r-reply"?: { set?: { memo?: UrbitMemo; seal?: UrbitSeal } }; - }; - }; - }; - }; -}; - +/** + * Resolve channel authorization by merging file config with settings store. + * Settings store takes precedence for fields it defines. + */ function resolveChannelAuthorization( cfg: OpenClawConfig, channelNest: string, + settings?: TlonSettingsStore, ): { mode: "restricted" | "open"; allowedShips: string[] } { const tlonConfig = cfg.channels?.tlon as | { @@ -72,16 +65,23 @@ function resolveChannelAuthorization( defaultAuthorizedShips?: string[]; } | undefined; - const rules = tlonConfig?.authorization?.channelRules ?? {}; - const rule = rules[channelNest]; - const allowedShips = rule?.allowedShips ?? tlonConfig?.defaultAuthorizedShips ?? []; + + // Merge channel rules: settings override file config + const fileRules = tlonConfig?.authorization?.channelRules ?? {}; + const settingsRules = settings?.channelRules ?? {}; + const rule = settingsRules[channelNest] ?? fileRules[channelNest]; + + // Merge default authorized ships: settings override file config + const defaultShips = settings?.defaultAuthorizedShips ?? tlonConfig?.defaultAuthorizedShips ?? []; + + const allowedShips = rule?.allowedShips ?? defaultShips; const mode = rule?.mode ?? "restricted"; return { mode, allowedShips }; } export async function monitorTlonProvider(opts: MonitorTlonOpts = {}): Promise { const core = getTlonRuntime(); - const cfg = core.config.loadConfig(); + const cfg = core.config.loadConfig() as OpenClawConfig; if (cfg.channels?.tlon?.enabled === false) { return; } @@ -104,41 +104,274 @@ export async function monitorTlonProvider(opts: MonitorTlonOpts = {}): Promise runtime.log?.(message), - error: (message) => runtime.error?.(message), - }, - }); - } catch (error) { - runtime.error?.(`[tlon] Failed to authenticate: ${formatError(error)}`); - throw error; - } + const ssrfPolicy = ssrfPolicyFromAllowPrivateNetwork(account.allowPrivateNetwork); - const processedTracker = createProcessedMessageTracker(2000); - let groupChannels: string[] = []; + // Store validated values for use in closures (TypeScript narrowing doesn't propagate) + const accountUrl = account.url; + const accountCode = account.code; - if (account.autoDiscoverChannels !== false) { - try { - const discoveredChannels = await fetchAllChannels(api, runtime); - if (discoveredChannels.length > 0) { - groupChannels = discoveredChannels; + // Helper to authenticate with retry logic + async function authenticateWithRetry(maxAttempts = 10): Promise { + for (let attempt = 1; ; attempt++) { + if (opts.abortSignal?.aborted) { + throw new Error("Aborted while waiting to authenticate"); + } + try { + runtime.log?.(`[tlon] Attempting authentication to ${accountUrl}...`); + return await authenticate(accountUrl, accountCode, { ssrfPolicy }); + } catch (error: any) { + runtime.error?.( + `[tlon] Failed to authenticate (attempt ${attempt}): ${error?.message ?? String(error)}`, + ); + if (attempt >= maxAttempts) { + throw error; + } + const delay = Math.min(30000, 1000 * Math.pow(2, attempt - 1)); + runtime.log?.(`[tlon] Retrying authentication in ${delay}ms...`); + await new Promise((resolve, reject) => { + const timer = setTimeout(resolve, delay); + if (opts.abortSignal) { + const onAbort = () => { + clearTimeout(timer); + reject(new Error("Aborted")); + }; + opts.abortSignal.addEventListener("abort", onAbort, { once: true }); + } + }); } - } catch (error) { - runtime.error?.(`[tlon] Auto-discovery failed: ${formatError(error)}`); } } - if (groupChannels.length === 0 && account.groupChannels.length > 0) { - groupChannels = account.groupChannels; - runtime.log?.(`[tlon] Using manual groupChannels config: ${groupChannels.join(", ")}`); + let api: UrbitSSEClient | null = null; + const cookie = await authenticateWithRetry(); + api = new UrbitSSEClient(account.url, cookie, { + ship: botShipName, + ssrfPolicy, + logger: { + log: (message) => runtime.log?.(message), + error: (message) => runtime.error?.(message), + }, + // Re-authenticate on reconnect in case the session expired + onReconnect: async (client) => { + runtime.log?.("[tlon] Re-authenticating on SSE reconnect..."); + const newCookie = await authenticateWithRetry(5); + client.updateCookie(newCookie); + runtime.log?.("[tlon] Re-authentication successful"); + }, + }); + + const processedTracker = createProcessedMessageTracker(2000); + let groupChannels: string[] = []; + let botNickname: string | null = null; + + // Settings store manager for hot-reloading config + const settingsManager = createSettingsManager(api, { + log: (msg) => runtime.log?.(msg), + error: (msg) => runtime.error?.(msg), + }); + + // Reactive state that can be updated via settings store + let effectiveDmAllowlist: string[] = account.dmAllowlist; + let effectiveShowModelSig: boolean = account.showModelSignature ?? false; + let effectiveAutoAcceptDmInvites: boolean = account.autoAcceptDmInvites ?? false; + let effectiveAutoAcceptGroupInvites: boolean = account.autoAcceptGroupInvites ?? false; + let effectiveGroupInviteAllowlist: string[] = account.groupInviteAllowlist; + let effectiveAutoDiscoverChannels: boolean = account.autoDiscoverChannels ?? false; + let effectiveOwnerShip: string | null = account.ownerShip + ? normalizeShip(account.ownerShip) + : null; + let pendingApprovals: PendingApproval[] = []; + let currentSettings: TlonSettingsStore = {}; + + // Track threads we've participated in (by parentId) - respond without mention requirement + const participatedThreads = new Set(); + + // Track DM senders per session to detect shared sessions (security warning) + const dmSendersBySession = new Map>(); + let sharedSessionWarningSent = false; + + // Fetch bot's nickname from contacts + try { + const selfProfile = await api.scry("/contacts/v1/self.json"); + if (selfProfile && typeof selfProfile === "object") { + const profile = selfProfile as { nickname?: { value?: string } }; + botNickname = profile.nickname?.value || null; + if (botNickname) { + runtime.log?.(`[tlon] Bot nickname: ${botNickname}`); + } + } + } catch (error: any) { + runtime.log?.(`[tlon] Could not fetch nickname: ${error?.message ?? String(error)}`); + } + + // Store init foreigns for processing after settings are loaded + let initForeigns: Foreigns | null = null; + + // Migrate file config to settings store (seed on first run) + async function migrateConfigToSettings() { + const migrations: Array<{ key: string; fileValue: unknown; settingsValue: unknown }> = [ + { + key: "dmAllowlist", + fileValue: account.dmAllowlist, + settingsValue: currentSettings.dmAllowlist, + }, + { + key: "groupInviteAllowlist", + fileValue: account.groupInviteAllowlist, + settingsValue: currentSettings.groupInviteAllowlist, + }, + { + key: "groupChannels", + fileValue: account.groupChannels, + settingsValue: currentSettings.groupChannels, + }, + { + key: "defaultAuthorizedShips", + fileValue: account.defaultAuthorizedShips, + settingsValue: currentSettings.defaultAuthorizedShips, + }, + { + key: "autoDiscoverChannels", + fileValue: account.autoDiscoverChannels, + settingsValue: currentSettings.autoDiscoverChannels, + }, + { + key: "autoAcceptDmInvites", + fileValue: account.autoAcceptDmInvites, + settingsValue: currentSettings.autoAcceptDmInvites, + }, + { + key: "autoAcceptGroupInvites", + fileValue: account.autoAcceptGroupInvites, + settingsValue: currentSettings.autoAcceptGroupInvites, + }, + { + key: "showModelSig", + fileValue: account.showModelSignature, + settingsValue: currentSettings.showModelSig, + }, + ]; + + for (const { key, fileValue, settingsValue } of migrations) { + // Only migrate if file has a value and settings store doesn't + const hasFileValue = Array.isArray(fileValue) ? fileValue.length > 0 : fileValue != null; + const hasSettingsValue = Array.isArray(settingsValue) + ? settingsValue.length > 0 + : settingsValue != null; + + if (hasFileValue && !hasSettingsValue) { + try { + await api!.poke({ + app: "settings", + mark: "settings-event", + json: { + "put-entry": { + "bucket-key": "tlon", + "entry-key": key, + value: fileValue, + desk: "moltbot", + }, + }, + }); + runtime.log?.(`[tlon] Migrated ${key} from config to settings store`); + } catch (err) { + runtime.log?.(`[tlon] Failed to migrate ${key}: ${String(err)}`); + } + } + } + } + + // Load settings from settings store (hot-reloadable config) + try { + currentSettings = await settingsManager.load(); + + // Migrate file config to settings store if not already present + await migrateConfigToSettings(); + + // Apply settings overrides + // Note: groupChannels from settings store are merged AFTER discovery runs (below) + if (currentSettings.defaultAuthorizedShips?.length) { + runtime.log?.( + `[tlon] Using defaultAuthorizedShips from settings store: ${currentSettings.defaultAuthorizedShips.join(", ")}`, + ); + } + if (currentSettings.autoDiscoverChannels !== undefined) { + effectiveAutoDiscoverChannels = currentSettings.autoDiscoverChannels; + runtime.log?.( + `[tlon] Using autoDiscoverChannels from settings store: ${effectiveAutoDiscoverChannels}`, + ); + } + if (currentSettings.dmAllowlist?.length) { + effectiveDmAllowlist = currentSettings.dmAllowlist; + runtime.log?.( + `[tlon] Using dmAllowlist from settings store: ${effectiveDmAllowlist.join(", ")}`, + ); + } + if (currentSettings.showModelSig !== undefined) { + effectiveShowModelSig = currentSettings.showModelSig; + } + if (currentSettings.autoAcceptDmInvites !== undefined) { + effectiveAutoAcceptDmInvites = currentSettings.autoAcceptDmInvites; + runtime.log?.( + `[tlon] Using autoAcceptDmInvites from settings store: ${effectiveAutoAcceptDmInvites}`, + ); + } + if (currentSettings.autoAcceptGroupInvites !== undefined) { + effectiveAutoAcceptGroupInvites = currentSettings.autoAcceptGroupInvites; + runtime.log?.( + `[tlon] Using autoAcceptGroupInvites from settings store: ${effectiveAutoAcceptGroupInvites}`, + ); + } + if (currentSettings.groupInviteAllowlist?.length) { + effectiveGroupInviteAllowlist = currentSettings.groupInviteAllowlist; + runtime.log?.( + `[tlon] Using groupInviteAllowlist from settings store: ${effectiveGroupInviteAllowlist.join(", ")}`, + ); + } + if (currentSettings.ownerShip) { + effectiveOwnerShip = normalizeShip(currentSettings.ownerShip); + runtime.log?.(`[tlon] Using ownerShip from settings store: ${effectiveOwnerShip}`); + } + if (currentSettings.pendingApprovals?.length) { + pendingApprovals = currentSettings.pendingApprovals; + runtime.log?.(`[tlon] Loaded ${pendingApprovals.length} pending approval(s) from settings`); + } + } catch (err) { + runtime.log?.(`[tlon] Settings store not available, using file config: ${String(err)}`); + } + + // Run channel discovery AFTER settings are loaded (so settings store value is used) + if (effectiveAutoDiscoverChannels) { + try { + const initData = await fetchInitData(api, runtime); + if (initData.channels.length > 0) { + groupChannels = initData.channels; + } + initForeigns = initData.foreigns; + } catch (error: any) { + runtime.error?.(`[tlon] Auto-discovery failed: ${error?.message ?? String(error)}`); + } + } + + // Merge manual config with auto-discovered channels + if (account.groupChannels.length > 0) { + for (const ch of account.groupChannels) { + if (!groupChannels.includes(ch)) { + groupChannels.push(ch); + } + } + runtime.log?.( + `[tlon] Added ${account.groupChannels.length} manual groupChannels to monitoring`, + ); + } + + // Also merge settings store groupChannels (may have been set via tlon settings command) + if (currentSettings.groupChannels?.length) { + for (const ch of currentSettings.groupChannels) { + if (!groupChannels.includes(ch)) { + groupChannels.push(ch); + } + } } if (groupChannels.length > 0) { @@ -149,142 +382,502 @@ export async function monitorTlonProvider(opts: MonitorTlonOpts = {}): Promise { - try { - const memo = update?.response?.add?.memo; - if (!memo) { - return; - } - - const messageId = update.id != null ? String(update.id) : undefined; - if (!processedTracker.mark(messageId)) { - return; - } - - const senderShip = normalizeShip(memo.author ?? ""); - if (!senderShip || senderShip === botShipName) { - return; - } - - const messageText = extractMessageText(memo.content); - if (!messageText) { - return; - } - - if (!isDmAllowed(senderShip, account.dmAllowlist)) { - runtime.log?.(`[tlon] Blocked DM from ${senderShip}: not in allowlist`); - return; - } - - await processMessage({ - messageId: messageId ?? "", - senderShip, - messageText, - isGroup: false, - timestamp: memo.sent || Date.now(), - }); - } catch (error) { - runtime.error?.(`[tlon] Error handling DM: ${formatError(error)}`); + // Helper to resolve cited message content + async function resolveCiteContent(cite: ParsedCite): Promise { + if (cite.type !== "chan" || !cite.nest || !cite.postId) { + return null; } - }; - const handleIncomingGroupMessage = (channelNest: string) => async (update: UrbitUpdate) => { try { - const parsed = parseChannelNest(channelNest); - if (!parsed) { - return; + // Scry for the specific post: /v4/{nest}/posts/post/{postId} + const scryPath = `/channels/v4/${cite.nest}/posts/post/${cite.postId}.json`; + runtime.log?.(`[tlon] Fetching cited post: ${scryPath}`); + + const data: any = await api!.scry(scryPath); + + // Extract text from the post's essay content + if (data?.essay?.content) { + const text = extractMessageText(data.essay.content); + return text || null; } - const post = update?.response?.post?.["r-post"]; - const essay = post?.set?.essay; - const memo = post?.reply?.["r-reply"]?.set?.memo; - if (!essay && !memo) { - return; - } - - const content = memo || essay; - if (!content) { - return; - } - const isThreadReply = Boolean(memo); - const rawMessageId = isThreadReply ? post?.reply?.id : update?.response?.post?.id; - const messageId = rawMessageId != null ? String(rawMessageId) : undefined; - - if (!processedTracker.mark(messageId)) { - return; - } - - const senderShip = normalizeShip(content.author ?? ""); - if (!senderShip || senderShip === botShipName) { - return; - } - - const messageText = extractMessageText(content.content); - if (!messageText) { - return; - } - - cacheMessage(channelNest, { - author: senderShip, - content: messageText, - timestamp: content.sent || Date.now(), - id: messageId, - }); - - const mentioned = isBotMentioned(messageText, botShipName); - if (!mentioned) { - return; - } - - const { mode, allowedShips } = resolveChannelAuthorization(cfg, channelNest); - if (mode === "restricted") { - if (allowedShips.length === 0) { - runtime.log?.(`[tlon] Access denied: ${senderShip} in ${channelNest} (no allowlist)`); - return; - } - const normalizedAllowed = allowedShips.map(normalizeShip); - if (!normalizedAllowed.includes(senderShip)) { - runtime.log?.( - `[tlon] Access denied: ${senderShip} in ${channelNest} (allowed: ${allowedShips.join(", ")})`, - ); - return; - } - } - - const seal = isThreadReply - ? update?.response?.post?.["r-post"]?.reply?.["r-reply"]?.set?.seal - : update?.response?.post?.["r-post"]?.set?.seal; - - const parentId = seal?.["parent-id"] || seal?.parent || null; - - await processMessage({ - messageId: messageId ?? "", - senderShip, - messageText, - isGroup: true, - groupChannel: channelNest, - groupName: `${parsed.hostShip}/${parsed.channelName}`, - timestamp: content.sent || Date.now(), - parentId, - }); - } catch (error) { - runtime.error?.(`[tlon] Error handling group message: ${formatError(error)}`); + return null; + } catch (err) { + runtime.log?.(`[tlon] Failed to fetch cited post: ${String(err)}`); + return null; } - }; + } + + // Resolve all cites in message content and return quoted text + async function resolveAllCites(content: unknown): Promise { + const cites = extractCites(content); + if (cites.length === 0) { + return ""; + } + + const resolved: string[] = []; + for (const cite of cites) { + const text = await resolveCiteContent(cite); + if (text) { + const author = cite.author || "unknown"; + resolved.push(`> ${author} wrote: ${text}`); + } + } + + return resolved.length > 0 ? resolved.join("\n") + "\n\n" : ""; + } + + // Helper to save pending approvals to settings store + async function savePendingApprovals(): Promise { + try { + await api!.poke({ + app: "settings", + mark: "settings-event", + json: { + "put-entry": { + desk: "moltbot", + "bucket-key": "tlon", + "entry-key": "pendingApprovals", + value: JSON.stringify(pendingApprovals), + }, + }, + }); + } catch (err) { + runtime.error?.(`[tlon] Failed to save pending approvals: ${String(err)}`); + } + } + + // Helper to update dmAllowlist in settings store + async function addToDmAllowlist(ship: string): Promise { + const normalizedShip = normalizeShip(ship); + if (!effectiveDmAllowlist.includes(normalizedShip)) { + effectiveDmAllowlist = [...effectiveDmAllowlist, normalizedShip]; + } + try { + await api!.poke({ + app: "settings", + mark: "settings-event", + json: { + "put-entry": { + desk: "moltbot", + "bucket-key": "tlon", + "entry-key": "dmAllowlist", + value: effectiveDmAllowlist, + }, + }, + }); + runtime.log?.(`[tlon] Added ${normalizedShip} to dmAllowlist`); + } catch (err) { + runtime.error?.(`[tlon] Failed to update dmAllowlist: ${String(err)}`); + } + } + + // Helper to update channelRules in settings store + async function addToChannelAllowlist(ship: string, channelNest: string): Promise { + const normalizedShip = normalizeShip(ship); + const channelRules = currentSettings.channelRules ?? {}; + const rule = channelRules[channelNest] ?? { mode: "restricted", allowedShips: [] }; + const allowedShips = [...(rule.allowedShips ?? [])]; // Clone to avoid mutation + + if (!allowedShips.includes(normalizedShip)) { + allowedShips.push(normalizedShip); + } + + const updatedRules = { + ...channelRules, + [channelNest]: { ...rule, allowedShips }, + }; + + // Update local state immediately (don't wait for settings subscription) + currentSettings = { ...currentSettings, channelRules: updatedRules }; + + try { + await api!.poke({ + app: "settings", + mark: "settings-event", + json: { + "put-entry": { + desk: "moltbot", + "bucket-key": "tlon", + "entry-key": "channelRules", + value: JSON.stringify(updatedRules), + }, + }, + }); + runtime.log?.(`[tlon] Added ${normalizedShip} to ${channelNest} allowlist`); + } catch (err) { + runtime.error?.(`[tlon] Failed to update channelRules: ${String(err)}`); + } + } + + // Helper to block a ship using Tlon's native blocking + async function blockShip(ship: string): Promise { + const normalizedShip = normalizeShip(ship); + try { + await api!.poke({ + app: "chat", + mark: "chat-block-ship", + json: { ship: normalizedShip }, + }); + runtime.log?.(`[tlon] Blocked ship ${normalizedShip}`); + } catch (err) { + runtime.error?.(`[tlon] Failed to block ship ${normalizedShip}: ${String(err)}`); + } + } + + // Check if a ship is blocked using Tlon's native block list + async function isShipBlocked(ship: string): Promise { + const normalizedShip = normalizeShip(ship); + try { + const blocked = (await api!.scry("/chat/blocked.json")) as string[] | undefined; + return Array.isArray(blocked) && blocked.some((s) => normalizeShip(s) === normalizedShip); + } catch (err) { + runtime.log?.(`[tlon] Failed to check blocked list: ${String(err)}`); + return false; + } + } + + // Get all blocked ships + async function getBlockedShips(): Promise { + try { + const blocked = (await api!.scry("/chat/blocked.json")) as string[] | undefined; + return Array.isArray(blocked) ? blocked : []; + } catch (err) { + runtime.log?.(`[tlon] Failed to get blocked list: ${String(err)}`); + return []; + } + } + + // Helper to unblock a ship using Tlon's native blocking + async function unblockShip(ship: string): Promise { + const normalizedShip = normalizeShip(ship); + try { + await api!.poke({ + app: "chat", + mark: "chat-unblock-ship", + json: { ship: normalizedShip }, + }); + runtime.log?.(`[tlon] Unblocked ship ${normalizedShip}`); + return true; + } catch (err) { + runtime.error?.(`[tlon] Failed to unblock ship ${normalizedShip}: ${String(err)}`); + return false; + } + } + + // Helper to send DM notification to owner + async function sendOwnerNotification(message: string): Promise { + if (!effectiveOwnerShip) { + runtime.log?.("[tlon] No ownerShip configured, cannot send notification"); + return; + } + try { + await sendDm({ + api: api!, + fromShip: botShipName, + toShip: effectiveOwnerShip, + text: message, + }); + runtime.log?.(`[tlon] Sent notification to owner ${effectiveOwnerShip}`); + } catch (err) { + runtime.error?.(`[tlon] Failed to send notification to owner: ${String(err)}`); + } + } + + // Queue a new approval request and notify the owner + async function queueApprovalRequest(approval: PendingApproval): Promise { + // Check if ship is blocked - silently ignore + if (await isShipBlocked(approval.requestingShip)) { + runtime.log?.(`[tlon] Ignoring request from blocked ship ${approval.requestingShip}`); + return; + } + + // Check for duplicate - if found, update it with new content and re-notify + const existingIndex = pendingApprovals.findIndex( + (a) => + a.type === approval.type && + a.requestingShip === approval.requestingShip && + (approval.type !== "channel" || a.channelNest === approval.channelNest) && + (approval.type !== "group" || a.groupFlag === approval.groupFlag), + ); + + if (existingIndex !== -1) { + // Update existing approval with new content (preserves the original ID) + const existing = pendingApprovals[existingIndex]; + if (approval.originalMessage) { + existing.originalMessage = approval.originalMessage; + existing.messagePreview = approval.messagePreview; + } + runtime.log?.( + `[tlon] Updated existing approval for ${approval.requestingShip} (${approval.type}) - re-sending notification`, + ); + await savePendingApprovals(); + const message = formatApprovalRequest(existing); + await sendOwnerNotification(message); + return; + } + + pendingApprovals.push(approval); + await savePendingApprovals(); + + const message = formatApprovalRequest(approval); + await sendOwnerNotification(message); + runtime.log?.( + `[tlon] Queued approval request: ${approval.id} (${approval.type} from ${approval.requestingShip})`, + ); + } + + // Process the owner's approval response + async function handleApprovalResponse(text: string): Promise { + const parsed = parseApprovalResponse(text); + if (!parsed) { + return false; + } + + const approval = findPendingApproval(pendingApprovals, parsed.id); + if (!approval) { + await sendOwnerNotification( + "No pending approval found" + (parsed.id ? ` for ID: ${parsed.id}` : ""), + ); + return true; // Still consumed the message + } + + if (parsed.action === "approve") { + switch (approval.type) { + case "dm": + await addToDmAllowlist(approval.requestingShip); + // Process the original message if available + if (approval.originalMessage) { + runtime.log?.( + `[tlon] Processing original message from ${approval.requestingShip} after approval`, + ); + await processMessage({ + messageId: approval.originalMessage.messageId, + senderShip: approval.requestingShip, + messageText: approval.originalMessage.messageText, + messageContent: approval.originalMessage.messageContent, + isGroup: false, + timestamp: approval.originalMessage.timestamp, + }); + } + break; + + case "channel": + if (approval.channelNest) { + await addToChannelAllowlist(approval.requestingShip, approval.channelNest); + // Process the original message if available + if (approval.originalMessage) { + const parsed = parseChannelNest(approval.channelNest); + runtime.log?.( + `[tlon] Processing original message from ${approval.requestingShip} in ${approval.channelNest} after approval`, + ); + await processMessage({ + messageId: approval.originalMessage.messageId, + senderShip: approval.requestingShip, + messageText: approval.originalMessage.messageText, + messageContent: approval.originalMessage.messageContent, + isGroup: true, + channelNest: approval.channelNest, + hostShip: parsed?.hostShip, + channelName: parsed?.channelName, + timestamp: approval.originalMessage.timestamp, + parentId: approval.originalMessage.parentId, + isThreadReply: approval.originalMessage.isThreadReply, + }); + } + } + break; + + case "group": + // Accept the group invite (don't add to allowlist - each invite requires approval) + if (approval.groupFlag) { + try { + await api!.poke({ + app: "groups", + mark: "group-join", + json: { + flag: approval.groupFlag, + "join-all": true, + }, + }); + runtime.log?.(`[tlon] Joined group ${approval.groupFlag} after approval`); + + // Immediately discover channels from the newly joined group + // Small delay to allow the join to propagate + setTimeout(async () => { + try { + const discoveredChannels = await fetchAllChannels(api!, runtime); + let newCount = 0; + for (const channelNest of discoveredChannels) { + if (!watchedChannels.has(channelNest)) { + watchedChannels.add(channelNest); + newCount++; + } + } + if (newCount > 0) { + runtime.log?.( + `[tlon] Discovered ${newCount} new channel(s) after joining group`, + ); + } + } catch (err) { + runtime.log?.(`[tlon] Channel discovery after group join failed: ${String(err)}`); + } + }, 2000); + } catch (err) { + runtime.error?.(`[tlon] Failed to join group ${approval.groupFlag}: ${String(err)}`); + } + } + break; + } + + await sendOwnerNotification(formatApprovalConfirmation(approval, "approve")); + } else if (parsed.action === "block") { + // Block the ship using Tlon's native blocking + await blockShip(approval.requestingShip); + await sendOwnerNotification(formatApprovalConfirmation(approval, "block")); + } else { + // Denied - just remove from pending, no notification to requester + await sendOwnerNotification(formatApprovalConfirmation(approval, "deny")); + } + + // Remove from pending + pendingApprovals = removePendingApproval(pendingApprovals, approval.id); + await savePendingApprovals(); + + return true; + } + + // Handle admin commands from owner (unblock, blocked, pending) + async function handleAdminCommand(text: string): Promise { + const command = parseAdminCommand(text); + if (!command) { + return false; + } + + switch (command.type) { + case "blocked": { + const blockedShips = await getBlockedShips(); + await sendOwnerNotification(formatBlockedList(blockedShips)); + runtime.log?.(`[tlon] Owner requested blocked ships list (${blockedShips.length} ships)`); + return true; + } + + case "pending": { + await sendOwnerNotification(formatPendingList(pendingApprovals)); + runtime.log?.( + `[tlon] Owner requested pending approvals list (${pendingApprovals.length} pending)`, + ); + return true; + } + + case "unblock": { + const shipToUnblock = command.ship; + const isBlocked = await isShipBlocked(shipToUnblock); + if (!isBlocked) { + await sendOwnerNotification(`${shipToUnblock} is not blocked.`); + return true; + } + const success = await unblockShip(shipToUnblock); + if (success) { + await sendOwnerNotification(`Unblocked ${shipToUnblock}.`); + } else { + await sendOwnerNotification(`Failed to unblock ${shipToUnblock}.`); + } + return true; + } + } + } + + // Check if a ship is the owner (always allowed to DM) + function isOwner(ship: string): boolean { + if (!effectiveOwnerShip) { + return false; + } + return normalizeShip(ship) === effectiveOwnerShip; + } + + /** + * Extract the DM partner ship from the 'whom' field. + * This is the canonical source for DM routing (more reliable than essay.author). + * Returns empty string if whom doesn't contain a valid patp-like value. + */ + function extractDmPartnerShip(whom: unknown): string { + const raw = + typeof whom === "string" + ? whom + : whom && typeof whom === "object" && "ship" in whom && typeof whom.ship === "string" + ? whom.ship + : ""; + const normalized = normalizeShip(raw); + // Keep DM routing strict: accept only patp-like values. + return /^~?[a-z-]+$/i.test(normalized) ? normalized : ""; + } const processMessage = async (params: { messageId: string; senderShip: string; messageText: string; + messageContent?: unknown; // Raw Tlon content for media extraction isGroup: boolean; - groupChannel?: string; - groupName?: string; + channelNest?: string; + hostShip?: string; + channelName?: string; timestamp: number; parentId?: string | null; + isThreadReply?: boolean; }) => { - const { messageId, senderShip, isGroup, groupChannel, groupName, timestamp, parentId } = params; + const { + messageId, + senderShip, + isGroup, + channelNest, + hostShip, + channelName, + timestamp, + parentId, + isThreadReply, + messageContent, + } = params; + const groupChannel = channelNest; // For compatibility let messageText = params.messageText; + // Download any images from the message content + let attachments: Array<{ path: string; contentType: string }> = []; + if (messageContent) { + try { + attachments = await downloadMessageImages(messageContent); + if (attachments.length > 0) { + runtime.log?.(`[tlon] Downloaded ${attachments.length} image(s) from message`); + } + } catch (error: any) { + runtime.log?.(`[tlon] Failed to download images: ${error?.message ?? String(error)}`); + } + } + + // Fetch thread context when entering a thread for the first time + if (isThreadReply && parentId && groupChannel) { + try { + const threadHistory = await fetchThreadHistory(api, groupChannel, parentId, 20, runtime); + if (threadHistory.length > 0) { + const threadContext = threadHistory + .slice(-10) // Last 10 messages for context + .map((msg) => `${msg.author}: ${msg.content}`) + .join("\n"); + + // Prepend thread context to the message + // Include note about ongoing conversation for agent judgment + const contextNote = `[Thread conversation - ${threadHistory.length} previous replies. You are participating in this thread. Only respond if relevant or helpful - you don't need to reply to every message.]`; + messageText = `${contextNote}\n\n[Previous messages]\n${threadContext}\n\n[Current message]\n${messageText}`; + runtime?.log?.( + `[tlon] Added thread context (${threadHistory.length} replies) to message`, + ); + } + } catch (error: any) { + runtime?.log?.(`[tlon] Could not fetch thread context: ${error?.message ?? String(error)}`); + // Continue without thread context - not critical + } + } + if (isGroup && groupChannel && isSummarizationRequest(messageText)) { try { const history = await getChannelHistory(api, groupChannel, 50, runtime); @@ -326,8 +919,8 @@ export async function monitorTlonProvider(opts: MonitorTlonOpts = {}): Promise 0 && !senders.has(senderShip)) { + // Log warning + runtime.log?.( + `[tlon] ⚠️ SECURITY: Multiple users sharing DM session. ` + + `Configure "session.dmScope: per-channel-peer" in OpenClaw config.`, + ); + + // Notify owner via DM (once per monitor session) + if (!sharedSessionWarningSent && effectiveOwnerShip) { + sharedSessionWarningSent = true; + const warningMsg = + `⚠️ Security Warning: Multiple users are sharing a DM session with this bot. ` + + `This can leak conversation context between users.\n\n` + + `Fix: Add to your OpenClaw config:\n` + + `session:\n dmScope: "per-channel-peer"\n\n` + + `Docs: https://docs.openclaw.ai/concepts/session#secure-dm-mode`; + + // Send async, don't block message processing + sendDm({ + api, + fromShip: botShipName, + toShip: effectiveOwnerShip, + text: warningMsg, + }).catch((err) => + runtime.error?.(`[tlon] Failed to send security warning to owner: ${err}`), + ); + } + } + senders.add(senderShip); + } + + const senderRole = isOwner(senderShip) ? "owner" : "user"; + const fromLabel = isGroup + ? `${senderShip} [${senderRole}] in ${channelNest}` + : `${senderShip} [${senderRole}]`; + + // Compute command authorization for slash commands (owner-only) + const shouldComputeAuth = core.channel.commands.shouldComputeCommandAuthorized( + messageText, + cfg, + ); + let commandAuthorized = false; + + if (shouldComputeAuth) { + const useAccessGroups = cfg.commands?.useAccessGroups !== false; + const senderIsOwner = isOwner(senderShip); + + commandAuthorized = core.channel.commands.resolveCommandAuthorizedFromAuthorizers({ + useAccessGroups, + authorizers: [{ configured: Boolean(effectiveOwnerShip), allowed: senderIsOwner }], + }); + + // Log when non-owner attempts a slash command (will be silently ignored by Gateway) + if (!commandAuthorized) { + console.log( + `[tlon] Command attempt denied: ${senderShip} is not owner (owner=${effectiveOwnerShip ?? "not configured"})`, + ); + } + } + + // Prepend attachment annotations to message body (similar to Signal format) + let bodyWithAttachments = messageText; + if (attachments.length > 0) { + const mediaLines = attachments + .map((a) => `[media attached: ${a.path} (${a.contentType}) | ${a.path}]`) + .join("\n"); + bodyWithAttachments = mediaLines + "\n" + messageText; + } + const body = core.channel.reply.formatAgentEnvelope({ channel: "Tlon", from: fromLabel, timestamp, - body: messageText, + body: bodyWithAttachments, }); + // Strip bot ship mention for CommandBody so "/status" is recognized as command-only + const commandBody = isGroup ? stripBotMention(messageText, botShipName) : messageText; + const ctxPayload = core.channel.reply.finalizeInboundContext({ Body: body, - BodyForAgent: messageText, RawBody: messageText, - CommandBody: messageText, + CommandBody: commandBody, From: isGroup ? `tlon:group:${groupChannel}` : `tlon:${senderShip}`, To: `tlon:${botShipName}`, SessionKey: route.sessionKey, @@ -377,28 +1047,33 @@ export async function monitorTlonProvider(opts: MonitorTlonOpts = {}): Promise 0 && { Attachments: attachments }), OriginatingChannel: "tlon", OriginatingTo: `tlon:${isGroup ? groupChannel : botShipName}`, + // Include thread context for automatic reply routing + ...(parentId && { ThreadId: String(parentId), ReplyToId: String(parentId) }), }); const dispatchStartTime = Date.now(); - const { onModelSelected, ...prefixOptions } = createReplyPrefixOptions({ + const responsePrefix = core.channel.reply.resolveEffectiveMessagesConfig( cfg, - agentId: route.agentId, - channel: "tlon", - accountId: route.accountId, - }); + route.agentId, + ).responsePrefix; const humanDelay = core.channel.reply.resolveHumanDelayConfig(cfg, route.agentId); await core.channel.reply.dispatchReplyWithBufferedBlockDispatcher({ ctx: ctxPayload, cfg, dispatcherOptions: { - ...prefixOptions, + responsePrefix, humanDelay, deliver: async (payload: ReplyPayload) => { let replyText = payload.text; @@ -406,8 +1081,8 @@ export async function monitorTlonProvider(opts: MonitorTlonOpts = {}): Promise(); - const subscribedDMs = new Set(); - - async function subscribeToChannel(channelNest: string) { - if (subscribedChannels.has(channelNest)) { - return; - } - const parsed = parseChannelNest(channelNest); - if (!parsed) { - runtime.error?.(`[tlon] Invalid channel format: ${channelNest}`); - return; - } + // Track which channels we're interested in for filtering firehose events + const watchedChannels = new Set(groupChannels); + const _watchedDMs = new Set(); + // Firehose handler for all channel messages (/v2) + const handleChannelsFirehose = async (event: any) => { try { - await api!.subscribe({ - app: "channels", - path: `/${channelNest}`, - event: (data: unknown) => { - handleIncomingGroupMessage(channelNest)(data as UrbitUpdate); - }, - err: (error) => { - runtime.error?.(`[tlon] Group subscription error for ${channelNest}: ${String(error)}`); - }, - quit: () => { - runtime.log?.(`[tlon] Group subscription ended for ${channelNest}`); - subscribedChannels.delete(channelNest); - }, + const nest = event?.nest; + if (!nest) { + return; + } + + // Only process channels we're watching + if (!watchedChannels.has(nest)) { + return; + } + + const response = event?.response; + if (!response) { + return; + } + + // Handle post responses (new posts and replies) + const essay = response?.post?.["r-post"]?.set?.essay; + const memo = response?.post?.["r-post"]?.reply?.["r-reply"]?.set?.memo; + if (!essay && !memo) { + return; + } + + const content = memo || essay; + const isThreadReply = Boolean(memo); + const messageId = isThreadReply ? response?.post?.["r-post"]?.reply?.id : response?.post?.id; + + if (!processedTracker.mark(messageId)) { + return; + } + + const senderShip = normalizeShip(content.author ?? ""); + if (!senderShip || senderShip === botShipName) { + return; + } + + // Resolve any cited/quoted messages first + const citedContent = await resolveAllCites(content.content); + const rawText = extractMessageText(content.content); + const messageText = citedContent + rawText; + if (!messageText.trim()) { + return; + } + + cacheMessage(nest, { + author: senderShip, + content: messageText, + timestamp: content.sent || Date.now(), + id: messageId, }); - subscribedChannels.add(channelNest); - runtime.log?.(`[tlon] Subscribed to group channel: ${channelNest}`); - } catch (error) { - runtime.error?.(`[tlon] Failed to subscribe to ${channelNest}: ${formatError(error)}`); - } - } - async function subscribeToDM(dmShip: string) { - if (subscribedDMs.has(dmShip)) { - return; - } - try { - await api!.subscribe({ - app: "chat", - path: `/dm/${dmShip}`, - event: (data: unknown) => { - handleIncomingDM(data as UrbitUpdate); - }, - err: (error) => { - runtime.error?.(`[tlon] DM subscription error for ${dmShip}: ${String(error)}`); - }, - quit: () => { - runtime.log?.(`[tlon] DM subscription ended for ${dmShip}`); - subscribedDMs.delete(dmShip); - }, - }); - subscribedDMs.add(dmShip); - runtime.log?.(`[tlon] Subscribed to DM with ${dmShip}`); - } catch (error) { - runtime.error?.(`[tlon] Failed to subscribe to DM with ${dmShip}: ${formatError(error)}`); - } - } + // Get thread info early for participation check + const seal = isThreadReply + ? response?.post?.["r-post"]?.reply?.["r-reply"]?.set?.seal + : response?.post?.["r-post"]?.set?.seal; + const parentId = seal?.["parent-id"] || seal?.parent || null; - async function refreshChannelSubscriptions() { - try { - const dmShips = await api!.scry("/chat/dm.json"); - if (Array.isArray(dmShips)) { - for (const dmShip of dmShips) { - await subscribeToDM(dmShip); + // Check if we should respond: + // 1. Direct mention always triggers response + // 2. Thread replies where we've participated - respond if relevant (let agent decide) + const mentioned = isBotMentioned(messageText, botShipName, botNickname ?? undefined); + const inParticipatedThread = + isThreadReply && parentId && participatedThreads.has(String(parentId)); + + if (!mentioned && !inParticipatedThread) { + return; + } + + // Log why we're responding + if (inParticipatedThread && !mentioned) { + runtime.log?.(`[tlon] Responding to thread we participated in (no mention): ${parentId}`); + } + + // Owner is always allowed + if (isOwner(senderShip)) { + runtime.log?.(`[tlon] Owner ${senderShip} is always allowed in channels`); + } else { + const { mode, allowedShips } = resolveChannelAuthorization(cfg, nest, currentSettings); + if (mode === "restricted") { + const normalizedAllowed = allowedShips.map(normalizeShip); + if (!normalizedAllowed.includes(senderShip)) { + // If owner is configured, queue approval request + if (effectiveOwnerShip) { + const approval = createPendingApproval({ + type: "channel", + requestingShip: senderShip, + channelNest: nest, + messagePreview: messageText.substring(0, 100), + originalMessage: { + messageId: messageId ?? "", + messageText, + messageContent: content.content, + timestamp: content.sent || Date.now(), + parentId: parentId ?? undefined, + isThreadReply, + }, + }); + await queueApprovalRequest(approval); + } else { + runtime.log?.( + `[tlon] Access denied: ${senderShip} in ${nest} (allowed: ${allowedShips.join(", ")})`, + ); + } + return; + } } } - if (account.autoDiscoverChannels !== false) { - const discoveredChannels = await fetchAllChannels(api!, runtime); - for (const channelNest of discoveredChannels) { - await subscribeToChannel(channelNest); + const parsed = parseChannelNest(nest); + await processMessage({ + messageId: messageId ?? "", + senderShip, + messageText, + messageContent: content.content, // Pass raw content for media extraction + isGroup: true, + channelNest: nest, + hostShip: parsed?.hostShip, + channelName: parsed?.channelName, + timestamp: content.sent || Date.now(), + parentId, + isThreadReply, + }); + } catch (error: any) { + runtime.error?.( + `[tlon] Error handling channel firehose event: ${error?.message ?? String(error)}`, + ); + } + }; + + // Firehose handler for all DM messages (/v3) + // Track which DM invites we've already processed to avoid duplicate accepts + const processedDmInvites = new Set(); + + const handleChatFirehose = async (event: any) => { + try { + // Handle DM invite lists (arrays) + if (Array.isArray(event)) { + for (const invite of event as DmInvite[]) { + const ship = normalizeShip(invite.ship || ""); + if (!ship || processedDmInvites.has(ship)) { + continue; + } + + // Owner is always allowed + if (isOwner(ship)) { + try { + await api.poke({ + app: "chat", + mark: "chat-dm-rsvp", + json: { ship, ok: true }, + }); + processedDmInvites.add(ship); + runtime.log?.(`[tlon] Auto-accepted DM invite from owner ${ship}`); + } catch (err) { + runtime.error?.(`[tlon] Failed to auto-accept DM from owner: ${String(err)}`); + } + continue; + } + + // Auto-accept if on allowlist and auto-accept is enabled + if (effectiveAutoAcceptDmInvites && isDmAllowed(ship, effectiveDmAllowlist)) { + try { + await api.poke({ + app: "chat", + mark: "chat-dm-rsvp", + json: { ship, ok: true }, + }); + processedDmInvites.add(ship); + runtime.log?.(`[tlon] Auto-accepted DM invite from ${ship}`); + } catch (err) { + runtime.error?.(`[tlon] Failed to auto-accept DM from ${ship}: ${String(err)}`); + } + continue; + } + + // If owner is configured and ship is not on allowlist, queue approval + if (effectiveOwnerShip && !isDmAllowed(ship, effectiveDmAllowlist)) { + const approval = createPendingApproval({ + type: "dm", + requestingShip: ship, + messagePreview: "(DM invite - no message yet)", + }); + await queueApprovalRequest(approval); + processedDmInvites.add(ship); // Mark as processed to avoid duplicate notifications + } + } + return; + } + if (!("whom" in event) || !("response" in event)) { + return; + } + + const whom = event.whom; // DM partner ship or club ID + const messageId = event.id; + const response = event.response; + + // Handle add events (new messages) + const essay = response?.add?.essay; + if (!essay) { + return; + } + + if (!processedTracker.mark(messageId)) { + return; + } + + const authorShip = normalizeShip(essay.author ?? ""); + const partnerShip = extractDmPartnerShip(whom); + const senderShip = partnerShip || authorShip; + + // Ignore the bot's own outbound DM events. + if (authorShip === botShipName) { + return; + } + if (!senderShip || senderShip === botShipName) { + return; + } + + // Log mismatch between author and partner for debugging + if (authorShip && partnerShip && authorShip !== partnerShip) { + runtime.log?.( + `[tlon] DM ship mismatch (author=${authorShip}, partner=${partnerShip}) - routing to partner`, + ); + } + + // Resolve any cited/quoted messages first + const citedContent = await resolveAllCites(essay.content); + const rawText = extractMessageText(essay.content); + const messageText = citedContent + rawText; + if (!messageText.trim()) { + return; + } + + // Check if this is the owner sending an approval response + if (isOwner(senderShip) && isApprovalResponse(messageText)) { + const handled = await handleApprovalResponse(messageText); + if (handled) { + runtime.log?.(`[tlon] Processed approval response from owner: ${messageText}`); + return; } } - } catch (error) { - runtime.error?.(`[tlon] Channel refresh failed: ${formatError(error)}`); + + // Check if this is the owner sending an admin command + if (isOwner(senderShip) && isAdminCommand(messageText)) { + const handled = await handleAdminCommand(messageText); + if (handled) { + runtime.log?.(`[tlon] Processed admin command from owner: ${messageText}`); + return; + } + } + + // Owner is always allowed to DM (bypass allowlist) + if (isOwner(senderShip)) { + runtime.log?.(`[tlon] Processing DM from owner ${senderShip}`); + await processMessage({ + messageId: messageId ?? "", + senderShip, + messageText, + messageContent: essay.content, + isGroup: false, + timestamp: essay.sent || Date.now(), + }); + return; + } + + // For DMs from others, check allowlist + if (!isDmAllowed(senderShip, effectiveDmAllowlist)) { + // If owner is configured, queue approval request + if (effectiveOwnerShip) { + const approval = createPendingApproval({ + type: "dm", + requestingShip: senderShip, + messagePreview: messageText.substring(0, 100), + originalMessage: { + messageId: messageId ?? "", + messageText, + messageContent: essay.content, + timestamp: essay.sent || Date.now(), + }, + }); + await queueApprovalRequest(approval); + } else { + runtime.log?.(`[tlon] Blocked DM from ${senderShip}: not in allowlist`); + } + return; + } + + await processMessage({ + messageId: messageId ?? "", + senderShip, + messageText, + messageContent: essay.content, // Pass raw content for media extraction + isGroup: false, + timestamp: essay.sent || Date.now(), + }); + } catch (error: any) { + runtime.error?.( + `[tlon] Error handling chat firehose event: ${error?.message ?? String(error)}`, + ); } - } + }; try { - runtime.log?.("[tlon] Subscribing to updates..."); + runtime.log?.("[tlon] Subscribing to firehose updates..."); - let dmShips: string[] = []; - try { - const dmList = await api.scry("/chat/dm.json"); - if (Array.isArray(dmList)) { - dmShips = dmList; - runtime.log?.(`[tlon] Found ${dmShips.length} DM conversation(s)`); + // Subscribe to channels firehose (/v2) + await api.subscribe({ + app: "channels", + path: "/v2", + event: handleChannelsFirehose, + err: (error) => { + runtime.error?.(`[tlon] Channels firehose error: ${String(error)}`); + }, + quit: () => { + runtime.log?.("[tlon] Channels firehose subscription ended"); + }, + }); + runtime.log?.("[tlon] Subscribed to channels firehose (/v2)"); + + // Subscribe to chat/DM firehose (/v3) + await api.subscribe({ + app: "chat", + path: "/v3", + event: handleChatFirehose, + err: (error) => { + runtime.error?.(`[tlon] Chat firehose error: ${String(error)}`); + }, + quit: () => { + runtime.log?.("[tlon] Chat firehose subscription ended"); + }, + }); + runtime.log?.("[tlon] Subscribed to chat firehose (/v3)"); + + // Subscribe to contacts updates to track nickname changes + await api.subscribe({ + app: "contacts", + path: "/v1/news", + event: (event: any) => { + try { + // Look for self profile updates + if (event?.self) { + const selfUpdate = event.self; + if (selfUpdate?.contact?.nickname?.value !== undefined) { + const newNickname = selfUpdate.contact.nickname.value || null; + if (newNickname !== botNickname) { + botNickname = newNickname; + runtime.log?.(`[tlon] Nickname updated: ${botNickname}`); + } + } + } + } catch (error: any) { + runtime.error?.( + `[tlon] Error handling contacts event: ${error?.message ?? String(error)}`, + ); + } + }, + err: (error) => { + runtime.error?.(`[tlon] Contacts subscription error: ${String(error)}`); + }, + quit: () => { + runtime.log?.("[tlon] Contacts subscription ended"); + }, + }); + runtime.log?.("[tlon] Subscribed to contacts updates (/v1/news)"); + + // Subscribe to settings store for hot-reloading config + settingsManager.onChange((newSettings) => { + currentSettings = newSettings; + + // Update watched channels if settings changed + if (newSettings.groupChannels?.length) { + const newChannels = newSettings.groupChannels; + for (const ch of newChannels) { + if (!watchedChannels.has(ch)) { + watchedChannels.add(ch); + runtime.log?.(`[tlon] Settings: now watching channel ${ch}`); + } + } + // Note: we don't remove channels from watchedChannels to avoid missing messages + // during transitions. The authorization check handles access control. } - } catch (error) { - runtime.error?.(`[tlon] Failed to fetch DM list: ${formatError(error)}`); + + // Update DM allowlist + if (newSettings.dmAllowlist !== undefined) { + effectiveDmAllowlist = + newSettings.dmAllowlist.length > 0 ? newSettings.dmAllowlist : account.dmAllowlist; + runtime.log?.(`[tlon] Settings: dmAllowlist updated to ${effectiveDmAllowlist.join(", ")}`); + } + + // Update model signature setting + if (newSettings.showModelSig !== undefined) { + effectiveShowModelSig = newSettings.showModelSig; + runtime.log?.(`[tlon] Settings: showModelSig = ${effectiveShowModelSig}`); + } + + // Update auto-accept DM invites setting + if (newSettings.autoAcceptDmInvites !== undefined) { + effectiveAutoAcceptDmInvites = newSettings.autoAcceptDmInvites; + runtime.log?.(`[tlon] Settings: autoAcceptDmInvites = ${effectiveAutoAcceptDmInvites}`); + } + + // Update auto-accept group invites setting + if (newSettings.autoAcceptGroupInvites !== undefined) { + effectiveAutoAcceptGroupInvites = newSettings.autoAcceptGroupInvites; + runtime.log?.( + `[tlon] Settings: autoAcceptGroupInvites = ${effectiveAutoAcceptGroupInvites}`, + ); + } + + // Update group invite allowlist + if (newSettings.groupInviteAllowlist !== undefined) { + effectiveGroupInviteAllowlist = + newSettings.groupInviteAllowlist.length > 0 + ? newSettings.groupInviteAllowlist + : account.groupInviteAllowlist; + runtime.log?.( + `[tlon] Settings: groupInviteAllowlist updated to ${effectiveGroupInviteAllowlist.join(", ")}`, + ); + } + + if (newSettings.defaultAuthorizedShips !== undefined) { + runtime.log?.( + `[tlon] Settings: defaultAuthorizedShips updated to ${(newSettings.defaultAuthorizedShips || []).join(", ")}`, + ); + } + + // Update auto-discover channels + if (newSettings.autoDiscoverChannels !== undefined) { + effectiveAutoDiscoverChannels = newSettings.autoDiscoverChannels; + runtime.log?.(`[tlon] Settings: autoDiscoverChannels = ${effectiveAutoDiscoverChannels}`); + } + + // Update owner ship + if (newSettings.ownerShip !== undefined) { + effectiveOwnerShip = newSettings.ownerShip + ? normalizeShip(newSettings.ownerShip) + : account.ownerShip + ? normalizeShip(account.ownerShip) + : null; + runtime.log?.(`[tlon] Settings: ownerShip = ${effectiveOwnerShip}`); + } + + // Update pending approvals + if (newSettings.pendingApprovals !== undefined) { + pendingApprovals = newSettings.pendingApprovals; + runtime.log?.( + `[tlon] Settings: pendingApprovals updated (${pendingApprovals.length} items)`, + ); + } + }); + + try { + await settingsManager.startSubscription(); + } catch (err) { + // Settings subscription is optional - don't fail if it doesn't work + runtime.log?.(`[tlon] Settings subscription not available: ${String(err)}`); } - for (const dmShip of dmShips) { - await subscribeToDM(dmShip); + // Subscribe to groups-ui for real-time channel additions (when invites are accepted) + try { + await api.subscribe({ + app: "groups", + path: "/groups/ui", + event: async (event: any) => { + try { + // Handle group/channel join events + // Event structure: { group: { flag: "~host/group-name", ... }, channels: { ... } } + if (event && typeof event === "object") { + // Check for new channels being added to groups + if (event.channels && typeof event.channels === "object") { + const channels = event.channels as Record; + for (const [channelNest, _channelData] of Object.entries(channels)) { + // Only monitor chat channels + if (!channelNest.startsWith("chat/")) { + continue; + } + + // If this is a new channel we're not watching yet, add it + if (!watchedChannels.has(channelNest)) { + watchedChannels.add(channelNest); + runtime.log?.( + `[tlon] Auto-detected new channel (invite accepted): ${channelNest}`, + ); + + // Persist to settings store so it survives restarts + if (effectiveAutoAcceptGroupInvites) { + try { + const currentChannels = currentSettings.groupChannels || []; + if (!currentChannels.includes(channelNest)) { + const updatedChannels = [...currentChannels, channelNest]; + // Poke settings store to persist + await api.poke({ + app: "settings", + mark: "settings-event", + json: { + "put-entry": { + "bucket-key": "tlon", + "entry-key": "groupChannels", + value: updatedChannels, + desk: "moltbot", + }, + }, + }); + runtime.log?.(`[tlon] Persisted ${channelNest} to settings store`); + } + } catch (err) { + runtime.error?.( + `[tlon] Failed to persist channel to settings: ${String(err)}`, + ); + } + } + } + } + } + + // Also check for the "join" event structure + if (event.join && typeof event.join === "object") { + const join = event.join as { group?: string; channels?: string[] }; + if (join.channels) { + for (const channelNest of join.channels) { + if (!channelNest.startsWith("chat/")) { + continue; + } + if (!watchedChannels.has(channelNest)) { + watchedChannels.add(channelNest); + runtime.log?.(`[tlon] Auto-detected joined channel: ${channelNest}`); + + // Persist to settings store + if (effectiveAutoAcceptGroupInvites) { + try { + const currentChannels = currentSettings.groupChannels || []; + if (!currentChannels.includes(channelNest)) { + const updatedChannels = [...currentChannels, channelNest]; + await api.poke({ + app: "settings", + mark: "settings-event", + json: { + "put-entry": { + "bucket-key": "tlon", + "entry-key": "groupChannels", + value: updatedChannels, + desk: "moltbot", + }, + }, + }); + runtime.log?.(`[tlon] Persisted ${channelNest} to settings store`); + } + } catch (err) { + runtime.error?.( + `[tlon] Failed to persist channel to settings: ${String(err)}`, + ); + } + } + } + } + } + } + } + } catch (error: any) { + runtime.error?.( + `[tlon] Error handling groups-ui event: ${error?.message ?? String(error)}`, + ); + } + }, + err: (error) => { + runtime.error?.(`[tlon] Groups-ui subscription error: ${String(error)}`); + }, + quit: () => { + runtime.log?.("[tlon] Groups-ui subscription ended"); + }, + }); + runtime.log?.("[tlon] Subscribed to groups-ui for real-time channel detection"); + } catch (err) { + // Groups-ui subscription is optional - channel discovery will still work via polling + runtime.log?.(`[tlon] Groups-ui subscription failed (will rely on polling): ${String(err)}`); } - for (const channelNest of groupChannels) { - await subscribeToChannel(channelNest); + // Subscribe to foreigns for auto-accepting group invites + // Always subscribe so we can hot-reload the setting via settings store + { + const processedGroupInvites = new Set(); + + // Helper to process pending invites + const processPendingInvites = async (foreigns: Foreigns) => { + if (!foreigns || typeof foreigns !== "object") { + return; + } + + for (const [groupFlag, foreign] of Object.entries(foreigns)) { + if (processedGroupInvites.has(groupFlag)) { + continue; + } + if (!foreign.invites || foreign.invites.length === 0) { + continue; + } + + const validInvite = foreign.invites.find((inv) => inv.valid); + if (!validInvite) { + continue; + } + + const inviterShip = validInvite.from; + const normalizedInviter = normalizeShip(inviterShip); + + // Owner invites are always accepted + if (isOwner(inviterShip)) { + try { + await api.poke({ + app: "groups", + mark: "group-join", + json: { + flag: groupFlag, + "join-all": true, + }, + }); + processedGroupInvites.add(groupFlag); + runtime.log?.(`[tlon] Auto-accepted group invite from owner: ${groupFlag}`); + } catch (err) { + runtime.error?.(`[tlon] Failed to accept group invite from owner: ${String(err)}`); + } + continue; + } + + // Skip if auto-accept is disabled + if (!effectiveAutoAcceptGroupInvites) { + // If owner is configured, queue approval + if (effectiveOwnerShip) { + const approval = createPendingApproval({ + type: "group", + requestingShip: inviterShip, + groupFlag, + }); + await queueApprovalRequest(approval); + processedGroupInvites.add(groupFlag); + } + continue; + } + + // Check if inviter is on allowlist + const isAllowed = + effectiveGroupInviteAllowlist.length > 0 + ? effectiveGroupInviteAllowlist + .map((s) => normalizeShip(s)) + .some((s) => s === normalizedInviter) + : false; // Fail-safe: empty allowlist means deny + + if (!isAllowed) { + // If owner is configured, queue approval + if (effectiveOwnerShip) { + const approval = createPendingApproval({ + type: "group", + requestingShip: inviterShip, + groupFlag, + }); + await queueApprovalRequest(approval); + processedGroupInvites.add(groupFlag); + } else { + runtime.log?.( + `[tlon] Rejected group invite from ${inviterShip} (not in groupInviteAllowlist): ${groupFlag}`, + ); + processedGroupInvites.add(groupFlag); + } + continue; + } + + // Inviter is on allowlist - accept the invite + try { + await api.poke({ + app: "groups", + mark: "group-join", + json: { + flag: groupFlag, + "join-all": true, + }, + }); + processedGroupInvites.add(groupFlag); + runtime.log?.( + `[tlon] Auto-accepted group invite: ${groupFlag} (from ${validInvite.from})`, + ); + } catch (err) { + runtime.error?.(`[tlon] Failed to auto-accept group ${groupFlag}: ${String(err)}`); + } + } + }; + + // Process existing pending invites from init data + if (initForeigns) { + await processPendingInvites(initForeigns); + } + + try { + await api.subscribe({ + app: "groups", + path: "/v1/foreigns", + event: (data: unknown) => { + void (async () => { + try { + await processPendingInvites(data as Foreigns); + } catch (error: any) { + runtime.error?.( + `[tlon] Error handling foreigns event: ${error?.message ?? String(error)}`, + ); + } + })(); + }, + err: (error) => { + runtime.error?.(`[tlon] Foreigns subscription error: ${String(error)}`); + }, + quit: () => { + runtime.log?.("[tlon] Foreigns subscription ended"); + }, + }); + runtime.log?.( + "[tlon] Subscribed to foreigns (/v1/foreigns) for auto-accepting group invites", + ); + } catch (err) { + runtime.log?.(`[tlon] Foreigns subscription failed: ${String(err)}`); + } + } + + // Discover channels to watch + if (effectiveAutoDiscoverChannels) { + const discoveredChannels = await fetchAllChannels(api, runtime); + for (const channelNest of discoveredChannels) { + watchedChannels.add(channelNest); + } + runtime.log?.(`[tlon] Watching ${watchedChannels.size} channel(s)`); + } + + // Log watched channels + for (const channelNest of watchedChannels) { + runtime.log?.(`[tlon] Watching channel: ${channelNest}`); } runtime.log?.("[tlon] All subscriptions registered, connecting to SSE stream..."); await api.connect(); - runtime.log?.("[tlon] Connected! All subscriptions active"); + runtime.log?.("[tlon] Connected! Firehose subscriptions active"); + // Periodically refresh channel discovery const pollInterval = setInterval( - () => { + async () => { if (!opts.abortSignal?.aborted) { - refreshChannelSubscriptions().catch((error) => { - runtime.error?.(`[tlon] Channel refresh error: ${formatError(error)}`); - }); + try { + if (effectiveAutoDiscoverChannels) { + const discoveredChannels = await fetchAllChannels(api, runtime); + for (const channelNest of discoveredChannels) { + if (!watchedChannels.has(channelNest)) { + watchedChannels.add(channelNest); + runtime.log?.(`[tlon] Now watching new channel: ${channelNest}`); + } + } + } + } catch (error: any) { + runtime.error?.(`[tlon] Channel refresh error: ${error?.message ?? String(error)}`); + } } }, 2 * 60 * 1000, @@ -589,8 +1918,8 @@ export async function monitorTlonProvider(opts: MonitorTlonOpts = {}): Promise { + try { + // Validate URL is http/https before fetching + const parsedUrl = new URL(url); + if (parsedUrl.protocol !== "http:" && parsedUrl.protocol !== "https:") { + console.warn(`[tlon-media] Rejected non-http(s) URL: ${url}`); + return null; + } + + // Ensure media directory exists + await mkdir(mediaDir, { recursive: true }); + + // Fetch with SSRF protection + // Use fetchWithSsrFGuard directly (not urbitFetch) to preserve the full URL path + const { response, release } = await fetchWithSsrFGuard({ + url, + init: { method: "GET" }, + policy: getDefaultSsrFPolicy(), + auditContext: "tlon-media-download", + }); + + try { + if (!response.ok) { + console.error(`[tlon-media] Failed to fetch ${url}: ${response.status}`); + return null; + } + + // Determine content type and extension + const contentType = response.headers.get("content-type") || "application/octet-stream"; + const ext = getExtensionFromContentType(contentType) || getExtensionFromUrl(url) || "bin"; + + // Generate unique filename + const filename = `${randomUUID()}.${ext}`; + const localPath = path.join(mediaDir, filename); + + // Stream to file + const body = response.body; + if (!body) { + console.error(`[tlon-media] No response body for ${url}`); + return null; + } + + const writeStream = createWriteStream(localPath); + await pipeline(Readable.fromWeb(body as any), writeStream); + + return { + localPath, + contentType, + originalUrl: url, + }; + } finally { + await release(); + } + } catch (error: any) { + console.error(`[tlon-media] Error downloading ${url}: ${error?.message ?? String(error)}`); + return null; + } +} + +function getExtensionFromContentType(contentType: string): string | null { + const map: Record = { + "image/jpeg": "jpg", + "image/jpg": "jpg", + "image/png": "png", + "image/gif": "gif", + "image/webp": "webp", + "image/svg+xml": "svg", + "video/mp4": "mp4", + "video/webm": "webm", + "audio/mpeg": "mp3", + "audio/ogg": "ogg", + }; + return map[contentType.split(";")[0].trim()] ?? null; +} + +function getExtensionFromUrl(url: string): string | null { + try { + const pathname = new URL(url).pathname; + const match = pathname.match(/\.([a-z0-9]+)$/i); + return match ? match[1].toLowerCase() : null; + } catch { + return null; + } +} + +/** + * Download all images from a message and return attachment metadata. + * Format matches OpenClaw's expected attachment structure. + */ +export async function downloadMessageImages( + content: unknown, + mediaDir?: string, +): Promise> { + const images = extractImageBlocks(content); + if (images.length === 0) { + return []; + } + + const attachments: Array<{ path: string; contentType: string }> = []; + + for (const image of images) { + const downloaded = await downloadMedia(image.url, mediaDir); + if (downloaded) { + attachments.push({ + path: downloaded.localPath, + contentType: downloaded.contentType, + }); + } + } + + return attachments; +} diff --git a/extensions/tlon/src/monitor/utils.ts b/extensions/tlon/src/monitor/utils.ts index 3c0103a7235..c0649dfbe85 100644 --- a/extensions/tlon/src/monitor/utils.ts +++ b/extensions/tlon/src/monitor/utils.ts @@ -1,12 +1,76 @@ import { normalizeShip } from "../targets.js"; +// Cite types for message references +export interface ChanCite { + chan: { nest: string; where: string }; +} +export interface GroupCite { + group: string; +} +export interface DeskCite { + desk: { flag: string; where: string }; +} +export interface BaitCite { + bait: { group: string; graph: string; where: string }; +} +export type Cite = ChanCite | GroupCite | DeskCite | BaitCite; + +export interface ParsedCite { + type: "chan" | "group" | "desk" | "bait"; + nest?: string; + author?: string; + postId?: string; + group?: string; + flag?: string; + where?: string; +} + +// Extract all cites from message content +export function extractCites(content: unknown): ParsedCite[] { + if (!content || !Array.isArray(content)) { + return []; + } + + const cites: ParsedCite[] = []; + + for (const verse of content) { + if (verse?.block?.cite && typeof verse.block.cite === "object") { + const cite = verse.block.cite; + + if (cite.chan && typeof cite.chan === "object") { + const { nest, where } = cite.chan; + const whereMatch = where?.match(/\/msg\/(~[a-z-]+)\/(.+)/); + cites.push({ + type: "chan", + nest, + where, + author: whereMatch?.[1], + postId: whereMatch?.[2], + }); + } else if (cite.group && typeof cite.group === "string") { + cites.push({ type: "group", group: cite.group }); + } else if (cite.desk && typeof cite.desk === "object") { + cites.push({ type: "desk", flag: cite.desk.flag, where: cite.desk.where }); + } else if (cite.bait && typeof cite.bait === "object") { + cites.push({ + type: "bait", + group: cite.bait.group, + nest: cite.bait.graph, + where: cite.bait.where, + }); + } + } + } + + return cites; +} + export function formatModelName(modelString?: string | null): string { if (!modelString) { return "AI"; } const modelName = modelString.includes("/") ? modelString.split("/")[1] : modelString; const modelMappings: Record = { - "claude-opus-4-6": "Claude Opus 4.6", "claude-opus-4-5": "Claude Opus 4.5", "claude-sonnet-4-5": "Claude Sonnet 4.5", "claude-sonnet-3-5": "Claude Sonnet 3.5", @@ -27,62 +91,234 @@ export function formatModelName(modelString?: string | null): string { .join(" "); } -export function isBotMentioned(messageText: string, botShipName: string): boolean { +export function isBotMentioned( + messageText: string, + botShipName: string, + nickname?: string, +): boolean { if (!messageText || !botShipName) { return false; } + + // Check for @all mention + if (/@all\b/i.test(messageText)) { + return true; + } + + // Check for ship mention const normalizedBotShip = normalizeShip(botShipName); const escapedShip = normalizedBotShip.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); const mentionPattern = new RegExp(`(^|\\s)${escapedShip}(?=\\s|$)`, "i"); - return mentionPattern.test(messageText); + if (mentionPattern.test(messageText)) { + return true; + } + + // Check for nickname mention (case-insensitive, word boundary) + if (nickname) { + const escapedNickname = nickname.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); + const nicknamePattern = new RegExp(`(^|\\s)${escapedNickname}(?=\\s|$|[,!?.])`, "i"); + if (nicknamePattern.test(messageText)) { + return true; + } + } + + return false; +} + +/** + * Strip bot ship mention from message text for command detection. + * "~bot-ship /status" → "/status" + */ +export function stripBotMention(messageText: string, botShipName: string): string { + if (!messageText || !botShipName) return messageText; + return messageText.replace(normalizeShip(botShipName), "").trim(); } export function isDmAllowed(senderShip: string, allowlist: string[] | undefined): boolean { if (!allowlist || allowlist.length === 0) { - return true; + return false; } const normalizedSender = normalizeShip(senderShip); return allowlist.map((ship) => normalizeShip(ship)).some((ship) => ship === normalizedSender); } +/** + * Check if a group invite from a ship should be auto-accepted. + * + * SECURITY: Fail-safe to deny. If allowlist is empty or undefined, + * ALL invites are rejected - even if autoAcceptGroupInvites is enabled. + * This prevents misconfigured bots from accepting malicious invites. + */ +export function isGroupInviteAllowed( + inviterShip: string, + allowlist: string[] | undefined, +): boolean { + // SECURITY: Fail-safe to deny when no allowlist configured + if (!allowlist || allowlist.length === 0) { + return false; + } + const normalizedInviter = normalizeShip(inviterShip); + return allowlist.map((ship) => normalizeShip(ship)).some((ship) => ship === normalizedInviter); +} + +// Helper to recursively extract text from inline content +function extractInlineText(items: any[]): string { + return items + .map((item: any) => { + if (typeof item === "string") { + return item; + } + if (item && typeof item === "object") { + if (item.ship) { + return item.ship; + } + if ("sect" in item) { + return `@${item.sect || "all"}`; + } + if (item["inline-code"]) { + return `\`${item["inline-code"]}\``; + } + if (item.code) { + return `\`${item.code}\``; + } + if (item.link && item.link.href) { + return item.link.content || item.link.href; + } + if (item.bold && Array.isArray(item.bold)) { + return `**${extractInlineText(item.bold)}**`; + } + if (item.italics && Array.isArray(item.italics)) { + return `*${extractInlineText(item.italics)}*`; + } + if (item.strike && Array.isArray(item.strike)) { + return `~~${extractInlineText(item.strike)}~~`; + } + } + return ""; + }) + .join(""); +} + export function extractMessageText(content: unknown): string { if (!content || !Array.isArray(content)) { return ""; } - return ( - content - // oxlint-disable-next-line typescript/no-explicit-any - .map((block: any) => { - if (block.inline && Array.isArray(block.inline)) { - return ( - block.inline - // oxlint-disable-next-line typescript/no-explicit-any - .map((item: any) => { - if (typeof item === "string") { - return item; - } - if (item && typeof item === "object") { - if (item.ship) { - return item.ship; - } - if (item.break !== undefined) { - return "\n"; - } - if (item.link && item.link.href) { - return item.link.href; - } - } - return ""; - }) - .join("") - ); + return content + .map((verse: any) => { + // Handle inline content (text, ships, links, etc.) + if (verse.inline && Array.isArray(verse.inline)) { + return verse.inline + .map((item: any) => { + if (typeof item === "string") { + return item; + } + if (item && typeof item === "object") { + if (item.ship) { + return item.ship; + } + // Handle sect (role mentions like @all) + if ("sect" in item) { + return `@${item.sect || "all"}`; + } + if (item.break !== undefined) { + return "\n"; + } + if (item.link && item.link.href) { + return item.link.href; + } + // Handle inline code (Tlon uses "inline-code" key) + if (item["inline-code"]) { + return `\`${item["inline-code"]}\``; + } + if (item.code) { + return `\`${item.code}\``; + } + // Handle bold/italic/strike - recursively extract text + if (item.bold && Array.isArray(item.bold)) { + return `**${extractInlineText(item.bold)}**`; + } + if (item.italics && Array.isArray(item.italics)) { + return `*${extractInlineText(item.italics)}*`; + } + if (item.strike && Array.isArray(item.strike)) { + return `~~${extractInlineText(item.strike)}~~`; + } + // Handle blockquote inline + if (item.blockquote && Array.isArray(item.blockquote)) { + return `> ${extractInlineText(item.blockquote)}`; + } + } + return ""; + }) + .join(""); + } + + // Handle block content (images, code blocks, etc.) + if (verse.block && typeof verse.block === "object") { + const block = verse.block; + + // Image blocks + if (block.image && block.image.src) { + const alt = block.image.alt ? ` (${block.image.alt})` : ""; + return `\n${block.image.src}${alt}\n`; } - return ""; - }) - .join("\n") - .trim() - ); + + // Code blocks + if (block.code && typeof block.code === "object") { + const lang = block.code.lang || ""; + const code = block.code.code || ""; + return `\n\`\`\`${lang}\n${code}\n\`\`\`\n`; + } + + // Header blocks + if (block.header && typeof block.header === "object") { + const text = + block.header.content + ?.map((item: any) => (typeof item === "string" ? item : "")) + .join("") || ""; + return `\n## ${text}\n`; + } + + // Cite/quote blocks - parse the reference structure + if (block.cite && typeof block.cite === "object") { + const cite = block.cite; + + // ChanCite - reference to a channel message + if (cite.chan && typeof cite.chan === "object") { + const { nest, where } = cite.chan; + // where is typically /msg/~author/timestamp + const whereMatch = where?.match(/\/msg\/(~[a-z-]+)\/(.+)/); + if (whereMatch) { + const [, author, _postId] = whereMatch; + return `\n> [quoted: ${author} in ${nest}]\n`; + } + return `\n> [quoted from ${nest}]\n`; + } + + // GroupCite - reference to a group + if (cite.group && typeof cite.group === "string") { + return `\n> [ref: group ${cite.group}]\n`; + } + + // DeskCite - reference to an app/desk + if (cite.desk && typeof cite.desk === "object") { + return `\n> [ref: ${cite.desk.flag}]\n`; + } + + // BaitCite - reference with group+graph context + if (cite.bait && typeof cite.bait === "object") { + return `\n> [ref: ${cite.bait.graph} in ${cite.bait.group}]\n`; + } + + return `\n> [quoted message]\n`; + } + } + + return ""; + }) + .join("\n") + .trim(); } export function isSummarizationRequest(messageText: string): boolean { diff --git a/extensions/tlon/src/security.test.ts b/extensions/tlon/src/security.test.ts new file mode 100644 index 00000000000..04fad337b14 --- /dev/null +++ b/extensions/tlon/src/security.test.ts @@ -0,0 +1,438 @@ +/** + * Security Tests for Tlon Plugin + * + * These tests ensure that security-critical behavior cannot regress: + * - DM allowlist enforcement + * - Channel authorization rules + * - Ship normalization consistency + * - Bot mention detection boundaries + */ + +import { describe, expect, it } from "vitest"; +import { + isDmAllowed, + isGroupInviteAllowed, + isBotMentioned, + extractMessageText, +} from "./monitor/utils.js"; +import { normalizeShip } from "./targets.js"; + +describe("Security: DM Allowlist", () => { + describe("isDmAllowed", () => { + it("rejects DMs when allowlist is empty", () => { + expect(isDmAllowed("~zod", [])).toBe(false); + expect(isDmAllowed("~sampel-palnet", [])).toBe(false); + }); + + it("rejects DMs when allowlist is undefined", () => { + expect(isDmAllowed("~zod", undefined)).toBe(false); + }); + + it("allows DMs from ships on the allowlist", () => { + const allowlist = ["~zod", "~bus"]; + expect(isDmAllowed("~zod", allowlist)).toBe(true); + expect(isDmAllowed("~bus", allowlist)).toBe(true); + }); + + it("rejects DMs from ships NOT on the allowlist", () => { + const allowlist = ["~zod", "~bus"]; + expect(isDmAllowed("~nec", allowlist)).toBe(false); + expect(isDmAllowed("~sampel-palnet", allowlist)).toBe(false); + expect(isDmAllowed("~random-ship", allowlist)).toBe(false); + }); + + it("normalizes ship names (with/without ~ prefix)", () => { + const allowlist = ["~zod"]; + expect(isDmAllowed("zod", allowlist)).toBe(true); + expect(isDmAllowed("~zod", allowlist)).toBe(true); + + const allowlistWithoutTilde = ["zod"]; + expect(isDmAllowed("~zod", allowlistWithoutTilde)).toBe(true); + expect(isDmAllowed("zod", allowlistWithoutTilde)).toBe(true); + }); + + it("handles galaxy, star, planet, and moon names", () => { + const allowlist = [ + "~zod", // galaxy + "~marzod", // star + "~sampel-palnet", // planet + "~dozzod-dozzod-dozzod-dozzod", // moon + ]; + + expect(isDmAllowed("~zod", allowlist)).toBe(true); + expect(isDmAllowed("~marzod", allowlist)).toBe(true); + expect(isDmAllowed("~sampel-palnet", allowlist)).toBe(true); + expect(isDmAllowed("~dozzod-dozzod-dozzod-dozzod", allowlist)).toBe(true); + + // Similar but different ships should be rejected + expect(isDmAllowed("~nec", allowlist)).toBe(false); + expect(isDmAllowed("~wanzod", allowlist)).toBe(false); + expect(isDmAllowed("~sampel-palned", allowlist)).toBe(false); + }); + + // NOTE: Ship names in Urbit are always lowercase by convention. + // This test documents current behavior - strict equality after normalization. + // If case-insensitivity is desired, normalizeShip should lowercase. + it("uses strict equality after normalization (case-sensitive)", () => { + const allowlist = ["~zod"]; + expect(isDmAllowed("~zod", allowlist)).toBe(true); + // Different case would NOT match with current implementation + expect(isDmAllowed("~Zod", ["~Zod"])).toBe(true); // exact match works + }); + + it("does not allow partial matches", () => { + const allowlist = ["~zod"]; + expect(isDmAllowed("~zod-extra", allowlist)).toBe(false); + expect(isDmAllowed("~extra-zod", allowlist)).toBe(false); + }); + + it("handles whitespace in ship names (normalized)", () => { + // Ships with leading/trailing whitespace are normalized by normalizeShip + const allowlist = [" ~zod ", "~bus"]; + expect(isDmAllowed("~zod", allowlist)).toBe(true); + expect(isDmAllowed(" ~zod ", allowlist)).toBe(true); + }); + }); +}); + +describe("Security: Group Invite Allowlist", () => { + describe("isGroupInviteAllowed", () => { + it("rejects invites when allowlist is empty (fail-safe)", () => { + // CRITICAL: Empty allowlist must DENY, not accept-all + expect(isGroupInviteAllowed("~zod", [])).toBe(false); + expect(isGroupInviteAllowed("~sampel-palnet", [])).toBe(false); + expect(isGroupInviteAllowed("~malicious-actor", [])).toBe(false); + }); + + it("rejects invites when allowlist is undefined (fail-safe)", () => { + // CRITICAL: Undefined allowlist must DENY, not accept-all + expect(isGroupInviteAllowed("~zod", undefined)).toBe(false); + expect(isGroupInviteAllowed("~sampel-palnet", undefined)).toBe(false); + }); + + it("accepts invites from ships on the allowlist", () => { + const allowlist = ["~nocsyx-lassul", "~malmur-halmex"]; + expect(isGroupInviteAllowed("~nocsyx-lassul", allowlist)).toBe(true); + expect(isGroupInviteAllowed("~malmur-halmex", allowlist)).toBe(true); + }); + + it("rejects invites from ships NOT on the allowlist", () => { + const allowlist = ["~nocsyx-lassul", "~malmur-halmex"]; + expect(isGroupInviteAllowed("~random-attacker", allowlist)).toBe(false); + expect(isGroupInviteAllowed("~malicious-ship", allowlist)).toBe(false); + expect(isGroupInviteAllowed("~zod", allowlist)).toBe(false); + }); + + it("normalizes ship names (with/without ~ prefix)", () => { + const allowlist = ["~nocsyx-lassul"]; + expect(isGroupInviteAllowed("nocsyx-lassul", allowlist)).toBe(true); + expect(isGroupInviteAllowed("~nocsyx-lassul", allowlist)).toBe(true); + + const allowlistWithoutTilde = ["nocsyx-lassul"]; + expect(isGroupInviteAllowed("~nocsyx-lassul", allowlistWithoutTilde)).toBe(true); + }); + + it("does not allow partial matches", () => { + const allowlist = ["~zod"]; + expect(isGroupInviteAllowed("~zod-moon", allowlist)).toBe(false); + expect(isGroupInviteAllowed("~pinser-botter-zod", allowlist)).toBe(false); + }); + + it("handles whitespace in allowlist entries", () => { + const allowlist = [" ~nocsyx-lassul ", "~malmur-halmex"]; + expect(isGroupInviteAllowed("~nocsyx-lassul", allowlist)).toBe(true); + }); + }); +}); + +describe("Security: Bot Mention Detection", () => { + describe("isBotMentioned", () => { + const botShip = "~sampel-palnet"; + const nickname = "nimbus"; + + it("detects direct ship mention", () => { + expect(isBotMentioned("hey ~sampel-palnet", botShip)).toBe(true); + expect(isBotMentioned("~sampel-palnet can you help?", botShip)).toBe(true); + expect(isBotMentioned("hello ~sampel-palnet how are you", botShip)).toBe(true); + }); + + it("detects @all mention", () => { + expect(isBotMentioned("@all please respond", botShip)).toBe(true); + expect(isBotMentioned("hey @all", botShip)).toBe(true); + expect(isBotMentioned("@ALL uppercase", botShip)).toBe(true); + }); + + it("detects nickname mention", () => { + expect(isBotMentioned("hey nimbus", botShip, nickname)).toBe(true); + expect(isBotMentioned("nimbus help me", botShip, nickname)).toBe(true); + expect(isBotMentioned("hello NIMBUS", botShip, nickname)).toBe(true); + }); + + it("does NOT trigger on random messages", () => { + expect(isBotMentioned("hello world", botShip)).toBe(false); + expect(isBotMentioned("this is a normal message", botShip)).toBe(false); + expect(isBotMentioned("hey everyone", botShip)).toBe(false); + }); + + it("does NOT trigger on partial ship matches", () => { + expect(isBotMentioned("~sampel-palnet-extra", botShip)).toBe(false); + expect(isBotMentioned("my~sampel-palnetfriend", botShip)).toBe(false); + }); + + it("does NOT trigger on substring nickname matches", () => { + // "nimbus" should not match "nimbusy" or "animbust" + expect(isBotMentioned("nimbusy", botShip, nickname)).toBe(false); + expect(isBotMentioned("prenimbus", botShip, nickname)).toBe(false); + }); + + it("handles empty/null inputs safely", () => { + expect(isBotMentioned("", botShip)).toBe(false); + expect(isBotMentioned("test", "")).toBe(false); + // @ts-expect-error testing null input + expect(isBotMentioned(null, botShip)).toBe(false); + }); + + it("requires word boundary for nickname", () => { + expect(isBotMentioned("nimbus, hello", botShip, nickname)).toBe(true); + expect(isBotMentioned("hello nimbus!", botShip, nickname)).toBe(true); + expect(isBotMentioned("nimbus?", botShip, nickname)).toBe(true); + }); + }); +}); + +describe("Security: Ship Normalization", () => { + describe("normalizeShip", () => { + it("adds ~ prefix if missing", () => { + expect(normalizeShip("zod")).toBe("~zod"); + expect(normalizeShip("sampel-palnet")).toBe("~sampel-palnet"); + }); + + it("preserves ~ prefix if present", () => { + expect(normalizeShip("~zod")).toBe("~zod"); + expect(normalizeShip("~sampel-palnet")).toBe("~sampel-palnet"); + }); + + it("trims whitespace", () => { + expect(normalizeShip(" ~zod ")).toBe("~zod"); + expect(normalizeShip(" zod ")).toBe("~zod"); + }); + + it("handles empty string", () => { + expect(normalizeShip("")).toBe(""); + expect(normalizeShip(" ")).toBe(""); + }); + }); +}); + +describe("Security: Message Text Extraction", () => { + describe("extractMessageText", () => { + it("extracts plain text", () => { + const content = [{ inline: ["hello world"] }]; + expect(extractMessageText(content)).toBe("hello world"); + }); + + it("extracts @all mentions from sect null", () => { + const content = [{ inline: [{ sect: null }] }]; + expect(extractMessageText(content)).toContain("@all"); + }); + + it("extracts ship mentions", () => { + const content = [{ inline: [{ ship: "~zod" }] }]; + expect(extractMessageText(content)).toContain("~zod"); + }); + + it("handles malformed input safely", () => { + expect(extractMessageText(null)).toBe(""); + expect(extractMessageText(undefined)).toBe(""); + expect(extractMessageText([])).toBe(""); + expect(extractMessageText([{}])).toBe(""); + expect(extractMessageText("not an array")).toBe(""); + }); + + it("does not execute injected code in inline content", () => { + // Ensure malicious content doesn't get executed + const maliciousContent = [{ inline: [""] }]; + const result = extractMessageText(maliciousContent); + expect(result).toBe(""); + // Just a string, not executed + }); + }); +}); + +describe("Security: Channel Authorization Logic", () => { + /** + * These tests document the expected behavior of channel authorization. + * The actual resolveChannelAuthorization function is internal to monitor/index.ts + * but these tests verify the building blocks and expected invariants. + */ + + it("default mode should be restricted (not open)", () => { + // This is a critical security invariant: if no mode is specified, + // channels should default to RESTRICTED, not open. + // If this test fails, someone may have changed the default unsafely. + + // The logic in resolveChannelAuthorization is: + // const mode = rule?.mode ?? "restricted"; + // We verify this by checking undefined rule gives restricted + type ModeRule = { mode?: "restricted" | "open" }; + const rule = undefined as ModeRule | undefined; + const mode = rule?.mode ?? "restricted"; + expect(mode).toBe("restricted"); + }); + + it("empty allowedShips with restricted mode should block all", () => { + // If a channel is restricted but has no allowed ships, + // no one should be able to send messages + const _mode = "restricted"; + const allowedShips: string[] = []; + const sender = "~random-ship"; + + const isAllowed = allowedShips.some((ship) => normalizeShip(ship) === normalizeShip(sender)); + expect(isAllowed).toBe(false); + }); + + it("open mode should not check allowedShips", () => { + // In open mode, any ship can send regardless of allowedShips + const mode: "open" | "restricted" = "open"; + // The check in monitor/index.ts is: + // if (mode === "restricted") { /* check ships */ } + // So open mode skips the ship check entirely + expect(mode).not.toBe("restricted"); + }); + + it("settings should override file config for channel rules", () => { + // Documented behavior: settingsRules[nest] ?? fileRules[nest] + // This means settings take precedence + type ChannelRule = { mode: "restricted" | "open" }; + const fileRules: Record = { "chat/~zod/test": { mode: "restricted" } }; + const settingsRules: Record = { "chat/~zod/test": { mode: "open" } }; + const nest = "chat/~zod/test"; + + const effectiveRule = settingsRules[nest] ?? fileRules[nest]; + expect(effectiveRule?.mode).toBe("open"); // settings wins + }); +}); + +describe("Security: Authorization Edge Cases", () => { + it("empty strings are not valid ships", () => { + expect(isDmAllowed("", ["~zod"])).toBe(false); + expect(isDmAllowed("~zod", [""])).toBe(false); + }); + + it("handles very long ship-like strings", () => { + const longName = "~" + "a".repeat(1000); + expect(isDmAllowed(longName, ["~zod"])).toBe(false); + }); + + it("handles special characters that could break regex", () => { + // These should not cause regex injection + const maliciousShip = "~zod.*"; + expect(isDmAllowed("~zodabc", [maliciousShip])).toBe(false); + + const allowlist = ["~zod"]; + expect(isDmAllowed("~zod.*", allowlist)).toBe(false); + }); + + it("protects against prototype pollution-style keys", () => { + const suspiciousShip = "__proto__"; + expect(isDmAllowed(suspiciousShip, ["~zod"])).toBe(false); + expect(isDmAllowed("~zod", [suspiciousShip])).toBe(false); + }); +}); + +describe("Security: Sender Role Identification", () => { + /** + * Tests for sender role identification (owner vs user). + * This prevents impersonation attacks where an approved user + * tries to claim owner privileges through prompt injection. + * + * SECURITY.md Section 9: Sender Role Identification + */ + + // Helper to compute sender role (mirrors logic in monitor/index.ts) + function getSenderRole(senderShip: string, ownerShip: string | null): "owner" | "user" { + if (!ownerShip) return "user"; + return normalizeShip(senderShip) === normalizeShip(ownerShip) ? "owner" : "user"; + } + + describe("owner detection", () => { + it("identifies owner when ownerShip matches sender", () => { + expect(getSenderRole("~nocsyx-lassul", "~nocsyx-lassul")).toBe("owner"); + expect(getSenderRole("nocsyx-lassul", "~nocsyx-lassul")).toBe("owner"); + expect(getSenderRole("~nocsyx-lassul", "nocsyx-lassul")).toBe("owner"); + }); + + it("identifies user when ownerShip does not match sender", () => { + expect(getSenderRole("~random-user", "~nocsyx-lassul")).toBe("user"); + expect(getSenderRole("~malicious-actor", "~nocsyx-lassul")).toBe("user"); + }); + + it("identifies everyone as user when ownerShip is null", () => { + expect(getSenderRole("~nocsyx-lassul", null)).toBe("user"); + expect(getSenderRole("~zod", null)).toBe("user"); + }); + + it("identifies everyone as user when ownerShip is empty string", () => { + // Empty string should be treated like null (no owner configured) + expect(getSenderRole("~nocsyx-lassul", "")).toBe("user"); + }); + }); + + describe("label format", () => { + // Helper to compute fromLabel (mirrors logic in monitor/index.ts) + function getFromLabel( + senderShip: string, + ownerShip: string | null, + isGroup: boolean, + channelNest?: string, + ): string { + const senderRole = getSenderRole(senderShip, ownerShip); + return isGroup + ? `${senderShip} [${senderRole}] in ${channelNest}` + : `${senderShip} [${senderRole}]`; + } + + it("DM from owner includes [owner] in label", () => { + const label = getFromLabel("~nocsyx-lassul", "~nocsyx-lassul", false); + expect(label).toBe("~nocsyx-lassul [owner]"); + expect(label).toContain("[owner]"); + }); + + it("DM from user includes [user] in label", () => { + const label = getFromLabel("~random-user", "~nocsyx-lassul", false); + expect(label).toBe("~random-user [user]"); + expect(label).toContain("[user]"); + }); + + it("group message from owner includes [owner] in label", () => { + const label = getFromLabel("~nocsyx-lassul", "~nocsyx-lassul", true, "chat/~host/general"); + expect(label).toBe("~nocsyx-lassul [owner] in chat/~host/general"); + expect(label).toContain("[owner]"); + }); + + it("group message from user includes [user] in label", () => { + const label = getFromLabel("~random-user", "~nocsyx-lassul", true, "chat/~host/general"); + expect(label).toBe("~random-user [user] in chat/~host/general"); + expect(label).toContain("[user]"); + }); + }); + + describe("impersonation prevention", () => { + it("approved user cannot get [owner] label through ship name tricks", () => { + // Even if someone has a ship name similar to owner, they should not get owner role + expect(getSenderRole("~nocsyx-lassul-fake", "~nocsyx-lassul")).toBe("user"); + expect(getSenderRole("~fake-nocsyx-lassul", "~nocsyx-lassul")).toBe("user"); + }); + + it("message content cannot change sender role", () => { + // The role is determined by ship identity, not message content + // This test documents that even if message contains "I am the owner", + // the actual senderShip determines the role + const senderShip = "~malicious-actor"; + const ownerShip = "~nocsyx-lassul"; + + // The role is always based on ship comparison, not message content + expect(getSenderRole(senderShip, ownerShip)).toBe("user"); + }); + }); +}); diff --git a/extensions/tlon/src/settings.ts b/extensions/tlon/src/settings.ts new file mode 100644 index 00000000000..8e74009049d --- /dev/null +++ b/extensions/tlon/src/settings.ts @@ -0,0 +1,391 @@ +/** + * Settings Store integration for hot-reloading Tlon plugin config. + * + * Settings are stored in Urbit's %settings agent under: + * desk: "moltbot" + * bucket: "tlon" + * + * This allows config changes via poke from any Landscape client + * without requiring a gateway restart. + */ + +import type { UrbitSSEClient } from "./urbit/sse-client.js"; + +/** Pending approval request stored for persistence */ +export type PendingApproval = { + id: string; + type: "dm" | "channel" | "group"; + requestingShip: string; + channelNest?: string; + groupFlag?: string; + messagePreview?: string; + /** Full message context for processing after approval */ + originalMessage?: { + messageId: string; + messageText: string; + messageContent: unknown; + timestamp: number; + parentId?: string; + isThreadReply?: boolean; + }; + timestamp: number; +}; + +export type TlonSettingsStore = { + groupChannels?: string[]; + dmAllowlist?: string[]; + autoDiscover?: boolean; + showModelSig?: boolean; + autoAcceptDmInvites?: boolean; + autoDiscoverChannels?: boolean; + autoAcceptGroupInvites?: boolean; + /** Ships allowed to invite us to groups (when autoAcceptGroupInvites is true) */ + groupInviteAllowlist?: string[]; + channelRules?: Record< + string, + { + mode?: "restricted" | "open"; + allowedShips?: string[]; + } + >; + defaultAuthorizedShips?: string[]; + /** Ship that receives approval requests for DMs, channel mentions, and group invites */ + ownerShip?: string; + /** Pending approval requests awaiting owner response */ + pendingApprovals?: PendingApproval[]; +}; + +export type TlonSettingsState = { + current: TlonSettingsStore; + loaded: boolean; +}; + +const SETTINGS_DESK = "moltbot"; +const SETTINGS_BUCKET = "tlon"; + +/** + * Parse channelRules - handles both JSON string and object formats. + * Settings-store doesn't support nested objects, so we store as JSON string. + */ +function parseChannelRules( + value: unknown, +): Record | undefined { + if (!value) { + return undefined; + } + + // If it's a string, try to parse as JSON + if (typeof value === "string") { + try { + const parsed = JSON.parse(value); + if (isChannelRulesObject(parsed)) { + return parsed; + } + } catch { + return undefined; + } + } + + // If it's already an object, use directly + if (isChannelRulesObject(value)) { + return value; + } + + return undefined; +} + +/** + * Parse settings from the raw Urbit settings-store response. + * The response shape is: { [bucket]: { [key]: value } } + */ +function parseSettingsResponse(raw: unknown): TlonSettingsStore { + if (!raw || typeof raw !== "object") { + return {}; + } + + const desk = raw as Record; + const bucket = desk[SETTINGS_BUCKET]; + if (!bucket || typeof bucket !== "object") { + return {}; + } + + const settings = bucket as Record; + + return { + groupChannels: Array.isArray(settings.groupChannels) + ? settings.groupChannels.filter((x): x is string => typeof x === "string") + : undefined, + dmAllowlist: Array.isArray(settings.dmAllowlist) + ? settings.dmAllowlist.filter((x): x is string => typeof x === "string") + : undefined, + autoDiscover: typeof settings.autoDiscover === "boolean" ? settings.autoDiscover : undefined, + showModelSig: typeof settings.showModelSig === "boolean" ? settings.showModelSig : undefined, + autoAcceptDmInvites: + typeof settings.autoAcceptDmInvites === "boolean" ? settings.autoAcceptDmInvites : undefined, + autoAcceptGroupInvites: + typeof settings.autoAcceptGroupInvites === "boolean" + ? settings.autoAcceptGroupInvites + : undefined, + groupInviteAllowlist: Array.isArray(settings.groupInviteAllowlist) + ? settings.groupInviteAllowlist.filter((x): x is string => typeof x === "string") + : undefined, + channelRules: parseChannelRules(settings.channelRules), + defaultAuthorizedShips: Array.isArray(settings.defaultAuthorizedShips) + ? settings.defaultAuthorizedShips.filter((x): x is string => typeof x === "string") + : undefined, + ownerShip: typeof settings.ownerShip === "string" ? settings.ownerShip : undefined, + pendingApprovals: parsePendingApprovals(settings.pendingApprovals), + }; +} + +function isChannelRulesObject( + val: unknown, +): val is Record { + if (!val || typeof val !== "object" || Array.isArray(val)) { + return false; + } + for (const [, rule] of Object.entries(val)) { + if (!rule || typeof rule !== "object") { + return false; + } + } + return true; +} + +/** + * Parse pendingApprovals - handles both JSON string and array formats. + * Settings-store stores complex objects as JSON strings. + */ +function parsePendingApprovals(value: unknown): PendingApproval[] | undefined { + if (!value) { + return undefined; + } + + // If it's a string, try to parse as JSON + let parsed: unknown = value; + if (typeof value === "string") { + try { + parsed = JSON.parse(value); + } catch { + return undefined; + } + } + + // Validate it's an array + if (!Array.isArray(parsed)) { + return undefined; + } + + // Filter to valid PendingApproval objects + return parsed.filter((item): item is PendingApproval => { + if (!item || typeof item !== "object") { + return false; + } + const obj = item as Record; + return ( + typeof obj.id === "string" && + (obj.type === "dm" || obj.type === "channel" || obj.type === "group") && + typeof obj.requestingShip === "string" && + typeof obj.timestamp === "number" + ); + }); +} + +/** + * Parse a single settings entry update event. + */ +function parseSettingsEvent(event: unknown): { key: string; value: unknown } | null { + if (!event || typeof event !== "object") { + return null; + } + + const evt = event as Record; + + // Handle put-entry events + if (evt["put-entry"]) { + const put = evt["put-entry"] as Record; + if (put.desk !== SETTINGS_DESK || put["bucket-key"] !== SETTINGS_BUCKET) { + return null; + } + return { + key: String(put["entry-key"] ?? ""), + value: put.value, + }; + } + + // Handle del-entry events + if (evt["del-entry"]) { + const del = evt["del-entry"] as Record; + if (del.desk !== SETTINGS_DESK || del["bucket-key"] !== SETTINGS_BUCKET) { + return null; + } + return { + key: String(del["entry-key"] ?? ""), + value: undefined, + }; + } + + return null; +} + +/** + * Apply a single settings update to the current state. + */ +function applySettingsUpdate( + current: TlonSettingsStore, + key: string, + value: unknown, +): TlonSettingsStore { + const next = { ...current }; + + switch (key) { + case "groupChannels": + next.groupChannels = Array.isArray(value) + ? value.filter((x): x is string => typeof x === "string") + : undefined; + break; + case "dmAllowlist": + next.dmAllowlist = Array.isArray(value) + ? value.filter((x): x is string => typeof x === "string") + : undefined; + break; + case "autoDiscover": + next.autoDiscover = typeof value === "boolean" ? value : undefined; + break; + case "showModelSig": + next.showModelSig = typeof value === "boolean" ? value : undefined; + break; + case "autoAcceptDmInvites": + next.autoAcceptDmInvites = typeof value === "boolean" ? value : undefined; + break; + case "autoAcceptGroupInvites": + next.autoAcceptGroupInvites = typeof value === "boolean" ? value : undefined; + break; + case "groupInviteAllowlist": + next.groupInviteAllowlist = Array.isArray(value) + ? value.filter((x): x is string => typeof x === "string") + : undefined; + break; + case "channelRules": + next.channelRules = parseChannelRules(value); + break; + case "defaultAuthorizedShips": + next.defaultAuthorizedShips = Array.isArray(value) + ? value.filter((x): x is string => typeof x === "string") + : undefined; + break; + case "ownerShip": + next.ownerShip = typeof value === "string" ? value : undefined; + break; + case "pendingApprovals": + next.pendingApprovals = parsePendingApprovals(value); + break; + } + + return next; +} + +export type SettingsLogger = { + log?: (msg: string) => void; + error?: (msg: string) => void; +}; + +/** + * Create a settings store subscription manager. + * + * Usage: + * const settings = createSettingsManager(api, logger); + * await settings.load(); + * settings.subscribe((newSettings) => { ... }); + */ +export function createSettingsManager(api: UrbitSSEClient, logger?: SettingsLogger) { + let state: TlonSettingsState = { + current: {}, + loaded: false, + }; + + const listeners = new Set<(settings: TlonSettingsStore) => void>(); + + const notify = () => { + for (const listener of listeners) { + try { + listener(state.current); + } catch (err) { + logger?.error?.(`[settings] Listener error: ${String(err)}`); + } + } + }; + + return { + /** + * Get current settings (may be empty if not loaded yet). + */ + get current(): TlonSettingsStore { + return state.current; + }, + + /** + * Whether initial settings have been loaded. + */ + get loaded(): boolean { + return state.loaded; + }, + + /** + * Load initial settings via scry. + */ + async load(): Promise { + try { + const raw = await api.scry("/settings/all.json"); + // Response shape: { all: { [desk]: { [bucket]: { [key]: value } } } } + const allData = raw as { all?: Record> }; + const deskData = allData?.all?.[SETTINGS_DESK]; + state.current = parseSettingsResponse(deskData ?? {}); + state.loaded = true; + logger?.log?.(`[settings] Loaded: ${JSON.stringify(state.current)}`); + return state.current; + } catch (err) { + // Settings desk may not exist yet - that's fine, use defaults + logger?.log?.(`[settings] No settings found (using defaults): ${String(err)}`); + state.current = {}; + state.loaded = true; + return state.current; + } + }, + + /** + * Subscribe to settings changes. + */ + async startSubscription(): Promise { + await api.subscribe({ + app: "settings", + path: "/desk/" + SETTINGS_DESK, + event: (event) => { + const update = parseSettingsEvent(event); + if (!update) { + return; + } + + logger?.log?.(`[settings] Update: ${update.key} = ${JSON.stringify(update.value)}`); + state.current = applySettingsUpdate(state.current, update.key, update.value); + notify(); + }, + err: (error) => { + logger?.error?.(`[settings] Subscription error: ${String(error)}`); + }, + quit: () => { + logger?.log?.("[settings] Subscription ended"); + }, + }); + logger?.log?.("[settings] Subscribed to settings updates"); + }, + + /** + * Register a listener for settings changes. + */ + onChange(listener: (settings: TlonSettingsStore) => void): () => void { + listeners.add(listener); + return () => listeners.delete(listener); + }, + }; +} diff --git a/extensions/tlon/src/targets.ts b/extensions/tlon/src/targets.ts index b93ede64bae..bacc6d576c0 100644 --- a/extensions/tlon/src/targets.ts +++ b/extensions/tlon/src/targets.ts @@ -1,5 +1,5 @@ export type TlonTarget = - | { kind: "direct"; ship: string } + | { kind: "dm"; ship: string } | { kind: "group"; nest: string; hostShip: string; channelName: string }; const SHIP_RE = /^~?[a-z-]+$/i; @@ -32,7 +32,7 @@ export function parseTlonTarget(raw?: string | null): TlonTarget | null { const dmPrefix = withoutPrefix.match(/^dm[/:](.+)$/i); if (dmPrefix) { - return { kind: "direct", ship: normalizeShip(dmPrefix[1]) }; + return { kind: "dm", ship: normalizeShip(dmPrefix[1]) }; } const groupPrefix = withoutPrefix.match(/^(group|room)[/:](.+)$/i); @@ -78,7 +78,7 @@ export function parseTlonTarget(raw?: string | null): TlonTarget | null { } if (SHIP_RE.test(withoutPrefix)) { - return { kind: "direct", ship: normalizeShip(withoutPrefix) }; + return { kind: "dm", ship: normalizeShip(withoutPrefix) }; } return null; diff --git a/extensions/tlon/src/types.ts b/extensions/tlon/src/types.ts index 9447e6c9b8a..81f38adc76b 100644 --- a/extensions/tlon/src/types.ts +++ b/extensions/tlon/src/types.ts @@ -11,8 +11,15 @@ export type TlonResolvedAccount = { allowPrivateNetwork: boolean | null; groupChannels: string[]; dmAllowlist: string[]; + /** Ships allowed to invite us to groups (security: prevent malicious group invites) */ + groupInviteAllowlist: string[]; autoDiscoverChannels: boolean | null; showModelSignature: boolean | null; + autoAcceptDmInvites: boolean | null; + autoAcceptGroupInvites: boolean | null; + defaultAuthorizedShips: string[]; + /** Ship that receives approval requests for DMs, channel mentions, and group invites */ + ownerShip: string | null; }; export function resolveTlonAccount( @@ -29,8 +36,12 @@ export function resolveTlonAccount( allowPrivateNetwork?: boolean; groupChannels?: string[]; dmAllowlist?: string[]; + groupInviteAllowlist?: string[]; autoDiscoverChannels?: boolean; showModelSignature?: boolean; + autoAcceptDmInvites?: boolean; + autoAcceptGroupInvites?: boolean; + ownerShip?: string; accounts?: Record>; } | undefined; @@ -47,8 +58,13 @@ export function resolveTlonAccount( allowPrivateNetwork: null, groupChannels: [], dmAllowlist: [], + groupInviteAllowlist: [], autoDiscoverChannels: null, showModelSignature: null, + autoAcceptDmInvites: null, + autoAcceptGroupInvites: null, + defaultAuthorizedShips: [], + ownerShip: null, }; } @@ -63,12 +79,25 @@ export function resolveTlonAccount( | null; const groupChannels = (account?.groupChannels ?? base.groupChannels ?? []) as string[]; const dmAllowlist = (account?.dmAllowlist ?? base.dmAllowlist ?? []) as string[]; + const groupInviteAllowlist = (account?.groupInviteAllowlist ?? + base.groupInviteAllowlist ?? + []) as string[]; const autoDiscoverChannels = (account?.autoDiscoverChannels ?? base.autoDiscoverChannels ?? null) as boolean | null; const showModelSignature = (account?.showModelSignature ?? base.showModelSignature ?? null) as | boolean | null; + const autoAcceptDmInvites = (account?.autoAcceptDmInvites ?? base.autoAcceptDmInvites ?? null) as + | boolean + | null; + const autoAcceptGroupInvites = (account?.autoAcceptGroupInvites ?? + base.autoAcceptGroupInvites ?? + null) as boolean | null; + const ownerShip = (account?.ownerShip ?? base.ownerShip ?? null) as string | null; + const defaultAuthorizedShips = ((account as Record)?.defaultAuthorizedShips ?? + (base as Record)?.defaultAuthorizedShips ?? + []) as string[]; const configured = Boolean(ship && url && code); return { @@ -82,8 +111,13 @@ export function resolveTlonAccount( allowPrivateNetwork, groupChannels, dmAllowlist, + groupInviteAllowlist, autoDiscoverChannels, showModelSignature, + autoAcceptDmInvites, + autoAcceptGroupInvites, + defaultAuthorizedShips, + ownerShip, }; } diff --git a/extensions/tlon/src/urbit/channel-client.ts b/extensions/tlon/src/urbit/channel-client.ts deleted file mode 100644 index 499860075b3..00000000000 --- a/extensions/tlon/src/urbit/channel-client.ts +++ /dev/null @@ -1,158 +0,0 @@ -import { randomUUID } from "node:crypto"; -import type { LookupFn, SsrFPolicy } from "openclaw/plugin-sdk"; -import { ensureUrbitChannelOpen, pokeUrbitChannel, scryUrbitPath } from "./channel-ops.js"; -import { getUrbitContext, normalizeUrbitCookie } from "./context.js"; -import { urbitFetch } from "./fetch.js"; - -export type UrbitChannelClientOptions = { - ship?: string; - ssrfPolicy?: SsrFPolicy; - lookupFn?: LookupFn; - fetchImpl?: (input: RequestInfo | URL, init?: RequestInit) => Promise; -}; - -export class UrbitChannelClient { - readonly baseUrl: string; - readonly cookie: string; - readonly ship: string; - readonly ssrfPolicy?: SsrFPolicy; - readonly lookupFn?: LookupFn; - readonly fetchImpl?: (input: RequestInfo | URL, init?: RequestInit) => Promise; - - private channelId: string | null = null; - - constructor(url: string, cookie: string, options: UrbitChannelClientOptions = {}) { - const ctx = getUrbitContext(url, options.ship); - this.baseUrl = ctx.baseUrl; - this.cookie = normalizeUrbitCookie(cookie); - this.ship = ctx.ship; - this.ssrfPolicy = options.ssrfPolicy; - this.lookupFn = options.lookupFn; - this.fetchImpl = options.fetchImpl; - } - - private get channelPath(): string { - const id = this.channelId; - if (!id) { - throw new Error("Channel not opened"); - } - return `/~/channel/${id}`; - } - - async open(): Promise { - if (this.channelId) { - return; - } - - const channelId = `${Math.floor(Date.now() / 1000)}-${randomUUID()}`; - this.channelId = channelId; - - try { - await ensureUrbitChannelOpen( - { - baseUrl: this.baseUrl, - cookie: this.cookie, - ship: this.ship, - channelId, - ssrfPolicy: this.ssrfPolicy, - lookupFn: this.lookupFn, - fetchImpl: this.fetchImpl, - }, - { - createBody: [], - createAuditContext: "tlon-urbit-channel-open", - }, - ); - } catch (error) { - this.channelId = null; - throw error; - } - } - - async poke(params: { app: string; mark: string; json: unknown }): Promise { - await this.open(); - const channelId = this.channelId; - if (!channelId) { - throw new Error("Channel not opened"); - } - return await pokeUrbitChannel( - { - baseUrl: this.baseUrl, - cookie: this.cookie, - ship: this.ship, - channelId, - ssrfPolicy: this.ssrfPolicy, - lookupFn: this.lookupFn, - fetchImpl: this.fetchImpl, - }, - { ...params, auditContext: "tlon-urbit-poke" }, - ); - } - - async scry(path: string): Promise { - return await scryUrbitPath( - { - baseUrl: this.baseUrl, - cookie: this.cookie, - ssrfPolicy: this.ssrfPolicy, - lookupFn: this.lookupFn, - fetchImpl: this.fetchImpl, - }, - { path, auditContext: "tlon-urbit-scry" }, - ); - } - - async getOurName(): Promise { - const { response, release } = await urbitFetch({ - baseUrl: this.baseUrl, - path: "/~/name", - init: { - method: "GET", - headers: { Cookie: this.cookie }, - }, - ssrfPolicy: this.ssrfPolicy, - lookupFn: this.lookupFn, - fetchImpl: this.fetchImpl, - timeoutMs: 30_000, - auditContext: "tlon-urbit-name", - }); - - try { - if (!response.ok) { - throw new Error(`Name request failed: ${response.status}`); - } - const text = await response.text(); - return text.trim(); - } finally { - await release(); - } - } - - async close(): Promise { - if (!this.channelId) { - return; - } - const channelPath = this.channelPath; - this.channelId = null; - - try { - const { response, release } = await urbitFetch({ - baseUrl: this.baseUrl, - path: channelPath, - init: { method: "DELETE", headers: { Cookie: this.cookie } }, - ssrfPolicy: this.ssrfPolicy, - lookupFn: this.lookupFn, - fetchImpl: this.fetchImpl, - timeoutMs: 30_000, - auditContext: "tlon-urbit-channel-close", - }); - try { - void response.body?.cancel(); - } finally { - await release(); - } - } catch { - // ignore cleanup errors - } - } -} diff --git a/extensions/tlon/src/urbit/context.ts b/extensions/tlon/src/urbit/context.ts index 90c2721c7b8..e5c78aeee7f 100644 --- a/extensions/tlon/src/urbit/context.ts +++ b/extensions/tlon/src/urbit/context.ts @@ -45,3 +45,12 @@ export function ssrfPolicyFromAllowPrivateNetwork( ): SsrFPolicy | undefined { return allowPrivateNetwork ? { allowPrivateNetwork: true } : undefined; } + +/** + * Get the default SSRF policy for image uploads. + * Uses a restrictive policy that blocks private networks by default. + */ +export function getDefaultSsrFPolicy(): SsrFPolicy | undefined { + // Default: block private networks for image uploads (safer default) + return undefined; +} diff --git a/extensions/tlon/src/urbit/foreigns.ts b/extensions/tlon/src/urbit/foreigns.ts new file mode 100644 index 00000000000..c9ce7c5002a --- /dev/null +++ b/extensions/tlon/src/urbit/foreigns.ts @@ -0,0 +1,49 @@ +/** + * Types for Urbit groups foreigns (group invites) + * Based on packages/shared/src/urbit/groups.ts from homestead + */ + +export interface GroupPreviewV7 { + meta: { + title: string; + description: string; + image: string; + cover: string; + }; + "channel-count": number; + "member-count": number; + admissions: { + privacy: "public" | "private" | "secret"; + }; +} + +export interface ForeignInvite { + flag: string; // group flag e.g. "~host/group-name" + time: number; // timestamp + from: string; // ship that sent invite + token: string | null; + note: string | null; + preview: GroupPreviewV7; + valid: boolean; // tracks if invite has been revoked +} + +export type Lookup = "preview" | "done" | "error"; +export type Progress = "ask" | "join" | "watch" | "done" | "error"; + +export interface Foreign { + invites: ForeignInvite[]; + lookup: Lookup | null; + preview: GroupPreviewV7 | null; + progress: Progress | null; + token: string | null; +} + +export interface Foreigns { + [flag: string]: Foreign; +} + +// DM invite structure from chat /v3 firehose +export interface DmInvite { + ship: string; + // Additional fields may be present +} diff --git a/extensions/tlon/src/urbit/send.ts b/extensions/tlon/src/urbit/send.ts index b848e99f4e4..70a16ce57d3 100644 --- a/extensions/tlon/src/urbit/send.ts +++ b/extensions/tlon/src/urbit/send.ts @@ -1,4 +1,5 @@ import { scot, da } from "@urbit/aura"; +import { markdownToStory, createImageBlock, isImageUrl, type Story } from "./story.js"; export type TlonPokeApi = { poke: (params: { app: string; mark: string; json: unknown }) => Promise; @@ -11,8 +12,19 @@ type SendTextParams = { text: string; }; +type SendStoryParams = { + api: TlonPokeApi; + fromShip: string; + toShip: string; + story: Story; +}; + export async function sendDm({ api, fromShip, toShip, text }: SendTextParams) { - const story = [{ inline: [text] }]; + const story: Story = markdownToStory(text); + return sendDmWithStory({ api, fromShip, toShip, story }); +} + +export async function sendDmWithStory({ api, fromShip, toShip, story }: SendStoryParams) { const sentAt = Date.now(); const idUd = scot("ud", da.fromUnix(sentAt)); const id = `${fromShip}/${idUd}`; @@ -52,6 +64,15 @@ type SendGroupParams = { replyToId?: string | null; }; +type SendGroupStoryParams = { + api: TlonPokeApi; + fromShip: string; + hostShip: string; + channelName: string; + story: Story; + replyToId?: string | null; +}; + export async function sendGroupMessage({ api, fromShip, @@ -60,13 +81,25 @@ export async function sendGroupMessage({ text, replyToId, }: SendGroupParams) { - const story = [{ inline: [text] }]; + const story: Story = markdownToStory(text); + return sendGroupMessageWithStory({ api, fromShip, hostShip, channelName, story, replyToId }); +} + +export async function sendGroupMessageWithStory({ + api, + fromShip, + hostShip, + channelName, + story, + replyToId, +}: SendGroupStoryParams) { const sentAt = Date.now(); // Format reply ID as @ud (with dots) - required for Tlon to recognize thread replies let formattedReplyId = replyToId; if (replyToId && /^\d+$/.test(replyToId)) { try { + // scot('ud', n) formats a number as @ud with dots formattedReplyId = scot("ud", BigInt(replyToId)); } catch { // Fall back to raw ID if formatting fails @@ -129,3 +162,27 @@ export function buildMediaText(text: string | undefined, mediaUrl: string | unde } return cleanText; } + +/** + * Build a story with text and optional media (image) + */ +export function buildMediaStory(text: string | undefined, mediaUrl: string | undefined): Story { + const story: Story = []; + const cleanText = text?.trim() ?? ""; + const cleanUrl = mediaUrl?.trim() ?? ""; + + // Add text content if present + if (cleanText) { + story.push(...markdownToStory(cleanText)); + } + + // Add image block if URL looks like an image + if (cleanUrl && isImageUrl(cleanUrl)) { + story.push(createImageBlock(cleanUrl, "")); + } else if (cleanUrl) { + // For non-image URLs, add as a link + story.push({ inline: [{ link: { href: cleanUrl, content: cleanUrl } }] }); + } + + return story.length > 0 ? story : [{ inline: [""] }]; +} diff --git a/extensions/tlon/src/urbit/sse-client.test.ts b/extensions/tlon/src/urbit/sse-client.test.ts index b37c3be05f8..5e4d34ebd13 100644 --- a/extensions/tlon/src/urbit/sse-client.test.ts +++ b/extensions/tlon/src/urbit/sse-client.test.ts @@ -1,44 +1,205 @@ -import type { LookupFn } from "openclaw/plugin-sdk"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { UrbitSSEClient } from "./sse-client.js"; -const mockFetch = vi.fn(); +// Mock urbitFetch to avoid real network calls +vi.mock("./fetch.js", () => ({ + urbitFetch: vi.fn(), +})); + +// Mock channel-ops to avoid real channel operations +vi.mock("./channel-ops.js", () => ({ + ensureUrbitChannelOpen: vi.fn().mockResolvedValue(undefined), + pokeUrbitChannel: vi.fn().mockResolvedValue(undefined), + scryUrbitPath: vi.fn().mockResolvedValue({}), +})); describe("UrbitSSEClient", () => { beforeEach(() => { - vi.stubGlobal("fetch", mockFetch); - mockFetch.mockReset(); + vi.clearAllMocks(); }); afterEach(() => { - vi.unstubAllGlobals(); + vi.restoreAllMocks(); }); - it("sends subscriptions added after connect", async () => { - mockFetch.mockResolvedValue({ ok: true, status: 200, text: async () => "" }); - const lookupFn = (async () => [{ address: "1.1.1.1", family: 4 }]) as unknown as LookupFn; + describe("subscribe", () => { + it("sends subscriptions added after connect", async () => { + const { urbitFetch } = await import("./fetch.js"); + const mockUrbitFetch = vi.mocked(urbitFetch); + mockUrbitFetch.mockResolvedValue({ + response: { ok: true, status: 200 } as unknown as Response, + finalUrl: "https://example.com", + release: vi.fn().mockResolvedValue(undefined), + }); - const client = new UrbitSSEClient("https://example.com", "urbauth-~zod=123", { - lookupFn, - }); - (client as { isConnected: boolean }).isConnected = true; + const client = new UrbitSSEClient("https://example.com", "urbauth-~zod=123"); + // Simulate connected state + (client as { isConnected: boolean }).isConnected = true; - await client.subscribe({ - app: "chat", - path: "/dm/~zod", - event: () => {}, + await client.subscribe({ + app: "chat", + path: "/dm/~zod", + event: () => {}, + }); + + expect(mockUrbitFetch).toHaveBeenCalledTimes(1); + const callArgs = mockUrbitFetch.mock.calls[0][0]; + expect(callArgs.path).toContain("/~/channel/"); + expect(callArgs.init?.method).toBe("PUT"); + + const body = JSON.parse(callArgs.init?.body as string); + expect(body).toHaveLength(1); + expect(body[0]).toMatchObject({ + action: "subscribe", + app: "chat", + path: "/dm/~zod", + }); }); - expect(mockFetch).toHaveBeenCalledTimes(1); - const [url, init] = mockFetch.mock.calls[0]; - expect(url).toBe(client.channelUrl); - expect(init.method).toBe("PUT"); - const body = JSON.parse(init.body as string); - expect(body).toHaveLength(1); - expect(body[0]).toMatchObject({ - action: "subscribe", - app: "chat", - path: "/dm/~zod", + it("queues subscriptions before connect", async () => { + const { urbitFetch } = await import("./fetch.js"); + const mockUrbitFetch = vi.mocked(urbitFetch); + + const client = new UrbitSSEClient("https://example.com", "urbauth-~zod=123"); + // Not connected yet + + await client.subscribe({ + app: "chat", + path: "/dm/~zod", + event: () => {}, + }); + + // Should not call urbitFetch since not connected + expect(mockUrbitFetch).not.toHaveBeenCalled(); + // But subscription should be queued + expect(client.subscriptions).toHaveLength(1); + expect(client.subscriptions[0]).toMatchObject({ + app: "chat", + path: "/dm/~zod", + }); + }); + }); + + describe("updateCookie", () => { + it("normalizes cookie when updating", () => { + const client = new UrbitSSEClient("https://example.com", "urbauth-~zod=123"); + + // Cookie with extra parts that should be stripped + client.updateCookie("urbauth-~zod=456; Path=/; HttpOnly"); + + expect(client.cookie).toBe("urbauth-~zod=456"); + }); + + it("handles simple cookie values", () => { + const client = new UrbitSSEClient("https://example.com", "urbauth-~zod=123"); + + client.updateCookie("urbauth-~zod=newvalue"); + + expect(client.cookie).toBe("urbauth-~zod=newvalue"); + }); + }); + + describe("reconnection", () => { + it("has autoReconnect enabled by default", () => { + const client = new UrbitSSEClient("https://example.com", "urbauth-~zod=123"); + expect(client.autoReconnect).toBe(true); + }); + + it("can disable autoReconnect via options", () => { + const client = new UrbitSSEClient("https://example.com", "urbauth-~zod=123", { + autoReconnect: false, + }); + expect(client.autoReconnect).toBe(false); + }); + + it("stores onReconnect callback", () => { + const onReconnect = vi.fn(); + const client = new UrbitSSEClient("https://example.com", "urbauth-~zod=123", { + onReconnect, + }); + expect(client.onReconnect).toBe(onReconnect); + }); + + it("resets reconnect attempts on successful connect", async () => { + const { urbitFetch } = await import("./fetch.js"); + const mockUrbitFetch = vi.mocked(urbitFetch); + + // Mock a response that returns a readable stream + const mockStream = new ReadableStream({ + start(controller) { + controller.close(); + }, + }); + + mockUrbitFetch.mockResolvedValue({ + response: { + ok: true, + status: 200, + body: mockStream, + } as unknown as Response, + finalUrl: "https://example.com", + release: vi.fn().mockResolvedValue(undefined), + }); + + const client = new UrbitSSEClient("https://example.com", "urbauth-~zod=123", { + autoReconnect: false, // Disable to prevent reconnect loop + }); + client.reconnectAttempts = 5; + + await client.connect(); + + expect(client.reconnectAttempts).toBe(0); + }); + }); + + describe("event acking", () => { + it("tracks lastHeardEventId and ackThreshold", () => { + const client = new UrbitSSEClient("https://example.com", "urbauth-~zod=123"); + + // Access private properties for testing + const lastHeardEventId = (client as unknown as { lastHeardEventId: number }).lastHeardEventId; + const ackThreshold = (client as unknown as { ackThreshold: number }).ackThreshold; + + expect(lastHeardEventId).toBe(-1); + expect(ackThreshold).toBeGreaterThan(0); + }); + }); + + describe("constructor", () => { + it("generates unique channel ID", () => { + const client1 = new UrbitSSEClient("https://example.com", "urbauth-~zod=123"); + const client2 = new UrbitSSEClient("https://example.com", "urbauth-~zod=123"); + + expect(client1.channelId).not.toBe(client2.channelId); + }); + + it("normalizes cookie in constructor", () => { + const client = new UrbitSSEClient( + "https://example.com", + "urbauth-~zod=123; Path=/; HttpOnly", + ); + + expect(client.cookie).toBe("urbauth-~zod=123"); + }); + + it("sets default reconnection parameters", () => { + const client = new UrbitSSEClient("https://example.com", "urbauth-~zod=123"); + + expect(client.maxReconnectAttempts).toBe(10); + expect(client.reconnectDelay).toBe(1000); + expect(client.maxReconnectDelay).toBe(30000); + }); + + it("allows overriding reconnection parameters", () => { + const client = new UrbitSSEClient("https://example.com", "urbauth-~zod=123", { + maxReconnectAttempts: 5, + reconnectDelay: 500, + maxReconnectDelay: 10000, + }); + + expect(client.maxReconnectAttempts).toBe(5); + expect(client.reconnectDelay).toBe(500); + expect(client.maxReconnectDelay).toBe(10000); }); }); }); diff --git a/extensions/tlon/src/urbit/sse-client.ts b/extensions/tlon/src/urbit/sse-client.ts index df128e51b87..897859d2fcd 100644 --- a/extensions/tlon/src/urbit/sse-client.ts +++ b/extensions/tlon/src/urbit/sse-client.ts @@ -55,6 +55,11 @@ export class UrbitSSEClient { fetchImpl?: (input: RequestInfo | URL, init?: RequestInit) => Promise; streamRelease: (() => Promise) | null = null; + // Event ack tracking - must ack every ~50 events to keep channel healthy + private lastHeardEventId = -1; + private lastAcknowledgedEventId = -1; + private readonly ackThreshold = 20; + constructor(url: string, cookie: string, options: UrbitSseOptions = {}) { const ctx = getUrbitContext(url, options.ship); this.url = ctx.baseUrl; @@ -249,8 +254,12 @@ export class UrbitSSEClient { processEvent(eventData: string) { const lines = eventData.split("\n"); let data: string | null = null; + let eventId: number | null = null; for (const line of lines) { + if (line.startsWith("id: ")) { + eventId = parseInt(line.substring(4), 10); + } if (line.startsWith("data: ")) { data = line.substring(6); } @@ -260,6 +269,21 @@ export class UrbitSSEClient { return; } + // Track event ID and send ack if needed + if (eventId !== null && !isNaN(eventId)) { + if (eventId > this.lastHeardEventId) { + this.lastHeardEventId = eventId; + if (eventId - this.lastAcknowledgedEventId > this.ackThreshold) { + this.logger.log?.( + `[SSE] Acking event ${eventId} (last acked: ${this.lastAcknowledgedEventId})`, + ); + this.ack(eventId).catch((err) => { + this.logger.error?.(`Failed to ack event ${eventId}: ${String(err)}`); + }); + } + } + } + try { const parsed = JSON.parse(data) as { id?: number; json?: unknown; response?: string }; @@ -318,17 +342,66 @@ export class UrbitSSEClient { ); } + /** + * Update the cookie used for authentication. + * Call this when re-authenticating after session expiry. + */ + updateCookie(newCookie: string): void { + this.cookie = normalizeUrbitCookie(newCookie); + } + + private async ack(eventId: number): Promise { + this.lastAcknowledgedEventId = eventId; + + const ackData = { + id: Date.now(), + action: "ack", + "event-id": eventId, + }; + + const { response, release } = await urbitFetch({ + baseUrl: this.url, + path: `/~/channel/${this.channelId}`, + init: { + method: "PUT", + headers: { + "Content-Type": "application/json", + Cookie: this.cookie, + }, + body: JSON.stringify([ackData]), + }, + ssrfPolicy: this.ssrfPolicy, + lookupFn: this.lookupFn, + fetchImpl: this.fetchImpl, + timeoutMs: 10_000, + auditContext: "tlon-urbit-ack", + }); + + try { + if (!response.ok) { + throw new Error(`Ack failed with status ${response.status}`); + } + } finally { + await release(); + } + } + async attemptReconnect() { if (this.aborted || !this.autoReconnect) { this.logger.log?.("[SSE] Reconnection aborted or disabled"); return; } + // If we've hit max attempts, wait longer then reset and keep trying if (this.reconnectAttempts >= this.maxReconnectAttempts) { - this.logger.error?.( - `[SSE] Max reconnection attempts (${this.maxReconnectAttempts}) reached. Giving up.`, + this.logger.log?.( + `[SSE] Max reconnection attempts (${this.maxReconnectAttempts}) reached. Waiting 10s before resetting...`, ); - return; + // Wait 10 seconds before resetting and trying again + const extendedBackoff = 10000; // 10 seconds + await new Promise((resolve) => setTimeout(resolve, extendedBackoff)); + this.reconnectAttempts = 0; // Reset counter to continue trying + this.logger.log?.("[SSE] Reconnection attempts reset, resuming reconnection..."); } this.reconnectAttempts += 1; diff --git a/extensions/tlon/src/urbit/story.ts b/extensions/tlon/src/urbit/story.ts new file mode 100644 index 00000000000..01a18c2eb09 --- /dev/null +++ b/extensions/tlon/src/urbit/story.ts @@ -0,0 +1,347 @@ +/** + * Tlon Story Format - Rich text converter + * + * Converts markdown-like text to Tlon's story format. + */ + +// Inline content types +export type StoryInline = + | string + | { bold: StoryInline[] } + | { italics: StoryInline[] } + | { strike: StoryInline[] } + | { blockquote: StoryInline[] } + | { "inline-code": string } + | { code: string } + | { ship: string } + | { link: { href: string; content: string } } + | { break: null } + | { tag: string }; + +// Block content types +export type StoryBlock = + | { header: { tag: "h1" | "h2" | "h3" | "h4" | "h5" | "h6"; content: StoryInline[] } } + | { code: { code: string; lang: string } } + | { image: { src: string; height: number; width: number; alt: string } } + | { rule: null } + | { listing: StoryListing }; + +export type StoryListing = + | { + list: { + type: "ordered" | "unordered" | "tasklist"; + items: StoryListing[]; + contents: StoryInline[]; + }; + } + | { item: StoryInline[] }; + +// A verse is either a block or inline content +export type StoryVerse = { block: StoryBlock } | { inline: StoryInline[] }; + +// A story is a list of verses +export type Story = StoryVerse[]; + +/** + * Parse inline markdown formatting (bold, italic, code, links, mentions) + */ +function parseInlineMarkdown(text: string): StoryInline[] { + const result: StoryInline[] = []; + let remaining = text; + + while (remaining.length > 0) { + // Ship mentions: ~sampel-palnet + const shipMatch = remaining.match(/^(~[a-z][-a-z0-9]*)/); + if (shipMatch) { + result.push({ ship: shipMatch[1] }); + remaining = remaining.slice(shipMatch[0].length); + continue; + } + + // Bold: **text** or __text__ + const boldMatch = remaining.match(/^\*\*(.+?)\*\*|^__(.+?)__/); + if (boldMatch) { + const content = boldMatch[1] || boldMatch[2]; + result.push({ bold: parseInlineMarkdown(content) }); + remaining = remaining.slice(boldMatch[0].length); + continue; + } + + // Italics: *text* or _text_ (but not inside words for _) + const italicsMatch = remaining.match(/^\*([^*]+?)\*|^_([^_]+?)_(?![a-zA-Z0-9])/); + if (italicsMatch) { + const content = italicsMatch[1] || italicsMatch[2]; + result.push({ italics: parseInlineMarkdown(content) }); + remaining = remaining.slice(italicsMatch[0].length); + continue; + } + + // Strikethrough: ~~text~~ + const strikeMatch = remaining.match(/^~~(.+?)~~/); + if (strikeMatch) { + result.push({ strike: parseInlineMarkdown(strikeMatch[1]) }); + remaining = remaining.slice(strikeMatch[0].length); + continue; + } + + // Inline code: `code` + const codeMatch = remaining.match(/^`([^`]+)`/); + if (codeMatch) { + result.push({ "inline-code": codeMatch[1] }); + remaining = remaining.slice(codeMatch[0].length); + continue; + } + + // Links: [text](url) + const linkMatch = remaining.match(/^\[([^\]]+)\]\(([^)]+)\)/); + if (linkMatch) { + result.push({ link: { href: linkMatch[2], content: linkMatch[1] } }); + remaining = remaining.slice(linkMatch[0].length); + continue; + } + + // Markdown images: ![alt](url) + const imageMatch = remaining.match(/^!\[([^\]]*)\]\(([^)]+)\)/); + if (imageMatch) { + // Return a special marker that will be hoisted to a block + result.push({ + __image: { src: imageMatch[2], alt: imageMatch[1] }, + } as unknown as StoryInline); + remaining = remaining.slice(imageMatch[0].length); + continue; + } + + // Plain URL detection + const urlMatch = remaining.match(/^(https?:\/\/[^\s<>"\]]+)/); + if (urlMatch) { + result.push({ link: { href: urlMatch[1], content: urlMatch[1] } }); + remaining = remaining.slice(urlMatch[0].length); + continue; + } + + // Hashtags: #tag - disabled, chat UI doesn't render them + // const tagMatch = remaining.match(/^#([a-zA-Z][a-zA-Z0-9_-]*)/); + // if (tagMatch) { + // result.push({ tag: tagMatch[1] }); + // remaining = remaining.slice(tagMatch[0].length); + // continue; + // } + + // Plain text: consume until next special character or URL start + // Exclude : and / to allow URL detection to work (stops before https://) + const plainMatch = remaining.match(/^[^*_`~[#~\n:/]+/); + if (plainMatch) { + result.push(plainMatch[0]); + remaining = remaining.slice(plainMatch[0].length); + continue; + } + + // Single special char that didn't match a pattern + result.push(remaining[0]); + remaining = remaining.slice(1); + } + + // Merge adjacent strings + return mergeAdjacentStrings(result); +} + +/** + * Merge adjacent string elements in an inline array + */ +function mergeAdjacentStrings(inlines: StoryInline[]): StoryInline[] { + const result: StoryInline[] = []; + for (const item of inlines) { + if (typeof item === "string" && typeof result[result.length - 1] === "string") { + result[result.length - 1] = (result[result.length - 1] as string) + item; + } else { + result.push(item); + } + } + return result; +} + +/** + * Create an image block + */ +export function createImageBlock( + src: string, + alt: string = "", + height: number = 0, + width: number = 0, +): StoryVerse { + return { + block: { + image: { src, height, width, alt }, + }, + }; +} + +/** + * Check if URL looks like an image + */ +export function isImageUrl(url: string): boolean { + const imageExtensions = /\.(jpg|jpeg|png|gif|webp|svg|bmp|ico)(\?.*)?$/i; + return imageExtensions.test(url); +} + +/** + * Process inlines and extract any image markers into blocks + */ +function processInlinesForImages(inlines: StoryInline[]): { + inlines: StoryInline[]; + imageBlocks: StoryVerse[]; +} { + const cleanInlines: StoryInline[] = []; + const imageBlocks: StoryVerse[] = []; + + for (const inline of inlines) { + if (typeof inline === "object" && "__image" in inline) { + const img = (inline as unknown as { __image: { src: string; alt: string } }).__image; + imageBlocks.push(createImageBlock(img.src, img.alt)); + } else { + cleanInlines.push(inline); + } + } + + return { inlines: cleanInlines, imageBlocks }; +} + +/** + * Convert markdown text to Tlon story format + */ +export function markdownToStory(markdown: string): Story { + const story: Story = []; + const lines = markdown.split("\n"); + let i = 0; + + while (i < lines.length) { + const line = lines[i]; + + // Code block: ```lang\ncode\n``` + if (line.startsWith("```")) { + const lang = line.slice(3).trim() || "plaintext"; + const codeLines: string[] = []; + i++; + while (i < lines.length && !lines[i].startsWith("```")) { + codeLines.push(lines[i]); + i++; + } + story.push({ + block: { + code: { + code: codeLines.join("\n"), + lang, + }, + }, + }); + i++; // skip closing ``` + continue; + } + + // Headers: # H1, ## H2, etc. + const headerMatch = line.match(/^(#{1,6})\s+(.+)$/); + if (headerMatch) { + const level = headerMatch[1].length as 1 | 2 | 3 | 4 | 5 | 6; + const tag = `h${level}` as "h1" | "h2" | "h3" | "h4" | "h5" | "h6"; + story.push({ + block: { + header: { + tag, + content: parseInlineMarkdown(headerMatch[2]), + }, + }, + }); + i++; + continue; + } + + // Horizontal rule: --- or *** + if (/^(-{3,}|\*{3,})$/.test(line.trim())) { + story.push({ block: { rule: null } }); + i++; + continue; + } + + // Blockquote: > text + if (line.startsWith("> ")) { + const quoteLines: string[] = []; + while (i < lines.length && lines[i].startsWith("> ")) { + quoteLines.push(lines[i].slice(2)); + i++; + } + const quoteText = quoteLines.join("\n"); + story.push({ + inline: [{ blockquote: parseInlineMarkdown(quoteText) }], + }); + continue; + } + + // Empty line - skip + if (line.trim() === "") { + i++; + continue; + } + + // Regular paragraph - collect consecutive non-empty lines + const paragraphLines: string[] = []; + while ( + i < lines.length && + lines[i].trim() !== "" && + !lines[i].startsWith("#") && + !lines[i].startsWith("```") && + !lines[i].startsWith("> ") && + !/^(-{3,}|\*{3,})$/.test(lines[i].trim()) + ) { + paragraphLines.push(lines[i]); + i++; + } + + if (paragraphLines.length > 0) { + const paragraphText = paragraphLines.join("\n"); + // Convert newlines within paragraph to break elements + const inlines = parseInlineMarkdown(paragraphText); + // Replace \n in strings with break elements + const withBreaks: StoryInline[] = []; + for (const inline of inlines) { + if (typeof inline === "string" && inline.includes("\n")) { + const parts = inline.split("\n"); + for (let j = 0; j < parts.length; j++) { + if (parts[j]) { + withBreaks.push(parts[j]); + } + if (j < parts.length - 1) { + withBreaks.push({ break: null }); + } + } + } else { + withBreaks.push(inline); + } + } + + // Extract any images from inlines and add as separate blocks + const { inlines: cleanInlines, imageBlocks } = processInlinesForImages(withBreaks); + + if (cleanInlines.length > 0) { + story.push({ inline: cleanInlines }); + } + story.push(...imageBlocks); + } + } + + return story; +} + +/** + * Convert plain text to simple story (no markdown parsing) + */ +export function textToStory(text: string): Story { + return [{ inline: [text] }]; +} + +/** + * Check if text contains markdown formatting + */ +export function hasMarkdown(text: string): boolean { + // Check for common markdown patterns + return /(\*\*|__|~~|`|^#{1,6}\s|^```|^\s*[-*]\s|\[.*\]\(.*\)|^>\s)/m.test(text); +} diff --git a/extensions/tlon/src/urbit/upload.test.ts b/extensions/tlon/src/urbit/upload.test.ts new file mode 100644 index 00000000000..3ff0e9fd1a0 --- /dev/null +++ b/extensions/tlon/src/urbit/upload.test.ts @@ -0,0 +1,188 @@ +import { describe, expect, it, vi, afterEach, beforeEach } from "vitest"; + +// Mock fetchWithSsrFGuard from plugin-sdk +vi.mock("openclaw/plugin-sdk", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + fetchWithSsrFGuard: vi.fn(), + }; +}); + +// Mock @tloncorp/api +vi.mock("@tloncorp/api", () => ({ + uploadFile: vi.fn(), +})); + +describe("uploadImageFromUrl", () => { + beforeEach(() => { + vi.clearAllMocks(); + }); + + afterEach(() => { + vi.restoreAllMocks(); + }); + + it("fetches image and calls uploadFile, returns uploaded URL", async () => { + const { fetchWithSsrFGuard } = await import("openclaw/plugin-sdk"); + const mockFetch = vi.mocked(fetchWithSsrFGuard); + + const { uploadFile } = await import("@tloncorp/api"); + const mockUploadFile = vi.mocked(uploadFile); + + // Mock fetchWithSsrFGuard to return a successful response with a blob + const mockBlob = new Blob(["fake-image"], { type: "image/png" }); + mockFetch.mockResolvedValue({ + response: { + ok: true, + headers: new Headers({ "content-type": "image/png" }), + blob: () => Promise.resolve(mockBlob), + } as unknown as Response, + finalUrl: "https://example.com/image.png", + release: vi.fn().mockResolvedValue(undefined), + }); + + // Mock uploadFile to return a successful upload + mockUploadFile.mockResolvedValue({ url: "https://memex.tlon.network/uploaded.png" }); + + const { uploadImageFromUrl } = await import("./upload.js"); + const result = await uploadImageFromUrl("https://example.com/image.png"); + + expect(result).toBe("https://memex.tlon.network/uploaded.png"); + expect(mockUploadFile).toHaveBeenCalledTimes(1); + expect(mockUploadFile).toHaveBeenCalledWith( + expect.objectContaining({ + blob: mockBlob, + contentType: "image/png", + }), + ); + }); + + it("returns original URL if fetch fails", async () => { + const { fetchWithSsrFGuard } = await import("openclaw/plugin-sdk"); + const mockFetch = vi.mocked(fetchWithSsrFGuard); + + // Mock fetchWithSsrFGuard to return a failed response + mockFetch.mockResolvedValue({ + response: { + ok: false, + status: 404, + } as unknown as Response, + finalUrl: "https://example.com/image.png", + release: vi.fn().mockResolvedValue(undefined), + }); + + const { uploadImageFromUrl } = await import("./upload.js"); + const result = await uploadImageFromUrl("https://example.com/image.png"); + + expect(result).toBe("https://example.com/image.png"); + }); + + it("returns original URL if upload fails", async () => { + const { fetchWithSsrFGuard } = await import("openclaw/plugin-sdk"); + const mockFetch = vi.mocked(fetchWithSsrFGuard); + + const { uploadFile } = await import("@tloncorp/api"); + const mockUploadFile = vi.mocked(uploadFile); + + // Mock fetchWithSsrFGuard to return a successful response + const mockBlob = new Blob(["fake-image"], { type: "image/png" }); + mockFetch.mockResolvedValue({ + response: { + ok: true, + headers: new Headers({ "content-type": "image/png" }), + blob: () => Promise.resolve(mockBlob), + } as unknown as Response, + finalUrl: "https://example.com/image.png", + release: vi.fn().mockResolvedValue(undefined), + }); + + // Mock uploadFile to throw an error + mockUploadFile.mockRejectedValue(new Error("Upload failed")); + + const { uploadImageFromUrl } = await import("./upload.js"); + const result = await uploadImageFromUrl("https://example.com/image.png"); + + expect(result).toBe("https://example.com/image.png"); + }); + + it("rejects non-http(s) URLs", async () => { + const { uploadImageFromUrl } = await import("./upload.js"); + + // file:// URL should be rejected + const result = await uploadImageFromUrl("file:///etc/passwd"); + expect(result).toBe("file:///etc/passwd"); + + // ftp:// URL should be rejected + const result2 = await uploadImageFromUrl("ftp://example.com/image.png"); + expect(result2).toBe("ftp://example.com/image.png"); + }); + + it("handles invalid URLs gracefully", async () => { + const { uploadImageFromUrl } = await import("./upload.js"); + + // Invalid URL should return original + const result = await uploadImageFromUrl("not-a-valid-url"); + expect(result).toBe("not-a-valid-url"); + }); + + it("extracts filename from URL path", async () => { + const { fetchWithSsrFGuard } = await import("openclaw/plugin-sdk"); + const mockFetch = vi.mocked(fetchWithSsrFGuard); + + const { uploadFile } = await import("@tloncorp/api"); + const mockUploadFile = vi.mocked(uploadFile); + + const mockBlob = new Blob(["fake-image"], { type: "image/jpeg" }); + mockFetch.mockResolvedValue({ + response: { + ok: true, + headers: new Headers({ "content-type": "image/jpeg" }), + blob: () => Promise.resolve(mockBlob), + } as unknown as Response, + finalUrl: "https://example.com/path/to/my-image.jpg", + release: vi.fn().mockResolvedValue(undefined), + }); + + mockUploadFile.mockResolvedValue({ url: "https://memex.tlon.network/uploaded.jpg" }); + + const { uploadImageFromUrl } = await import("./upload.js"); + await uploadImageFromUrl("https://example.com/path/to/my-image.jpg"); + + expect(mockUploadFile).toHaveBeenCalledWith( + expect.objectContaining({ + fileName: "my-image.jpg", + }), + ); + }); + + it("uses default filename when URL has no path", async () => { + const { fetchWithSsrFGuard } = await import("openclaw/plugin-sdk"); + const mockFetch = vi.mocked(fetchWithSsrFGuard); + + const { uploadFile } = await import("@tloncorp/api"); + const mockUploadFile = vi.mocked(uploadFile); + + const mockBlob = new Blob(["fake-image"], { type: "image/png" }); + mockFetch.mockResolvedValue({ + response: { + ok: true, + headers: new Headers({ "content-type": "image/png" }), + blob: () => Promise.resolve(mockBlob), + } as unknown as Response, + finalUrl: "https://example.com/", + release: vi.fn().mockResolvedValue(undefined), + }); + + mockUploadFile.mockResolvedValue({ url: "https://memex.tlon.network/uploaded.png" }); + + const { uploadImageFromUrl } = await import("./upload.js"); + await uploadImageFromUrl("https://example.com/"); + + expect(mockUploadFile).toHaveBeenCalledWith( + expect.objectContaining({ + fileName: expect.stringMatching(/^upload-\d+\.png$/), + }), + ); + }); +}); diff --git a/extensions/tlon/src/urbit/upload.ts b/extensions/tlon/src/urbit/upload.ts new file mode 100644 index 00000000000..0c01483991b --- /dev/null +++ b/extensions/tlon/src/urbit/upload.ts @@ -0,0 +1,60 @@ +/** + * Upload an image from a URL to Tlon storage. + */ +import { uploadFile } from "@tloncorp/api"; +import { fetchWithSsrFGuard } from "openclaw/plugin-sdk"; +import { getDefaultSsrFPolicy } from "./context.js"; + +/** + * Fetch an image from a URL and upload it to Tlon storage. + * Returns the uploaded URL, or falls back to the original URL on error. + * + * Note: configureClient must be called before using this function. + */ +export async function uploadImageFromUrl(imageUrl: string): Promise { + try { + // Validate URL is http/https before fetching + const url = new URL(imageUrl); + if (url.protocol !== "http:" && url.protocol !== "https:") { + console.warn(`[tlon] Rejected non-http(s) URL: ${imageUrl}`); + return imageUrl; + } + + // Fetch the image with SSRF protection + // Use fetchWithSsrFGuard directly (not urbitFetch) to preserve the full URL path + const { response, release } = await fetchWithSsrFGuard({ + url: imageUrl, + init: { method: "GET" }, + policy: getDefaultSsrFPolicy(), + auditContext: "tlon-upload-image", + }); + + try { + if (!response.ok) { + console.warn(`[tlon] Failed to fetch image from ${imageUrl}: ${response.status}`); + return imageUrl; + } + + const contentType = response.headers.get("content-type") || "image/png"; + const blob = await response.blob(); + + // Extract filename from URL or use a default + const urlPath = new URL(imageUrl).pathname; + const fileName = urlPath.split("/").pop() || `upload-${Date.now()}.png`; + + // Upload to Tlon storage + const result = await uploadFile({ + blob, + fileName, + contentType, + }); + + return result.url; + } finally { + await release(); + } + } catch (err) { + console.warn(`[tlon] Failed to upload image, using original URL: ${err}`); + return imageUrl; + } +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index fca1f7935e9..8286468513f 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -436,9 +436,18 @@ importers: extensions/tlon: dependencies: + '@tloncorp/api': + specifier: github:tloncorp/api-beta#main + version: https://codeload.github.com/tloncorp/api-beta/tar.gz/7eede1c1a756977b09f96aa14a92e2b06318ae87 + '@tloncorp/tlon-skill': + specifier: 0.1.9 + version: 0.1.9 '@urbit/aura': specifier: ^3.0.0 version: 3.0.0 + '@urbit/http-api': + specifier: ^3.0.0 + version: 3.0.0 extensions/twitch: dependencies: @@ -556,6 +565,12 @@ packages: resolution: {integrity: sha512-nLbCWqQNgUiwwtFsen1AdzAtvuLRsQS8rYgMuxCrdKf9kOssamGLuPwyTY9wyYblNr9+1XM8v6zoDTPPSIeANg==} engines: {node: '>=16.0.0'} + '@aws-crypto/crc32c@5.2.0': + resolution: {integrity: sha512-+iWb8qaHLYKrNvGRbiYRHSdKRWhto5XlZUEBwDjYNf+ly5SVYG6zEoYIdxvf5R3zyeP16w4PLBn3rH1xc74Rag==} + + '@aws-crypto/sha1-browser@5.2.0': + resolution: {integrity: sha512-OH6lveCFfcDjX4dbAvCFSYUjJZjDr/3XJ3xHtjn3Oj5b9RjojQo8npoLeA/bNwkOkrSQ0wgrHzXk4tDRxGKJeg==} + '@aws-crypto/sha256-browser@5.2.0': resolution: {integrity: sha512-AXfN/lGotSQwu6HNcEsIASo7kWXZ5HYWvfOmSNKDsEqC4OashTp8alTmaz+F7TC2L083SFv5RdB+qU3Vs1kZqw==} @@ -577,10 +592,18 @@ packages: resolution: {integrity: sha512-wGU8uJXrPW/hZuHdPNVe1kAFIBiKcslBcoDBN0eYBzS13um8p5jJiQJ9WsD1nSpKCmyx7qZXc6xjcbIQPyOrrA==} engines: {node: '>=20.0.0'} + '@aws-sdk/client-s3@3.1000.0': + resolution: {integrity: sha512-7kPy33qNGq3NfwHC0412T6LDK1bp4+eiPzetX0sVd9cpTSXuQDKpoOFnB0Njj6uZjJDcLS3n2OeyarwwgkQ0Ow==} + engines: {node: '>=20.0.0'} + '@aws-sdk/core@3.973.15': resolution: {integrity: sha512-AlC0oQ1/mdJ8vCIqu524j5RB7M8i8E24bbkZmya1CuiQxkY7SdIZAyw7NDNMGaNINQFq/8oGRMX0HeOfCVsl/A==} engines: {node: '>=20.0.0'} + '@aws-sdk/crc64-nvme@3.972.3': + resolution: {integrity: sha512-UExeK+EFiq5LAcbHm96CQLSia+5pvpUVSAsVApscBzayb7/6dJBJKwV4/onsk4VbWSmqxDMcfuTD+pC4RxgZHg==} + engines: {node: '>=20.0.0'} + '@aws-sdk/credential-provider-env@3.972.13': resolution: {integrity: sha512-6ljXKIQ22WFKyIs1jbORIkGanySBHaPPTOI4OxACP5WXgbcR0nDYfqNJfXEGwCK7IzHdNbCSFsNKKs0qCexR8Q==} engines: {node: '>=20.0.0'} @@ -617,14 +640,30 @@ packages: resolution: {integrity: sha512-mKPiiVssgFDWkAXdEDh8+wpr2pFSX/fBn2onXXnrfIAYbdZhYb4WilKbZ3SJMUnQi+Y48jZMam5J0RrgARluaA==} engines: {node: '>=20.0.0'} + '@aws-sdk/middleware-bucket-endpoint@3.972.6': + resolution: {integrity: sha512-3H2bhvb7Cb/S6WFsBy/Dy9q2aegC9JmGH1inO8Lb2sWirSqpLJlZmvQHPE29h2tIxzv6el/14X/tLCQ8BQU6ZQ==} + engines: {node: '>=20.0.0'} + '@aws-sdk/middleware-eventstream@3.972.6': resolution: {integrity: sha512-mB2+3G/oxRC+y9WRk0KCdradE2rSfxxJpcOSmAm+vDh3ex3WQHVLZ1catNIe1j5NQ+3FLBsNMRPVGkZ43PRpjw==} engines: {node: '>=20.0.0'} + '@aws-sdk/middleware-expect-continue@3.972.6': + resolution: {integrity: sha512-QMdffpU+GkSGC+bz6WdqlclqIeCsOfgX8JFZ5xvwDtX+UTj4mIXm3uXu7Ko6dBseRcJz1FA6T9OmlAAY6JgJUg==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/middleware-flexible-checksums@3.973.1': + resolution: {integrity: sha512-QLXsxsI6VW8LuGK+/yx699wzqP/NMCGk/hSGP+qtB+Lcff+23UlbahyouLlk+nfT7Iu021SkXBhnAuVd6IZcPw==} + engines: {node: '>=20.0.0'} + '@aws-sdk/middleware-host-header@3.972.6': resolution: {integrity: sha512-5XHwjPH1lHB+1q4bfC7T8Z5zZrZXfaLcjSMwTd1HPSPrCmPFMbg3UQ5vgNWcVj0xoX4HWqTGkSf2byrjlnRg5w==} engines: {node: '>=20.0.0'} + '@aws-sdk/middleware-location-constraint@3.972.6': + resolution: {integrity: sha512-XdZ2TLwyj3Am6kvUc67vquQvs6+D8npXvXgyEUJAdkUDx5oMFJKOqpK+UpJhVDsEL068WAJl2NEGzbSik7dGJQ==} + engines: {node: '>=20.0.0'} + '@aws-sdk/middleware-logger@3.972.6': resolution: {integrity: sha512-iFnaMFMQdljAPrvsCVKYltPt2j40LQqukAbXvW7v0aL5I+1GO7bZ/W8m12WxW3gwyK5p5u1WlHg8TSAizC5cZw==} engines: {node: '>=20.0.0'} @@ -633,6 +672,14 @@ packages: resolution: {integrity: sha512-dY4v3of5EEMvik6+UDwQ96KfUFDk8m1oZDdkSc5lwi4o7rFrjnv0A+yTV+gu230iybQZnKgDLg/rt2P3H+Vscw==} engines: {node: '>=20.0.0'} + '@aws-sdk/middleware-sdk-s3@3.972.15': + resolution: {integrity: sha512-WDLgssevOU5BFx1s8jA7jj6cE5HuImz28sy9jKOaVtz0AW1lYqSzotzdyiybFaBcQTs5zxXOb2pUfyMxgEKY3Q==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/middleware-ssec@3.972.6': + resolution: {integrity: sha512-acvMUX9jF4I2Ew+Z/EA6gfaFaz9ehci5wxBmXCZeulLuv8m+iGf6pY9uKz8TPjg39bdAz3hxoE0eLP8Qz+IYlA==} + engines: {node: '>=20.0.0'} + '@aws-sdk/middleware-user-agent@3.972.15': resolution: {integrity: sha512-ABlFVcIMmuRAwBT+8q5abAxOr7WmaINirDJBnqGY5b5jSDo00UMlg/G4a0xoAgwm6oAECeJcwkvDlxDwKf58fQ==} engines: {node: '>=20.0.0'} @@ -649,6 +696,14 @@ packages: resolution: {integrity: sha512-Aa5PusHLXAqLTX1UKDvI3pHQJtIsF7Q+3turCHqfz/1F61/zDMWfbTC8evjhrrYVAtz9Vsv3SJ/waSUeu7B6gw==} engines: {node: '>=20.0.0'} + '@aws-sdk/s3-request-presigner@3.1000.0': + resolution: {integrity: sha512-DP6EbwCD0CKzBwBnT1X6STB5i+bY765CxjMbWCATDhCgOB343Q6AHM9c1S/300Uc5waXWtI/Wdeak9Ru56JOvg==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/signature-v4-multi-region@3.996.3': + resolution: {integrity: sha512-gQYI/Buwp0CAGQxY7mR5VzkP56rkWq2Y1ROkFuXh5XY94DsSjJw62B3I0N0lysQmtwiL2ht2KHI9NylM/RP4FA==} + engines: {node: '>=20.0.0'} + '@aws-sdk/token-providers@3.1000.0': resolution: {integrity: sha512-eOI+8WPtWpLdlYBGs8OCK3k5uIMUHVsNG3AFO4kaRaZcKReJ/2OO6+2O2Dd/3vTzM56kRjSKe7mBOCwa4PdYqg==} engines: {node: '>=20.0.0'} @@ -661,6 +716,10 @@ packages: resolution: {integrity: sha512-RW60aH26Bsc016Y9B98hC0Plx6fK5P2v/iQYwMzrSjiDh1qRMUCP6KrXHYEHe3uFvKiOC93Z9zk4BJsUi6Tj1Q==} engines: {node: '>=20.0.0'} + '@aws-sdk/util-arn-parser@3.972.2': + resolution: {integrity: sha512-VkykWbqMjlSgBFDyrY3nOSqupMc6ivXuGmvci6Q3NnLq5kC+mKQe2QBZ4nrWRE/jqOxeFP2uYzLtwncYYcvQDg==} + engines: {node: '>=20.0.0'} + '@aws-sdk/util-endpoints@3.996.3': resolution: {integrity: sha512-yWIQSNiCjykLL+ezN5A+DfBb1gfXTytBxm57e64lYmwxDHNmInYHRJYYRAGWG1o77vKEiWaw4ui28e3yb1k5aQ==} engines: {node: '>=20.0.0'} @@ -2526,6 +2585,14 @@ packages: resolution: {integrity: sha512-qocxM/X4XGATqQtUkbE9SPUB6wekBi+FyJOMbPj0AhvyvFGYEmOlz6VB22iMePCQsFmMIvFSeViDvA7mZJG47g==} engines: {node: '>=18.0.0'} + '@smithy/chunked-blob-reader-native@4.2.2': + resolution: {integrity: sha512-QzzYIlf4yg0w5TQaC9VId3B3ugSk1MI/wb7tgcHtd7CBV9gNRKZrhc2EPSxSZuDy10zUZ0lomNMgkc6/VVe8xg==} + engines: {node: '>=18.0.0'} + + '@smithy/chunked-blob-reader@5.2.1': + resolution: {integrity: sha512-y5d4xRiD6TzeP5BWlb+Ig/VFqF+t9oANNhGeMqyzU7obw7FYgTgVi50i5JqBTeKp+TABeDIeeXFZdz65RipNtA==} + engines: {node: '>=18.0.0'} + '@smithy/config-resolver@4.4.9': resolution: {integrity: sha512-ejQvXqlcU30h7liR9fXtj7PIAau1t/sFbJpgWPfiYDs7zd16jpH0IsSXKcba2jF6ChTXvIjACs27kNMc5xxE2Q==} engines: {node: '>=18.0.0'} @@ -2562,10 +2629,18 @@ packages: resolution: {integrity: sha512-wbTRjOxdFuyEg0CpumjZO0hkUl+fetJFqxNROepuLIoijQh51aMBmzFLfoQdwRjxsuuS2jizzIUTjPWgd8pd7g==} engines: {node: '>=18.0.0'} + '@smithy/hash-blob-browser@4.2.11': + resolution: {integrity: sha512-DrcAx3PM6AEbWZxsKl6CWAGnVwiz28Wp1ZhNu+Hi4uI/6C1PIZBIaPM2VoqBDAsOWbM6ZVzOEQMxFLLdmb4eBQ==} + engines: {node: '>=18.0.0'} + '@smithy/hash-node@4.2.10': resolution: {integrity: sha512-1VzIOI5CcsvMDvP3iv1vG/RfLJVVVc67dCRyLSB2Hn9SWCZrDO3zvcIzj3BfEtqRW5kcMg5KAeVf1K3dR6nD3w==} engines: {node: '>=18.0.0'} + '@smithy/hash-stream-node@4.2.10': + resolution: {integrity: sha512-w78xsYrOlwXKwN5tv1GnKIRbHb1HygSpeZMP6xDxCPGf1U/xDHjCpJu64c5T35UKyEPwa0bPeIcvU69VY3khUA==} + engines: {node: '>=18.0.0'} + '@smithy/invalid-dependency@4.2.10': resolution: {integrity: sha512-vy9KPNSFUU0ajFYk0sDZIYiUlAWGEAhRfehIr5ZkdFrRFTAuXEPUd41USuqHU6vvLX4r6Q9X7MKBco5+Il0Org==} engines: {node: '>=18.0.0'} @@ -2578,6 +2653,10 @@ packages: resolution: {integrity: sha512-Yfu664Qbf1B4IYIsYgKoABt010daZjkaCRvdU/sPnZG6TtHOB0md0RjNdLGzxe5UIdn9js4ftPICzmkRa9RJ4Q==} engines: {node: '>=18.0.0'} + '@smithy/md5-js@4.2.10': + resolution: {integrity: sha512-Op+Dh6dPLWTjWITChFayDllIaCXRofOed8ecpggTC5fkh8yXes0vAEX7gRUfjGK+TlyxoCAA05gHbZW/zB9JwQ==} + engines: {node: '>=18.0.0'} + '@smithy/middleware-content-length@4.2.10': resolution: {integrity: sha512-TQZ9kX5c6XbjhaEBpvhSvMEZ0klBs1CFtOdPFwATZSbC9UeQfKHPLPN9Y+I6wZGMOavlYTOlHEPDrt42PMSH9w==} engines: {node: '>=18.0.0'} @@ -2710,6 +2789,10 @@ packages: resolution: {integrity: sha512-DSIwNaWtmzrNQHv8g7DBGR9mulSit65KSj5ymGEIAknmIN8IpbZefEep10LaMG/P/xquwbmJ1h9ectz8z6mV6g==} engines: {node: '>=18.0.0'} + '@smithy/util-waiter@4.2.10': + resolution: {integrity: sha512-4eTWph/Lkg1wZEDAyObwme0kmhEb7J/JjibY2znJdrYRgKbKqB7YoEhhJVJ4R1g/SYih4zuwX7LpJaM8RsnTVg==} + engines: {node: '>=18.0.0'} + '@smithy/uuid@1.1.1': resolution: {integrity: sha512-dSfDCeihDmZlV2oyr0yWPTUfh07suS+R5OB+FZGiv/hHyK3hrFBW5rR1UYjfa57vBsrP9lciFkRPzebaV1Qujw==} engines: {node: '>=18.0.0'} @@ -2819,6 +2902,38 @@ packages: resolution: {integrity: sha512-5Kc5CM2Ysn3vTTArBs2vESUt0AQiWZA86yc1TI3B+lxXmtEq133C1nxXNOgnzhrivdPZIh3zLj5gDnZjoLL5GA==} engines: {node: '>=12.17.0'} + '@tloncorp/api@https://codeload.github.com/tloncorp/api-beta/tar.gz/7eede1c1a756977b09f96aa14a92e2b06318ae87': + resolution: {tarball: https://codeload.github.com/tloncorp/api-beta/tar.gz/7eede1c1a756977b09f96aa14a92e2b06318ae87} + version: 0.0.2 + + '@tloncorp/tlon-skill-darwin-arm64@0.1.9': + resolution: {integrity: sha512-qhsblq0zx6Ugsf7++IGY+ai3uQYAS4XsFLCnQqxbenzPcnWLnDFvzpn+cBVMmXYJXxmOIUjI9Vk929vUkPQbTw==} + cpu: [arm64] + os: [darwin] + hasBin: true + + '@tloncorp/tlon-skill-darwin-x64@0.1.9': + resolution: {integrity: sha512-tmEZv1fx86Rt7Y9OpTG+zTpHisjHcI7c6D0+p9kellPE9fa6qGG2lC4lcYNMsPXSjzmzznJNWcd0ltQW4/NHEQ==} + cpu: [x64] + os: [darwin] + hasBin: true + + '@tloncorp/tlon-skill-linux-arm64@0.1.9': + resolution: {integrity: sha512-+EXkUmlcMTY1DkAkQTE+eRHAyrWunAgOthaTVG4zYU9B4eyXC3MstMId6EaAXkv89HZ3vMqAAW4CCDxpxIzg5Q==} + cpu: [arm64] + os: [linux] + hasBin: true + + '@tloncorp/tlon-skill-linux-x64@0.1.9': + resolution: {integrity: sha512-x09fR3H2kSCfzTsB2e2ajRLlN8ANSeTHvyXEy+emHhohlLHMacSoHLgYccR4oK7TrE8iCexYZYLGypXSk8FmZQ==} + cpu: [x64] + os: [linux] + hasBin: true + + '@tloncorp/tlon-skill@0.1.9': + resolution: {integrity: sha512-uBLh2GLX8X9Dbyv84FakNbZwsrA4vEBBGzSXwevQtO/7ttbHU18zQsQKv9NFTWrTJtQ8yUkZjb5F4bmYHuXRIw==} + hasBin: true + '@tokenizer/inflate@0.4.1': resolution: {integrity: sha512-2mAv+8pkG6GIZiF1kNg1jAjh27IDxEPKwdGul3snfztFerfPGI1LjDezZp3i7BElXompqEtPmoPx6c2wgtWsOA==} engines: {node: '>=18'} @@ -3033,6 +3148,12 @@ packages: resolution: {integrity: sha512-N8/FHc/lmlMDCumMuTXyRHCxlov5KZY6unmJ9QR2GOw+OpROZMBsXYGwE+ZMtvN21ql9+Xb8KhGNBj08IrG3Wg==} engines: {node: '>=16', npm: '>=8'} + '@urbit/http-api@3.0.0': + resolution: {integrity: sha512-EmyPbWHWXhfYQ/9wWFcLT53VvCn8ct9ljd6QEe+UBjNPEhUPOFBLpDsDp3iPLQgg8ykSU8JMMHxp95LHCorExA==} + + '@urbit/nockjs@1.6.0': + resolution: {integrity: sha512-f2xCIxoYQh+bp/p6qztvgxnhGsnUwcrSSvW2CUKX7BPPVkDNppQCzCVPWo38TbqgChE7wh6rC1pm6YNCOyFlQA==} + '@vector-im/matrix-bot-sdk@0.8.0-element.3': resolution: {integrity: sha512-2FFo/Kz2vTnOZDv59Q0s803LHf7KzuQ2EwOYYAtO0zUKJ8pV5CPsVC/IHyFb+Fsxl3R9XWFiX529yhslb4v9cQ==} engines: {node: '>=22.0.0'} @@ -3194,6 +3315,10 @@ packages: resolution: {integrity: sha512-HqZ5rWlFjGiV0tDm3UxxgNRqsOTniqoKZu0pIAfh7TZQMGuZK+hH0drySty0si0QXj1ieop4+SkSfPZBPPkHig==} engines: {node: '>=14'} + any-ascii@0.3.3: + resolution: {integrity: sha512-8hm+zPrc1VnlxD5eRgMo9F9k2wEMZhbZVLKwA/sPKIt6ywuz7bI9uV/yb27uvc8fv8q6Wl2piJT51q1saKX0Jw==} + engines: {node: '>=12.20'} + any-promise@1.3.0: resolution: {integrity: sha512-7UvmKalWRt1wgjL1RrGxoSJW/0QZFIegpeGvZG9kjp8vrRu55XTHbwnqq2GpXm9uLbcuhxm3IqX9OB4MZR1b2A==} @@ -3317,6 +3442,10 @@ packages: before-after-hook@4.0.0: resolution: {integrity: sha512-q6tR3RPqIB1pMiTRMFcZwuG5T8vwp+vUvEG0vuI6B+Rikh5BfPp2fQ82c925FOs+b0lcFQ8CFrL+KbilfZFhOQ==} + big-integer@1.6.52: + resolution: {integrity: sha512-QxD8cf2eVqJOOz63z6JIN9BzvVs/dlySa5HGSBH5xtR8dPteIRQnBxxKqkNTiT6jbDTF6jAfrd4oMcND9RGbQg==} + engines: {node: '>=0.6'} + bignumber.js@9.3.1: resolution: {integrity: sha512-Ko0uX15oIUS7wJ3Rb30Fs6SkVbLmPBAKdlm7q9+ak9bbIeFf0MwuBsQV6z7+X768/cHsfg+WlysDWJcmthjsjQ==} @@ -3347,6 +3476,12 @@ packages: resolution: {integrity: sha512-fy6KJm2RawA5RcHkLa1z/ScpBeA762UF9KmZQxwIbDtRJrgLzM10depAiEQ+CXYcoiqW1/m96OAAoke2nE9EeA==} engines: {node: 18 || 20 || >=22} + browser-or-node@1.3.0: + resolution: {integrity: sha512-0F2z/VSnLbmEeBcUrSuDH5l0HxTXdQQzLjkmBR4cYfvg1zJrKSlmIZFqyFR8oX0NrwPhy3c3HQ6i3OxMbew4Tg==} + + browser-or-node@3.0.0: + resolution: {integrity: sha512-iczIdVJzGEYhP5DqQxYM9Hh7Ztpqqi+CXZpSmX8ALFs9ecXkQIeqRyM6TfxEfMVpwhl3dSuDvxdzzo9sUOIVBQ==} + buffer-crc32@0.2.13: resolution: {integrity: sha512-VO9Ht/+p3SN7SKWqcrgEzjGbRSJYTx+Q1pTQC0wrWqHx0vpJraQ6GtHx8tvcg1rlK1byhU5gccxgOgj7B0TDkQ==} @@ -3356,6 +3491,9 @@ packages: buffer-from@1.1.2: resolution: {integrity: sha512-E+XQCRwSbaaiChtv6k6Dwgc+bx+Bs6vuKJHHl5kox/BaKbhiXzqQOwK4cO22yElGp2OCmjwVhT3HmxgyPGnJfQ==} + buffer@6.0.3: + resolution: {integrity: sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA==} + bun-types@1.3.9: resolution: {integrity: sha512-+UBWWOakIP4Tswh0Bt0QD0alpTY8cb5hvgiYeWCMet9YukHbzuruIEeXC2D7nMJPB12kbh8C7XJykSexEqGKJg==} @@ -3520,6 +3658,9 @@ packages: resolution: {integrity: sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w==} engines: {node: '>= 0.6'} + core-js@3.48.0: + resolution: {integrity: sha512-zpEHTy1fjTMZCKLHUZoVeylt9XrzaIN2rbPXEt0k+q7JE5CkCZdo6bNq55bn24a69CH7ErAVLKijxJja4fw+UQ==} + core-util-is@1.0.2: resolution: {integrity: sha512-3lqz5YjWTYnW6dlDa5TLaTCcShfar1e40rmcJVwCBJC6mWlFuj0eCHIElmG1g5kyuJ/GD+8Wn4FFCcz4gJPfaQ==} @@ -3562,6 +3703,9 @@ packages: resolution: {integrity: sha512-7hvf7/GW8e86rW0ptuwS3OcBGDjIi6SZva7hCyWC0yYry2cOPmLIjXAUHI6DK2HsnwJd9ifmt57i8eV2n4YNpw==} engines: {node: '>= 14'} + date-fns@3.6.0: + resolution: {integrity: sha512-fRHTG8g/Gif+kSh50gaGEdToemgfj74aRX3swtiouboip5JDLAyDE9F11nHMIcvOaXeOC6D7SpNhi7uFyB7Uww==} + debug@2.6.9: resolution: {integrity: sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==} peerDependencies: @@ -3784,6 +3928,9 @@ packages: resolution: {integrity: sha512-knvyeauYhqjOYvQ66MznSMs83wmHrCycNEN6Ao+2AeYEfxUIkuiVxdEa1qlGEPK+We3n0THiDciYSsCcgW/DoA==} engines: {node: '>=12.0.0'} + exponential-backoff@3.1.3: + resolution: {integrity: sha512-ZgEeZXj30q+I0EN+CbSSpIyPaJ5HVQD18Z1m+u1FXbAeT94mr1zw50q4q6jiiC447Nl/YTcIYSAftiGqetwXCA==} + express@4.22.1: resolution: {integrity: sha512-F2X8g9P1X7uCPZMA3MVf9wcTqlyNp7IhH5qPCI0izhaOIYXaW9L535tGA3qmjRzpH+bZczqq7hVKxTR4NWnu+g==} engines: {node: '>= 0.10.0'} @@ -4295,6 +4442,9 @@ packages: leac@0.6.0: resolution: {integrity: sha512-y+SqErxb8h7nE/fiEX07jsbuhrpO9lL8eca7/Y1nuWV2moNlXhyd59iDGcRf6moVyDMbmTNzL40SUyrFU/yDpg==} + libphonenumber-js@1.12.38: + resolution: {integrity: sha512-vwzxmasAy9hZigxtqTbFEwp8ZdZ975TiqVDwj5bKx5sR+zi5ucUQy9mbVTkKM9GzqdLdxux/hTw2nmN5J7POMA==} + lie@3.3.0: resolution: {integrity: sha512-UaiMJzeWRlEujzAuw5LokY1L5ecNQYZKfmyZ9L7wDHb/p5etKaxXhohBcrw0EYby+G/NA52vRSN4N39dxHAIwQ==} @@ -5370,6 +5520,9 @@ packages: sonic-boom@4.2.1: resolution: {integrity: sha512-w6AxtubXa2wTXAUsZMMWERrsIRAdrK0Sc+FUytWvYAhBJLyuI4llrMIC1DtlNSdI99EI86KZum2MMq3EAZlF9Q==} + sorted-btree@1.8.1: + resolution: {integrity: sha512-395+XIP+wqNn3USkFSrNz7G3Ss/MXlZEqesxvzCRFwL14h6e8LukDHdLBePn5pwbm5OQ9vGu8mDyz2lLDIqamQ==} + source-map-js@1.2.1: resolution: {integrity: sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==} engines: {node: '>=0.10.0'} @@ -5743,6 +5896,10 @@ packages: resolution: {integrity: sha512-hVDIBwsRruT73PbK7uP5ebUt+ezEtCmzZz3F59BSr2F6OVFnJ/6h8liuvdLrQ88Xmnk6/+xGGuq+pG9WwTuy3A==} engines: {node: ^20.17.0 || >=22.9.0} + validator@13.15.26: + resolution: {integrity: sha512-spH26xU080ydGggxRyR1Yhcbgx+j3y5jbNXk/8L+iRvdIEQ4uTRH2Sgf2dokud6Q4oAtsbNvJ1Ft+9xmm6IZcA==} + engines: {node: '>= 0.10'} + vary@1.1.2: resolution: {integrity: sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==} engines: {node: '>= 0.8'} @@ -5967,6 +6124,21 @@ snapshots: '@aws-sdk/types': 3.973.4 tslib: 2.8.1 + '@aws-crypto/crc32c@5.2.0': + dependencies: + '@aws-crypto/util': 5.2.0 + '@aws-sdk/types': 3.973.4 + tslib: 2.8.1 + + '@aws-crypto/sha1-browser@5.2.0': + dependencies: + '@aws-crypto/supports-web-crypto': 5.2.0 + '@aws-crypto/util': 5.2.0 + '@aws-sdk/types': 3.973.4 + '@aws-sdk/util-locate-window': 3.965.4 + '@smithy/util-utf8': 2.3.0 + tslib: 2.8.1 + '@aws-crypto/sha256-browser@5.2.0': dependencies: '@aws-crypto/sha256-js': 5.2.0 @@ -6090,6 +6262,66 @@ snapshots: transitivePeerDependencies: - aws-crt + '@aws-sdk/client-s3@3.1000.0': + dependencies: + '@aws-crypto/sha1-browser': 5.2.0 + '@aws-crypto/sha256-browser': 5.2.0 + '@aws-crypto/sha256-js': 5.2.0 + '@aws-sdk/core': 3.973.15 + '@aws-sdk/credential-provider-node': 3.972.14 + '@aws-sdk/middleware-bucket-endpoint': 3.972.6 + '@aws-sdk/middleware-expect-continue': 3.972.6 + '@aws-sdk/middleware-flexible-checksums': 3.973.1 + '@aws-sdk/middleware-host-header': 3.972.6 + '@aws-sdk/middleware-location-constraint': 3.972.6 + '@aws-sdk/middleware-logger': 3.972.6 + '@aws-sdk/middleware-recursion-detection': 3.972.6 + '@aws-sdk/middleware-sdk-s3': 3.972.15 + '@aws-sdk/middleware-ssec': 3.972.6 + '@aws-sdk/middleware-user-agent': 3.972.15 + '@aws-sdk/region-config-resolver': 3.972.6 + '@aws-sdk/signature-v4-multi-region': 3.996.3 + '@aws-sdk/types': 3.973.4 + '@aws-sdk/util-endpoints': 3.996.3 + '@aws-sdk/util-user-agent-browser': 3.972.6 + '@aws-sdk/util-user-agent-node': 3.973.0 + '@smithy/config-resolver': 4.4.9 + '@smithy/core': 3.23.6 + '@smithy/eventstream-serde-browser': 4.2.10 + '@smithy/eventstream-serde-config-resolver': 4.3.10 + '@smithy/eventstream-serde-node': 4.2.10 + '@smithy/fetch-http-handler': 5.3.11 + '@smithy/hash-blob-browser': 4.2.11 + '@smithy/hash-node': 4.2.10 + '@smithy/hash-stream-node': 4.2.10 + '@smithy/invalid-dependency': 4.2.10 + '@smithy/md5-js': 4.2.10 + '@smithy/middleware-content-length': 4.2.10 + '@smithy/middleware-endpoint': 4.4.20 + '@smithy/middleware-retry': 4.4.37 + '@smithy/middleware-serde': 4.2.11 + '@smithy/middleware-stack': 4.2.10 + '@smithy/node-config-provider': 4.3.10 + '@smithy/node-http-handler': 4.4.12 + '@smithy/protocol-http': 5.3.10 + '@smithy/smithy-client': 4.12.0 + '@smithy/types': 4.13.0 + '@smithy/url-parser': 4.2.10 + '@smithy/util-base64': 4.3.1 + '@smithy/util-body-length-browser': 4.2.1 + '@smithy/util-body-length-node': 4.2.2 + '@smithy/util-defaults-mode-browser': 4.3.36 + '@smithy/util-defaults-mode-node': 4.2.39 + '@smithy/util-endpoints': 3.3.1 + '@smithy/util-middleware': 4.2.10 + '@smithy/util-retry': 4.2.10 + '@smithy/util-stream': 4.5.15 + '@smithy/util-utf8': 4.2.1 + '@smithy/util-waiter': 4.2.10 + tslib: 2.8.1 + transitivePeerDependencies: + - aws-crt + '@aws-sdk/core@3.973.15': dependencies: '@aws-sdk/types': 3.973.4 @@ -6106,6 +6338,11 @@ snapshots: '@smithy/util-utf8': 4.2.1 tslib: 2.8.1 + '@aws-sdk/crc64-nvme@3.972.3': + dependencies: + '@smithy/types': 4.13.0 + tslib: 2.8.1 + '@aws-sdk/credential-provider-env@3.972.13': dependencies: '@aws-sdk/core': 3.973.15 @@ -6217,6 +6454,16 @@ snapshots: '@smithy/types': 4.13.0 tslib: 2.8.1 + '@aws-sdk/middleware-bucket-endpoint@3.972.6': + dependencies: + '@aws-sdk/types': 3.973.4 + '@aws-sdk/util-arn-parser': 3.972.2 + '@smithy/node-config-provider': 4.3.10 + '@smithy/protocol-http': 5.3.10 + '@smithy/types': 4.13.0 + '@smithy/util-config-provider': 4.2.1 + tslib: 2.8.1 + '@aws-sdk/middleware-eventstream@3.972.6': dependencies: '@aws-sdk/types': 3.973.4 @@ -6224,6 +6471,30 @@ snapshots: '@smithy/types': 4.13.0 tslib: 2.8.1 + '@aws-sdk/middleware-expect-continue@3.972.6': + dependencies: + '@aws-sdk/types': 3.973.4 + '@smithy/protocol-http': 5.3.10 + '@smithy/types': 4.13.0 + tslib: 2.8.1 + + '@aws-sdk/middleware-flexible-checksums@3.973.1': + dependencies: + '@aws-crypto/crc32': 5.2.0 + '@aws-crypto/crc32c': 5.2.0 + '@aws-crypto/util': 5.2.0 + '@aws-sdk/core': 3.973.15 + '@aws-sdk/crc64-nvme': 3.972.3 + '@aws-sdk/types': 3.973.4 + '@smithy/is-array-buffer': 4.2.1 + '@smithy/node-config-provider': 4.3.10 + '@smithy/protocol-http': 5.3.10 + '@smithy/types': 4.13.0 + '@smithy/util-middleware': 4.2.10 + '@smithy/util-stream': 4.5.15 + '@smithy/util-utf8': 4.2.1 + tslib: 2.8.1 + '@aws-sdk/middleware-host-header@3.972.6': dependencies: '@aws-sdk/types': 3.973.4 @@ -6231,6 +6502,12 @@ snapshots: '@smithy/types': 4.13.0 tslib: 2.8.1 + '@aws-sdk/middleware-location-constraint@3.972.6': + dependencies: + '@aws-sdk/types': 3.973.4 + '@smithy/types': 4.13.0 + tslib: 2.8.1 + '@aws-sdk/middleware-logger@3.972.6': dependencies: '@aws-sdk/types': 3.973.4 @@ -6245,6 +6522,29 @@ snapshots: '@smithy/types': 4.13.0 tslib: 2.8.1 + '@aws-sdk/middleware-sdk-s3@3.972.15': + dependencies: + '@aws-sdk/core': 3.973.15 + '@aws-sdk/types': 3.973.4 + '@aws-sdk/util-arn-parser': 3.972.2 + '@smithy/core': 3.23.6 + '@smithy/node-config-provider': 4.3.10 + '@smithy/protocol-http': 5.3.10 + '@smithy/signature-v4': 5.3.10 + '@smithy/smithy-client': 4.12.0 + '@smithy/types': 4.13.0 + '@smithy/util-config-provider': 4.2.1 + '@smithy/util-middleware': 4.2.10 + '@smithy/util-stream': 4.5.15 + '@smithy/util-utf8': 4.2.1 + tslib: 2.8.1 + + '@aws-sdk/middleware-ssec@3.972.6': + dependencies: + '@aws-sdk/types': 3.973.4 + '@smithy/types': 4.13.0 + tslib: 2.8.1 + '@aws-sdk/middleware-user-agent@3.972.15': dependencies: '@aws-sdk/core': 3.973.15 @@ -6321,6 +6621,26 @@ snapshots: '@smithy/types': 4.13.0 tslib: 2.8.1 + '@aws-sdk/s3-request-presigner@3.1000.0': + dependencies: + '@aws-sdk/signature-v4-multi-region': 3.996.3 + '@aws-sdk/types': 3.973.4 + '@aws-sdk/util-format-url': 3.972.6 + '@smithy/middleware-endpoint': 4.4.20 + '@smithy/protocol-http': 5.3.10 + '@smithy/smithy-client': 4.12.0 + '@smithy/types': 4.13.0 + tslib: 2.8.1 + + '@aws-sdk/signature-v4-multi-region@3.996.3': + dependencies: + '@aws-sdk/middleware-sdk-s3': 3.972.15 + '@aws-sdk/types': 3.973.4 + '@smithy/protocol-http': 5.3.10 + '@smithy/signature-v4': 5.3.10 + '@smithy/types': 4.13.0 + tslib: 2.8.1 + '@aws-sdk/token-providers@3.1000.0': dependencies: '@aws-sdk/core': 3.973.15 @@ -6350,6 +6670,10 @@ snapshots: '@smithy/types': 4.13.0 tslib: 2.8.1 + '@aws-sdk/util-arn-parser@3.972.2': + dependencies: + tslib: 2.8.1 + '@aws-sdk/util-endpoints@3.996.3': dependencies: '@aws-sdk/types': 3.973.4 @@ -8138,6 +8462,15 @@ snapshots: '@smithy/types': 4.13.0 tslib: 2.8.1 + '@smithy/chunked-blob-reader-native@4.2.2': + dependencies: + '@smithy/util-base64': 4.3.1 + tslib: 2.8.1 + + '@smithy/chunked-blob-reader@5.2.1': + dependencies: + tslib: 2.8.1 + '@smithy/config-resolver@4.4.9': dependencies: '@smithy/node-config-provider': 4.3.10 @@ -8206,6 +8539,13 @@ snapshots: '@smithy/util-base64': 4.3.1 tslib: 2.8.1 + '@smithy/hash-blob-browser@4.2.11': + dependencies: + '@smithy/chunked-blob-reader': 5.2.1 + '@smithy/chunked-blob-reader-native': 4.2.2 + '@smithy/types': 4.13.0 + tslib: 2.8.1 + '@smithy/hash-node@4.2.10': dependencies: '@smithy/types': 4.13.0 @@ -8213,6 +8553,12 @@ snapshots: '@smithy/util-utf8': 4.2.1 tslib: 2.8.1 + '@smithy/hash-stream-node@4.2.10': + dependencies: + '@smithy/types': 4.13.0 + '@smithy/util-utf8': 4.2.1 + tslib: 2.8.1 + '@smithy/invalid-dependency@4.2.10': dependencies: '@smithy/types': 4.13.0 @@ -8226,6 +8572,12 @@ snapshots: dependencies: tslib: 2.8.1 + '@smithy/md5-js@4.2.10': + dependencies: + '@smithy/types': 4.13.0 + '@smithy/util-utf8': 4.2.1 + tslib: 2.8.1 + '@smithy/middleware-content-length@4.2.10': dependencies: '@smithy/protocol-http': 5.3.10 @@ -8433,6 +8785,12 @@ snapshots: '@smithy/util-buffer-from': 4.2.1 tslib: 2.8.1 + '@smithy/util-waiter@4.2.10': + dependencies: + '@smithy/abort-controller': 4.2.10 + '@smithy/types': 4.13.0 + tslib: 2.8.1 + '@smithy/uuid@1.1.1': dependencies: tslib: 2.8.1 @@ -8514,6 +8872,45 @@ snapshots: '@tinyhttp/content-disposition@2.2.4': {} + '@tloncorp/api@https://codeload.github.com/tloncorp/api-beta/tar.gz/7eede1c1a756977b09f96aa14a92e2b06318ae87': + dependencies: + '@aws-sdk/client-s3': 3.1000.0 + '@aws-sdk/s3-request-presigner': 3.1000.0 + '@urbit/aura': 3.0.0 + '@urbit/nockjs': 1.6.0 + any-ascii: 0.3.3 + big-integer: 1.6.52 + browser-or-node: 3.0.0 + buffer: 6.0.3 + date-fns: 3.6.0 + emoji-regex: 10.6.0 + exponential-backoff: 3.1.3 + libphonenumber-js: 1.12.38 + lodash: 4.17.23 + sorted-btree: 1.8.1 + validator: 13.15.26 + transitivePeerDependencies: + - aws-crt + + '@tloncorp/tlon-skill-darwin-arm64@0.1.9': + optional: true + + '@tloncorp/tlon-skill-darwin-x64@0.1.9': + optional: true + + '@tloncorp/tlon-skill-linux-arm64@0.1.9': + optional: true + + '@tloncorp/tlon-skill-linux-x64@0.1.9': + optional: true + + '@tloncorp/tlon-skill@0.1.9': + optionalDependencies: + '@tloncorp/tlon-skill-darwin-arm64': 0.1.9 + '@tloncorp/tlon-skill-darwin-x64': 0.1.9 + '@tloncorp/tlon-skill-linux-arm64': 0.1.9 + '@tloncorp/tlon-skill-linux-x64': 0.1.9 + '@tokenizer/inflate@0.4.1': dependencies: debug: 4.4.3 @@ -8780,6 +9177,14 @@ snapshots: '@urbit/aura@3.0.0': {} + '@urbit/http-api@3.0.0': + dependencies: + '@babel/runtime': 7.28.6 + browser-or-node: 1.3.0 + core-js: 3.48.0 + + '@urbit/nockjs@1.6.0': {} + '@vector-im/matrix-bot-sdk@0.8.0-element.3(@cypress/request@3.0.10)': dependencies: '@matrix-org/matrix-sdk-crypto-nodejs': 0.4.0 @@ -9007,6 +9412,8 @@ snapshots: ansis@4.2.0: {} + any-ascii@0.3.3: {} + any-promise@1.3.0: {} apache-arrow@18.1.0: @@ -9126,6 +9533,8 @@ snapshots: before-after-hook@4.0.0: {} + big-integer@1.6.52: {} + bignumber.js@9.3.1: {} birpc@4.0.0: {} @@ -9173,12 +9582,21 @@ snapshots: dependencies: balanced-match: 4.0.4 + browser-or-node@1.3.0: {} + + browser-or-node@3.0.0: {} + buffer-crc32@0.2.13: {} buffer-equal-constant-time@1.0.1: {} buffer-from@1.1.2: {} + buffer@6.0.3: + dependencies: + base64-js: 1.5.1 + ieee754: 1.2.1 + bun-types@1.3.9: dependencies: '@types/node': 25.3.3 @@ -9337,6 +9755,8 @@ snapshots: cookie@0.7.2: {} + core-js@3.48.0: {} + core-util-is@1.0.2: {} core-util-is@1.0.3: {} @@ -9373,6 +9793,8 @@ snapshots: data-uri-to-buffer@6.0.2: {} + date-fns@3.6.0: {} + debug@2.6.9: dependencies: ms: 2.0.0 @@ -9567,6 +9989,8 @@ snapshots: expect-type@1.3.0: {} + exponential-backoff@3.1.3: {} + express@4.22.1: dependencies: accepts: 1.3.8 @@ -10233,6 +10657,8 @@ snapshots: leac@0.6.0: {} + libphonenumber-js@1.12.38: {} + lie@3.3.0: dependencies: immediate: 3.0.6 @@ -11573,6 +11999,8 @@ snapshots: dependencies: atomic-sleep: 1.0.0 + sorted-btree@1.8.1: {} + source-map-js@1.2.1: {} source-map-support@0.5.21: @@ -11928,6 +12356,8 @@ snapshots: validate-npm-package-name@7.0.2: {} + validator@13.15.26: {} + vary@1.1.2: {} verror@1.10.0: