* feat(imessage): always-on inbound recovery, deprecate catchup
Replaces the opt-in catchup subsystem with always-on inbound replay
protection that brings iMessage in line with the other channels, and
fixes#89237 (stale backlog dispatched as fresh after bridge recovery).
- New inbound-dedupe.ts: persistent claimable GUID dedupe (claim/commit/
release) plus a stale-backlog age fence that suppresses live rows whose
send date is materially older than arrival (logged, never silent).
- monitor-provider: claim at ingestion, carry the exact claimed key on the
debouncer entry, commit on successful flush / release on dispatch failure
(per-unit so a coalesced bucket cannot strand a sibling claim). Keeps the
local startup since_rowid watermark so startup-window rows are not skipped.
- Deprecate catchup: delete catchup.ts + catchup-bridge.ts, remove the
channels.imessage.catchup schema, cursor migration, and config-guard nag.
Back-compat: strip the retired key before validation; new imessage doctor
contract reports + removes it on doctor --fix.
- Docs updated for the new recovery model.
Net -947 prod LOC.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(imessage): recover downtime messages via since_rowid replay
Builds downtime recovery on the new inbound dedupe instead of restoring the
old catchup subsystem. On startup the monitor passes the last dispatched rowid
(a persisted per-account cursor) to imsg watch.subscribe as since_rowid, so imsg
replays the messages that landed while the gateway was down, then tails live.
The GUID dedupe drops anything already handled, so no cursor/retry bookkeeping
is needed.
- recovery-cursor.ts: minimal persisted per-account lastDispatchedRowid.
- monitor-provider: since_rowid = cursor (capped to the most recent
IMESSAGE_RECOVERY_MAX_ROWS); split the age fence on the startup rowid boundary
so replayed rows (<= boundary) use the wider recovery window and live rows
(> boundary) keep the tight #89237 fence; advance the cursor on commit.
- Local only: remote SSH cliPath cannot read chat.db, so it tails from the
current rowid (suppress-and-move-on) as before.
Restores missed-message recovery that the catchup removal dropped, with no
config and a fraction of the old LOC.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(imessage): make recovery cursor advance failure- and suppression-safe
Addresses two cursor-state regressions in the downtime-recovery path:
- Failed replay rows could be skipped forever: a released (failed) row keeps
its dedupe claim for retry, but a later successful row in the same flush
advanced the cursor past it, so the next startup's since_rowid skipped it.
Hold a per-session floor at the lowest released rowid and never advance the
cursor past it.
- Suppressed live backlog could be re-delivered after a restart: a live row
suppressed under the tight live fence was not recorded, so after a restart it
fell under the wider recovery window (its rowid now below the new boundary)
and was delivered. Commit its dedupe key on suppression so the recovery
replay treats it as already handled.
Both caught by Codex autoreview. Adds regression tests for the floor and the
suppression record.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(imessage): bound the GUID-less replay key length
Hash the composite fallback key's variable parts (conversation, sender,
created_at, text) so the key is length-bounded regardless of message text.
The persistent dedupe store already hashes keys internally, so this was not a
live overflow, but the bounded key removes the dependency on that and keeps the
fallback fail-open. Flagged by autoreview.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(imessage): recover downtime messages on remote cliPath setups too
The since_rowid replay runs over the imsg RPC client, so driving it from the
persisted recovery cursor (not the local chat.db boundary) makes downtime
recovery work for remote SSH cliPath gateways — the topology the old RPC-based
catchup served and that the rowid-boundary-only version regressed. Local setups
keep the wider, capped recovery window via the chat.db boundary; remote uses the
live age-fence window. Flagged by autoreview.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(imessage): seed recovery cursor from retired catchup cursor on upgrade
A one-time, self-cleaning migration: when the recovery cursor is empty on the
first startup after upgrade, seed it from the retired imessage.catchup-cursors
lastSeenRowid and consume the legacy entry. Without this a user who had catchup
enabled would not replay messages missed across the upgrade restart. Flagged by
autoreview.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(imessage): preserve catchup recovery on upgrade
---------
Co-authored-by: Omar Shahine <10343873+omarshahine@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Gate iMessage same-sender DM split-send coalescing on imsg's structural
`balloon_bundle_id` URL-balloon marker (openclaw/imsg#137) instead of timing/
text-shape inference, with a session capability latch and a back-compat path:
- URL-balloon marker present -> merge (precise split-send).
- Build known to emit balloon metadata (session latch) -> keep non-marker
buckets separate (the precision win).
- Build that never emits balloon metadata (older imsg) -> preserve the legacy
unconditional merge, so split-send users do not regress to two turns.
Never merges more than shipped main already did. Verified live end-to-end: the
patched gateway, watching a real chat.db via an imsg #137 build, merged a real
iPhone-sent `Dump <url>` split-send into one turn. Client-side removal once imsg
coalesces upstream is tracked in #91243 (openclaw/imsg#141).
Closes#90795
* refactor: move imessage monitor state to sqlite
* test: use OpenClaw temp root in iMessage state helper
* test: avoid pending promise lint in chat tests
* test: harden gateway ci flakes
* test: align session list merge expectation
* feat(imessage): support thumb approval reactions
Mirrors openclaw#85477 (WhatsApp) for the iMessage channel. iMessage can now
deliver exec/plugin approval prompts via the existing imsg/BlueBubbles
transport and resolve approvals from 👍 (allow-once) / 👎 (deny) tapbacks.
Allow-always remains on the manual /approve <id> allow-always fallback.
What changed:
- New approval surfaces under extensions/imessage/src/:
approval-auth.ts, approval-resolver.ts, approval-reactions.ts,
approval-handler.runtime.ts, approval-native.ts (+ tests for each).
- channel.ts wires base.approvalCapability to the new iMessage capability.
- send.ts appends the 👍/👎 hint to outbound /approve prompts and registers
the reaction binding (keyed by accountId + chat_guid/chat_identifier/
chat_id/handle + messageId) after a successful send.
- monitor/monitor-provider.ts resolves approval reactions ahead of the
normal inbound decision pipeline so resolution bypasses
reactionNotifications gating and runs its own actor authorization.
- runtime.ts now exports getIMessageRuntime / getOptionalIMessageRuntime so
approval-reactions can open a persistent keyed store for binding state
across gateway restarts.
What did NOT change:
- Core approval surfaces in src/gateway/server-methods/* and src/infra/*
remain channel-agnostic; the channels.imessage.allowFrom field already
exists and is reused as the approver list for reactions.
- Other channels and the manual /approve sender-authorized path are
untouched.
* fix(imessage): address codex review findings on thumb approvals
Addresses 15 findings from the multi-angle codex review:
Critical (correctness / blocking):
- Register CHANNEL_APPROVAL_NATIVE_RUNTIME_CONTEXT_CAPABILITY in the iMessage
monitor so the gateway can actually deliver native approval prompts via
approval-handler.runtime.ts (it was dead code without the context lease).
- DM tapback approvals never resolved because send keyed by handle while
inbound preferred chat_guid. Register and look up under EVERY available
conversation key (chat_guid / chat_identifier / chat_id / handle); inbound
probes them all and accepts the first hit.
- Reaction binding now requires the bridge's GUID string (rejecting numeric
ROWIDs) so the binding key matches inbound reacted_to_guid.
- Outbound regex now requires both a canonical `ID: <approvalId>` header AND
a matching `/approve <id> <decision>` line, so non-approval messages that
legitimately mention /approve syntax no longer get a phantom reaction
binding (and can no longer resolve a colliding live approval).
- Drop is_from_me reaction events so cross-device echoes of the operator's
own tap cannot self-approve when their handle is in allowFrom.
High (operability / cleanup):
- Non-ApprovalNotFound errors now log at warn via the runtime child logger
(no longer hidden behind OPENCLAW_LOG_LEVEL=debug).
- In-memory binding is cleared on successful resolve so a toggle 👍→👎 (or
chat.db replay) does not refire and emit a misleading 'expired approval'
log line. Removed tapbacks are also owned by the shortcut and not surfaced
as noisy reaction system events.
- Move resolveIMessageReactionContext (and its helpers) to a slim
monitor/reaction-context.ts so approval-reactions.ts no longer transitively
pulls monitor/inbound-processing.ts (14+ heavy runtime modules) into the
hot channel.ts entrypoint per extensions/CLAUDE.md.
Medium (consistency / future-proofing):
- Native runtime exec pending payload now passes agentId, ask, and
sessionKey through buildExecApprovalPendingReplyPayload so the two
delivery routes produce identical operator-visible prompts.
- Both delivery paths now use addIMessageApprovalReactionHintToText (single
insertion point after ID:) so the hint cannot be double-emitted by the
native runtime path bypassing the idempotency guard.
- Extract replaceApprovalIdPlaceholder into a shared approval-text.ts that
escapes `$` in the replacement string so an approvalId containing
`$&`/`$1`-`$9`/`$$` cannot interpolate into the outbound text.
- In-memory Map now stores TTL alongside each entry and prunes expired
bindings on each register so the gateway no longer accumulates an
unbounded reaction-target Map.
- bindPending refuses to bind when accountId is missing or the approval is
already expired, with explicit error logs instead of silent no-ops.
- Reject chat_id=0 as a synthetic key value (chat.db ROWIDs start at 1).
- Drop dead getIMessageRuntime export — only the optional accessor is used.
Documentation:
- docs/channels/imessage.md gains an 'Approval reactions (👍 / 👎)' accordion
documenting the reaction emoji map, allowFrom approver requirement, the
/approve <id> allow-always manual fallback, and the deliberate change to
/approve command authorization for users with non-empty allowFrom.
- CHANGELOG.md entry added under 2026.5.24.
Tests: 411 iMessage tests pass (was 406). Added explicit coverage for the
DM key-mismatch fix, the regex-tightening fix, the is_from_me guard, the
clear-on-success behavior, and the approval-id `$` escape.
* test(imessage): match WhatsApp approval-native test coverage
Backfills the nine cases from extensions/whatsapp/src/approval-native.test.ts
that weren't mirrored in iMessage:
- target-mode exec + plugin prompt rendering with the canonical hint
- target-mode availability when no iMessage target matches
- agentFilter / sessionFilter applied to native handling
- account-scoped target enabled/disabled per account
- shouldSuppressForwardingFallback session-origin exact-match cases
- shouldSuppressForwardingFallback off when native cannot bind (locks down
the targets-only forwarding path the Lobster live deploy exercised)
- both-mode explicit + unscoped target suppression
- group-origin tapback approvals require explicit approvers
Tests: extensions/imessage/src/approval-native.test.ts 21 passed (was 11).
Total iMessage approval-specific cases now 49 (was 40).
* fix(imessage): preserve service-prefixed direct handles as approvers
ClawSweeper P1 review finding on #85952. normalizeIMessageApproverId was
calling looksLikeIMessageExplicitTargetId() to reject conversation-target
prefixes, but that helper also matches the imessage:/sms:/auto: service
prefixes — which are valid direct-handle forms. Any allowFrom entry like
'imessage:+15551230000' dropped to undefined, leaving approvers empty,
which:
- silently denied reaction resolution ('reactions require explicit
approvers'), and
- let text /approve fall back to implicit same-chat authorization.
Fix: normalize first via normalizeIMessageHandle (strips the service
prefix), then reject only chat_id:/chat_guid:/chat_identifier:
conversation-target shapes that remain after normalization.
Tests:
- approval-auth.test.ts: assert the resolved approver list contains the
normalized handle, plus the corollary that a non-matching sender is
explicitly rejected (no longer masked by the implicit-same-chat
fallback). Add a separate case covering chat_id/chat_guid/
chat_identifier rejection (with and without a service prefix).
- approval-reactions.test.ts: reaction resolution end-to-end with a
service-prefixed allowFrom entry — proves resolveIMessageApproval is
called rather than silently denied.
Focused suite: 48 passed (was 47).
* test(imessage): satisfy strict buildPendingPayload signature in render tests
CI check:test-types caught that the render.exec/render.plugin
buildPendingPayload calls were passing accountId (not in the type
signature). The signature is { cfg, request, target, nowMs }. Replace
accountId with target on the four render-test sites so the strict
test-types pass matches the SDK contract:
- it('renders thumbs-only reaction hints in exec approval prompts')
- it('renders thumbs-only reaction hints in plugin approval prompts ...')
- it('renders target-mode exec prompts with concrete thumbs-only ...')
- it('renders target-mode plugin prompts with concrete thumbs-only ...')
Verified locally with pnpm check:test-types (tsgo:core:test +
tsgo:extensions:test). 49 approval-specific tests still pass.
* fix(imessage): probe every tapback GUID form for approval lookup
ClawSweeper P1 review finding on #85952. readApprovalReactionEvent was
only using reaction.targetGuid (the first/normalized form), but
resolveIMessageReactionContext produces reaction.targetGuids = [normalized,
raw] for both `abc-123` and `p:0/abc-123` forms. If the imsg bridge
returned 'p:0/<guid>' from send() and send.ts registered the binding under
that prefixed key, the inbound resolver probing only the unprefixed form
would miss and the tapback would silently fall through.
Fix:
- Surface every GUID candidate in IMessageApprovalReactionEvent
(messageIdCandidates).
- maybeResolveIMessageApprovalReaction now probes each candidate in
precedence order; first hit wins.
- On success / ApprovalNotFoundError, clear the binding under all
candidate keys so toggle/replay does not refire.
Tests: extensions/imessage/src/approval-reactions.test.ts gains a
'resolves a reaction when the binding was registered under a p:0/…
prefixed GUID and the tapback surfaces both forms' regression case;
22/22 reaction tests pass. Full iMessage suite: 424/424.
* fix(imessage): native approval binding requires GUID, not numeric id
ClawSweeper third P1 review finding on #85952. approval-handler.runtime.ts
deliverPending was using result.messageId as the approval-reaction binding
key, but that field can be a numeric ROWID coerced to a string ('12345')
when the imsg bridge returns only message_id. Inbound tapbacks carry
reacted_to_guid which is always a GUID, so a numeric-id binding can never
match.
Fix mirrors the send.ts forwarding-path treatment:
- IMessageSendResult now exposes a separate guid?: string field, populated
from the same resolveOutboundMessageGuid helper send.ts already uses for
the forwarding-path binding. The generic messageId field is unchanged so
reply-cache, echo-cache, and receipt-building paths still see the
broadest id form.
- deliverPending now binds against result.guid; when it's undefined (numeric
ROWID or 'ok'/'unknown' placeholders), the function returns null instead
of binding against an id the inbound tapback can't possibly match.
Tests: approval-handler.runtime.test.ts gets a deliverPending GUID-only
binding describe block with three regression cases (numeric ROWID refused,
GUID accepted, ok/unknown placeholders refused). vi.mock isolates
sendMessageIMessage so the cases run synchronously without spawning imsg.
11 tests pass across handler.runtime + send specs.
---------
Co-authored-by: Omar Shahine <10343873+omarshahine@users.noreply.github.com>
Summary:
- The PR adds a Warning block to `docs/channels/imessage.md` explaining that iMessage `cliPath` wrappers and SSH proxies must stream long-lived JSON-RPC stdin/stdout incrementally.
- Reproducibility: not applicable. for this docs-only PR. Source inspection verifies the runtime uses long-lived line-framed stdio, and current main lacks the operator warning being added.
Automerge notes:
- PR branch already contained follow-up commit before automerge: docs(imessage): warn that cliPath wrappers must stream JSON-RPC stdio…
Validation:
- ClawSweeper review passed for head a371ee998e.
- Required merge gates passed before the squash merge.
Prepared head SHA: a371ee998e
Review: https://github.com/openclaw/openclaw/pull/84420#issuecomment-4494313781
Co-authored-by: HCL <chenglunhu@gmail.com>
Co-authored-by: clawsweeper <274271284+clawsweeper[bot]@users.noreply.github.com>
Co-authored-by: clawsweeper[bot] <274271284+clawsweeper[bot]@users.noreply.github.com>
Approved-by: takhoffman
Co-authored-by: takhoffman <781889+takhoffman@users.noreply.github.com>
Summary:
- Add `openclaw channels status --channel <name>` filtering through CLI, gateway protocol, and fallback status rendering.
- Document the BlueBubbles-to-iMessage cutover path so operators can probe iMessage without starting both monitors.
- Refresh generated Swift protocol model for the new optional channel status parameter.
Verification:
- `pnpm test src/gateway/server-methods/channels.status.test.ts src/commands/channels.status.command-flow.test.ts src/cli/program/routes.test.ts -- --reporter=verbose`
- `CI=true pnpm check:docs`
- `pnpm protocol:check`
- `git diff --check`
- `node scripts/check-changelog-attributions.mjs`
- CI head `45b27e3866`: focused/docs/protocol shards green locally; GitHub broad/scanner jobs queued for runners at merge attempt time; `Real behavior proof` failure is the maintainer-ignorable external-real-proof complaint.
Closes#78649. Adds opt-in inbound iMessage catchup that recovers messages landing in chat.db while the gateway is offline (crash, restart, mac sleep). Mirrors the design of the retired BlueBubbles catchup, adapted for the imsg JSON-RPC chats.list + messages.history fetch path.
- Schema: new channels.imessage.catchup block with enabled / maxAgeMinutes (1..720) / perRunLimit (1..500) / firstRunLookbackMinutes (1..720) / maxFailureRetries (1..1000). Disabled by default — opt-in.
- Cursor + replay loop (extensions/imessage/src/monitor/catchup.ts): per-account state under <openclawStateDir>/imessage/catchup/. Walks rows oldest-first, advances on success/give-up, holds at failed.rowid - 1 when a failure is below maxFailureRetries (cannot leapfrog held failures even when later rows in the same batch succeed). Watermark floor for parse-rejected rows.
- Bridge (extensions/imessage/src/monitor/catchup-bridge.ts): live chats.list + per-chat messages.history fetch adapter; dispatch adapter routes through the live handleMessageNow path so allowlists / group policy / dedupe / echo cache behave identically on replayed and live messages. Watermark clamped to last dispatched rowid when the cap truncates.
- Monitor wiring (extensions/imessage/src/monitor/monitor-provider.ts): catchup runs once between watch.subscribe and the live dispatch loop when enabled. Bypasses the inbound debouncer for serial per-row dispatch.
- Echo-cache TTL bumped 2 min → 12 h so own outbound rows from before a gap are not re-fed as inbound on replay.
- Generated bundled-channel-config-metadata.generated.ts so the runtime AJV schema accepts the new catchup block.
- Docs: new "Catching up after gateway downtime" section + BlueBubbles migration parity update.
Tests: 322/322 in extensions/imessage/, including 5 regression tests covering the cursor-leapfrog, parse-rejected stall, watermark vs held failure, and cap-truncation-cursor-floor edge cases that codex (gpt-5.4) and clawsweeper (gpt-5.5) found during review. Live-tested end-to-end against the running gateway: replayed=1 fetchedCount=1, agent reply observed, cursor persisted at the test row's exact rowid.
Co-authored-by: Omar Shahine <10343873+omarshahine@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add consistent Related sections to 17 channel pages that had none,
linking to: Channels Overview, Pairing, Groups, Channel Routing, Security.
Add Groups and Security links to 4 channel pages (discord, slack,
telegram, whatsapp) that already had partial Related sections.
* feat: Make BlueBubbles the primary iMessage integration
- Remove old imsg skill (skills/imsg/SKILL.md)
- Create new BlueBubbles skill (skills/bluebubbles/SKILL.md) with message tool examples
- Add keep-alive script documentation for VM/headless setups to docs/channels/bluebubbles.md
- AppleScript that pokes Messages.app every 5 minutes
- LaunchAgent configuration for automatic execution
- Prevents Messages.app from going idle in VM environments
- Update all documentation to prioritize BlueBubbles over legacy imsg:
- Mark imsg channel as legacy throughout docs
- Update README.md channel lists
- Update wizard, hubs, pairing, and index docs
- Update FAQ to recommend BlueBubbles for iMessage
- Update RPC docs to note imsg as legacy pattern
- Update Chinese documentation (zh-CN)
- Replace imsg examples with generic macOS skill examples where appropriate
BlueBubbles is now the recommended first-class iMessage integration,
with the legacy imsg integration marked for potential future removal.
* refactor: Update import paths and improve code formatting
- Adjusted import paths in session-status-tool.ts, whatsapp-heartbeat.ts, and heartbeat-runner.ts for consistency.
- Reformatted code for better readability by aligning and grouping related imports and function parameters.
- Enhanced error messages and conditional checks for clarity in heartbeat-runner.ts.
* skills: restore imsg skill and align bluebubbles skill
* docs: update FAQ for clarity and formatting
- Adjusted the formatting of the FAQ section to ensure consistent bullet point alignment.
- No content changes were made, only formatting improvements for better readability.
* style: oxfmt touched files
* fix: preserve BlueBubbles developer reference (#8415) (thanks @tyler6204)
