* feat(ui): redesign session observer HUD with labeled status and visibility-gated digests
* chore(protocol): regenerate app protocol bindings for sessions.observer.visibility
* feat(android,gateway): expose mobile.ui.observe/act as node commands (PR 2/3)
Wire the PR1 AccessibilityService executor over the existing node.invoke
transport. Additive: no gateway protocol version bump.
- New commands mobile.ui.observe / mobile.ui.act (capability mobileUI),
generated into the protocol constants.
- thirdParty MobileUiHandler owns one mutex-serialized AccessibilityActionExecutor
and bridges JSON <-> the PR1 snapshot/action model; play flavor ships a
permanently-unavailable no-op stub (Play APK stays accessibility-free).
- Commands advertised only when accessibilityControlEnabled and the service is
connected; NodeRuntime refreshes the advertised surface on connect/disconnect.
- Classified dangerous in node-command-policy (declarable-but-armed, mirroring
computer.act): the transport cannot invoke them until gateway.nodes.allowCommands
explicitly arms them. Screen reads are treated as dangerous too, so observe is
gated as well as act.
Both flavors assemble; thirdParty/play unit tests, ktlint, android lint, and the
gateway node-command-policy tests pass; protocol generator is idempotent; Play
APK verified to contain zero accessibility classes. Emulator: no regression to
PR1 dev-screen observe.
* fix(gateway): migrate mobile.ui arming test/comment to gateway.nodes.commands.allow
Main renamed the node-command arming config from gateway.nodes.allowCommands to
gateway.nodes.commands.allow; update the mobile.ui policy test config shape and
the comment to the current contract. Production resolution already reads the new
path.
* chore(android): update native i18n baseline for PR2 line-number shifts
* feat(android): AccessibilityService UI executor (thirdParty, PR 1/3)
Add a thirdParty-flavor-only AccessibilityService that observes the active
app's UI as a bounded semantic snapshot and performs typed actions, exercised
via a local developer screen. No gateway/agent wiring yet (PR 2 adds
node.invoke commands, PR 3 the agent tool + policy gates).
- Play APK stays accessibility-free: service, config, executor, and UI live
entirely under src/thirdParty; SensitiveFeatureConfig.accessibilityControlEnabled
gates it. Verified: 0 accessibility refs in every merged Play manifest and
0 accessibility classes in the Play dex.
- Semantic-first executor: observe() returns a bounded snapshot (node/depth/text
caps, deterministic order, password + sensitive-field redaction, stable action
vocabulary, generation-scoped refs); act() performs one typed action with a
closed ActionOutcomeCode result set.
- Safety model: coordinate gestures gated by package-match (fail-closed) + a UI
epoch advanced on window/content/scroll/text mutation events; node actions
gated by package-match + per-node refresh(); global actions ungated. Capture
runs off the main thread. Dev UI honestly disables cross-app node controls
(only reachable while the target is foreground; validated via the remote path).
Built and emulator-tested on API 36: live connection status, immediate observe,
a 103-node cross-app Settings capture, and global Home. Both flavors assemble;
thirdParty unit tests, ktlint, and android lint pass.
* feat(android): gate accessibility control behind an off-by-default opt-in
Make landing the accessibility feature a no-op for existing thirdParty users:
the service is invisible and inert until the user explicitly opts in.
- The AccessibilityService and its dev activity are declared
android:enabled="false", so a fresh install exposes NO new accessibility
service in system settings and nothing can bind it (verified on device:
absent from the installed-services list and unbindable; shell cannot enable
it either — only the app can).
- New thirdParty-only "Control other apps" toggle (persisted in the existing
openclaw.node prefs, default OFF). Turning it on enables both components via
PackageManager.setComponentEnabledSetting and deep-links to Accessibility
settings so the user can grant it; turning it off disables them again
(DONT_KILL_APP). A disclosure describes what enabling does.
- Play flavor is a no-op (FlavorPhoneCapabilitiesSettings = Unit); no
accessibility component/controller/toggle references reach the Play APK.
- No new permissions or dependencies. The compile-time flavor gate and the
gateway dangerous-command arming remain as additional layers.
* chore(android): update native i18n baseline for accessibility control strings
* feat(gateway,ui): ask-the-observer card input over sessions.observer.ask
* refactor(ui): single home for observer run-identity helper after restack
* test(ui): drop duplicated observer hud test after restack
* test(ui): give the observer ask flow its own colocated suite
* refactor(gateway): leaf observer contract and ask module split for ci gates
* refactor(gateway): drop observer contract re-export shims
* feat(gateway): session observer digests over the utility model
* fix(gateway): split session-observer modules and satisfy ci gates
* fix(gateway): observer reads session entries without materializing agent state
* feat(mobile): session dashboard screens on iOS and Android via authenticated Control UI webview
* fix(android): keep configured Control UI base path in session dashboard URL
* docs(android): note system-trust boundary of the shared Control UI webview
* fix(android): origin-only document-start rule for Control UI auth script
* chore(i18n): refresh native inventory on rebased head
* fix(ios): swiftlint closure form in session dashboard toolbar
* fix(i18n): tolerate workflow-owned pending native rows in PR alignment checks
* fix(android): KTX toUri per lint and refresh native inventory
* fix(android): ktlint import order incl. main-inherited fleet test, refresh inventory
* feat: add session discussion panel seam
* fix: keep discussion iframe cookie-capable and changelog release-owned
* test: cover cookie-capable discussion iframe sandbox
* fix: stretch discussion panel host so the embed fills the rail
* fix: keep provider failures retryable and probe discussion availability before showing the action
* fix: block same-origin discussion embeds, show action on catalog sessions, close stale panel on reconnect
* fix: scope discussion probes and panel callbacks to the issuing connection
* fix: dedupe in-flight discussion probes per session
* fix: retry superseded discussion probes and key-scope panel results
* fix: regenerate Swift protocol models and extend advertised-method expectations
* style: format chat-pane-header
* fix: regenerate Kotlin protocol models and date discussion methods in the 2026.7 train
* chore: restore release-owned changelog to main state
* chore: keep changelog untouched relative to merge-base
* feat(clients): adopt model controls, session management, and keyboard history recall on iOS
* chore(i18n): translate iOS adoption strings and realign locale artifacts
* refactor(clients): drop test-only group response inits flagged by the shared dead-code intersect
* chore: retrigger ci
* chore(i18n): resync inventory after voice-unification main rebase
* chore(i18n): resync inventory after main rebase
* chore(i18n): resync inventory after main rebase
* chore(i18n): realign locale artifacts after main rebase
* fix(clients): preserve native chat API and locale copy
* fix(ios): retain public session DTO initializers
* refactor(clients): delete test-only session DTO inits flagged dead instead of suppressing
Live-append voice transcripts into the agent session and persist a per-agent SQLite call record across relay and client transcript paths.
Add run-scoped spoken confirmation for high-impact actions, mutation digests, bootstrap-context injection, talk.client.transcript and talk.client.close protocol methods, and Control UI adoption. This adds zero new configuration.
Co-authored-by: Clifton King <clifton@users.noreply.github.com>
The Nodes & Devices screen showed pending pairing counts but pointed users
at a CLI command on the gateway host. An already-trusted phone can now act:
approve/reject pending requests (gated on hello methods plus the
operator.pairing or admin scope, mirroring the gateway's second
authorization gate for requested roles/scopes) and remove paired devices
(admin-gated, matching the gateway's removal contract). The confirm dialog
is the consent surface: it renders the complete requested roles/scopes
unbounded and scrollable with control/bidi sanitization, keyed to the
gateway identity so a pending confirmation cannot survive a gateway switch.
Mutations capture the gateway scope at claim time, require the exact write
acknowledgement plus a canonical device.pair.list readback before claiming
success, surface definitive gateway denials verbatim, and take over the
refresh generation with the verified snapshot so stale refreshes cannot
republish pre-mutation state. Bootstrap-requested scopes stay on the
canonical limited profile; extending it with operator.pairing is a
deliberate gateway-side product decision left open.
Publicly-CA-valid gateway hosts (e.g. Tailscale Funnel with Let's Encrypt)
no longer force fingerprint pinning: a real platform-trust handshake with
HTTPS hostname verification and SNI connects without prompting, ending the
changed-pin re-prompt on every cert renewal. Candidate gating excludes
.local, IP literals, and single-label hosts so LAN endpoints keep pinning.
Existing pins stay authoritative: a valid public chain never silently clears
a stored pin - the changed-pin prompt gains an explicit one-time 'Use system
trust' switch instead. Discovered endpoints still require a stored pin for
reconnect eligibility (discovery data is attacker-controllable). When the
TLS probe cannot capture a certificate, the trust dialog now accepts an
out-of-band SHA-256 fingerprint paste (sha256:/colon tolerant, 64-hex
validated) which is then enforced on connect. Malformed stored pins now
fail closed instead of silently degrading. Both probe attempts share one
connect+handshake budget so stalled hosts cannot double the probe latency.
* feat(android): accept audio and document attachments via share sheet and composer
Extends Android share-sheet intake beyond text/images: audio plus a curated
document set (PDF, OOXML, CSV, Markdown) with code-side mime enforcement,
provider-type revalidation while the URI grant is live, bounded streaming
reads, per-kind size budgets aligned with the outbox command limit, and a
document/audio picker in the composer. Share parsing moves off the main
thread behind a bounded slot queue; draft-not-autosend and drop/failure
reporting behavior unchanged.
* test(android): cover audio and document share intake
* test(android): align composer draft test with attachment kinds
* chore(android): refresh native i18n inventory
* chore(android): rebuild native locale artifacts for inventory order
* fix(android): stop push-to-talk from dropping speech after mid-hold pauses
The SpeechRecognizer PTT path ran one session with a 2.5s silence window,
kept only the last final, and never restarted during a hold - speech after
a mid-hold pause was silently dropped. Capture now uses an ordered rung
ladder: API 33+ app-owned AudioRecord piped via EXTRA_AUDIO_SOURCE with
EXTRA_SEGMENTED_SESSION, then silence-keyed segmented sessions, then a
restart-during-hold single-session fallback for older devices; failed holds
only descend. Final segments accumulate and merge locale-aware (no ASCII
separators after CJK/Arabic punctuation), with a trailing live partial
deduped against the last final. Release waits one bounded grace for the
terminal recognizer callback so late finals are not truncated; rapid
re-press drains the prior release instead of cancelling it; cancellation
mid-release still tears capture down, resumes realtime talk, and resolves
the stop payload. Raw-rung release also stops the recognizer since
EXTRA_AUDIO_SOURCE is optional and a service may run its own microphone.
Input level meters from our PCM stream in the raw rung where onRmsChanged
never fires.
* fix(android): API-33 recognizer extras behind RequiresApi and inventory refresh
Min-SDK lint (InlinedApi) flagged the API 33 RecognizerIntent extras; they
now live in RequiresApi(TIRAMISU) helpers behind explicit SDK gates, and
the native i18n inventory is refreshed for the moved lines.
* chore(android): rebuild native locale artifacts for inventory order
* feat(apple): copy or save rendered widgets as images from the chat transcript
* chore(i18n): regenerate native locale artifacts for widget export strings
* chore(i18n): resync native locale artifacts after main i18n advance
