* feat(android,gateway): expose mobile.ui.observe/act as node commands (PR 2/3)
Wire the PR1 AccessibilityService executor over the existing node.invoke
transport. Additive: no gateway protocol version bump.
- New commands mobile.ui.observe / mobile.ui.act (capability mobileUI),
generated into the protocol constants.
- thirdParty MobileUiHandler owns one mutex-serialized AccessibilityActionExecutor
and bridges JSON <-> the PR1 snapshot/action model; play flavor ships a
permanently-unavailable no-op stub (Play APK stays accessibility-free).
- Commands advertised only when accessibilityControlEnabled and the service is
connected; NodeRuntime refreshes the advertised surface on connect/disconnect.
- Classified dangerous in node-command-policy (declarable-but-armed, mirroring
computer.act): the transport cannot invoke them until gateway.nodes.allowCommands
explicitly arms them. Screen reads are treated as dangerous too, so observe is
gated as well as act.
Both flavors assemble; thirdParty/play unit tests, ktlint, android lint, and the
gateway node-command-policy tests pass; protocol generator is idempotent; Play
APK verified to contain zero accessibility classes. Emulator: no regression to
PR1 dev-screen observe.
* fix(gateway): migrate mobile.ui arming test/comment to gateway.nodes.commands.allow
Main renamed the node-command arming config from gateway.nodes.allowCommands to
gateway.nodes.commands.allow; update the mobile.ui policy test config shape and
the comment to the current contract. Production resolution already reads the new
path.
* chore(android): update native i18n baseline for PR2 line-number shifts
* feat(android): AccessibilityService UI executor (thirdParty, PR 1/3)
Add a thirdParty-flavor-only AccessibilityService that observes the active
app's UI as a bounded semantic snapshot and performs typed actions, exercised
via a local developer screen. No gateway/agent wiring yet (PR 2 adds
node.invoke commands, PR 3 the agent tool + policy gates).
- Play APK stays accessibility-free: service, config, executor, and UI live
entirely under src/thirdParty; SensitiveFeatureConfig.accessibilityControlEnabled
gates it. Verified: 0 accessibility refs in every merged Play manifest and
0 accessibility classes in the Play dex.
- Semantic-first executor: observe() returns a bounded snapshot (node/depth/text
caps, deterministic order, password + sensitive-field redaction, stable action
vocabulary, generation-scoped refs); act() performs one typed action with a
closed ActionOutcomeCode result set.
- Safety model: coordinate gestures gated by package-match (fail-closed) + a UI
epoch advanced on window/content/scroll/text mutation events; node actions
gated by package-match + per-node refresh(); global actions ungated. Capture
runs off the main thread. Dev UI honestly disables cross-app node controls
(only reachable while the target is foreground; validated via the remote path).
Built and emulator-tested on API 36: live connection status, immediate observe,
a 103-node cross-app Settings capture, and global Home. Both flavors assemble;
thirdParty unit tests, ktlint, and android lint pass.
* feat(android): gate accessibility control behind an off-by-default opt-in
Make landing the accessibility feature a no-op for existing thirdParty users:
the service is invisible and inert until the user explicitly opts in.
- The AccessibilityService and its dev activity are declared
android:enabled="false", so a fresh install exposes NO new accessibility
service in system settings and nothing can bind it (verified on device:
absent from the installed-services list and unbindable; shell cannot enable
it either — only the app can).
- New thirdParty-only "Control other apps" toggle (persisted in the existing
openclaw.node prefs, default OFF). Turning it on enables both components via
PackageManager.setComponentEnabledSetting and deep-links to Accessibility
settings so the user can grant it; turning it off disables them again
(DONT_KILL_APP). A disclosure describes what enabling does.
- Play flavor is a no-op (FlavorPhoneCapabilitiesSettings = Unit); no
accessibility component/controller/toggle references reach the Play APK.
- No new permissions or dependencies. The compile-time flavor gate and the
gateway dangerous-command arming remain as additional layers.
* chore(android): update native i18n baseline for accessibility control strings
