mirror of
https://github.com/openclaw/openclaw.git
synced 2026-07-23 15:01:16 +00:00
* fix(android): restore full Wear companion UI * chore(android): sync native i18n inventory * fix(android): negotiate Wear companion capabilities * chore(android): sync generated Japanese localization * fix(android): preserve Wear reply notification setup Co-authored-by: JARVIS-Glasses <whatsskilll@gmail.com> * fix(android): harden Wear conversation controls Co-authored-by: JARVIS-Glasses <whatsskilll@gmail.com> * fix(android): preserve Wear control context Co-authored-by: JARVIS-Glasses <whatsskilll@gmail.com> * fix(android): keep Wear sessions agent-scoped Co-authored-by: JARVIS-Glasses <whatsskilll@gmail.com> * fix(android): isolate Wear conversation context Co-authored-by: JARVIS-Glasses <whatsskilll@gmail.com> --------- Co-authored-by: Peter Steinberger <steipete@gmail.com>
384 lines
17 KiB
Markdown
384 lines
17 KiB
Markdown
## OpenClaw Android App
|
|
|
|
OpenClaw Android is the officially released Google Play app. It connects to an OpenClaw Gateway as a companion node for chat, voice, approvals, screen, and device-aware automation.
|
|
|
|
### Current App Surface
|
|
|
|
- [x] New 4-step onboarding flow
|
|
- [x] Connect tab with `Setup Code` + `Manual` modes
|
|
- [x] Encrypted persistence for gateway setup/auth state
|
|
- [x] Chat UI restyled
|
|
- [x] Settings UI restyled and de-duplicated (gateway controls moved to Connect)
|
|
- [x] QR code scanning in onboarding
|
|
- [x] Performance improvements
|
|
- [x] Streaming support in chat UI
|
|
- [x] Dedicated per-device Android chat session created/adopted on connect without resetting history
|
|
- [x] Request camera/location and other permissions in onboarding/settings flow
|
|
- [x] Push notifications for gateway/chat status updates
|
|
- [x] Security hardening (biometric lock, token handling, safer defaults)
|
|
- [x] Authenticated background presence beacons
|
|
- [x] Voice tab full functionality
|
|
- [x] Foreground on-device Voice Wake with Gateway-synced wake words
|
|
- [x] Screen tab full functionality
|
|
- [x] Skill Workshop settings can filter proposals, inspect proposal content, and apply/reject/quarantine drafts through Gateway RPCs
|
|
- [x] Skills settings can search installed skills, enable or disable them, and install Gateway-verified ClawHub releases
|
|
- [x] Per-app language selection for translated resources follows Android system settings and persistence
|
|
- [x] Cron job settings support details, run history, run now, edits, enable/disable, and deletion with admin-scoped Gateway access
|
|
- [x] Wear OS companion proxies sessions, transcripts, replies, aborts, and realtime Talk through the paired phone without storing Gateway credentials on the watch
|
|
|
|
## Open in Android Studio
|
|
|
|
- Open the folder `apps/android`.
|
|
|
|
## Wear OS companion
|
|
|
|
The `wear` app is a paired-phone companion with the same application ID and signing identity as the phone app. The watch discovers the phone through Wear OS Data Layer, then uses the phone's existing authenticated operator session. It never receives or stores Gateway tokens, passwords, TLS pins, or device-signing identity.
|
|
|
|
The watch supports agent and session selection, bounded text-only transcript history, streaming reply state, text and voice replies, abort, realtime Talk within the selected session, paired-phone Gateway controls, local reply notifications, theme and automatic-speech settings, and a launch Tile. Realtime Talk streams watch microphone and playback audio over a temporary Wear OS Data Layer channel; it still uses the phone's authenticated Gateway session and closes when the selected phone or Gateway connection changes. A missing Data Layer event sequence or changed phone-process epoch triggers a fresh history request instead of applying uncertain deltas. Agent and Gateway controls are capability-negotiated so an older paired phone remains usable during staggered updates.
|
|
|
|
```bash
|
|
cd apps/android
|
|
./gradlew :wear:testDebugUnitTest :wear:assembleDebug :wear:lintDebug :wear:ktlintCheck
|
|
```
|
|
|
|
## Build / Run
|
|
|
|
```bash
|
|
cd apps/android
|
|
./gradlew :app:assemblePlayDebug
|
|
./gradlew :app:installPlayDebug
|
|
./gradlew :app:testPlayDebugUnitTest
|
|
cd ../..
|
|
pnpm android:release:archive
|
|
```
|
|
|
|
Third-party debug flavor:
|
|
|
|
```bash
|
|
cd apps/android
|
|
./gradlew :app:assembleThirdPartyDebug
|
|
./gradlew :app:installThirdPartyDebug
|
|
./gradlew :app:testThirdPartyDebugUnitTest
|
|
```
|
|
|
|
Repository-backed debug Gradle invocations, including `pnpm android:run` and
|
|
`pnpm android:screenshots`, stamp the full checkout commit and capture one UTC
|
|
build timestamp shared by every debug variant in that invocation. Release
|
|
tasks still require explicit `openclawBuildCommit` and
|
|
`openclawBuildTimestamp` properties so signed artifacts remain reproducible.
|
|
|
|
Android release archives use the pinned version in `apps/android/version.json`. Update it with:
|
|
|
|
```bash
|
|
pnpm android:version
|
|
pnpm android:version:check
|
|
pnpm android:version:pin -- --from-gateway
|
|
pnpm android:version:pin -- --version 2026.6.5 --version-code 2026060501
|
|
```
|
|
|
|
Release-owner signing sync:
|
|
|
|
```bash
|
|
pnpm android:release:signing:plan
|
|
MATCH_PASSWORD=<signing repo password> pnpm android:release:signing:sync:pull
|
|
MATCH_PASSWORD=<signing repo password> pnpm android:release:signing:check
|
|
```
|
|
|
|
The signing sync pulls encrypted Android upload-key assets from the shared `apps-signing` repo and materializes decrypted files under `apps/android/build/release-signing/`.
|
|
Standalone release APK verification also requires that key's public certificate SHA-256 fingerprint to match `Config/ReleaseSigning.json`.
|
|
|
|
Generate raw Google Play screenshots:
|
|
|
|
```bash
|
|
pnpm android:screenshots
|
|
```
|
|
|
|
The screenshot script defaults to a retained `OpenClaw_Screenshots_API36` AVD
|
|
created from Android's no-cutout Pixel 2 profile. It creates the AVD when
|
|
missing, boots it headlessly, waits for Android to finish booting, disables
|
|
animations, captures the screenshots, then shuts down the emulator it started.
|
|
The API 36 Google APIs system image must be installed in the local Android SDK.
|
|
Use `ANDROID_SCREENSHOT_AVD` or `--avd` to select another AVD, or `--device` to
|
|
explicitly use a connected emulator.
|
|
|
|
`pnpm android:release:archive` builds signed release artifacts into `apps/android/build/release-artifacts/` and writes `.sha256` checksum files:
|
|
|
|
- Play build: `openclaw-<version>-play-release.aab`
|
|
- Wear build: `openclaw-<version>-wear-release.aab`
|
|
- Third-party build: `openclaw-<version>-third-party-release.apk`
|
|
|
|
`pnpm android:bundle:release` is an alias for the same Fastlane archive lane.
|
|
|
|
Regular final and correction OpenClaw releases publish the signed third-party APK as `OpenClaw-Android.apk` with a checksum manifest and GitHub Actions provenance. `.github/workflows/android-release.yml` is the only automated GitHub Release upload path; `OpenClaw Release Publish` dispatches it while the canonical release is still a draft and blocks publication until the uploaded asset contract verifies.
|
|
|
|
The protected `android-release` environment supplies `MATCH_PASSWORD`; the repository's read-only GitHub App token checks out encrypted material from `openclaw/apps-signing`. The workflow builds the exact release tag, refuses to replace different existing bytes, and re-downloads the APK for checksum, certificate, and provenance verification.
|
|
|
|
`pnpm android:release:archive` is for local archive validation only. It is not a
|
|
fallback upload path after `pnpm android:release:upload` fails.
|
|
|
|
Agent-driven Google Play uploads must use `pnpm android:release:upload` as the
|
|
only release path. If that command fails, stop and fix the failing screenshot,
|
|
metadata, signing, validation, archive, or upload step before trying again. Do
|
|
not upload archived artifacts through direct Fastlane lanes, Gradle artifacts,
|
|
Google Play API commands, or Play Console mutation commands.
|
|
|
|
The release lane uploads the phone and Wear bundles in one atomic Google Play
|
|
edit. It publishes the phone bundle to `GOOGLE_PLAY_TRACK` and maps the Wear
|
|
bundle to the corresponding form-factor track (`wear:qa` for the default
|
|
internal channel, otherwise `wear:<track>`).
|
|
|
|
See `apps/android/VERSIONING.md` and `apps/android/fastlane/SETUP.md` for the release workflow.
|
|
|
|
Prefer `pnpm android:release:archive`, which stamps and validates the full Git commit and one UTC build timestamp before signing. Flavor-specific direct Gradle release tasks must pass the same metadata explicitly:
|
|
|
|
```bash
|
|
cd apps/android
|
|
commit="$(git -C ../.. rev-parse HEAD)"
|
|
built_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
|
./gradlew -PopenclawBuildCommit="$commit" -PopenclawBuildTimestamp="$built_at" :app:bundlePlayRelease
|
|
./gradlew -PopenclawBuildCommit="$commit" -PopenclawBuildTimestamp="$built_at" :wear:bundleRelease
|
|
./gradlew -PopenclawBuildCommit="$commit" -PopenclawBuildTimestamp="$built_at" :app:bundleThirdPartyRelease
|
|
```
|
|
|
|
## Kotlin Lint + Format
|
|
|
|
```bash
|
|
pnpm android:lint
|
|
pnpm android:format
|
|
```
|
|
|
|
Android framework/resource lint (separate pass):
|
|
|
|
```bash
|
|
pnpm android:lint:android
|
|
```
|
|
|
|
Direct Gradle tasks:
|
|
|
|
```bash
|
|
cd apps/android
|
|
./gradlew :app:ktlintCheck :benchmark:ktlintCheck
|
|
./gradlew :app:ktlintFormat :benchmark:ktlintFormat
|
|
./gradlew :app:lintPlayDebug :app:lintThirdPartyDebug
|
|
```
|
|
|
|
`gradlew` auto-detects the Android SDK at `~/Library/Android/sdk` (macOS default) if `ANDROID_SDK_ROOT` / `ANDROID_HOME` are unset.
|
|
|
|
## Macrobenchmark (Startup + Frame Timing)
|
|
|
|
```bash
|
|
cd apps/android
|
|
./gradlew :benchmark:connectedDebugAndroidTest
|
|
```
|
|
|
|
Reports are written under:
|
|
|
|
- `apps/android/benchmark/build/reports/androidTests/connected/`
|
|
|
|
## Perf CLI (low-noise)
|
|
|
|
Deterministic startup measurement + hotspot extraction with compact CLI output:
|
|
|
|
```bash
|
|
cd apps/android
|
|
./scripts/perf-startup-benchmark.sh
|
|
./scripts/perf-startup-hotspots.sh
|
|
```
|
|
|
|
Benchmark script behavior:
|
|
|
|
- Runs only `StartupMacrobenchmark#coldStartup` (10 iterations).
|
|
- Prints median/min/max/COV in one line.
|
|
- Writes timestamped snapshot JSON to `apps/android/benchmark/results/`.
|
|
- Auto-compares with previous local snapshot (or pass explicit baseline: `--baseline <old-benchmarkData.json>`).
|
|
|
|
Hotspot script behavior:
|
|
|
|
- Ensures debug app installed, captures startup `simpleperf` data for `.MainActivity`.
|
|
- Prints top DSOs, top symbols, and key app-path clues (Compose/MainActivity/WebView).
|
|
- Writes raw `perf.data` path for deeper follow-up if needed.
|
|
|
|
## Run on a Real Android Phone (USB)
|
|
|
|
1) On phone, enable **Developer options** + **USB debugging**.
|
|
2) Connect by USB and accept the debugging trust prompt on phone.
|
|
3) Verify ADB can see the device:
|
|
|
|
```bash
|
|
adb devices -l
|
|
```
|
|
|
|
4) Install + launch debug build:
|
|
|
|
```bash
|
|
pnpm android:install
|
|
pnpm android:run
|
|
```
|
|
|
|
If `adb devices -l` shows `unauthorized`, re-plug and accept the trust prompt again.
|
|
|
|
### USB-only gateway testing (no LAN dependency)
|
|
|
|
Use `adb reverse` so Android `localhost:18789` tunnels to your laptop `localhost:18789`.
|
|
|
|
Terminal A (gateway):
|
|
|
|
```bash
|
|
pnpm openclaw gateway --port 18789 --verbose
|
|
```
|
|
|
|
Terminal B (USB tunnel):
|
|
|
|
```bash
|
|
adb reverse tcp:18789 tcp:18789
|
|
```
|
|
|
|
Then in app **Connect → Manual**:
|
|
|
|
- Host: `127.0.0.1`
|
|
- Port: `18789`
|
|
- TLS: off
|
|
|
|
## Hot Reload / Fast Iteration
|
|
|
|
This app is native Kotlin + Jetpack Compose.
|
|
|
|
- For Compose UI edits: use Android Studio **Live Edit** on a debug build (works on physical devices; project `minSdk=31` already meets API requirement).
|
|
- For many non-structural code/resource changes: use Android Studio **Apply Changes**.
|
|
- For structural/native/manifest/Gradle changes: do full reinstall (`pnpm android:run`).
|
|
- Canvas web content already supports live reload when loaded from Gateway `__openclaw__/canvas/` (see `docs/platforms/android.md`).
|
|
|
|
## Connect / Pair
|
|
|
|
1) Start the gateway (on your main machine):
|
|
|
|
```bash
|
|
pnpm openclaw gateway --port 18789 --verbose
|
|
```
|
|
|
|
2) In the Android app:
|
|
|
|
- Open the **Connect** tab.
|
|
- Use **Setup Code** or **Manual** mode to connect.
|
|
|
|
3) Approve pairing (on the gateway machine):
|
|
|
|
```bash
|
|
openclaw devices list
|
|
openclaw devices approve <requestId>
|
|
```
|
|
|
|
More details: `docs/platforms/android.md`.
|
|
|
|
## Permissions
|
|
|
|
- Discovery:
|
|
- Android 13+ (`API 33+`): `NEARBY_WIFI_DEVICES`
|
|
- Android 12 and below: `ACCESS_FINE_LOCATION` (required for NSD scanning)
|
|
- Location:
|
|
- Both flavors: `ACCESS_FINE_LOCATION` / `ACCESS_COARSE_LOCATION` for foreground checks.
|
|
- Third-party flavor only: `ACCESS_BACKGROUND_LOCATION` plus `FOREGROUND_SERVICE_LOCATION` for user-enabled `Always` checks.
|
|
- Foreground service notification (Android 13+): `POST_NOTIFICATIONS`
|
|
- Camera:
|
|
- `CAMERA` for `camera.snap` and `camera.clip`
|
|
- `RECORD_AUDIO` for `camera.clip` when `includeAudio=true`
|
|
|
|
## Google Play Restricted Permissions
|
|
|
|
As of March 19, 2026, these manifest permissions are the main Google Play policy risk for this app:
|
|
|
|
- `READ_SMS`
|
|
- `SEND_SMS`
|
|
- `READ_CALL_LOG`
|
|
|
|
Why these matter:
|
|
|
|
- Google Play treats SMS and Call Log access as highly restricted. In most cases, Play only allows them for the default SMS app, default Phone app, default Assistant, or a narrow policy exception.
|
|
- Review usually involves a `Permissions Declaration Form`, policy justification, and demo video evidence in Play Console.
|
|
- The Play build removes these behind the `play` flavor.
|
|
- Photo library access is also removed from the Play build. Use third-party builds for `photos.latest`.
|
|
|
|
Current OpenClaw Android implication:
|
|
|
|
- APK / sideload build can keep SMS, Call Log, and recent-photo features.
|
|
- Google Play build excludes SMS send/search, Call Log search, and recent-photo access unless the product is intentionally positioned and approved under the relevant policy exception.
|
|
- The repo now ships this split as Android product flavors:
|
|
- `play`: removes `READ_SMS`, `SEND_SMS`, `READ_CALL_LOG`, `READ_MEDIA_IMAGES`, `READ_MEDIA_VISUAL_USER_SELECTED`, `READ_EXTERNAL_STORAGE`, and background location; hides SMS, Call Log, Photos, and `Always` location surfaces.
|
|
- Installed-app listing is user controlled. `device.apps` is advertised only after the user enables **Settings > Phone Capabilities > Installed Apps**. The command defaults to launcher-visible apps and does not require `QUERY_ALL_PACKAGES`.
|
|
- `thirdParty`: keeps the full permission set and the existing SMS / Call Log / Photos functionality, and offers explicit `Always` location opt-in through Android settings.
|
|
|
|
Policy links:
|
|
|
|
- [Google Play SMS and Call Log policy](https://support.google.com/googleplay/android-developer/answer/10208820?hl=en)
|
|
- [Google Play sensitive permissions policy hub](https://support.google.com/googleplay/android-developer/answer/16558241)
|
|
- [Android default handlers guide](https://developer.android.com/guide/topics/permissions/default-handlers)
|
|
|
|
Other Play-restricted surfaces to watch if added later:
|
|
|
|
- `ACCESS_BACKGROUND_LOCATION`
|
|
- `MANAGE_EXTERNAL_STORAGE`
|
|
- `QUERY_ALL_PACKAGES`
|
|
- `REQUEST_INSTALL_PACKAGES`
|
|
- `AccessibilityService`
|
|
|
|
Reference links:
|
|
|
|
- [Background location policy](https://support.google.com/googleplay/android-developer/answer/9799150)
|
|
- [AccessibilityService policy](https://support.google.com/googleplay/android-developer/answer/10964491?hl=en-GB)
|
|
- [Photo and Video Permissions policy](https://support.google.com/googleplay/android-developer/answer/14594990)
|
|
|
|
## Integration Capability Test (Preconditioned)
|
|
|
|
This suite assumes setup is already done manually. It does **not** install/run/pair automatically.
|
|
|
|
Pre-req checklist:
|
|
|
|
1) Gateway is running and reachable from the Android app.
|
|
2) Android app is connected to that gateway and `openclaw nodes status` shows it as paired + connected.
|
|
3) App stays unlocked and in foreground for the whole run.
|
|
4) Open the app **Screen** tab and keep it active during the run (canvas/A2UI commands require the canvas WebView attached there).
|
|
5) Grant runtime permissions for capabilities you expect to pass (camera/mic/location/notification listener/location, etc.).
|
|
6) No interactive system dialogs should be pending before test start.
|
|
7) Canvas host is enabled and reachable from the device for remote Canvas checks (do not run gateway with `OPENCLAW_SKIP_CANVAS_HOST=1`; startup logs should include `canvas host mounted at .../__openclaw__/`).
|
|
8) Local operator test client pairing is approved. If first run fails with `pairing required`, preview the latest pending request, approve the printed request ID, then rerun:
|
|
9) For A2UI checks, keep the app on **Screen** tab; the node uses its bundled app-owned A2UI page for message application.
|
|
|
|
```bash
|
|
openclaw devices list
|
|
openclaw devices approve --latest # preview only; copy the requestId from output
|
|
openclaw devices approve <requestId>
|
|
```
|
|
|
|
Run:
|
|
|
|
```bash
|
|
pnpm android:test:integration
|
|
```
|
|
|
|
Optional overrides:
|
|
|
|
- `OPENCLAW_ANDROID_GATEWAY_URL=ws://...` (default: from your local OpenClaw config)
|
|
- `OPENCLAW_ANDROID_GATEWAY_TOKEN=...`
|
|
- `OPENCLAW_ANDROID_GATEWAY_PASSWORD=...`
|
|
- `OPENCLAW_ANDROID_NODE_ID=...` or `OPENCLAW_ANDROID_NODE_NAME=...`
|
|
|
|
What it does:
|
|
|
|
- Reads `node.describe` command list from the selected Android node.
|
|
- Invokes advertised non-interactive commands.
|
|
- Skips `screen.record` in this suite (Android requires interactive per-invocation screen-capture consent).
|
|
- Asserts command contracts (success or expected deterministic error for safe-invalid calls like `sms.send` and `notifications.actions`).
|
|
|
|
Common failure quick-fixes:
|
|
|
|
- `pairing required` before tests start:
|
|
- list pending requests (`openclaw devices list`), then approve with the exact ID (`openclaw devices approve <requestId>`) and rerun.
|
|
- `A2UI host not reachable` / `A2UI_HOST_UNAVAILABLE`:
|
|
- keep the app foregrounded on the **Screen** tab and rerun. A2UI commands use the bundled app-owned A2UI page; the Gateway Canvas host is still needed for remote Canvas checks, but not for A2UI message application.
|
|
- `NODE_BACKGROUND_UNAVAILABLE: canvas unavailable`:
|
|
- app is not effectively ready for canvas commands; keep app foregrounded and **Screen** tab active.
|
|
|
|
## Contributions
|
|
|
|
Maintainer: @obviyus. For issues/questions/contributions, please open an issue or reach out on Discord.
|