diff --git a/docs/plugins/google-meet.md b/docs/plugins/google-meet.md index 92186901728..45af386aa2f 100644 --- a/docs/plugins/google-meet.md +++ b/docs/plugins/google-meet.md @@ -104,6 +104,18 @@ openclaw googlemeet create openclaw googlemeet join https://meet.google.com/new-abcd-xyz --transport chrome-node ``` +`googlemeet create` has two paths: + +- API create: used when Google Meet OAuth credentials are configured. This is + the most deterministic path and does not depend on browser UI state. +- Browser fallback: used when OAuth credentials are absent. OpenClaw uses the + pinned Chrome node, opens `https://meet.google.com/new`, waits for Google to + redirect to a real meeting-code URL, then returns that URL. This path requires + the OpenClaw Chrome profile on the node to already be signed in to Google. + +The command output includes a `source` field (`api` or `browser`) so agents can +explain which path was used. + Or tell an agent: "Create a Google Meet, join it with realtime voice, and send me the link." The agent should call `google_meet` with `action: "create"`, copy the returned `meetingUri`, then call `google_meet` with `action: "join"` and @@ -400,7 +412,11 @@ openclaw googlemeet join https://meet.google.com/abc-defg-hij \ ## OAuth and preflight -Google Meet Media API access uses a personal OAuth client first. Configure +OAuth is optional for creating a Meet link because `googlemeet create` can fall +back to browser automation. Configure OAuth when you want official API create, +space resolution, or Meet Media API preflight checks. + +Google Meet API access uses a personal OAuth client first. Configure `oauth.clientId` and optionally `oauth.clientSecret`, then run: ```bash @@ -411,11 +427,15 @@ The command prints an `oauth` config block with a refresh token. It uses PKCE, localhost callback on `http://localhost:8085/oauth2callback`, and a manual copy/paste flow with `--manual`. -The OAuth consent includes Meet space creation, Meet space read access, and -Meet conference media read access. If you authenticated before meeting creation +The OAuth consent includes Meet space creation, Meet space read access, and Meet +conference media read access. If you authenticated before meeting creation support existed, rerun `openclaw googlemeet auth login --json` so the refresh token has the `meetings.space.created` scope. +No OAuth credentials are needed for the browser fallback. In that mode, Google +auth comes from the signed-in Chrome profile on the selected node, not from +OpenClaw config. + These environment variables are accepted as fallbacks: - `OPENCLAW_GOOGLE_MEET_CLIENT_ID` or `GOOGLE_MEET_CLIENT_ID` @@ -439,21 +459,50 @@ Run preflight before media work: openclaw googlemeet preflight --meeting https://meet.google.com/abc-defg-hij ``` -Create a fresh Meet space with the same OAuth config: +Create a fresh Meet space: ```bash openclaw googlemeet create ``` -The command prints the new `meeting uri` and `space`. Agents can use the +The command prints the new `meeting uri` and source. With OAuth credentials it +uses the official Google Meet API. Without OAuth credentials it uses the pinned +Chrome node's signed-in browser profile as a fallback. Agents can use the `google_meet` tool with `action: "create"` to create a meeting, then call `action: "join"` with the returned `meetingUri`. -Creating a Meet space only creates the meeting URL. The Chrome or Chrome-node -transport still needs a signed-in Google Chrome profile to join through the -browser. If the profile is signed out, OpenClaw reports -`manualActionRequired: true` and asks the operator to finish Google login before -retrying the join. +Example JSON output from the browser fallback: + +```json +{ + "source": "browser", + "meetingUri": "https://meet.google.com/abc-defg-hij", + "browser": { + "nodeId": "ba0f4e4bc...", + "targetId": "tab-1" + } +} +``` + +Example JSON output from API create: + +```json +{ + "source": "api", + "meetingUri": "https://meet.google.com/abc-defg-hij", + "space": { + "name": "spaces/abc-defg-hij", + "meetingCode": "abc-defg-hij", + "meetingUri": "https://meet.google.com/abc-defg-hij" + } +} +``` + +Creating a Meet only creates or discovers the meeting URL. The Chrome or +Chrome-node transport still needs a signed-in Google Chrome profile to join +through the browser. If the profile is signed out, OpenClaw reports +`manualActionRequired: true` or a browser fallback error and asks the operator +to finish Google login before retrying. Set `preview.enrollmentAcknowledged: true` only after confirming your Cloud project, OAuth principal, and meeting participants are enrolled in the Google @@ -740,15 +789,20 @@ Common manual actions: ### Meeting creation fails -`googlemeet create` uses the Google Meet API `spaces.create` endpoint. Confirm: +`googlemeet create` first uses the Google Meet API `spaces.create` endpoint +when OAuth credentials are configured. Without OAuth credentials it falls back +to the pinned Chrome node browser. Confirm: -- `oauth.clientId` and `oauth.refreshToken` are configured, or matching - `OPENCLAW_GOOGLE_MEET_*` environment variables are present. -- The refresh token was minted after create support was added. Older tokens may - be missing the `meetings.space.created` scope; rerun +- For API creation: `oauth.clientId` and `oauth.refreshToken` are configured, + or matching `OPENCLAW_GOOGLE_MEET_*` environment variables are present. +- For API creation: the refresh token was minted after create support was + added. Older tokens may be missing the `meetings.space.created` scope; rerun `openclaw googlemeet auth login --json` and update plugin config. -- The Google Cloud project and OAuth principal are allowed to use the required - Google Meet API scopes. +- For browser fallback: `defaultTransport: "chrome-node"` and + `chromeNode.node` point at a connected node with `browser.proxy` and + `googlemeet.chrome`. +- For browser fallback: the OpenClaw Chrome profile on that node is signed in + to Google and can open `https://meet.google.com/new`. ### Agent joins but does not talk diff --git a/extensions/google-meet/index.test.ts b/extensions/google-meet/index.test.ts index 7e272336ed8..0528b8c0a6f 100644 --- a/extensions/google-meet/index.test.ts +++ b/extensions/google-meet/index.test.ts @@ -777,6 +777,72 @@ describe("google-meet plugin", () => { } }); + it("creates a Meet through browser fallback when OAuth is not configured", async () => { + const { methods, nodesInvoke } = setup( + { + defaultTransport: "chrome-node", + chromeNode: { node: "parallels-macos" }, + }, + { + nodesInvokeHandler: async (params) => { + const proxy = params.params as { path?: string; body?: { url?: string } }; + if (proxy.path === "/tabs/open") { + return { + payload: { + result: { + targetId: "tab-1", + title: "Meet", + url: proxy.body?.url, + }, + }, + }; + } + if (proxy.path === "/act") { + return { + payload: { + result: { + ok: true, + targetId: "tab-1", + result: { + meetingUri: "https://meet.google.com/browser-made-url", + browserUrl: "https://meet.google.com/browser-made-url", + browserTitle: "Meet", + }, + }, + }, + }; + } + return { payload: { result: { ok: true } } }; + }, + }, + ); + const handler = methods.get("googlemeet.create") as + | ((ctx: { + params: Record; + respond: ReturnType; + }) => Promise) + | undefined; + const respond = vi.fn(); + + await handler?.({ params: {}, respond }); + + expect(respond.mock.calls[0]?.[0]).toBe(true); + expect(respond.mock.calls[0]?.[1]).toMatchObject({ + source: "browser", + meetingUri: "https://meet.google.com/browser-made-url", + browser: { nodeId: "node-1", targetId: "tab-1" }, + }); + expect(nodesInvoke).toHaveBeenCalledWith( + expect.objectContaining({ + command: "browser.proxy", + params: expect.objectContaining({ + path: "/tabs/open", + body: { url: "https://meet.google.com/new" }, + }), + }), + ); + }); + it("launches Chrome after the BlackHole check", async () => { const originalPlatform = process.platform; Object.defineProperty(process, "platform", { value: "darwin" }); diff --git a/extensions/google-meet/index.ts b/extensions/google-meet/index.ts index e1a4e9eeed8..85fb7b1514d 100644 --- a/extensions/google-meet/index.ts +++ b/extensions/google-meet/index.ts @@ -18,6 +18,7 @@ import { import { handleGoogleMeetNodeHostCommand } from "./src/node-host.js"; import { resolveGoogleMeetAccessToken } from "./src/oauth.js"; import { GoogleMeetRuntime } from "./src/runtime.js"; +import { createMeetWithBrowserProxyOnNode } from "./src/transports/chrome.js"; const googleMeetConfigSchema = { parse(value: unknown) { @@ -227,7 +228,45 @@ async function createSpaceFromParams(config: GoogleMeetConfig, raw: Record): boolean { + return Boolean( + normalizeOptionalString(raw.accessToken) ?? + normalizeOptionalString(raw.refreshToken) ?? + config.oauth.accessToken ?? + config.oauth.refreshToken, + ); +} + +async function createMeetFromParams(params: { + config: GoogleMeetConfig; + runtime: OpenClawPluginApi["runtime"]; + raw: Record; +}) { + if (hasGoogleMeetOAuth(params.config, params.raw)) { + const { token: _token, ...result } = await createSpaceFromParams(params.config, params.raw); + return result; + } + const browser = await createMeetWithBrowserProxyOnNode({ + runtime: params.runtime, + config: params.config, + }); + return { + source: browser.source, + meetingUri: browser.meetingUri, + space: { + name: `browser/${browser.meetingUri.split("/").pop()}`, + meetingUri: browser.meetingUri, + }, + browser: { + nodeId: browser.nodeId, + targetId: browser.targetId, + browserUrl: browser.browserUrl, + browserTitle: browser.browserTitle, + }, + }; } export default definePluginEntry({ @@ -284,8 +323,7 @@ export default definePluginEntry({ async ({ params, respond }: GatewayRequestHandlerOptions) => { try { const raw = asParamRecord(params); - const { token: _token, ...result } = await createSpaceFromParams(config, raw); - respond(true, result); + respond(true, await createMeetFromParams({ config, runtime: api.runtime, raw })); } catch (err) { sendError(respond, err); } @@ -395,8 +433,7 @@ export default definePluginEntry({ ); } case "create": { - const { token: _token, ...result } = await createSpaceFromParams(config, raw); - return json(result); + return json(await createMeetFromParams({ config, runtime: api.runtime, raw })); } case "test_speech": { const rt = await ensureRuntime(); diff --git a/extensions/google-meet/src/cli.ts b/extensions/google-meet/src/cli.ts index a2f6fa8efe3..862ca7ff310 100644 --- a/extensions/google-meet/src/cli.ts +++ b/extensions/google-meet/src/cli.ts @@ -145,6 +145,15 @@ function resolveCreateTokenOptions( }; } +function hasCreateOAuth(config: GoogleMeetConfig, options: CreateOptions): boolean { + return Boolean( + options.accessToken?.trim() || + options.refreshToken?.trim() || + config.oauth.accessToken || + config.oauth.refreshToken, + ); +} + export function registerGoogleMeetCli(params: { program: Command; config: GoogleMeetConfig; @@ -226,6 +235,28 @@ export function registerGoogleMeetCli(params: { .option("--expires-at ", "Cached access token expiry as unix epoch milliseconds") .option("--json", "Print JSON output", false) .action(async (options: CreateOptions) => { + if (!hasCreateOAuth(params.config, options)) { + const rt = await params.ensureRuntime(); + const result = await rt.createViaBrowser(); + const payload = { + source: result.source, + meetingUri: result.meetingUri, + browser: { + nodeId: result.nodeId, + targetId: result.targetId, + browserUrl: result.browserUrl, + browserTitle: result.browserTitle, + }, + }; + if (options.json) { + writeStdoutJson(payload); + return; + } + writeStdoutLine("meeting uri: %s", result.meetingUri); + writeStdoutLine("source: browser"); + writeStdoutLine("node: %s", result.nodeId); + return; + } const token = await resolveGoogleMeetAccessToken( resolveCreateTokenOptions(params.config, options), ); diff --git a/extensions/google-meet/src/runtime.ts b/extensions/google-meet/src/runtime.ts index 5d9ef08935a..92b122cca41 100644 --- a/extensions/google-meet/src/runtime.ts +++ b/extensions/google-meet/src/runtime.ts @@ -5,7 +5,11 @@ import type { PluginRuntime, RuntimeLogger } from "openclaw/plugin-sdk/plugin-ru import { normalizeOptionalString } from "openclaw/plugin-sdk/text-runtime"; import type { GoogleMeetConfig, GoogleMeetMode, GoogleMeetTransport } from "./config.js"; import { getGoogleMeetSetupStatus } from "./setup.js"; -import { launchChromeMeet, launchChromeMeetOnNode } from "./transports/chrome.js"; +import { + createMeetWithBrowserProxyOnNode, + launchChromeMeet, + launchChromeMeetOnNode, +} from "./transports/chrome.js"; import { buildMeetDtmfSequence, normalizeDialInNumber } from "./transports/twilio.js"; import type { GoogleMeetChromeHealth, @@ -84,6 +88,13 @@ export class GoogleMeetRuntime { return getGoogleMeetSetupStatus(this.params.config, { fullConfig: this.params.fullConfig }); } + async createViaBrowser() { + return createMeetWithBrowserProxyOnNode({ + runtime: this.params.runtime, + config: this.params.config, + }); + } + async join(request: GoogleMeetJoinRequest): Promise { const url = normalizeMeetUrl(request.url); const transport = resolveTransport(request.transport, this.params.config); diff --git a/extensions/google-meet/src/transports/chrome.ts b/extensions/google-meet/src/transports/chrome.ts index b43605f0e96..f46324efb95 100644 --- a/extensions/google-meet/src/transports/chrome.ts +++ b/extensions/google-meet/src/transports/chrome.ts @@ -231,6 +231,15 @@ type BrowserTab = { url?: string; }; +export type GoogleMeetBrowserCreateResult = { + meetingUri: string; + nodeId: string; + targetId?: string; + browserUrl?: string; + browserTitle?: string; + source: "browser"; +}; + function unwrapNodeInvokePayload(raw: unknown): unknown { const record = raw && typeof raw === "object" ? (raw as Record) : {}; if (typeof record.payloadJSON === "string" && record.payloadJSON.trim()) { @@ -283,6 +292,110 @@ function readBrowserTab(result: unknown): BrowserTab | undefined { return result && typeof result === "object" ? (result as BrowserTab) : undefined; } +function readBrowserCreateResult(result: unknown): { + meetingUri?: string; + browserUrl?: string; + browserTitle?: string; + manualAction?: string; +} { + const record = result && typeof result === "object" ? (result as Record) : {}; + const nested = + record.result && typeof record.result === "object" + ? (record.result as Record) + : record; + return { + meetingUri: typeof nested.meetingUri === "string" ? nested.meetingUri : undefined, + browserUrl: typeof nested.browserUrl === "string" ? nested.browserUrl : undefined, + browserTitle: typeof nested.browserTitle === "string" ? nested.browserTitle : undefined, + manualAction: typeof nested.manualAction === "string" ? nested.manualAction : undefined, + }; +} + +const CREATE_MEET_FROM_BROWSER_SCRIPT = `async () => { + const meetUrlPattern = /^https:\\/\\/meet\\.google\\.com\\/[a-z]{3}-[a-z]{4}-[a-z]{3}(?:$|[/?#])/i; + const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); + const current = () => location.href; + if (!current().startsWith("https://meet.google.com/")) { + return { + manualAction: "Sign in to Google in the OpenClaw browser profile, then retry meeting creation.", + browserUrl: current(), + browserTitle: document.title, + }; + } + for (let i = 0; i < 80; i += 1) { + const href = current(); + if (meetUrlPattern.test(href)) { + return { meetingUri: href, browserUrl: href, browserTitle: document.title }; + } + const text = document.body?.innerText ?? ""; + if (/sign in|use your google account|couldn't create|unable to create/i.test(text)) { + return { + manualAction: "Sign in to Google in the OpenClaw browser profile or resolve the Meet page prompt, then retry meeting creation.", + browserUrl: href, + browserTitle: document.title, + }; + } + await sleep(500); + } + return { + manualAction: "Google Meet did not return a meeting URL from the browser create flow before timeout.", + browserUrl: current(), + browserTitle: document.title, + }; +}`; + +export async function createMeetWithBrowserProxyOnNode(params: { + runtime: PluginRuntime; + config: GoogleMeetConfig; +}): Promise { + const nodeId = await resolveChromeNode({ + runtime: params.runtime, + requestedNode: params.config.chromeNode.node, + }); + const timeoutMs = Math.max(15_000, params.config.chrome.joinTimeoutMs); + const tab = readBrowserTab( + await callBrowserProxyOnNode({ + runtime: params.runtime, + nodeId, + method: "POST", + path: "/tabs/open", + body: { url: "https://meet.google.com/new" }, + timeoutMs, + }), + ); + const targetId = tab?.targetId; + if (!targetId) { + throw new Error("Browser fallback opened Google Meet but did not return a targetId."); + } + const evaluated = await callBrowserProxyOnNode({ + runtime: params.runtime, + nodeId, + method: "POST", + path: "/act", + body: { + kind: "evaluate", + targetId, + fn: CREATE_MEET_FROM_BROWSER_SCRIPT, + }, + timeoutMs, + }); + const result = readBrowserCreateResult(evaluated); + if (result.meetingUri) { + return { + source: "browser", + nodeId, + targetId, + meetingUri: result.meetingUri, + browserUrl: result.browserUrl, + browserTitle: result.browserTitle, + }; + } + throw new Error( + result.manualAction ?? + "Browser fallback could not create a Google Meet URL. Sign in to the OpenClaw browser profile, then retry.", + ); +} + function parseMeetBrowserStatus(result: unknown): GoogleMeetChromeHealth | undefined { const record = result && typeof result === "object" ? (result as Record) : {}; const raw = record.result;