mirror of
https://github.com/openclaw/openclaw.git
synced 2026-05-06 07:10:43 +00:00
feat(google-meet): add browser create fallback
This commit is contained in:
@@ -104,6 +104,18 @@ openclaw googlemeet create
|
||||
openclaw googlemeet join https://meet.google.com/new-abcd-xyz --transport chrome-node
|
||||
```
|
||||
|
||||
`googlemeet create` has two paths:
|
||||
|
||||
- API create: used when Google Meet OAuth credentials are configured. This is
|
||||
the most deterministic path and does not depend on browser UI state.
|
||||
- Browser fallback: used when OAuth credentials are absent. OpenClaw uses the
|
||||
pinned Chrome node, opens `https://meet.google.com/new`, waits for Google to
|
||||
redirect to a real meeting-code URL, then returns that URL. This path requires
|
||||
the OpenClaw Chrome profile on the node to already be signed in to Google.
|
||||
|
||||
The command output includes a `source` field (`api` or `browser`) so agents can
|
||||
explain which path was used.
|
||||
|
||||
Or tell an agent: "Create a Google Meet, join it with realtime voice, and send
|
||||
me the link." The agent should call `google_meet` with `action: "create"`, copy
|
||||
the returned `meetingUri`, then call `google_meet` with `action: "join"` and
|
||||
@@ -400,7 +412,11 @@ openclaw googlemeet join https://meet.google.com/abc-defg-hij \
|
||||
|
||||
## OAuth and preflight
|
||||
|
||||
Google Meet Media API access uses a personal OAuth client first. Configure
|
||||
OAuth is optional for creating a Meet link because `googlemeet create` can fall
|
||||
back to browser automation. Configure OAuth when you want official API create,
|
||||
space resolution, or Meet Media API preflight checks.
|
||||
|
||||
Google Meet API access uses a personal OAuth client first. Configure
|
||||
`oauth.clientId` and optionally `oauth.clientSecret`, then run:
|
||||
|
||||
```bash
|
||||
@@ -411,11 +427,15 @@ The command prints an `oauth` config block with a refresh token. It uses PKCE,
|
||||
localhost callback on `http://localhost:8085/oauth2callback`, and a manual
|
||||
copy/paste flow with `--manual`.
|
||||
|
||||
The OAuth consent includes Meet space creation, Meet space read access, and
|
||||
Meet conference media read access. If you authenticated before meeting creation
|
||||
The OAuth consent includes Meet space creation, Meet space read access, and Meet
|
||||
conference media read access. If you authenticated before meeting creation
|
||||
support existed, rerun `openclaw googlemeet auth login --json` so the refresh
|
||||
token has the `meetings.space.created` scope.
|
||||
|
||||
No OAuth credentials are needed for the browser fallback. In that mode, Google
|
||||
auth comes from the signed-in Chrome profile on the selected node, not from
|
||||
OpenClaw config.
|
||||
|
||||
These environment variables are accepted as fallbacks:
|
||||
|
||||
- `OPENCLAW_GOOGLE_MEET_CLIENT_ID` or `GOOGLE_MEET_CLIENT_ID`
|
||||
@@ -439,21 +459,50 @@ Run preflight before media work:
|
||||
openclaw googlemeet preflight --meeting https://meet.google.com/abc-defg-hij
|
||||
```
|
||||
|
||||
Create a fresh Meet space with the same OAuth config:
|
||||
Create a fresh Meet space:
|
||||
|
||||
```bash
|
||||
openclaw googlemeet create
|
||||
```
|
||||
|
||||
The command prints the new `meeting uri` and `space`. Agents can use the
|
||||
The command prints the new `meeting uri` and source. With OAuth credentials it
|
||||
uses the official Google Meet API. Without OAuth credentials it uses the pinned
|
||||
Chrome node's signed-in browser profile as a fallback. Agents can use the
|
||||
`google_meet` tool with `action: "create"` to create a meeting, then call
|
||||
`action: "join"` with the returned `meetingUri`.
|
||||
|
||||
Creating a Meet space only creates the meeting URL. The Chrome or Chrome-node
|
||||
transport still needs a signed-in Google Chrome profile to join through the
|
||||
browser. If the profile is signed out, OpenClaw reports
|
||||
`manualActionRequired: true` and asks the operator to finish Google login before
|
||||
retrying the join.
|
||||
Example JSON output from the browser fallback:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": "browser",
|
||||
"meetingUri": "https://meet.google.com/abc-defg-hij",
|
||||
"browser": {
|
||||
"nodeId": "ba0f4e4bc...",
|
||||
"targetId": "tab-1"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Example JSON output from API create:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": "api",
|
||||
"meetingUri": "https://meet.google.com/abc-defg-hij",
|
||||
"space": {
|
||||
"name": "spaces/abc-defg-hij",
|
||||
"meetingCode": "abc-defg-hij",
|
||||
"meetingUri": "https://meet.google.com/abc-defg-hij"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Creating a Meet only creates or discovers the meeting URL. The Chrome or
|
||||
Chrome-node transport still needs a signed-in Google Chrome profile to join
|
||||
through the browser. If the profile is signed out, OpenClaw reports
|
||||
`manualActionRequired: true` or a browser fallback error and asks the operator
|
||||
to finish Google login before retrying.
|
||||
|
||||
Set `preview.enrollmentAcknowledged: true` only after confirming your Cloud
|
||||
project, OAuth principal, and meeting participants are enrolled in the Google
|
||||
@@ -740,15 +789,20 @@ Common manual actions:
|
||||
|
||||
### Meeting creation fails
|
||||
|
||||
`googlemeet create` uses the Google Meet API `spaces.create` endpoint. Confirm:
|
||||
`googlemeet create` first uses the Google Meet API `spaces.create` endpoint
|
||||
when OAuth credentials are configured. Without OAuth credentials it falls back
|
||||
to the pinned Chrome node browser. Confirm:
|
||||
|
||||
- `oauth.clientId` and `oauth.refreshToken` are configured, or matching
|
||||
`OPENCLAW_GOOGLE_MEET_*` environment variables are present.
|
||||
- The refresh token was minted after create support was added. Older tokens may
|
||||
be missing the `meetings.space.created` scope; rerun
|
||||
- For API creation: `oauth.clientId` and `oauth.refreshToken` are configured,
|
||||
or matching `OPENCLAW_GOOGLE_MEET_*` environment variables are present.
|
||||
- For API creation: the refresh token was minted after create support was
|
||||
added. Older tokens may be missing the `meetings.space.created` scope; rerun
|
||||
`openclaw googlemeet auth login --json` and update plugin config.
|
||||
- The Google Cloud project and OAuth principal are allowed to use the required
|
||||
Google Meet API scopes.
|
||||
- For browser fallback: `defaultTransport: "chrome-node"` and
|
||||
`chromeNode.node` point at a connected node with `browser.proxy` and
|
||||
`googlemeet.chrome`.
|
||||
- For browser fallback: the OpenClaw Chrome profile on that node is signed in
|
||||
to Google and can open `https://meet.google.com/new`.
|
||||
|
||||
### Agent joins but does not talk
|
||||
|
||||
|
||||
@@ -777,6 +777,72 @@ describe("google-meet plugin", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("creates a Meet through browser fallback when OAuth is not configured", async () => {
|
||||
const { methods, nodesInvoke } = setup(
|
||||
{
|
||||
defaultTransport: "chrome-node",
|
||||
chromeNode: { node: "parallels-macos" },
|
||||
},
|
||||
{
|
||||
nodesInvokeHandler: async (params) => {
|
||||
const proxy = params.params as { path?: string; body?: { url?: string } };
|
||||
if (proxy.path === "/tabs/open") {
|
||||
return {
|
||||
payload: {
|
||||
result: {
|
||||
targetId: "tab-1",
|
||||
title: "Meet",
|
||||
url: proxy.body?.url,
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
if (proxy.path === "/act") {
|
||||
return {
|
||||
payload: {
|
||||
result: {
|
||||
ok: true,
|
||||
targetId: "tab-1",
|
||||
result: {
|
||||
meetingUri: "https://meet.google.com/browser-made-url",
|
||||
browserUrl: "https://meet.google.com/browser-made-url",
|
||||
browserTitle: "Meet",
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
return { payload: { result: { ok: true } } };
|
||||
},
|
||||
},
|
||||
);
|
||||
const handler = methods.get("googlemeet.create") as
|
||||
| ((ctx: {
|
||||
params: Record<string, unknown>;
|
||||
respond: ReturnType<typeof vi.fn>;
|
||||
}) => Promise<void>)
|
||||
| undefined;
|
||||
const respond = vi.fn();
|
||||
|
||||
await handler?.({ params: {}, respond });
|
||||
|
||||
expect(respond.mock.calls[0]?.[0]).toBe(true);
|
||||
expect(respond.mock.calls[0]?.[1]).toMatchObject({
|
||||
source: "browser",
|
||||
meetingUri: "https://meet.google.com/browser-made-url",
|
||||
browser: { nodeId: "node-1", targetId: "tab-1" },
|
||||
});
|
||||
expect(nodesInvoke).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
command: "browser.proxy",
|
||||
params: expect.objectContaining({
|
||||
path: "/tabs/open",
|
||||
body: { url: "https://meet.google.com/new" },
|
||||
}),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("launches Chrome after the BlackHole check", async () => {
|
||||
const originalPlatform = process.platform;
|
||||
Object.defineProperty(process, "platform", { value: "darwin" });
|
||||
|
||||
@@ -18,6 +18,7 @@ import {
|
||||
import { handleGoogleMeetNodeHostCommand } from "./src/node-host.js";
|
||||
import { resolveGoogleMeetAccessToken } from "./src/oauth.js";
|
||||
import { GoogleMeetRuntime } from "./src/runtime.js";
|
||||
import { createMeetWithBrowserProxyOnNode } from "./src/transports/chrome.js";
|
||||
|
||||
const googleMeetConfigSchema = {
|
||||
parse(value: unknown) {
|
||||
@@ -227,7 +228,45 @@ async function createSpaceFromParams(config: GoogleMeetConfig, raw: Record<strin
|
||||
expiresAt: typeof raw.expiresAt === "number" ? raw.expiresAt : config.oauth.expiresAt,
|
||||
});
|
||||
const result = await createGoogleMeetSpace({ accessToken: token.accessToken });
|
||||
return { token, ...result };
|
||||
return { source: "api" as const, token, ...result };
|
||||
}
|
||||
|
||||
function hasGoogleMeetOAuth(config: GoogleMeetConfig, raw: Record<string, unknown>): boolean {
|
||||
return Boolean(
|
||||
normalizeOptionalString(raw.accessToken) ??
|
||||
normalizeOptionalString(raw.refreshToken) ??
|
||||
config.oauth.accessToken ??
|
||||
config.oauth.refreshToken,
|
||||
);
|
||||
}
|
||||
|
||||
async function createMeetFromParams(params: {
|
||||
config: GoogleMeetConfig;
|
||||
runtime: OpenClawPluginApi["runtime"];
|
||||
raw: Record<string, unknown>;
|
||||
}) {
|
||||
if (hasGoogleMeetOAuth(params.config, params.raw)) {
|
||||
const { token: _token, ...result } = await createSpaceFromParams(params.config, params.raw);
|
||||
return result;
|
||||
}
|
||||
const browser = await createMeetWithBrowserProxyOnNode({
|
||||
runtime: params.runtime,
|
||||
config: params.config,
|
||||
});
|
||||
return {
|
||||
source: browser.source,
|
||||
meetingUri: browser.meetingUri,
|
||||
space: {
|
||||
name: `browser/${browser.meetingUri.split("/").pop()}`,
|
||||
meetingUri: browser.meetingUri,
|
||||
},
|
||||
browser: {
|
||||
nodeId: browser.nodeId,
|
||||
targetId: browser.targetId,
|
||||
browserUrl: browser.browserUrl,
|
||||
browserTitle: browser.browserTitle,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export default definePluginEntry({
|
||||
@@ -284,8 +323,7 @@ export default definePluginEntry({
|
||||
async ({ params, respond }: GatewayRequestHandlerOptions) => {
|
||||
try {
|
||||
const raw = asParamRecord(params);
|
||||
const { token: _token, ...result } = await createSpaceFromParams(config, raw);
|
||||
respond(true, result);
|
||||
respond(true, await createMeetFromParams({ config, runtime: api.runtime, raw }));
|
||||
} catch (err) {
|
||||
sendError(respond, err);
|
||||
}
|
||||
@@ -395,8 +433,7 @@ export default definePluginEntry({
|
||||
);
|
||||
}
|
||||
case "create": {
|
||||
const { token: _token, ...result } = await createSpaceFromParams(config, raw);
|
||||
return json(result);
|
||||
return json(await createMeetFromParams({ config, runtime: api.runtime, raw }));
|
||||
}
|
||||
case "test_speech": {
|
||||
const rt = await ensureRuntime();
|
||||
|
||||
@@ -145,6 +145,15 @@ function resolveCreateTokenOptions(
|
||||
};
|
||||
}
|
||||
|
||||
function hasCreateOAuth(config: GoogleMeetConfig, options: CreateOptions): boolean {
|
||||
return Boolean(
|
||||
options.accessToken?.trim() ||
|
||||
options.refreshToken?.trim() ||
|
||||
config.oauth.accessToken ||
|
||||
config.oauth.refreshToken,
|
||||
);
|
||||
}
|
||||
|
||||
export function registerGoogleMeetCli(params: {
|
||||
program: Command;
|
||||
config: GoogleMeetConfig;
|
||||
@@ -226,6 +235,28 @@ export function registerGoogleMeetCli(params: {
|
||||
.option("--expires-at <ms>", "Cached access token expiry as unix epoch milliseconds")
|
||||
.option("--json", "Print JSON output", false)
|
||||
.action(async (options: CreateOptions) => {
|
||||
if (!hasCreateOAuth(params.config, options)) {
|
||||
const rt = await params.ensureRuntime();
|
||||
const result = await rt.createViaBrowser();
|
||||
const payload = {
|
||||
source: result.source,
|
||||
meetingUri: result.meetingUri,
|
||||
browser: {
|
||||
nodeId: result.nodeId,
|
||||
targetId: result.targetId,
|
||||
browserUrl: result.browserUrl,
|
||||
browserTitle: result.browserTitle,
|
||||
},
|
||||
};
|
||||
if (options.json) {
|
||||
writeStdoutJson(payload);
|
||||
return;
|
||||
}
|
||||
writeStdoutLine("meeting uri: %s", result.meetingUri);
|
||||
writeStdoutLine("source: browser");
|
||||
writeStdoutLine("node: %s", result.nodeId);
|
||||
return;
|
||||
}
|
||||
const token = await resolveGoogleMeetAccessToken(
|
||||
resolveCreateTokenOptions(params.config, options),
|
||||
);
|
||||
|
||||
@@ -5,7 +5,11 @@ import type { PluginRuntime, RuntimeLogger } from "openclaw/plugin-sdk/plugin-ru
|
||||
import { normalizeOptionalString } from "openclaw/plugin-sdk/text-runtime";
|
||||
import type { GoogleMeetConfig, GoogleMeetMode, GoogleMeetTransport } from "./config.js";
|
||||
import { getGoogleMeetSetupStatus } from "./setup.js";
|
||||
import { launchChromeMeet, launchChromeMeetOnNode } from "./transports/chrome.js";
|
||||
import {
|
||||
createMeetWithBrowserProxyOnNode,
|
||||
launchChromeMeet,
|
||||
launchChromeMeetOnNode,
|
||||
} from "./transports/chrome.js";
|
||||
import { buildMeetDtmfSequence, normalizeDialInNumber } from "./transports/twilio.js";
|
||||
import type {
|
||||
GoogleMeetChromeHealth,
|
||||
@@ -84,6 +88,13 @@ export class GoogleMeetRuntime {
|
||||
return getGoogleMeetSetupStatus(this.params.config, { fullConfig: this.params.fullConfig });
|
||||
}
|
||||
|
||||
async createViaBrowser() {
|
||||
return createMeetWithBrowserProxyOnNode({
|
||||
runtime: this.params.runtime,
|
||||
config: this.params.config,
|
||||
});
|
||||
}
|
||||
|
||||
async join(request: GoogleMeetJoinRequest): Promise<GoogleMeetJoinResult> {
|
||||
const url = normalizeMeetUrl(request.url);
|
||||
const transport = resolveTransport(request.transport, this.params.config);
|
||||
|
||||
@@ -231,6 +231,15 @@ type BrowserTab = {
|
||||
url?: string;
|
||||
};
|
||||
|
||||
export type GoogleMeetBrowserCreateResult = {
|
||||
meetingUri: string;
|
||||
nodeId: string;
|
||||
targetId?: string;
|
||||
browserUrl?: string;
|
||||
browserTitle?: string;
|
||||
source: "browser";
|
||||
};
|
||||
|
||||
function unwrapNodeInvokePayload(raw: unknown): unknown {
|
||||
const record = raw && typeof raw === "object" ? (raw as Record<string, unknown>) : {};
|
||||
if (typeof record.payloadJSON === "string" && record.payloadJSON.trim()) {
|
||||
@@ -283,6 +292,110 @@ function readBrowserTab(result: unknown): BrowserTab | undefined {
|
||||
return result && typeof result === "object" ? (result as BrowserTab) : undefined;
|
||||
}
|
||||
|
||||
function readBrowserCreateResult(result: unknown): {
|
||||
meetingUri?: string;
|
||||
browserUrl?: string;
|
||||
browserTitle?: string;
|
||||
manualAction?: string;
|
||||
} {
|
||||
const record = result && typeof result === "object" ? (result as Record<string, unknown>) : {};
|
||||
const nested =
|
||||
record.result && typeof record.result === "object"
|
||||
? (record.result as Record<string, unknown>)
|
||||
: record;
|
||||
return {
|
||||
meetingUri: typeof nested.meetingUri === "string" ? nested.meetingUri : undefined,
|
||||
browserUrl: typeof nested.browserUrl === "string" ? nested.browserUrl : undefined,
|
||||
browserTitle: typeof nested.browserTitle === "string" ? nested.browserTitle : undefined,
|
||||
manualAction: typeof nested.manualAction === "string" ? nested.manualAction : undefined,
|
||||
};
|
||||
}
|
||||
|
||||
const CREATE_MEET_FROM_BROWSER_SCRIPT = `async () => {
|
||||
const meetUrlPattern = /^https:\\/\\/meet\\.google\\.com\\/[a-z]{3}-[a-z]{4}-[a-z]{3}(?:$|[/?#])/i;
|
||||
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
const current = () => location.href;
|
||||
if (!current().startsWith("https://meet.google.com/")) {
|
||||
return {
|
||||
manualAction: "Sign in to Google in the OpenClaw browser profile, then retry meeting creation.",
|
||||
browserUrl: current(),
|
||||
browserTitle: document.title,
|
||||
};
|
||||
}
|
||||
for (let i = 0; i < 80; i += 1) {
|
||||
const href = current();
|
||||
if (meetUrlPattern.test(href)) {
|
||||
return { meetingUri: href, browserUrl: href, browserTitle: document.title };
|
||||
}
|
||||
const text = document.body?.innerText ?? "";
|
||||
if (/sign in|use your google account|couldn't create|unable to create/i.test(text)) {
|
||||
return {
|
||||
manualAction: "Sign in to Google in the OpenClaw browser profile or resolve the Meet page prompt, then retry meeting creation.",
|
||||
browserUrl: href,
|
||||
browserTitle: document.title,
|
||||
};
|
||||
}
|
||||
await sleep(500);
|
||||
}
|
||||
return {
|
||||
manualAction: "Google Meet did not return a meeting URL from the browser create flow before timeout.",
|
||||
browserUrl: current(),
|
||||
browserTitle: document.title,
|
||||
};
|
||||
}`;
|
||||
|
||||
export async function createMeetWithBrowserProxyOnNode(params: {
|
||||
runtime: PluginRuntime;
|
||||
config: GoogleMeetConfig;
|
||||
}): Promise<GoogleMeetBrowserCreateResult> {
|
||||
const nodeId = await resolveChromeNode({
|
||||
runtime: params.runtime,
|
||||
requestedNode: params.config.chromeNode.node,
|
||||
});
|
||||
const timeoutMs = Math.max(15_000, params.config.chrome.joinTimeoutMs);
|
||||
const tab = readBrowserTab(
|
||||
await callBrowserProxyOnNode({
|
||||
runtime: params.runtime,
|
||||
nodeId,
|
||||
method: "POST",
|
||||
path: "/tabs/open",
|
||||
body: { url: "https://meet.google.com/new" },
|
||||
timeoutMs,
|
||||
}),
|
||||
);
|
||||
const targetId = tab?.targetId;
|
||||
if (!targetId) {
|
||||
throw new Error("Browser fallback opened Google Meet but did not return a targetId.");
|
||||
}
|
||||
const evaluated = await callBrowserProxyOnNode({
|
||||
runtime: params.runtime,
|
||||
nodeId,
|
||||
method: "POST",
|
||||
path: "/act",
|
||||
body: {
|
||||
kind: "evaluate",
|
||||
targetId,
|
||||
fn: CREATE_MEET_FROM_BROWSER_SCRIPT,
|
||||
},
|
||||
timeoutMs,
|
||||
});
|
||||
const result = readBrowserCreateResult(evaluated);
|
||||
if (result.meetingUri) {
|
||||
return {
|
||||
source: "browser",
|
||||
nodeId,
|
||||
targetId,
|
||||
meetingUri: result.meetingUri,
|
||||
browserUrl: result.browserUrl,
|
||||
browserTitle: result.browserTitle,
|
||||
};
|
||||
}
|
||||
throw new Error(
|
||||
result.manualAction ??
|
||||
"Browser fallback could not create a Google Meet URL. Sign in to the OpenClaw browser profile, then retry.",
|
||||
);
|
||||
}
|
||||
|
||||
function parseMeetBrowserStatus(result: unknown): GoogleMeetChromeHealth | undefined {
|
||||
const record = result && typeof result === "object" ? (result as Record<string, unknown>) : {};
|
||||
const raw = record.result;
|
||||
|
||||
Reference in New Issue
Block a user