From 8a916652e8b856fdaf9d3bbb24ad11bf32a86b2b Mon Sep 17 00:00:00 2001 From: Vincent Koc Date: Mon, 30 Mar 2026 16:26:02 +0900 Subject: [PATCH] docs: add Background Tasks page and clean up cross-references MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New page: docs/automation/tasks.md — comprehensive reference for the task system covering lifecycle, delivery, notifications, audit, CLI commands, storage, maintenance, and how tasks relate to cron/heartbeat/sessions. - Add to Mintlify navigation (docs.json) under Automation group - Clean up engineer's earlier scattered additions in cron-jobs.md, cron-vs-heartbeat.md, and heartbeat.md to be concise and link to the new canonical tasks page - Replace verbose inline explanations with cross-reference links --- docs/automation/cron-jobs.md | 4 +- docs/automation/cron-vs-heartbeat.md | 13 +- docs/automation/tasks.md | 218 +++++++++++++++++++++++++++ docs/docs.json | 1 + docs/gateway/heartbeat.md | 9 +- 5 files changed, 232 insertions(+), 13 deletions(-) create mode 100644 docs/automation/tasks.md diff --git a/docs/automation/cron-jobs.md b/docs/automation/cron-jobs.md index 4b756b2bd32..136f756d1d5 100644 --- a/docs/automation/cron-jobs.md +++ b/docs/automation/cron-jobs.md @@ -160,7 +160,7 @@ They must use `payload.kind = "systemEvent"`. This is the best fit when you want the normal heartbeat prompt + main-session context. See [Heartbeat](/gateway/heartbeat). -Main-session cron jobs do **not** create background task records. +Main-session cron jobs do **not** create [background task](/automation/tasks) records. #### Isolated jobs (dedicated cron sessions) @@ -183,7 +183,7 @@ Key behaviors: Use isolated jobs for noisy, frequent, or "background chores" that shouldn't spam your main chat history. -These detached runs appear in `openclaw tasks` and inherit task audit and maintenance behavior. +These detached runs create [background task](/automation/tasks) records visible in `openclaw tasks` and subject to task audit and maintenance. ### Payload shapes (what runs) diff --git a/docs/automation/cron-vs-heartbeat.md b/docs/automation/cron-vs-heartbeat.md index ff27ee6dbdb..7c11813906f 100644 --- a/docs/automation/cron-vs-heartbeat.md +++ b/docs/automation/cron-vs-heartbeat.md @@ -13,11 +13,11 @@ Both heartbeats and cron jobs let you run tasks on a schedule. This guide helps One important distinction: -- **Heartbeat** is a scheduled **main-session turn**. -- **Cron (main)** is a scheduled **system event into the main session**. -- **Cron (isolated)** is a scheduled **background run**. +- **Heartbeat** is a scheduled **main-session turn** — no task record created. +- **Cron (main)** is a scheduled **system event into the main session** — no task record created. +- **Cron (isolated)** is a scheduled **background run** — tracked in `openclaw tasks`. -Only detached background runs show up in `openclaw tasks`. Normal heartbeat runs and main-session cron reminders do not. +Only detached background runs (isolated cron, ACP, subagents) appear in the [task ledger](/automation/tasks). Heartbeat turns and main-session cron reminders stay in session history. ## Quick Decision Guide @@ -48,7 +48,7 @@ Heartbeats run in the **main session** at a regular interval (default: 30 min). - **Context-aware**: The agent knows what you've been working on and can prioritize accordingly. - **Smart suppression**: If nothing needs attention, the agent replies `HEARTBEAT_OK` and no message is delivered. - **Natural timing**: Drifts slightly based on queue load, which is fine for most monitoring. -- **No background task ledger**: heartbeat turns stay in main-session history instead of creating a separate task run. +- **No task record**: heartbeat turns stay in main-session history (see [Background Tasks](/automation/tasks)). ### Heartbeat example: HEARTBEAT.md checklist @@ -107,7 +107,7 @@ per-job offset in a 0-5 minute window. - **Immediate delivery**: Announce mode posts directly without waiting for heartbeat. - **No agent context needed**: Runs even if main session is idle or compacted. - **One-shot support**: `--at` for precise future timestamps. -- **Task visibility for detached runs**: isolated jobs show up in `openclaw tasks`, `openclaw tasks audit`, and maintenance surfaces. +- **Task tracking**: isolated jobs create [background task](/automation/tasks) records visible in `openclaw tasks` and `openclaw tasks audit`. ### Cron example: Daily morning briefing @@ -294,4 +294,5 @@ openclaw cron add \ - [Heartbeat](/gateway/heartbeat) - full heartbeat configuration - [Cron jobs](/automation/cron-jobs) - full cron CLI and API reference +- [Background Tasks](/automation/tasks) - task ledger, audit, and lifecycle - [System](/cli/system) - system events + heartbeat controls diff --git a/docs/automation/tasks.md b/docs/automation/tasks.md new file mode 100644 index 00000000000..ad8e6a0ef64 --- /dev/null +++ b/docs/automation/tasks.md @@ -0,0 +1,218 @@ +--- +summary: "Background task tracking for ACP runs, subagents, isolated cron jobs, and CLI operations" +read_when: + - Inspecting background work in progress or recently completed + - Debugging delivery failures for detached agent runs + - Understanding how background runs relate to sessions, cron, and heartbeat +title: "Background Tasks" +--- + +# Background Tasks + +Background tasks track work that runs **outside your main conversation session**: ACP runs, subagent spawns, isolated cron job executions, and CLI-initiated operations. + + +Not every agent run creates a task. Heartbeat turns and main-session cron reminders stay in main-session history. Only **detached** work (isolated cron, ACP, subagents, CLI operations) appears in the task ledger. + + +## Quick start + +```bash +# List all tasks (newest first) +openclaw tasks list + +# Show details for a specific task +openclaw tasks show + +# Cancel a running task +openclaw tasks cancel + +# Change notification policy +openclaw tasks notify state_changes + +# Run a health audit +openclaw tasks audit +``` + +## What creates a task + +| Source | Runtime | When a task is created | +|---|---|---| +| ACP background runs | `acp` | Spawning a child ACP session | +| Subagent orchestration | `subagent` | Spawning a subagent via `sessions_spawn` | +| Isolated cron jobs | `cron` | Each execution of a `sessionTarget: "isolated"` or custom-session cron job | +| CLI operations | `cli` | Background CLI commands that run through the gateway | + +**Not tracked as tasks:** + +- Heartbeat turns (main-session; see [Heartbeat](/gateway/heartbeat)) +- Main-session cron reminders (`sessionTarget: "main"`; see [Cron Jobs](/automation/cron-jobs)) +- Normal interactive chat turns + +## Task lifecycle + +``` +queued ──→ running ──→ succeeded + ├──→ failed + ├──→ timed_out + └──→ cancelled + + (any active state) ──→ lost +``` + +| Status | Meaning | +|---|---| +| `queued` | Created, waiting to start | +| `running` | Actively executing | +| `succeeded` | Completed successfully | +| `failed` | Completed with an error | +| `timed_out` | Exceeded the configured timeout | +| `cancelled` | Cancelled by the operator (`openclaw tasks cancel`) | +| `lost` | Backing session disappeared (detected after a 5-minute grace period) | + +Tasks automatically transition from `running` to their terminal state when the associated agent run ends. + +## Delivery and notifications + +When a task finishes, OpenClaw can notify you through two mechanisms: + +### Direct delivery + +If the task has a `requesterOrigin` (channel target), the completion message is delivered directly to that channel (Telegram, Discord, Slack, etc.). + +### Session-queued delivery + +If direct delivery fails or no origin is set, the update is queued as a system event in the requester session and delivered on the next heartbeat. + +### Notification policies + +Control how much you hear about a task: + +| Policy | Behavior | +|---|---| +| `done_only` (default) | Notify only when the task reaches a terminal state | +| `state_changes` | Notify on every state transition and progress update | +| `silent` | No notifications at all | + +Change the policy for a running task: + +```bash +openclaw tasks notify state_changes +``` + +## Inspecting tasks + +### List all tasks + +```bash +openclaw tasks list +``` + +Output columns: Task ID, Kind (runtime), Status, Delivery status, Run ID, Child Session, Summary. + +Filter by runtime or status: + +```bash +openclaw tasks list --runtime acp +openclaw tasks list --status running +openclaw tasks list --json +``` + +### Show task details + +```bash +openclaw tasks show +``` + +The lookup token can be a task ID, run ID, or session key. Shows full task record including timing, delivery state, and terminal summary. + +### Cancel a task + +```bash +openclaw tasks cancel +``` + +For ACP and subagent tasks, this kills the child session. The task status transitions to `cancelled`. + +## Task audit + +The audit surfaces operational issues with background tasks: + +```bash +openclaw tasks audit +openclaw tasks audit --json +``` + +| Finding | Severity | Condition | +|---|---|---| +| `stale_queued` | warn | Queued for more than 10 minutes | +| `stale_running` | error | Running for more than 30 minutes | +| `lost` | error | Status is `lost` (backing session gone) | +| `delivery_failed` | warn | Delivery failed and notify policy is not `silent` | +| `missing_cleanup` | warn | Terminal but no cleanup timestamp set | +| `inconsistent_timestamps` | warn | Timeline violations (ended before started, etc.) | + +Task audit findings also appear in `openclaw status` output when issues are detected. + +## Task pressure (status integration) + +The task system reports an "at a glance" summary in `openclaw status`: + +``` +Tasks: 3 queued · 2 running · 1 issues +``` + +This includes: + +- **active**: count of `queued` + `running` tasks +- **failures**: count of `failed` + `timed_out` + `lost` tasks +- **byRuntime**: breakdown by `acp`, `subagent`, `cron`, `cli` + +## Storage and maintenance + +### Where tasks are stored + +Task records persist in SQLite at `$OPENCLAW_STATE_DIR/tasks/runs.sqlite`. The registry loads into memory at gateway start and syncs writes to SQLite for durability across restarts. + +### Automatic cleanup + +A maintenance sweeper runs every 60 seconds: + +1. **Reconciliation**: checks if active tasks' backing sessions still exist. If a session is gone for more than 5 minutes, the task is marked `lost`. +2. **Cleanup stamping**: terminal tasks get a `cleanupAfter` timestamp set to 7 days after completion. +3. **Pruning**: records past their `cleanupAfter` are deleted. + +**Retention**: terminal task records are kept for **7 days** for historical inspection, then automatically pruned. + +## How tasks relate to other systems + +### Tasks and cron + +- A cron job **definition** lives in `~/.openclaw/cron/jobs.json`. +- Each **execution** of an isolated cron job creates a task record. +- Main-session cron jobs (`sessionTarget: "main"`) do **not** create task records. +- See [Cron Jobs](/automation/cron-jobs) for scheduling and [Cron vs Heartbeat](/automation/cron-vs-heartbeat) for choosing the right mechanism. + +### Tasks and heartbeat + +- Heartbeat runs are main-session turns — they do **not** create task records. +- When a detached task completes, it can enqueue a system event and trigger an immediate heartbeat wake so you see the result quickly. +- See [Heartbeat](/gateway/heartbeat) for configuration. + +### Tasks and sessions + +- A task may have an associated `childSessionKey` (the session where work happens). +- The `requesterSessionKey` identifies who initiated the task (the parent session). +- Sessions are conversation context; tasks are activity tracking records. + +### Tasks and agent runs + +- A task's `runId` links to the agent run executing the work. +- Agent lifecycle events (start, end, error) automatically update task status. + +## Related + +- [Cron Jobs](/automation/cron-jobs) — scheduling background work +- [Cron vs Heartbeat](/automation/cron-vs-heartbeat) — choosing the right mechanism +- [Heartbeat](/gateway/heartbeat) — periodic main-session turns +- [CLI: Tasks](/cli/index#tasks) — CLI command reference diff --git a/docs/docs.json b/docs/docs.json index aa4f160cc3e..49dbca45ff7 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -1118,6 +1118,7 @@ "automation/standing-orders", "automation/cron-jobs", "automation/cron-vs-heartbeat", + "automation/tasks", "automation/troubleshooting", "automation/webhook", "automation/gmail-pubsub", diff --git a/docs/gateway/heartbeat.md b/docs/gateway/heartbeat.md index 40827db0a2a..22fedfc1b73 100644 --- a/docs/gateway/heartbeat.md +++ b/docs/gateway/heartbeat.md @@ -13,9 +13,8 @@ title: "Heartbeat" Heartbeat runs **periodic agent turns** in the main session so the model can surface anything that needs attention without spamming you. -Heartbeat is not the background task executor. It is a scheduled main-session turn. -Background task records are for detached work such as ACP runs, subagents, background CLI work, -and isolated cron jobs. +Heartbeat is a scheduled main-session turn — it does **not** create [background task](/automation/tasks) records. +Task records are for detached work (ACP runs, subagents, isolated cron jobs). Troubleshooting: [/automation/troubleshooting](/automation/troubleshooting) @@ -69,7 +68,7 @@ The default prompt is intentionally broad: occasional lightweight “anything you need?” message, but avoids night-time spam by using your configured local timezone (see [/concepts/timezone](/concepts/timezone)). -Heartbeat can react to detached tasks, but a heartbeat run itself does not create a task record. +Heartbeat can react to completed [background tasks](/automation/tasks), but a heartbeat run itself does not create a task record. If you want a heartbeat to do something very specific (e.g. “check Gmail PubSub stats” or “verify gateway health”), set `agents.defaults.heartbeat.prompt` (or @@ -259,7 +258,7 @@ Use `accountId` to target a specific account on multi-account channels like Tele outbound message is sent. - Heartbeat-only replies do **not** keep the session alive; the last `updatedAt` is restored so idle expiry behaves normally. -- Detached work can enqueue a system event and wake heartbeat when the main session should notice something quickly. That wake still does not make the heartbeat run a background task. +- Detached [background tasks](/automation/tasks) can enqueue a system event and wake heartbeat when the main session should notice something quickly. That wake does not make the heartbeat run a background task. ## Visibility controls