35 KiB
summary, title, read_when
| summary | title | read_when | |||
|---|---|---|---|---|---|
| Export OpenClaw diagnostics to OpenTelemetry collectors or stdout JSONL via the diagnostics-otel plugin | OpenTelemetry export |
|
OpenClaw exports diagnostics through the official diagnostics-otel plugin
using OTLP/HTTP (protobuf). Logs can also be written as stdout JSONL for
container and sandbox log pipelines. Any collector or backend that accepts
OTLP/HTTP works without code changes. For local file logs, see
Logging.
- Diagnostics events are structured, in-process records emitted by the Gateway and bundled plugins for model runs, message flow, sessions, queues, and exec.
diagnostics-otelsubscribes to those events and exports them as OpenTelemetry metrics, traces, and logs over OTLP/HTTP, and can mirror log records to stdout JSONL.- Provider calls receive a W3C
traceparentheader from OpenClaw's trusted model-call span context when the provider transport accepts custom headers. Plugin-emitted trace context is not propagated. - Exporters attach only when both the diagnostics surface and the plugin are enabled, so in-process cost stays near zero by default.
Quick start
openclaw plugins install clawhub:@openclaw/diagnostics-otel
{
plugins: {
allow: ["diagnostics-otel"],
entries: {
"diagnostics-otel": { enabled: true },
},
},
diagnostics: {
enabled: true,
otel: {
enabled: true,
endpoint: "http://otel-collector:4318",
protocol: "http/protobuf",
serviceName: "openclaw-gateway",
traces: true,
metrics: true,
logs: true,
sampleRate: 0.2,
flushIntervalMs: 60000,
},
},
}
Or enable the plugin from the CLI: openclaw plugins enable diagnostics-otel.
Signals exported
| Signal | What goes in it |
|---|---|
| Metrics | Counters/histograms for token usage, cost, run duration, failover, skill usage, message flow, Talk events, queue lanes, session state/recovery, tool execution, exec, memory, liveness, and exporter health. |
| Traces | Spans for model usage, model calls, harness lifecycle, skill usage, tool execution, exec, webhook/message processing, context assembly, and tool loops. |
| Logs | Structured logging.file records exported over OTLP or stdout JSONL when diagnostics.otel.logs is enabled; log bodies are withheld unless content capture is explicitly enabled. |
Toggle traces, metrics, and logs independently. Traces and metrics
default to on when diagnostics.otel.enabled is true; logs default to off
and export only when diagnostics.otel.logs is explicitly true. Log export
defaults to OTLP; set diagnostics.otel.logsExporter to stdout for JSONL on
stdout, or both for both.
Configuration reference
{
diagnostics: {
enabled: true,
otel: {
enabled: true,
endpoint: "http://otel-collector:4318",
tracesEndpoint: "http://otel-collector:4318/v1/traces",
metricsEndpoint: "http://otel-collector:4318/v1/metrics",
logsEndpoint: "http://otel-collector:4318/v1/logs",
protocol: "http/protobuf", // grpc disables OTLP export
serviceName: "openclaw-gateway", // unset falls back to OTEL_SERVICE_NAME, then "openclaw"
headers: { "x-collector-token": "..." },
traces: true,
metrics: true,
logs: true,
logsExporter: "otlp", // otlp | stdout | both
sampleRate: 0.2, // root-span sampler, 0.0..1.0
flushIntervalMs: 60000, // metric export interval (min 1000ms)
captureContent: {
enabled: false,
inputMessages: false,
outputMessages: false,
toolInputs: false,
toolOutputs: false,
systemPrompt: false,
toolDefinitions: false,
},
},
},
}
Environment variables
| Variable | Purpose |
|---|---|
OTEL_EXPORTER_OTLP_ENDPOINT |
Fallback for diagnostics.otel.endpoint when the config key is unset. |
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT / OTEL_EXPORTER_OTLP_METRICS_ENDPOINT / OTEL_EXPORTER_OTLP_LOGS_ENDPOINT |
Signal-specific endpoint fallbacks used when the matching diagnostics.otel.*Endpoint config key is unset. Signal-specific config wins over signal-specific env, which wins over the shared endpoint. |
OTEL_SERVICE_NAME |
Fallback for diagnostics.otel.serviceName when the config key is unset. Default service name is openclaw. |
OTEL_EXPORTER_OTLP_PROTOCOL |
Fallback for the wire protocol when diagnostics.otel.protocol is unset. Only http/protobuf enables export. |
OTEL_SEMCONV_STABILITY_OPT_IN |
Set to gen_ai_latest_experimental to emit the latest GenAI inference span shape: {gen_ai.operation.name} {gen_ai.request.model} span names, CLIENT span kind, and gen_ai.provider.name instead of the legacy gen_ai.system. GenAI metrics always use bounded, low-cardinality attributes regardless. |
OPENCLAW_OTEL_PRELOADED |
Set to 1 when another preload or host process already registered the global OpenTelemetry SDK. The plugin then skips its own NodeSDK lifecycle but still wires diagnostic listeners and honors traces/metrics/logs. |
Privacy and content capture
Raw model/tool content is not exported by default. Spans carry bounded
identifiers (channel, provider, model, error category, hash-only request ids,
tool source, tool owner, skill name/source) and never include prompt text,
response text, tool inputs, tool outputs, skill file paths, or session keys.
Values that look like scoped agent session keys (for example starting with
agent:) are replaced with unknown on low-cardinality attributes. OTLP log
records keep severity, logger, code location, trusted trace context, and
sanitized attributes by default; the raw log message body is exported only
when diagnostics.otel.captureContent is boolean true. Granular
captureContent.* subkeys never enable log bodies. Talk metrics export only
bounded event metadata (mode, transport, provider, event type) - no
transcripts, audio payloads, session ids, turn ids, call ids, room ids, or
handoff tokens.
Outbound model requests may include a W3C traceparent header generated only
from OpenClaw-owned diagnostic trace context for the active model call.
Existing caller-supplied traceparent headers are replaced, so plugins or
custom provider options cannot spoof cross-service trace ancestry.
Set diagnostics.otel.captureContent.* to true only when your collector
and retention policy are approved for prompt, response, tool, or
system-prompt text. Each subkey is independent:
inputMessages- user prompt content.outputMessages- model response content.toolInputs- tool argument payloads.toolOutputs- tool result payloads.systemPrompt- assembled system/developer prompt.toolDefinitions- model tool names, descriptions, and schemas.
When any subkey is enabled, model and tool spans get bounded, redacted
openclaw.content.* attributes for that class only.
toolInputs/toolOutputs content is captured for the built-in agent
runtime's tool executions (openclaw.content.tool_input and
gen_ai.tool.call.arguments on completed/error spans;
openclaw.content.tool_output and gen_ai.tool.call.result on completed
spans). The openclaw.content.* names remain the stable OpenClaw attribute
names; the gen_ai.tool.call.* copies mirror them for semconv-native viewers.
External harness tool calls (Codex, Claude CLI) emit
tool.execution.* spans without content payloads. Captured content travels on a
trusted, listener-only channel and is never placed on the public diagnostic event
bus.
Sampling and flushing
- Traces:
diagnostics.otel.sampleRatesets aTraceIdRatioBasedSampleron the root span only (0.0drops all,1.0keeps all). Unset uses the OpenTelemetry SDK default (always-on). - Metrics:
diagnostics.otel.flushIntervalMs(clamped to a minimum of1000); unset uses the SDK's periodic-export default. - Logs: OTLP logs respect
logging.level(file log level) and use the diagnostic log-record redaction path, not console formatting. High-volume installs should prefer OTLP collector sampling/filtering over local sampling. Setdiagnostics.otel.logsExporter: "stdout"when your platform already ships stdout/stderr to a log processor and you have no OTLP logs collector. Stdout records are one JSON object per line withts,signal,service.name, severity, body, redacted attributes, and trusted trace fields when available. - File-log correlation: JSONL file logs include top-level
traceId,spanId,parentSpanId, andtraceFlagswhen the log call carries a valid diagnostic trace context, letting log processors join local log lines with exported spans. - Request correlation: Gateway HTTP requests and WebSocket frames create
an internal request trace scope. Logs and diagnostic events inside that
scope inherit the request trace by default, while agent run and model-call
spans are created as children so provider
traceparentheaders stay on the same trace. - Model-call correlation:
openclaw.model.callspans include safe prompt component sizes by default and per-call token attributes when the provider result exposes usage.openclaw.model.usageremains the run-level accounting span for aggregate cost, context, and channel dashboards, and stays on the same diagnostic trace when the emitting runtime has trusted trace context.
Model-call observation units
Every openclaw.model.call span identifies what its lifecycle measures through
openclaw.model_call.observation_unit:
request- one observable model/provider request. Native embedded model calls use this unit, and exporters treat a missing value asrequestfor compatibility with older or external emitters.turn- one opaque agent CLI turn that may contain hidden model requests, retries, tool work, or background work. Claude Code CLI and Codex app-server calls use this unit.
Both units remain model-call spans so trace backends can render model input,
output, usage, and hierarchy. Request spans use the API-derived GenAI operation
(chat, generate_content, or text_completion), while turn spans use
gen_ai.operation.name = invoke_agent. Both contribute to
gen_ai.client.operation.duration, where the operation name keeps direct
request latency separate from full-turn latency. OpenClaw's OTEL model-call
metrics also include openclaw.model_call.observation_unit; the Prometheus
model-call metrics expose the equivalent observation_unit label.
Claude Code CLI model-call fidelity
Claude Code CLI turns emit one synthetic, turn-level openclaw.model.call
span. These are not Anthropic HTTP request spans. They use openclaw.api = claude-code, openclaw.model_call.observation_unit = turn, and identify
the operation as gen_ai.operation.name = invoke_agent. They identify
OpenClaw's CLI boundary through
openclaw.transport:
stdio- one-shot local Claude Code process.stdio-live- one turn on a managed persistent Claude stdio session.paired-node-cli- one-shot Claude Code execution delegated to a paired node.
Claude CLI diagnostics are instantiated only while the process diagnostic dispatcher is enabled and an internal or trusted event listener is attached. With no observability plugin or other listener active, Claude CLI turns skip the synthetic trace hierarchy, content buffers, and diagnostic stream-byte accounting. When content capture is enabled, prompt and system-prompt fields are capped at 128 KiB each; assistant output is capped at 128 KiB across at most 200 envelopes, with 16 KiB and one item reserved for a final visible fallback response. A marker records truncation when the limit is reached.
OpenClaw gives Claude CLI turns the same ownership hierarchy used by other
agent runtimes: openclaw.harness.run (openclaw.harness.id = claude-cli)
contains openclaw.run, which contains the Claude openclaw.model.call
span. The harness and run spans are synthetic OpenClaw turn boundaries, not
Claude Code internal phases. One-shot and managed stdio turns use the same
hierarchy; a real fresh-session retry creates another model-call child inside
the same OpenClaw run.
The span starts when OpenClaw admits the prepared CLI turn and ends only after that turn succeeds or fails. For managed sessions, an interim success result does not end the span while Claude reports result-holding background agents or workflows; the final post-drain result does. Abort, timeout, process failure, output/parse failure, and other turn failures end the same span with an error.
Claude Code reports per-assistant-message usage and may also report cumulative usage on its terminal result. OpenClaw reply accounting continues to use the last assistant message so existing cost semantics do not change; the turn-level model-call span uses terminal cumulative usage when available, including cache-read and cache-creation tokens.
For these CLI spans, byte and timing fields describe the observable OpenClaw CLI boundary:
openclaw.model_call.request_bytesis the UTF-8 size of the prompt value sent over one-shot stdin/argv, or the managed stdio JSONL user envelope. It is not the size of Claude Code's hidden model request.openclaw.model_call.response_bytesis the UTF-8 size of Claude CLI stdout observed during the turn. It is not Anthropic HTTP response size.openclaw.model_call.time_to_first_byte_msis time to the first observable Claude CLI stdout or stderr output. It is not network TTFB.
With the matching granular captureContent fields enabled, the span exports
the effective prompt OpenClaw sends to Claude Code, OpenClaw's appended system
prompt, and visible assistant text/reasoning/tool-call identity through
gen_ai.input.messages, gen_ai.output.messages, and
gen_ai.system_instructions. Tool arguments, opaque thinking signatures, and
tool results are omitted from the Claude assistant envelope. OpenClaw does not
claim access to Claude Code's private system prompt, hidden resumed or
compacted request payload, native internal tool schemas, raw Anthropic HTTP
request, internal retries, upstream request id, or true network TTFB. Because
Claude Code does not expose its effective native tool definitions accurately,
these spans do not populate gen_ai.tool.definitions.
External Claude harness tool spans remain metadata-only even when tool content capture is enabled. As with every model span, captured Claude CLI content uses the trusted listener-only path and the exporter's existing redaction and size bounds; content remains off by default.
Exported metrics
Model usage
openclaw.tokens(counter, attrs:openclaw.token,openclaw.channel,openclaw.provider,openclaw.model,openclaw.agent)openclaw.cost.usd(counter, attrs:openclaw.channel,openclaw.provider,openclaw.model)openclaw.run.duration_ms(histogram, attrs:openclaw.channel,openclaw.provider,openclaw.model)openclaw.context.tokens(histogram, attrs:openclaw.context,openclaw.channel,openclaw.provider,openclaw.model)gen_ai.client.token.usage(histogram, GenAI semantic-conventions metric, attrs:gen_ai.token.type=input/output,gen_ai.provider.name,gen_ai.operation.name,gen_ai.request.model)gen_ai.client.operation.duration(histogram, seconds, GenAI semantic-conventions metric for model requests and synthetic agent turns; attrs:gen_ai.provider.name,gen_ai.operation.name,gen_ai.request.model, optionalerror.type; turn observations usegen_ai.operation.name = invoke_agent)openclaw.model_call.duration_ms(histogram, attrs:openclaw.provider,openclaw.model,openclaw.api,openclaw.transport,openclaw.model_call.observation_unit, plusopenclaw.errorCategoryandopenclaw.failureKindon classified errors)openclaw.model_call.request_bytes(histogram, UTF-8 byte size of the final model request payload; for Claude Code CLI, the observable prompt input/envelope described above; no raw payload content)openclaw.model_call.response_bytes(histogram, UTF-8 byte size of streamed response chunk payloads; high-frequency text, thinking, and tool-call deltas count only incrementaldeltabytes; for Claude Code CLI, observed stdout bytes; no raw response content)openclaw.model_call.time_to_first_byte_ms(histogram, elapsed time before the first streamed response event; for Claude Code CLI, first observable CLI output rather than network TTFB)openclaw.model.failover(counter, attrs:openclaw.provider,openclaw.model,openclaw.failover.to_provider,openclaw.failover.to_model,openclaw.failover.reason,openclaw.failover.suspended,openclaw.lane)openclaw.skill.used(counter, attrs:openclaw.skill.name,openclaw.skill.source,openclaw.skill.activation, optionalopenclaw.agent, optionalopenclaw.toolName)
Message flow
openclaw.webhook.received(counter, attrs:openclaw.channel,openclaw.webhook)openclaw.webhook.error(counter, attrs:openclaw.channel,openclaw.webhook)openclaw.webhook.duration_ms(histogram, attrs:openclaw.channel,openclaw.webhook)openclaw.message.queued(counter, attrs:openclaw.channel,openclaw.source)openclaw.message.received(counter, attrs:openclaw.channel,openclaw.source)openclaw.message.dispatch.started(counter, attrs:openclaw.channel,openclaw.source)openclaw.message.dispatch.completed(counter, attrs:openclaw.channel,openclaw.outcome,openclaw.reason,openclaw.source)openclaw.message.dispatch.duration_ms(histogram, attrs:openclaw.channel,openclaw.outcome,openclaw.reason,openclaw.source)openclaw.message.processed(counter, attrs:openclaw.channel,openclaw.outcome)openclaw.message.duration_ms(histogram, attrs:openclaw.channel,openclaw.outcome)openclaw.message.delivery.started(counter, attrs:openclaw.channel,openclaw.delivery.kind)openclaw.message.delivery.duration_ms(histogram, attrs:openclaw.channel,openclaw.delivery.kind,openclaw.outcome,openclaw.errorCategory)
Talk
openclaw.talk.event(counter, attrs:openclaw.talk.event_type,openclaw.talk.mode,openclaw.talk.transport,openclaw.talk.brain,openclaw.talk.provider)openclaw.talk.event.duration_ms(histogram, attrs: same asopenclaw.talk.event; emitted when a Talk event reports duration)openclaw.talk.audio.bytes(histogram, attrs: same asopenclaw.talk.event; emitted for Talk audio frame events that report byte length)
Queues and sessions
openclaw.queue.lane.enqueue(counter, attrs:openclaw.lane)openclaw.queue.lane.dequeue(counter, attrs:openclaw.lane)openclaw.queue.depth(histogram, attrs:openclaw.laneoropenclaw.channel=heartbeat)openclaw.queue.wait_ms(histogram, attrs:openclaw.lane)openclaw.session.state(counter, attrs:openclaw.state,openclaw.reason)openclaw.session.stuck(counter, attrs:openclaw.state; emitted for recoverable stale session bookkeeping)openclaw.session.stuck_age_ms(histogram, attrs:openclaw.state; emitted for recoverable stale session bookkeeping)openclaw.session.turn.created(counter, attrs:openclaw.agent,openclaw.channel,openclaw.trigger)openclaw.session.recovery.requested(counter, attrs:openclaw.state,openclaw.action,openclaw.active_work_kind,openclaw.reason)openclaw.session.recovery.completed(counter, attrs:openclaw.state,openclaw.action,openclaw.status,openclaw.active_work_kind,openclaw.reason)openclaw.session.recovery.age_ms(histogram, attrs: same as the matching recovery counter)openclaw.run.attempt(counter, attrs:openclaw.attempt)
Session liveness telemetry
A processing session does not age toward the built-in liveness threshold while OpenClaw observes reply, tool, status, block, or ACP runtime progress. Typing keepalives do not count as progress, so a silent model or harness can still be detected.
OpenClaw classifies sessions by the work it can still observe:
session.long_running: active embedded work, model calls, or tool calls are still making progress. Owned silent model calls also report as long-running before the built-in abort threshold, so slow or non-streaming model providers do not look like stalled gateway sessions while abort-observable.session.stalled: active work exists, but the active run has not reported recent progress. Owned model calls switch fromsession.long_runningtosession.stalledat or after the built-in abort threshold; ownerless stale model/tool activity is not treated as harmless long-running work. Stalled embedded runs stay observe-only at first, then abort-drain after the abort threshold with no progress so queued turns behind the lane can resume.session.stuck: stale session bookkeeping with no active work, or an idle queued session with stale ownerless model/tool activity. This releases the affected session lane immediately after recovery gates pass.
Recovery emits structured session.recovery.requested and
session.recovery.completed events. Diagnostic session state is marked idle
only after a mutating recovery outcome (aborted or released) and only if
the same processing generation is still current.
Only session.stuck emits the openclaw.session.stuck counter, the
openclaw.session.stuck_age_ms histogram, and the openclaw.session.stuck
span. Repeated session.stuck diagnostics back off while the session remains
unchanged, so dashboards should alert on sustained increases rather than
every heartbeat tick. For the config knob and defaults, see
Configuration reference.
Liveness warnings also emit:
openclaw.liveness.warning(counter, attrs:openclaw.liveness.reason)openclaw.liveness.event_loop_delay_p99_ms(histogram, attrs:openclaw.liveness.reason)openclaw.liveness.event_loop_delay_max_ms(histogram, attrs:openclaw.liveness.reason)openclaw.liveness.event_loop_utilization(histogram, attrs:openclaw.liveness.reason)openclaw.liveness.cpu_core_ratio(histogram, attrs:openclaw.liveness.reason)
Harness lifecycle
openclaw.harness.duration_ms(histogram, attrs:openclaw.harness.id,openclaw.harness.plugin,openclaw.outcome,openclaw.harness.phaseon errors)
Tool execution and loop detection
openclaw.tool.execution.duration_ms(histogram, attrs:gen_ai.tool.name,openclaw.toolName,openclaw.tool.source,openclaw.tool.owner,openclaw.tool.params.kind, plusopenclaw.errorCategoryon errors)openclaw.tool.execution.blocked(counter, attrs:gen_ai.tool.name,openclaw.toolName,openclaw.tool.source,openclaw.tool.owner,openclaw.tool.params.kind,openclaw.deniedReason)openclaw.tool.loop(counter, attrs:openclaw.toolName,openclaw.loop.level,openclaw.loop.action,openclaw.loop.detector,openclaw.loop.count, optionalopenclaw.loop.paired_tool; emitted when a repetitive tool-call loop is detected)
Exec
openclaw.exec.duration_ms(histogram, attrs:openclaw.exec.target,openclaw.exec.mode,openclaw.outcome,openclaw.failureKind)
Diagnostics internals (memory, payloads, exporter health)
openclaw.payload.large(counter, attrs:openclaw.payload.surface,openclaw.payload.action,openclaw.channel,openclaw.plugin,openclaw.reason)openclaw.payload.large_bytes(histogram, attrs: same asopenclaw.payload.large)openclaw.memory.rss_bytes/openclaw.memory.heap_used_bytes/openclaw.memory.heap_total_bytes/openclaw.memory.external_bytes/openclaw.memory.array_buffers_bytes(histograms, no attrs; process memory samples)openclaw.memory.pressure(counter, attrs:openclaw.memory.level,openclaw.memory.reason)openclaw.diagnostic.async_queue.dropped(counter, attrs:openclaw.diagnostic.async_queue.drop_class; internal diagnostic-queue backpressure drops)openclaw.telemetry.exporter.events(counter, attrs:openclaw.exporter,openclaw.signal,openclaw.status, optionalopenclaw.reason, optionalopenclaw.errorCategory; exporter lifecycle/failure self-telemetry)
Exported spans
openclaw.model.usageopenclaw.channel,openclaw.provider,openclaw.modelopenclaw.tokens.*(input/output/cache_read/cache_write/total)gen_ai.systemby default, orgen_ai.provider.namewhen the latest GenAI semantic conventions are opted ingen_ai.request.model,gen_ai.operation.name,gen_ai.usage.*
openclaw.runopenclaw.outcome,openclaw.channel,openclaw.provider,openclaw.model,openclaw.errorCategory
openclaw.model.callgen_ai.systemby default, orgen_ai.provider.namewhen the latest GenAI semantic conventions are opted ingen_ai.request.model,gen_ai.operation.name,openclaw.provider,openclaw.model,openclaw.api,openclaw.transport,openclaw.model_call.observation_unit(requestorturn)openclaw.errorCategory,error.type, and optionalopenclaw.failureKindon errorsopenclaw.model_call.request_bytes,openclaw.model_call.response_bytes,openclaw.model_call.time_to_first_byte_msopenclaw.model_call.prompt.input_messages_count,openclaw.model_call.prompt.input_messages_chars,openclaw.model_call.prompt.system_prompt_chars,openclaw.model_call.prompt.tool_definitions_count,openclaw.model_call.prompt.tool_definitions_chars,openclaw.model_call.prompt.total_chars(safe component sizes only, no prompt text)openclaw.model_call.usage.*andgen_ai.usage.*when the result carries usage for that request or aggregate turn- Span event
openclaw.provider.requestwith attributeopenclaw.upstreamRequestIdHash(bounded, hash-based) when the upstream provider result exposes a request id; raw ids are never exported - With
OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental, request spans use the latest GenAI inference span name{gen_ai.operation.name} {gen_ai.request.model}. Turn spans useinvoke_agentbecause OpenClaw does not claim a native agent name from the opaque CLI boundary. Both useCLIENTspan kind instead ofopenclaw.model.call.
openclaw.harness.runopenclaw.harness.id,openclaw.harness.plugin,openclaw.outcome,openclaw.provider,openclaw.model,openclaw.channel- On completion:
openclaw.harness.result_classification,openclaw.harness.yield_detected,openclaw.harness.items.started,openclaw.harness.items.completed,openclaw.harness.items.active - On error:
openclaw.harness.phase,openclaw.errorCategory, optionalopenclaw.harness.cleanup_failed
openclaw.tool.executiongen_ai.tool.name,gen_ai.operation.name(execute_tool),openclaw.toolName,openclaw.tool.source, optionalgen_ai.tool.call.id,openclaw.tool.owner,openclaw.tool.params.*- Optional
openclaw.errorCategory/openclaw.errorCodeon errors,openclaw.deniedReasonandopenclaw.outcome=blockedwhen denied by policy or sandbox
openclaw.execopenclaw.exec.target,openclaw.exec.mode,openclaw.outcome,openclaw.failureKind,openclaw.exec.command_length,openclaw.exec.exit_code,openclaw.exec.exit_signal,openclaw.exec.timed_out
openclaw.webhook.processedopenclaw.channel,openclaw.webhook
openclaw.webhook.erroropenclaw.channel,openclaw.webhook,openclaw.error
openclaw.message.processedopenclaw.channel,openclaw.outcome,openclaw.reason
openclaw.message.deliveryopenclaw.channel,openclaw.delivery.kind,openclaw.outcome,openclaw.errorCategory,openclaw.delivery.result_count
openclaw.session.stuckopenclaw.state,openclaw.ageMs,openclaw.queueDepth
openclaw.context.assembledopenclaw.prompt.size,openclaw.history.size,openclaw.context.tokens,openclaw.errorCategory(no prompt, history, response, or session-key content)
openclaw.tool.loopopenclaw.toolName,openclaw.loop.level,openclaw.loop.action,openclaw.loop.detector,openclaw.loop.count, optionalopenclaw.loop.paired_tool(no loop messages, params, or tool output)
openclaw.memory.pressureopenclaw.memory.level,openclaw.memory.reason,openclaw.memory.rss_bytes,openclaw.memory.heap_used_bytes,openclaw.memory.heap_total_bytes,openclaw.memory.external_bytes,openclaw.memory.array_buffers_bytes, optionalopenclaw.memory.threshold_bytes/openclaw.memory.rss_growth_bytes/openclaw.memory.window_ms
When content capture is explicitly enabled, model and tool spans can also
include bounded, redacted openclaw.content.* attributes for the specific
content classes you opted into.
Diagnostic event catalog
The events below back the metrics and spans above or are available for direct
plugin subscription. run.progress and run.execution_phase are direct-only
lifecycle signals; the diagnostics-otel plugin does not export them as
standalone OTLP signals. Event kinds and run.execution_phase.phase values are
additive. TypeScript consumers should keep default branches instead of assuming
either union is permanently exhaustive.
Model usage
model.usage- tokens, cost, duration, context, provider/model/channel, session ids.usageis provider/turn accounting for cost and telemetry;context.usedis the current prompt/context snapshot and can be lower than providerusage.totalwhen cached input or tool-loop calls are involved.
Message flow
webhook.received/webhook.processed/webhook.errormessage.queued/message.processedmessage.delivery.started/message.delivery.completed/message.delivery.error
Queue and session
queue.lane.enqueue/queue.lane.dequeuesession.state/session.long_running/session.stalled/session.stuckrun.attempt/run.progressrun.execution_phase(public, session-correlated embedded-runner startup milestones)diagnostic.heartbeat(aggregate counters: webhooks/queue/session)
Harness lifecycle
harness.run.started/harness.run.completed/harness.run.error- per-run lifecycle for the agent harness. IncludesharnessId, optionalpluginId, provider/model/channel, and run id. Completion addsdurationMs,outcome, optionalresultClassification,yieldDetected, anditemLifecyclecounts. Errors addphase(prepare/start/send/resolve/cleanup),errorCategory, and optionalcleanupFailed.
Exec
exec.process.completed- terminal outcome, duration, target, mode, exit code, and failure kind. Command text and working directories are not included.exec.approval.followup_suppressed- stale approval follow-up dropped after a session rebound. IncludesapprovalId,reason(session_rebound),phase(direct_deliveryorgateway_preflight), and the dispatcher timestamp. Session keys, routes, and command text are not included.
Without an exporter
Keep diagnostics events available to plugins or custom sinks without running
diagnostics-otel:
{
diagnostics: { enabled: true },
}
For targeted debug output without raising logging.level, use diagnostics
flags. Flags are case-insensitive and support wildcards (telegram.* or
*):
{
diagnostics: { flags: ["telegram.http"] },
}
Or as a one-off env override:
OPENCLAW_DIAGNOSTICS=telegram.http,telegram.payload openclaw gateway
Flag output goes to the standard log file (logging.file) and is still
redacted by logging.redactSensitive. Full guide:
Diagnostics flags.
Disable
{
diagnostics: { otel: { enabled: false } },
}
Or leave diagnostics-otel out of plugins.allow, or run
openclaw plugins disable diagnostics-otel.
Related
- Logging - file logs, console output, CLI tailing, and the Control UI Logs tab
- Gateway logging internals - WS log styles, subsystem prefixes, and console capture
- Diagnostics flags - targeted debug-log flags
- Diagnostics export - operator support-bundle tool (separate from OTEL export)
- Configuration reference - full
diagnostics.*field reference