mirror of
https://github.com/openclaw/openclaw.git
synced 2026-08-02 13:01:34 +00:00
f1ee2a3098
* fix(diagnostics-otel): keep parent span links across long turns Retained trusted span contexts translate a completed lifecycle span's diagnostic span id to the real OTel span context. That table was evicted 5s after run.completed, so children of a longer turn missed their parent and OTel minted a fresh trace id for each one, silently splitting a turn into single-span traces. Retention is now identity-based and lives for the service lifetime, bounded by MAX_RETAINED_TRUSTED_SPAN_CONTEXTS, which removes the timing race and the drain/timeout cleanup subsystem. Also nests openclaw.exec under its run: the exec event now carries a trusted trace context and the recorder resolves the ambient run scope. * fix(diagnostics-otel): key retained span contexts by diagnostic trace id Retained lifecycle span contexts were stored under the span's OTel trace id but looked up with the event's diagnostic trace id. Those are different id spaces, so every post-completion parent lookup missed and each straggler span started a brand-new trace. The mocked suite could not catch this: its tracer reports the same trace id the test feeds in, collapsing both id spaces into one value. Adds a boundary test that runs the real OTel SDK and OTLP/protobuf exporter against a local receiver and asserts on exported span bytes. * fix(diagnostics-otel): harden span-linking contract after review Review findings on the parent-span linking fix: - completeTrackedLifecycleSpan now takes the DiagnosticTraceContext instead of a bare trace id beside a span id. Two adjacent hex strings let a transposition compile clean and silently reinstate split traces; DiagnosticTraceContext is the only shape an OTel SpanContext cannot satisfy (traceFlags string vs number). - internalOrTrustedTraceContext composes normalizedTrustedTraceContext rather than spelling the trust predicate a third time. - Corrected comments that claimed OTel ids never equal diagnostic ids. Spans parented from an upstream traceparent do adopt the diagnostic trace id; the removed guard only ever misfired on root lifecycle spans. - Exec comment now states the real parent: the openclaw harness opens no run scope, so exec nests under openclaw.harness.run. Test gaps closed: - The core emitter switch was untested; reverting it kept every test green. The pty-fallback test now asserts the emitted metadata. - The boundary test covered only run.completed of three lifecycle owners; it now covers harness.run.completed and message.processed too, each with a straggler. - Dropped the service-context cast for the typed helper, which caught a bogus context.assembled field, and moved teardown into afterEach so a failed assertion cannot leak the real NodeSDK across files. - Scoped the one-trace assertion to this turn's spans; faked Date so a lazy Date.now expiry cannot pass. Docs: exec and tool spans now listed as staying on an upstream request trace. * fix(diagnostics-otel): keep late children on the trace on every terminator Native review found three correctness gaps in the previous commit: - Exec spans could be exported naming a parent span id nothing ever emits. The ambient lookup fell through to a remote-parent fallback built from diagnostic ids, which is a regression from the previous clean-root behavior and breaks waterfalls and parent-id-keyed backends. Exec now resolves only spans this process exported and stays a root on a miss; message spans keep the remote fallback because their context can come from an inbound traceparent. - harness.run.error never retained its span context, so an aborted turn still split. For the openclaw harness that span is the only ancestor a late child has, and aborted turns emit no run.completed. - completeTrackedLifecycleSpan no longer takes a redundant spanId. The boundary test now runs through the OPENCLAW_OTEL_PRELOADED seam with an in-memory exporter and releases the global tracer provider in teardown. A NodeSDK cannot be unregistered, so the old version would have made any later real-SDK test silently export nothing. That also deletes the hand-rolled OTLP protobuf decoder. Added guards for the aborted-turn path and for exec staying parentless; both are mutation-verified. * fix(diagnostics-otel): age retained parent contexts out after 10 minutes Retention had no time bound at all, so a command backgrounded by a turn could finish hours later and still attach to a parent that ended long before. Backends derive trace latency from the span envelope, so a 30s turn rendered as hours with a child extending past its parent. The horizon is checked lazily on lookup rather than on a timer: a timer firing mid-turn is exactly what silently split traces before. Past the horizon a straggler starts its own trace, which only loses causality for a span that would have skewed its parent's duration anyway. The two bounds now pin each other - shrinking the horizon to the old 5s window fails the original regression tests, and removing it fails the new horizon test. * fix(diagnostics-otel): retain parent contexts until capacity
@openclaw/diagnostics-otel
Official OpenTelemetry diagnostics exporter for OpenClaw.
This plugin exports OpenClaw Gateway traces, metrics, and logs to an OTLP collector for observability stacks such as Grafana, Datadog, Honeycomb, New Relic, Tempo, and compatible collectors. It can also write diagnostic log records as stdout JSONL for container log pipelines.
Install
openclaw plugins install @openclaw/diagnostics-otel
Restart the Gateway after installing or updating the plugin.
Configure
Enable the plugin and set the OTLP endpoint in plugins.entries.diagnostics-otel.config.
The full config surface, metric names, span names, and collector examples live in the docs:
Package
- Plugin id:
diagnostics-otel - Package:
@openclaw/diagnostics-otel - Minimum OpenClaw host:
2026.4.25