vultr/openclaw
1
0
Fork 0
mirror of https://github.com/openclaw/openclaw.git synced 2026-06-29 17:03:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
60,102 Commits 1,843 Branches 149 Tags
0710eebec2b0e86f3bc885ba8eecf51fdfd435df
Commit Graph

62 Commits

Author SHA1 Message Date
Dallin Romney
e32929e12c Add slim evidence mode for QA profile evidence (#93179) 
* test(qa): compact profile evidence execution metadata

* docs(qa): document compact profile evidence

* test(qa): support compact evidence mode

* test(qa): rename compact evidence mode to slim

* docs(qa): trim slim evidence wording

* fix(qa): avoid commander runtime import
 2026-06-15 14:50:40 -07:00
Dallin Romney
e8db9c3bc0 test(qa): add qa run --profile and unified output summary/evidence (#91587) 
* test(qa): add mapped qa run profiles

* test(qa): document mapped profile runner

* test(qa): validate run profiles from mapping

* test(qa): preserve root profile parsing

* test(qa): simplify taxonomy profile dispatch

* test(qa): align tool coverage CLI expectation

* test(qa): fix profile dispatch fixture type

* test(qa): share profile runner option types

* test(qa): split shared cli runner options

* test(qa): unify profile suite artifacts

* fix(qa): filter profile scenarios by provider lane

* test(qa): drop native scenario subreports

* fix(qa): keep native log refs repo-relative

* fix(cli): preserve qa run root profile parsing

* fix(qa): avoid qa profile flag collision

* fix(qa): reject profile flags without qa profile
 2026-06-14 18:08:42 -07:00
Dallin Romney
fef8394079 Convert QA scenarios to YAML files (#92915) 
* refactor: load QA scenarios from YAML

* docs: update personal QA scenario docs

* test: keep QA scenarios YAML-only
 2026-06-14 17:31:18 -07:00
Dallin Romney
a3e9dfee0e Simplify QA scorecard mapping shape (#92558) 
* test(qa): simplify scorecard mapping shape

* test(qa): use typed scorecard evidence refs

* test(qa): map scorecard categories by coverage id

* feat: align qa coverage with taxonomy features

* refactor: keep qa coverage ids canonical

* refactor: minimize qa coverage id churn

* test: align qa coverage id assertions

* test: update qa evidence coverage expectations

* refactor qa taxonomy coverage ids

* style qa taxonomy coverage ids

* test qa coverage lint fix

* test qa coverage type fix
 2026-06-14 00:16:33 -07:00
Dallin Romney
561b293c7a Run Vitest and Playwright scenarios from qa suite (#92606) 
* test(qa): run vitest and playwright scenarios from qa suite

* fix(qa): harden scenario suite dispatch

* refactor(qa): share scenario path utilities

* refactor(qa): share test file scenario runner

* refactor(qa): route test file scenarios through suite runtime

* refactor(qa): use explicit suite runtime result kind

* test(qa): write suite evidence artifact

* refactor(qa): clarify suite execution dispatch

* fix(qa): keep test-file scenarios out of flow-only runners

* refactor(qa): export mixed scenario suite runner
 2026-06-13 01:06:10 -07:00
Peter Steinberger
58912f8fd8 docs: document channel extension sources 2026-06-04 21:59:00 -04:00
Peter Steinberger
28a2043f51 fix(qa-lab): centralize cli integer parsing 2026-05-29 06:20:43 -04:00
Peter Steinberger
cd80b4efca fix: parse qa cli integers strictly 2026-05-28 14:36:47 -04:00
Peter Steinberger
bb46b79d3c refactor: internalize OpenClaw agent runtime (#85341) 
* refactor: extract agent core package

Introduce packages/agent-core as the OpenClaw-owned home for reusable agent loop, harness, session, prompt, and runtime dependency contracts.

* refactor: extract shared llm runtime

Move provider model registries, stream wrappers, OAuth helpers, and LLM utilities into src/llm with plugin-sdk barrels instead of depending on the old embedded runtime layout.

* refactor: remove pi runtime internals

Rename remaining Pi-shaped agent surfaces to OpenClaw agent runtime names, delete obsolete Pi docs and package graph checks, and add the third-party notice for incorporated code.

* refactor: tighten agent session runtime

Make agent-core/runtime dependencies explicit, consolidate compaction and session transcript helpers, and move model/session helpers behind OpenClaw-owned contracts.

* refactor: remove static model and pi auth paths

Drop static model catalogs and Pi auth bridges, move model/provider facts to manifest-owned runtime contracts, and harden internal embedded-agent utilities.

* refactor: remove legacy provider compat paths

* docs: remove agent parity notes

* fix: skip provider wildcard metadata parsing

* refactor: share session extension sdk loading

* refactor: inline acpx proxy error formatter

* refactor: fold edit recovery into edit tool

* fix: accept extension batch separator

* test: align startup provider plugin expectations

* fix: restore provider-scoped release discovery

* test: align static asset packaging expectations

* fix: run static provider catalogs during scoped discovery

* fix: add provider entry catalogs for scoped live discovery

* fix: load lightweight provider catalog entries

* fix: refresh provider-scoped plugin metadata

* fix: keep provider catalog entries on release live path

* fix: keep static manifest models in release live checks

* fix: harden release model discovery

* fix: reduce OpenAI live cache probe reasoning

* fix: disable OpenAI cache probe reasoning

* ci: extend OpenAI gateway live timeout

* fix: extend live gateway model budget

* fix: stabilize release validation regressions

* fix: honor provider aliases in model rows

* fix: stabilize release validation lanes

* fix: stabilize release memory qa

* ci: stabilize release validation lanes

* ci: prefer ipv4 for live docker node calls

* fix: restore shared tool-call stream wrapper

* ci: remove legacy pi test shard alias

* fix: clean up embedded agent test drift

* fix: stabilize runtime alias status

* fix: clean up embedded agent ci drift

* fix: restore release ci invariants

* fix: clean up post-rebase runtime drift

* fix: restore release ci checks

* fix: restore release ci after rebase

* fix: remove stale pi runtime path

* test: align compaction runtime expectations

* test: update plugin prerelease expectations

* fix: handle claude live tool approvals

* fix: stabilize release validation gates

* fix: finish agent runtime import

* test: finish post-rebase agent runtime mocks

* fix: keep codex compaction native

* fix: stabilize codex app-server hook tests

* test: isolate codex diagnostic active run

* test: remove codex diagnostic completion race

# Conflicts:
#	extensions/codex/src/app-server/run-attempt.test.ts

* ci: fix full release manifest performance run id

* refactor: narrow llm plugin sdk boundary

* chore: drop generated google boundary stamps

* fix: repair rebase fallout

* fix: clean up rebased runtime references

* fix: decode codex jwt payloads as base64url

* fix: preserve shipped pi runtime alias

* fix: add scoped sdk virtual modules

* fix: decode llm codex oauth jwt as base64url

* fix: avoid stale vertex adc negative cache

* fix: harden tool arg decoding and codeql path

* fix: keep vertex adc negative checks live

* refactor: consolidate codex jwt and edit helpers

* fix: await codex oauth node runtime imports

* fix: preserve sdk tool and notice contracts

* fix: preserve shipped compat config boundaries

* fix: align codex oauth callback host

* fix: terminate agent-core loop streams on failure

* fix: keep codex oauth callback alive during fallback

* ci: include session tools in critical codeql scans

* fix: keep Cloudflare Anthropic provider auth header

* docs: redirect legacy pi runtime pages

* fix: honor bundled web provider compat discovery

* fix: protect session output spill files

* fix: keep legacy agent dir env blocked

* fix: contain auto-discovered skill symlinks

* fix: harden agent core sdk proxy surfaces

* fix: restore approval reaction sdk compat

* fix: keep live docker runs bounded

* fix: keep codex oauth redirect host aligned

* fix: resolve post-rebase agent runtime drift

* fix: redact anthropic oauth parse failures

* fix: preserve responses strict tool shaping

* fix: repair agent runtime rebase cleanup

* docs: redirect retired parity pages

* fix: bound auto-discovered resources to roots

* fix: repair post-rebase agent test drift

* fix: preserve bundled provider allowlist migration

* fix: preserve manifest-owned provider aliases

* fix: declare photon image dependency

* fix: keep provider headers out of proxy body

* fix: preserve shipped env aliases

* fix: refresh control ui i18n generated state

* fix: quote read fallback paths

* fix: preview edits through configured backend

* test: satisfy core test typecheck

* fix: preserve ZAI usage auth fallback

* test: repair codex diagnostic test

* fix: repair agent runtime rebase drift

* test: finish embedded runner import rename

* fix: repair agent runtime rebase integrations

* test: align compaction oauth fallback expectations

* fix: allow sdk-auth session models

* fix: update doctor tool schema import

* fix: preserve bedrock plugin region

* fix: stream harmony-like prose immediately

* ci: include session runtime in codeql shards

* fix: repair latest rebase integrations

* fix: honor explicit codex websocket transport

* fix: keep openai-compatible credentials provider-scoped

* fix: refresh sdk api baseline after rebase

* fix: route cli runtime aliases through openclaw harness

* test: rename stale harness mock expectation

* test: rename embedded agent overflow calls

* test: clean embedded auth test wording

* test: use openclaw stream types in deepinfra cache test

* fix: refresh sdk api baseline on latest main

* fix: honor bundled discovery compat allowlists

* fix: refresh sdk api baseline after latest rebase

* fix: remove stale rebase imports

* test: rename stale model catalog mock

* test: mock renamed doctor runtime modules

* fix: map canonical kimi env auth

* fix: use internal model registry in bench script

* fix: migrate deepinfra provider catalog entry

* fix: enforce builtin tool suppression

* fix: route compaction auth and proxy payloads safely

* refactor: prune unused llm registry leftovers

* test: update codex hooks session import

* test: fix model picker ci coverage

* test: align model picker auth mock types
 2026-05-27 19:24:04 +01:00
Vincent Koc
f6a49a4e8a test(qa-lab): add runtime confidence reports 2026-05-25 22:00:21 +01:00
Peter Steinberger
a1fe86a0ff feat(qa): add coverage scenario matching 2026-05-25 10:22:51 +01:00
Vincent Koc
7f05be041e fix(diagnostics): harden observability exports and smokes (#85371) 
* test(diagnostics): widen observability smokes

* fix(diagnostics): sanitize observability exports

* docs(diagnostics): format otel export docs
 2026-05-23 15:27:43 +08:00
Vincent Koc
cf0657852f feat(qa-lab): add jsonl replay harness 2026-05-21 23:03:51 +08:00
Vincent Koc
1300b22630 fix(qa-lab): classify runtime token efficiency 2026-05-18 11:09:08 +08:00
Vincent Koc
79212f9869 feat(qa-lab): select runtime parity tiers 2026-05-18 00:21:13 +08:00
Vincent Koc
d3a7348ad9 fix(qa-lab): keep bootstrap tokens private 2026-05-17 21:05:01 +08:00
Vincent Koc
1f9d8c1e9d fix(qa-lab): wire tool coverage report command 2026-05-17 17:12:10 +08:00
Vincent Koc
da8afe359d feat(qa-lab): add scenario pack selector 2026-05-17 09:23:48 +08:00
Vincent Koc
f345b54d04 test(qa-lab): add runtime parity axis 2026-05-17 03:32:50 +08:00
Peter Steinberger
0bf06e953f feat: add Mantis Discord smoke runner (#76696) 
* docs: add Mantis QA system design

* feat: add Mantis Discord smoke runner

* fix: harden Mantis Discord smoke

* fix: redact Mantis Discord artifacts

* fix: satisfy Mantis redaction lint

* fix: redact Mantis mismatch failures

* test: avoid promise assertions in Mantis tests
 2026-05-03 15:25:56 +01:00
Vincent Koc
a6dfaaeb4e test(plugins): add gateway gauntlet 2026-04-28 16:44:10 -07:00
Peter Steinberger
88c91675e2 test: stabilize qa suite concurrency 2026-04-24 20:39:33 +01:00
Peter Steinberger
903308dbf2 fix: stabilize qa lab mock suite 2026-04-24 02:46:33 +01:00
Val Alexander
dab46a7e98 qa: harden parity gate execution (#70045) 2026-04-22 03:08:25 -05:00
Peter Steinberger
6ce17db11a fix: gate max thinking by model support 2026-04-21 07:02:43 +01:00
Josh Avant
d5b326523f qa-lab: make live lanes CI-ready for v1 E2E automation (#69122) 
* qa-lab: harden CI defaults and failure semantics for live lanes

* qa-lab: add unit tests for suite progress logging defaults

* qa-lab: cover malformed multipass summary edge cases

* qa-lab: share suite summary failure counting helper

* qa-lab: test allow-failures parse wiring and sanitize progress ids

* fix: note qa CI live-lane defaults in changelog (#69122) (thanks @joshavant)
 2026-04-19 21:13:27 -05:00
Gustavo Madeira Santana
3a1e469732 QA: track scenario coverage intent 2026-04-17 14:05:49 -04:00
Gustavo Madeira Santana
bb7e9823a8 QA Lab: add provider registry 
Move mock and live provider behavior behind provider-owned definitions so suite, manual, Matrix, and transport lanes share defaults, auth staging, model config, and standalone server startup.

Add AIMock as a first-class local provider mode while keeping mock-openai as the scenario-aware deterministic lane.
 2026-04-16 21:24:59 -04:00
Gustavo Madeira Santana
82a2db71e8 refactor(qa): split Matrix QA into optional plugin (#66723) 
Merged via squash.

Prepared head SHA: 27241bd089
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
 2026-04-14 16:28:57 -04:00
Josh Avant
3d07dfbb65 feat(qa-lab): add Convex credential broker and admin CLI (#65596) 
* QA Lab: add Convex credential source for Telegram lane

* QA Lab: scaffold Convex credential broker

* QA Lab: add Convex credential admin CLI

* QA Lab: harden Convex credential security paths

* QA Broker: validate Telegram payloads on admin add

* fix: note QA Convex credential broker in changelog (#65596) (thanks @joshavant)
 2026-04-12 22:03:42 -05:00
Peter Steinberger
fcee268373 feat(qa-lab): support scenario-defined plugin runs 2026-04-12 11:59:50 -07:00
Marcus Castro
000fc7f233 refactor(qa): add shared QA channel contract and harden worker startup (#64562) 
* refactor(qa): add shared transport contract and suite migration

* refactor(qa): harden worker gateway startup

* fix(qa): scope waits and sanitize shutdown artifacts

* fix(qa): confine artifacts and redact preserved logs

* fix(qa): block symlink escapes in artifact paths

* fix(gateway): clear shutdown race timers

* fix(qa): harden shutdown cleanup paths

* fix(qa): sanitize gateway logs in thrown errors

* fix(qa): harden suite startup and artifact paths

* fix(qa): stage bundled plugins from mutated config

* fix(qa): broaden gateway log bearer redaction

* fix(qa-channel): restore runtime export

* fix(qa): stop failed gateway startups as a process tree

* fix(qa-channel): load runtime hook from api surface
 2026-04-12 15:02:57 -03:00
Peter Steinberger
057fe786bd style: apply formatter drift 2026-04-11 21:25:24 +01:00
Eva
3211aa2540 fix(qa): surface missing required scenarios in parity report 2026-04-11 14:22:48 +01:00
Eva
67fdd3b4df benchmarks: add agentic parity report gate 2026-04-11 14:22:48 +01:00
Eva
d9c7ddb099 test: add agentic parity scenario pack 2026-04-11 14:22:48 +01:00
Peter Steinberger
cd89892b1f fix(release): keep private QA bundles out of npm pack 2026-04-11 13:13:11 +01:00
Gustavo Madeira Santana
25445a9f2e qa-lab: add Matrix live transport QA lane (#64489) 
Merged via squash.

Prepared head SHA: ae9bb37751
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
 2026-04-10 19:35:08 -04:00
Ayaan Zaidi
d69cc5da5c fix(qa-lab): address remaining review comments 2026-04-10 21:53:31 +05:30
Ayaan Zaidi
2aaf5a3baa fix(qa-lab): address telegram qa review comments 2026-04-10 21:53:31 +05:30
Ayaan Zaidi
e093cb6c93 feat(qa-lab): add telegram live qa lane 2026-04-10 21:53:31 +05:30
Peter Steinberger
07e7222e28 test: split Claude CLI QA auth modes 2026-04-10 14:56:36 +01:00
Peter Steinberger
4c14f55c62 test: parallelize QA suite scenarios 2026-04-10 13:45:57 +01:00
Shakker
def2eadb1d feat: add multipass runner to qa suite 2026-04-09 23:53:13 +01:00
Peter Steinberger
39cc6b7dc7 fix: stabilize character eval and Qwen model routing 2026-04-09 01:04:09 +01:00
Peter Steinberger
21ef1bf8de feat: parallelize character eval runs 2026-04-08 20:05:55 +01:00
Peter Steinberger
4a51a1031d feat: add character eval model options 2026-04-08 17:05:30 +01:00
Peter Steinberger
3101d81053 feat: add QA character eval reports 2026-04-08 15:52:49 +01:00
Vincent Koc
76bde3d42b fix(qa): support neutral-cwd docker commands 2026-04-07 20:32:42 +01:00
Vincent Koc
816a3eae8a chore(qa): align qa cli provider input types 2026-04-07 20:32:42 +01:00