feat(jev): add TypeSafe System One client, Spring Boot starter, and example middlewares

[jujn]

Summary

This PR adds a dedicated Jev extension for TypeSafe System One (https://docs.typesafe.ai/introduction),
along with a Spring Boot starter and three reference middlewares.

Jev is intentionally not registered as a ChatModel provider. It is a structured decision model,
not a chat model, so the extension exposes it as a typed decision primitive (Noul, Choice,
Score) for middleware and application logic.

Changes

• Added agentscope-extensions-jev
  • JevClient: HTTP client for POST /v1/systemone, configurable base URL /
    model / transport / retry / timeout, and response validation for answer keys, types,
    probabilities, and score legends
  • Typed request/response DTOs for NoulQuestion, ChoiceQuestion, ScoreQuestion and their
    corresponding answers
  • JevException with HTTP status, response body, and retryability
• Added agentscope-jev-spring-boot-starter
  • Auto-configures a JevClient bean from agentscope.jev.* properties
  • Supports JevClientBuilderCustomizer for advanced programmatic configuration
  • Falls back to TYPESAFE_API_KEY / JEV_API_KEY environment variables when no explicit key
    is configured
• Added three example middlewares in io.agentscope.extensions.judge.jev.example(for reference only):
  • JevToolSelectionMiddleware (onReasoning)
    • Ranks optional tools with Jev and only sends those above the __none__ threshold to the
      primary model, up to maxTools
    • Always preserves core tools (load_skill_through_path, reset_tools, generate_response)
    • Chunks and reranks tool sets larger than 254 candidates
  • JevModelRouterMiddleware (onAgent / onModelCall)
    • Asks Jev to choose between configured models once per agent invocation, based on
      per-candidate routing criteria
    • Falls back to the agent's configured model when confidence is below threshold, Jev fails,
      or no user text is available
    • Only replaces ModelCallInput.model; messages, tools, and options pass through
  • JevAutoModeMiddleware (onActing)
    • Asks Jev a yes/no risk question before executing guarded tool calls (e.g. bash)
    • Denies calls whose P(safe) is below safetyThreshold by writing a synthetic DENIED
ToolResultBlock and emitting tool-result events, without invoking the toolkit
    • Skips calls already confirmed as ALLOWED via HITL, so human approval takes precedence
    • Batches multiple guarded calls into a single Jev request
• Added shared selection helpers (JevSelectionSupport) for message extraction, criteria
  building, and chunking/reranking
• Added English and Chinese integration docs covering the client, starter, and all three example
  middlewares:
  • English
  • Chinese
• Registered the new modules in the extension aggregator, BOM, and all-in-one distribution

Manual Testing

Three manual runners are included for end-to-end validation against the real Jev and DashScope
APIs. They are plain Java classes with main() methods (not JUnit tests), located in
agentscope-java\agentscope-examples\documentation\src\main\java\io\agentscope\examples\documentation2\jev\:

• JevAutoModeMiddlewareTest — runs a ReActAgent with a simulated bash tool; verifies that
  ls passes the Jev safety check while rm -rf . gets denied
• JevModelRouterMiddlewareTest — runs a ReActAgent that routes between two models based on
  task complexity
• JevToolSelectionMiddlewareTest — runs a ReActAgent with a larger tool registry; verifies
  that only relevant tools are sent to the model

To run them:

export TYPESAFE_API_KEY=...
export DASHSCOPE_API_KEY=...
mvn -pl agentscope-examples/documentation -am \
    compile exec:java -DskipTests \
    -Dexec.mainClass=io.agentscope.examples.documentation2.jev.JevAutoModeMiddlewareTest

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
agentscope-java	🟡 Building	–	Sep 21, 2026, 5:22 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

[Copilot]

Copilot review overview

🟡 Changes recommended

Chunked selection and response-validation paths contain correctness gaps, including invalid __none__ handling and malformed score responses.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 5 · 3

Open (8)

• Guard against null deserialization results · New
• Validate legend values against question criteria levels · New
• Treat blank primary API keys as unset · New
• Prevent none from colliding with user-defined candidates · New
• Do not select representatives when no-match probability is highest · New
• Translate the non-English source comment · New
• Preserve API key fallback in the quickstart · New
• 避免快速示例绕过 API 密钥回退 · New

What changed in this PR

Adds a TypeSafe System One Jev client, typed decision models, selection middlewares, tests, documentation, and distribution integration.

Changes:

• Added HTTP client with retries, timeouts, authentication, and response validation.
• Added skill suggestion and tool selection middlewares with chunking and fail-open behavior.
• Registered the extension in Maven aggregators, BOM, all-in-one distribution, and bilingual docs.

File	Description
docs/​v2/​zh/​integration/​ecosystem/​jev.md	Chinese Jev integration guide
docs/​v2/​en/​integration/​ecosystem/​jev.md	English Jev integration guide
docs/​docs.json	Documentation navigation and redirects
agentscope-extensions/​pom.xml	Registers the extension module
agentscope-extensions/​agentscope-extensions-jev/​pom.xml	Jev module dependencies
agentscope-distribution/​agentscope-bom/​pom.xml	BOM registration
agentscope-distribution/​agentscope-all/​pom.xml	All-in-one distribution registration
.../​JevClient.java	HTTP client and validation
.../​JevException.java	Jev exception type
.../​JevRetryPolicy.java	Retry configuration
.../​SystemOneRequest.java	Request model and builder
.../​SystemOneResult.java	Response model
.../​Usage.java	Token usage model
.../​Question.java	Question polymorphism
.../​Answer.java	Answer polymorphism
.../​NoulQuestion.java	Noul question model
.../​NoulAnswer.java	Noul answer model
.../​ChoiceQuestion.java	Choice question model
.../​ChoiceAnswer.java	Choice answer model
.../​ScoreQuestion.java	Score question model
.../​ScoreAnswer.java	Score answer model
.../​JevSelectionSupport.java	Shared selection helpers
.../​JevSkillSuggestionMiddleware.java	Skill suggestion middleware
.../​JevToolSelectionMiddleware.java	Tool selection middleware
.../​JevClientTest.java	Client behavior tests
.../​JevDtoTest.java	DTO serialization tests
.../​JevExceptionTest.java	Exception tests
.../​JevRetryPolicyTest.java	Retry policy tests
.../​JevSkillSuggestionMiddlewareTest.java	Skill middleware tests
.../​JevToolSelectionMiddlewareTest.java	Tool middleware tests

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[codecov]

Codecov Report

❌ Patch coverage is 72.47706% with 210 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
.../io/agentscope/extensions/judge/jev/JevClient.java	57.66%	55 Missing and 50 partials ⚠️
...ns/judge/jev/example/JevModelRouterMiddleware.java	73.78%	16 Missing and 11 partials ⚠️
...sions/judge/jev/example/JevAutoModeMiddleware.java	84.67%	10 Missing and 11 partials ⚠️
.../judge/jev/example/JevToolSelectionMiddleware.java	81.30%	12 Missing and 8 partials ⚠️
...ensions/judge/jev/example/JevSelectionSupport.java	61.70%	5 Missing and 13 partials ⚠️
...entscope/spring/boot/jev/JevAutoConfiguration.java	69.23%	1 Missing and 7 partials ⚠️
.../agentscope/extensions/judge/jev/JevException.java	75.00%	5 Missing ⚠️
...gentscope/extensions/judge/jev/ChoiceQuestion.java	60.00%	1 Missing and 1 partial ⚠️
...gentscope/extensions/judge/jev/JevRetryPolicy.java	87.50%	0 Missing and 1 partial ⚠️
...agentscope/extensions/judge/jev/ScoreQuestion.java	66.66%	0 Missing and 1 partial ⚠️
... and 2 more

📢 Thoughts on this report? Let us know!

[oss-maintainer]

Summary

Adds a new agentscope-extensions-jev module: a typed client for TypeSafe's System One endpoint (JevClient + sealed Question/Answer DTOs + JevRetryPolicy), plus two middlewares that use it to rank skills (JevSkillSuggestionMiddleware) and shrink the tool schema sent to the primary model (JevToolSelectionMiddleware), with docs in both languages. ~3.3k lines, all additive, with 1.2k lines of tests — good coverage of validation, retry and fail-open paths, and the module is registered in the extensions aggregator, agentscope-all and the BOM.

Overall this looks well-built and I'd land it after a round of fixes on the runtime-behaviour items below. My main theme: the remote call is on the reasoning hot path — one (or two, for >254 candidates) round trips per reasoning step with a per-attempt timeout that is restarted by retryWhen, so worst-case latency added to a ReAct turn is well over a minute before fail-open engages, and a low-confidence answer degrades to zero optional tools rather than to "no filtering".

Findings

• [Critical] middleware/JevToolSelectionMiddleware.java:160 — empty selection (Set.of(), also from the confidenceThreshold path in JevSelectionSupport.selectedNames) removes every optional tool for that step, which is the opposite of the failOpen intent.
• [Warning] middleware/JevToolSelectionMiddleware.java:106 — latency budget: 2 sequential systemOne() calls per step, each 3 x timeout + backoff (default JevRetryPolicy gives ~19.5s, i.e. ~39s worst case). Suggest an outer total deadline plus per-turn memoization.
• [Warning] middleware/JevSelectionSupport.java:57 — raw Msg objects are serialized through the SNAKE_CASE JevClient.MAPPER, so message JSON on the wire is not the canonical AgentScope shape; it also ships full conversation content (incl. tool results) to a third party with no projection/redaction hook.
• [Warning] middleware/JevToolSelectionMiddleware.java:52 — load_skill_through_path / reset_tools / generate_response are literals owned by SkillToolFactory and ReActAgent.STRUCTURED_OUTPUT_TOOL_NAME; a rename upstream silently removes them from the schema.
• [Warning] middleware/JevSkillSuggestionMiddleware.java:223 — repository.getAllSkills() runs per agent invocation; blocking I/O fan-out on the hot path, worth caching.
• [Warning] JevClient.java:310 — null usage aborts an otherwise-valid response; it is a reporting field, treat it as optional.
• [Warning] JevClient.java:403 — absolute 1e-6 probability-sum tolerance over up to 255 options will reject legitimately rounded responses. Scale the tolerance with option count or normalize before comparing.
• [Info] JevClient.java:95 — systemOneBlocking uses block(), which throws on NonBlocking threads; guard or document.
• [Info] middleware/JevSkillSuggestionMiddleware.java:231 — swallowed RuntimeException with no logger makes a permanently broken repository indistinguishable from "no skills".
• [Info] docs/docs.json:331 — the page is registered but missing from the Ecosystem list in docs/v2/{en,zh}/integration/overview.md.

Suggestions

For the hot-path items, a shape that would address both at once:

return selectTools(state, optionalTools)
        .timeout(totalBudget)                       // hard ceiling across retries
        .onErrorResume(e -> failOpen ? Mono.just(allNames) : Mono.error(e))
        .map(names -> names.isEmpty() ? allNames : names)   // "unsure" => no filtering

and in retrySpec, Retry.backoff(...).handle(...)-style accounting or an outer .timeout() so timeout bounds the whole call rather than each attempt.

Notes

• I did not build or run the module locally (no local checkout sync in this run), so the review is static: the findings above come from reading the diff against agentscope-core's HttpTransport / MiddlewareBase / skill contracts.
• Nice touches: sealed DTOs with explicit Jackson subtype names, strict request/response validation, failOpen as a builder flag, builder(Function) seams that let the middleware tests avoid a real transport, and matched en/zh docs.

---

Automated review by github-manager-bot

[DennisWLX]

这个 PR 提供了有价值的 Jev 集成。我建议分别评估 Skill 推荐和 Tool 筛选，因为两者的错误代价不同：Skill 中间件保留完整目录、追加相关性建议；Tool 中间件则会真正移除发送给主模型的候选。
对当前 Tool 筛选实现，主要有以下疑问：

1. Choice 的单选语义与多工具召回存在差异。 Choice 的概率表示候选之间的相对选择分布，并不等于各工具独立适用的概率。多步骤任务可能同时需要多个工具，直接据此截取 Top-K，需要验证必要工具的召回率。逐候选 Noul 或统一标准的 Score，可能更适合判断一组工具是否有用。
2. 超过 254 个候选后，每块只保留 Top-1，会造成不可恢复的候选损失。 同一块中可能有多个必要工具，但第二轮只能看到其中一个。例如 maxTools=3 时，254 个候选可以选出三个；增加到 255 个后，分成两块，第二轮最多只剩两个。筛选结果因此可能受注册顺序和分块边界影响。建议每块保留多个候选，或先检索召回，再统一重排。
3. fail-open 无法覆盖“高置信度但不完整”的选择。 当前异常、低置信度或空结果可以回退，但非空结果遗漏必要工具时仍会过滤。建议保留工具搜索或扩展候选入口，让主模型能主动找回未展示的能力。
4. 需要验证端到端收益。 Tool 筛选每轮 reasoning 都会执行，大候选集需要两次串行请求，而且客户端超时是单次尝试的上限。建议评估任务完成率、必要工具召回率、P95 延迟和每个成功任务的成本，而不只看 Schema token 减少量。测试应包含 254/255/256 个候选、随机调整顺序、多工具组合、中文和无匹配任务。
   对于数百至数千个能力，更建议采用“检索召回 → 可选 Jev 重排 → 主模型选择，并允许继续搜索”的流程。AgentScope 已有 Higress searchTools(query, topK) 可作为 MCP 工具召回来源；本地 Tool 和 Skill 仍需要补充通用检索层。
   另外，TypeSafe 官方的 Skill 实验使用了候选详情复核和 Noul 适用性判断，最终保留完整技能目录。因此该实验支持辅助推荐的价值，但不能直接作为当前 Tool 硬过滤策略的效果依据。

[Aias00]

Review notes

Reviewed at head 559ce7358bafe9073aebcfa32dfef953b502bfe8.

1. High - Jev-denied tool calls bypass the standard acting lifecycle.

   JevAutoModeMiddleware.onActing writes denied results directly into AgentState and returns Flux.empty() when all calls are denied. Those calls therefore never reach ReActAgent's actingStream / resultHolder path. Unlike the native permission-denial path, this skips ToolResult* events, post-acting hooks, and tracing for the denied calls. A streaming consumer can observe a tool call without its terminal denial event.

   Could the synthetic denial be routed through the same core result/event path used by permission denials? A ReAct-level regression test should assert that a Jev-denied tool emits the standard terminal events, invokes post-acting middleware, persists exactly one denied result, and never executes the tool.

2. Medium - score legend values are not checked against the request rubric.

   validateScoreAnswer verifies only that legend and probabilities have matching level keys. For criteria ['Low', 'High'], a response legend such as {'0':'High','1':'Low'} is accepted. TypeSafe's score contract defines the legend as the mapping from numeric levels back to the submitted criteria, so accepting a mismatched legend exposes a misleading rubric to callers.

   Please compare each legend entry with its corresponding request criterion and add a negative test with swapped legend values.

3. Medium - nullable retry properties can fail startup with a bare NPE.

   JevAutoConfiguration passes nullable Integer maxRetries and nullable Duration initialBackoff directly into JevRetryPolicy; unboxing a cleared maxRetries or accepting a null backoff fails context refresh without a useful configuration error. This overlaps the existing unresolved thread. Falling back field-by-field to JevRetryPolicy.defaults() (or validating explicitly) would make the behavior deterministic. The starter test should also assert the effective timeout and retry policy, not only URL/auth/model.

4. Low - the quickstart overrides the documented API-key fallback.

   The quickstart calls .apiKey(System.getenv("TYPESAFE_API_KEY")). If that variable is absent, this overwrites the builder's JEV_API_KEY fallback with null. Omitting the setter, or resolving both variables before calling it, keeps the sample consistent with the documented client behavior. The Chinese page has the same issue.

5. Low - manual runners are included in the production artifact.

   The three live-API runners are under src/main/java/io/agentscope/extensions/judge/jev/test, so their main() methods, credential reads, and System.exit(1) calls are packaged in agentscope-extensions-jev. Moving them to an examples module or a test source set would keep the published API surface clean and reduce uncovered production code.

Verification

The focused reactor tests passed:

mvn -pl agentscope-extensions/agentscope-extensions-judge/agentscope-extensions-jev,agentscope-extensions/agentscope-spring-boot-starters/agentscope-jev-spring-boot-starter -am test

This covered the eight-module reactor, including 49 Jev tests and 5 starter tests. git diff --check also passed. Current CI is green for Ubuntu/Windows builds, docs, license, CLA, and module sync; codecov/patch is the remaining failed check.

[Buktal]

The overall approach looks good: Jev is exposed as a standalone structured-decision client. The following runtime-contract issues should be addressed before merging.

Blocking issues

1. Null RuntimeContext causes model routing to fail. Both branches of JevModelRouterMiddleware.onAgent() call ctx.put(), but the default agent.call() / streamEvents() paths allow a null context. Support those entry points and propagate the routing decision to the same invocation’s onModelCall(). Creating a method-local context without propagating it is insufficient.

2. Denied events do not reach the public event stream. ReActAgent.acting() only handles RequestStopEvent and consumes the remaining events through then(). External publication happens inside actingStream(), which middleware-synthesized events bypass. Connect the synthesized events to the publication path without duplicating core-emitted events. Currently, conversation state records DENIED, but external consumers receive no corresponding tool-completion event.

3. Partial denial reorders the remaining tool calls. Collecting all unguarded calls before approved guarded calls turns [approved guarded A, unguarded B, denied C] into [B, A]. This can break order-dependent execution, such as writing a file before reading it. Filter the original list while preserving the relative order of retained calls.

4. A valid __none__ decision is treated as fallback. When selectedNames.isEmpty(), ToolSelection returns the original tool list. Consequently, even a high-confidence __none__ decision sends all tools to the model. Distinguish a successful zero-tool selection, which should retain only always-included tools, from an unusable decision that requires fallback.

5. Filtering can invalidate an explicit tool_choice. Tool schemas are filtered while generation options pass through unchanged. This can leave ToolChoice.Specific("search") pointing to a removed tool. Preserve the explicitly requested tool or bypass automatic filtering when a specific tool is required.

6. The manual runners lack their required model-provider dependency. All three call ModelRegistry.resolve("openai:..."), but the module does not include agentscope-extensions-model-openai, and core does not provide that provider. Add the dependency in the appropriate scope and update the documented execution commands.

Additional improvements

7. Consider moving the three manual runners out of src/main, either into src/test as *ManualRunner or into an example module. This avoids shipping manual entry points containing System.exit in the production JAR and avoids duplicate simple class names with unit tests. If moved to test sources, update dependency scopes and the exec:java classpath configuration.

8. Clarify the status of jev.example: supported public API with an explicit stability contract, or reference implementations intended for copying and customization with documented compatibility expectations. The documentation already describes them as references; align packaging and API expectations with that positioning.

9. Make the missing-key error identify agentscope.jev.api-key and the TYPESAFE_API_KEY / JEV_API_KEY fallback options. Do not reject an absent property before accounting for environment variables and builder customizers.

10. Validate explicitly null retry subfields or define a default-value fallback. This avoids obscure failures from Integer unboxing or the JevRetryPolicy constructor. Existing defaults already cover ordinary omitted or partially specified configuration.

11. Consider making JevClient.MAPPER private and letting tests construct their own mapper. It is currently package-private, and no reconfiguration path was found, so this is an encapsulation improvement rather than an established concurrency defect.

12. Consider defensively copying the maps in ChoiceAnswer and ScoreAnswer, and make the mutability contract of SystemOneResult.answers consistent. If iteration order matters, use an unmodifiable LinkedHashMap copy.

13. AutoMode’s Javadoc should state that fail-open is enabled by default and that the state == null branch allows calls unconditionally, even when failOpen(false) is configured.

Please also add integration coverage through real ReActAgent.call() / streamEvents() entry points for default-context handling, denied-event visibility, mixed-tool ordering, and explicit tool-choice preservation. Direct hook tests with a stubbed next function do not exercise these lifecycle interactions.