GW2 Priory

Appearance

Research 029 — Natural-language assistant (the seam + closest-to-craft) ​

Status: complete Step 1.5 output, written between the spec draft and the approval gate. open until every [NEEDS VERIFICATION] marker in spec.md has a verdict here; complete once they do. The three markers (V1–V3) have verdicts below; none are refuted, so the spec does not go back to step 1. Two findings turned up alongside (F4, F5) — both simplify or sharpen the plan rather than blocking it.

The two [NEEDS CLARIFICATION] items were human decisions, resolved in spec.md and transcribed on the user's "go" (2026-08-20): C1 → eval-only (NLU quality held by a manual/periodic eval set, not a CI gate) and C2 → require the key up front (400 when Authorization is absent). They are recorded here for completeness; they are not verification items.

Evidence is cited, not recalled. Repo claims cite a file and line; outside-world claims cite the claude-api skill docs bundled this session.

Verified against, and when. Findings dated 2026-08-20. Repository: branch 029-nl-assistant at the commit that adds spec.md. Node v26.5.0, pnpm 11.15.1. apps/api on NestJS 11 with nestjs-zod + zod already in the dependency graph (ranking.schema.ts uses both). Anthropic facts come from the bundled claude-api skill (TypeScript SDK reference), not a live SDK install. No discovery spike was run — every marker was answerable from the repo plus the skill docs.

V1 — Is @anthropic-ai/sdk available, and what is its structured-output mechanism + a reliable Zod→schema derivation? ​

Question. R3 assumes the intent parse can force the model into a Zod discriminated union (closest_to_craft | unsupported) via the SDK's structured output, with the schema derived from one Zod source of truth.

Verdict. Confirmed — and simpler than the spec assumed. @anthropic-ai/sdk is not yet a dependency (must be added), and it ships a first-class structured-output path that takes a Zod schema directly — so no separate zod-to-json-schema dependency is needed (this refines R3/R8; see F4).

Evidence.

  • Absent today: a grep of package.json, apps/api/package.json, apps/web/package.json for anthropic / zod-to-json-schema / openai returned no matches (recon, 2026-08-20).
  • Mechanism (claude-api skill → typescript/claude-api/tool-use.md § Structured Outputs, and typescript/claude-api/README.md): client.messages.parse({ …, output_config: { format: zodOutputFormat(Schema) } }), with zodOutputFormat imported from @anthropic-ai/sdk/helpers/zod; the validated object is on response.parsed_output (null if parsing failed — must be guarded).
  • Model support (skill → shared/tool-use-concepts.md § Structured Outputs): structured outputs are supported on Claude Opus 4.8, Sonnet 5, Haiku 4.5, Fable 5 — so a small/fast model (Haiku 4.5) still supports the mechanism (relevant to F5).
  • Incompatibilities (same source): structured outputs are incompatible with message prefilling and citations. This spec uses neither, so there is no conflict.

Caveat. Verified from the skill's SDK reference, not a compiled call against an installed SDK. The plan's first task should add @anthropic-ai/sdk, write one throwaway parse against the real model, and confirm parsed_output validates — the same "test on a single request first" the skill recommends.

V2 — What is the repo's config/secret convention for an outbound API key? ​

Question. R8 needs to know where the Anthropic key lives so the LLM-client wrapper can read it, and so the Render deploy can supply it.

Verdict. Confirmed. Config is read directly from process.env (no @nestjs/config); the Anthropic SDK's zero-arg client auto-reads ANTHROPIC_API_KEY, which matches the repo's existing pattern.

Evidence.

  • The repo reads env directly through small pure resolvers, not a config module: apps/api/src/config/port.ts:1-4 (resolvePort(env) off env.PORT), consumed at apps/api/src/main.ts:31 (resolvePort(process.env)); the MCP token is read the same way at apps/api/src/mcp/mcp.guard.ts:34 (resolveMcpToken(process.env)). A grep for @nestjs/config across apps/api/src returned no matches (recon, 2026-08-20).
  • SDK auto-read (claude-api skill → typescript/claude-api/README.md, Client Initialization): a zero-arg new Anthropic() resolves credentials from ANTHROPIC_API_KEY (then ANTHROPIC_AUTH_TOKEN, then a CLI profile).

Caveat. The deploy step (setting ANTHROPIC_API_KEY as a Render secret) is out of this file's scope; docs/architecture/deploy.md owns the deploy shape. The plan should add a resolveAnthropicKey(process.env) that mirrors port.ts and fails fast at boot when the key is missing, so a mis-provisioned deploy errors on startup rather than on the first /assistant/ask.

V3 — What latency/cost does the endpoint inherit from RankingService.rank()? ​

Question. R4/R8: the closest_to_craft capability calls rank(), which the MCP tool description (mcp.tools.ts:239) already flags as expensive. How slow, and what does the endpoint inherit?

Verdict. Confirmed — expensive, and it is a floor, not a ceiling. rank() fans out one account read per character (via getOwnedItems), measured at ~31 s on a 19-character account in spec 017's discovery; the assistant endpoint inherits that latency on a cold cache, then is fast on repeat because the account reads are cached.

Evidence.

  • RankingService.rank() calls this.account.getOwnedItems(apiKey) (apps/api/src/legendaries/ranking.service.ts:118).
  • Measurement: spec 017 research timed getOwnedItems end-to-end — "31 seconds, and that is the floor, not the figure" on a 19-character account (specs/017-mcp-server/research.md:567-578).
  • Cache behaviour: the MCP tool description states a retry within five minutes is usually fast because the account reads are cached, and a server restart clears that cache (apps/api/src/mcp/mcp.tools.ts:239).

Caveat. The 31 s figure is spec 017's spike on one account, not re-measured this session, and it timed getOwnedItems only (the ranking arithmetic is additional). The endpoint's single LLM parse call is cheap by comparison, and the SDK's default request timeout is 10 min, so the model call is not the bottleneck — the account fan-out is. A "still working…" UX and an explicit client/Render request timeout are worth a note in the plan, but bounding rank() itself is a spec-017 concern, not this feature's.

F4 — The SDK's zodOutputFormat removes a dependency the spec implied ​

Turned up while resolving V1. The spec (R3, R8's [NEEDS VERIFICATION]) hedged that a "reliable zod→json-schema derivation" was needed. It is not: @anthropic-ai/sdk/helpers/zod exports zodOutputFormat, which the SDK feeds into output_config.format and validates for you. zod is already in the dependency graph (ranking.schema.ts), so the only new dependency is @anthropic-ai/sdk.

Why it matters. Simplifies R3/R8 and the plan's dependency step — one package to add, one Zod source of truth for both the response DTO and the model's output schema, no schema-derivation library to vet.

F5 — The parse model is a plan-time decision with a real cost/latency lever ​

Turned up while resolving V1. The spec says "a small/fast Claude, fixed in plan.md." The claude-api skill is firm that the default is claude-opus-4-8 and that downgrading for cost is the human's call, not the agent's. For a bounded intent classification, claude-haiku-4-5 is a legitimate choice — cheapest, lowest latency, and confirmed to support the structured-output mechanism (V1) — and the human endorsed "small/fast" in brainstorming.

Why it matters. Touches R8. The plan must pin an exact model id; record it there as a human decision (recommended claude-haiku-4-5 for a cheap, low-latency classification; claude-opus-4-8 if parse quality on adversarial phrasing proves insufficient in the eval). It is not baked as a hard requirement here so the choice stays visible and reversible.

Refuted claims ​

None. V1–V3 are all confirmed; F4/F5 refine the plan without contradicting a spec premise, so the spec does not return to step 1.

Graduation ​

Candidates to move to docs/architecture/ at step 6 (so the next LLM-touching spec does not re-discover them):

  • The @anthropic-ai/sdk integration shape — env key via a process.env resolver that fails fast at boot; structured output via client.messages.parse() + zodOutputFormat over a shared Zod schema; the injectable, mockable LLM-client boundary. A short docs/architecture/llm.md, or a section in docs/architecture/nestjs.md.
  • The assistant seam — free-text in, typed blocks[]/envelope out; the LLM as a swappable front door over a deterministic capability layer. Worth a durable note once the block-union spec lands.