OpenResponses Translation Conformance

by hand — edit tests/conformance/openresponses/manifest.yaml and regenerate. –>
On this page

This suite runs the external OpenResponses conformance oracle against the responses_to_chat_completions translation filter — never against native Responses passthrough.

Coverage

7 / 7 in-scope templates passing (100.0%). 10 of 17 suite templates are out of scope and excluded from the denominator.

Coverage counts templates the translation filter is responsible for (supported + unsupported). Fixing a translation gap promotes an unsupported template to supported and bumps this number.

  • Supported (7): basic-response, system-prompt, multi-turn, assistant-phase, tool-calling, streaming-response, response-output-phase-schema
  • Unsupported (0):
  • Inapplicable (10, excluded): image-input, compact-response, compact-missing-model, websocket-response, websocket-sequential-responses, websocket-continuation, websocket-reconnect-store-false-recovery, websocket-previous-response-not-found, websocket-failed-continuation-evicts-cache, websocket-compact-new-chain

Per-template rationale lives in the triage manifest (tests/conformance/openresponses/manifest.yaml); the launcher fails if any suite template is missing from it.

What runs

A Praxis listener loads the shared examples/configs/openai/responses/responses-to-chat-completions.yaml example, which translates POST /v1/responses into POST /v1/chat/completions and forwards to a Chat-Completions-only backend (vLLM CPU, Qwen/Qwen3-0.6B). A translation-witness shim sits between Praxis and vLLM and asserts the backend only ever receives POST /v1/chat/completions (criterion f).

Pins

  • Suite commit: 92c12d96d7b61d6d15e2214daa5e9c6000ab6e1c
  • Runner: bun 1.3.12, zod 3.25.76
  • Backend: vLLM CPU, Qwen/Qwen3-0.6B

Running locally

make test-responses-conformance

Requires bun, uv, and a reachable vLLM backend (VLLM_BASE_URL, default http://127.0.0.1:8000).