01 / THE QUESTION
Does API compatibility produce a usable coding workflow?
A client connecting successfully is different from a complete coding workflow. Tool calls, context accounting, streaming and compaction all need to behave consistently across the client and model server.
02 / SETUP
Make the conditions explicit.
Pin the client and API adapter versions, model identifier, context limit, output allowance and supported capabilities. Use an isolated fixture repository with no credentials or private data.
03 / PROPOSED METHOD
A repeatable path.
- Start with a plain text response, then verify streaming completion and cancellation.
- Run a harmless, clearly specified tool task only when both the client and server support tool calling.
- Use a fixed multi-turn conversation to observe context accounting and compaction behavior.
- Inspect actual API messages and final files. Distinguish executed tools from commands merely printed in the response.
04 / WHAT TO MEASURE
More than a single number.
- Plain text and streaming completion
- Tool-call structure and completion
- Compaction and repeated-response behavior
- Final task outcome and reproducibility
05 / LIMITATIONS
What this would not prove.
Passing one harness does not establish compatibility with others. A client update, prompt change or different output allowance can change behavior. Chat-only mode must not be described as a fully supported coding agent.
06 / NEXT STEP
Before collecting evidence.
Create the version-pinned fixture and acceptance checks before publishing a compatibility claim.
Download draft protocol ↓