ABLITAI LAB — DRAFT PROTOCOL
PROPOSED STUDY — NO RESULTS

When a longer context stops being useful

Question
Can a model use the context it accepts?

Setup
Fix the model and its exact revision, tokenizer, serving runtime, GPU, cache settings and generation settings. Record the context allowance and reserve the same output budget for every comparable case.

Proposed method
1. Create fixed documents with facts placed near the beginning, middle and end. Keep a reference answer for every question.
2. Repeat comparable cases at increasing input lengths. Include questions that need evidence from more than one section.
3. Include missing-information controls. A correct response should acknowledge when the answer is absent.
4. Preserve the full outputs and score correctness separately from first-token delay, generation speed and memory use.

Measures
- Answer correctness and source grounding
- Recall by fact position and input length
- Time to first token and output tokens per second
- Peak memory and request failures

Limitations
A small set of synthetic documents cannot establish performance on every real workload. Exact-match recall, multi-step reasoning and code work need separate reporting. Changing the model, quantization or runtime invalidates direct comparisons.

Next step
Freeze the evaluation cases and reference answers before collecting a publishable result.