In this field report
Romeo's local model configuration changed substantially during our agent work. The earlier arrangement served a Q4 dense model alongside a separate coding model. The newer configuration serves a Q8 model with a loaded context of 262,144 tokens. That change exposed a practical problem: a model can be present in the server while an agent still requests an obsolete identifier.
This is a configuration field report, not a Q4-versus-Q8 benchmark. Reported generation rates varied across sessions, but those observations did not hold workload, context, runtime and concurrency constant. We cannot assign the difference to quantization alone.
Start with the serving identity
The old configuration requested qwen3.8-27b@q4_k_m. The local server's newer exposed identifier was qwen3.8-27b. With just-in-time loading disabled, the obsolete request failed instead of selecting the already-loaded model.
Those strings are local serving identifiers. They are not evidence of an upstream model release, architecture or supported context specification. A reproducible configuration must also identify the actual weight file and runtime. A friendly model name is useful for navigation, but inadequate as a benchmark identifier.
The repair had to account for more than a new-chat default. Agent profiles, provider metadata, auxiliary model assignments and saved-session configuration can retain different values. Updating one dropdown does not establish that every request path now resolves correctly. Our migration tooling therefore treated model assignments and context metadata as explicit configuration changes, with backed-up state and drift checks.
The practical starting point is a read-only model inventory, followed by comparison with the identifier the failing client actually sends. LM Studio's model-list documentation describes the compatible endpoint. A successful list response proves reachability and an exposed identifier. It does not prove that a text or image request will finish.
Separate load settings from request settings
Context length and model offload concern the loaded instance. Temperature, sampling and output bounds concern the request. Treating them as one undifferentiated settings panel makes troubleshooting harder.
LM Studio supports persistent defaults for individual models, including context size and GPU offload. These defaults apply on subsequent loads and can be overridden when loading. They are useful for reproducibility, but they should not be confused with a change already applied to a running instance. Per-model defaults
The native load API can return the final applied configuration. That is better evidence than a screenshot of the intended values: record what the instance actually accepted. Its documented settings include context length, evaluation batch size and GPU KV-cache offload. Native model loading
These controls represent different limits. The illustration is not a recommended universal preset.
Context is capacity, not a promise of responsiveness
The September 14 metadata check reported a loaded Q8 instance with a 262,144-token context. This establishes an allocated configuration, not an evaluation of answer quality at every position in a full-length conversation.
A large context does not mean each request contains that many tokens. Nor does it mean the agent will never summarize its history. The client can have its own context policy and output reservation. The server's loaded capacity and the agent's conversation policy must be examined separately.
Romeo has two 32GB R9700 cards and 128GB of host RAM. That is substantial capacity, but aggregate GPU memory is not a single automatically shared device. A settings value does not establish where the weights and cache reside. To claim a particular two-card placement, retain the loaded-instance configuration and per-device residency evidence.
It is reasonable to design a shorter-context everyday profile and a longer-context investigation profile. It is not reasonable to claim that the shorter profile is faster on this machine until both are tested with the same tasks. The proposed profiles are an experiment, not a published performance finding.
Output budgets are not timeout budgets
Our current auxiliary image route was configured with a 1,024-token output cap, thinking disabled and a 120-second HTTP timeout. These settings address different stages of the operation.
The output cap bounds generated analysis. It does not bound queue delay or guarantee fast image processing. The HTTP deadline limits how long the client waits. A timeout does not by itself prove whether the server was queued, processing input, generating slowly or stalled. It also does not establish that server-side work was cancelled.
That distinction appeared in actual tests. A small neutral image succeeded through the registered handler in 31.8 seconds. The next, larger neutral image timed out after 121.43 seconds. We retained those as a success followed by an unresolved failure, not as proof that vision was reliably repaired. The earlier model-routing error was corrected; repeatable inference remained a separate check.
Measure the useful answer
For an agent, the endpoint of interest is a usable result: a completed answer, an executable patch or a verified tool action. The first streamed token is not that endpoint.
A profile comparison should retain the artifact identity, runtime/backend, applied load configuration, prompt size, sampling settings, reasoning configuration, output cap and request concurrency. Run the same neutral tasks across profiles. Separate time to first output from time to final content, and grade the final deliverable independently.
Keep provider-reported generation timing distinct from client elapsed time. If a request times out without a final result or usage totals, record the timeout. Do not manufacture a throughput result from an incomplete stream.
The configuration outcome
The useful lesson from this migration is not that Q8 or maximum context is universally better. It is that a local agent needs an explicit serving contract: the correct identifier, the actual loaded configuration, bounded requests and a defined successful result.
Our current observations support a Q8/262,144-context configuration case study and an explanation of the stale-identifier failure. They do not support a controlled quantization ranking, a guaranteed latency improvement or a claim that all image requests now succeed. Those remain experiments to run and report on their own terms.