Lab Notes
Our AI Agent Used the Right Tool—and Still Got the Answer Wrong
A hands-on NeMo Relay and Arize Phoenix field report: a successful tool call, a wrong final answer, and the privacy checks needed before tracing live agents.
Benchmark Command LLC
Independent local-AI field reports
Local AI.
Real hardware.
Clear evidence.
Inside the systems, software and practical work behind local AI. Explore our Intel and AMD builds, read the field reports, and see how we separate observations from conclusions.
Settings, versions and limitations matter as much as the headline result.
The physical lab
A closer look
at the hardware.
Two builds, photographed in our workspace. The AMD system pairs two Radeon cards; the Intel system houses two Arc Pro B70 cards.
Original build photographs from Benchmark Command. Hardware pictured is not, by itself, a performance claim; test conditions and results belong with each report.
Published work
Latest field reports
Explore published field reports on local AI, workstation hardware and the details that shape real-world results.
Explore by subject
Local AI benchmarks
Inference profiles, measured model behavior and the limits behind the results.
Local AI Benchmarks
RTX 3090 GPU Unlock: Our Stock–Unlock–Stock Results at 300 W
Our RTX 3090 stock–unlock–stock dense GEMM test at a 300 W cap measured BF16 +57.3%, FP16 +51.4% and TF32 +60.2%; strict FP32 did not improve. Raw…
Guides
Glimmer Q8 on Two B70s: It Loaded, but Delivered No Code
Sentinel ran a verified Q8 Glimmer instance across two 32GiB B70s. Three neutral one-shot coding tests returned no final code, exposing the gap between load success…
Guides
LM Studio Profiles: Quantization, Context and the Time to a Useful Answer
Lessons from migrating a local agent host to Q8 and a 262,144-token context, including why model identifiers, response limits and latency must be tested separately.
Guides
Running Qwen3.6-27B Locally: A Complete Inference Guide
The Qwen3.6-27B model occupies a specific niche in the local LLM landscape: it is dense enough to outperform most 14B-class models on coding and reasoning benchmarks,…
Explore by subject
Hardware builds
The workstation configurations, components and practical decisions behind local AI.
Explore by subject
Fleet & lab notes
Monitoring, agent reliability and the private-lab infrastructure behind the work.
Guides
Building Fleet Command: What Online Actually Means
How we repaired a private lab dashboard by separating machine identity, service reachability, fresh telemetry and explicitly authorized controls.
Guides
Versioned agent context: Markdown facts, receipts and a pointer-last handoff
A dated Markdown knowledge system separates facts from plans and history, generates bounded context, and verifies a staged handoff without pretending it trains a model.
Guides
An alive process, an unresponsive agent: the Hermes regex outage
A September 6 recovery traced an unresponsive Hermes backend to one costly approval regex, then repaired its scan behavior without weakening the policy.
Evidence before conclusions.
The editorial treatment distinguishes what was measured, what was inferred, what remains planned, and where readers can inspect the reporting standard.
STEP 01
Record
Capture the workload, settings, versions, and observed output.
STEP 02
Repeat
Use consistent conditions and retain individual runs instead of a lone headline number.
STEP 03
Bound
Label observations, inferences, plans, and unresolved checks separately.
STEP 04
Review
Keep staging and publication distinct so evidence can be checked before release.