Clinescope Build Log

Day 1: Diff-quality scoring: the metric I killed before shipping

I built a diff-quality scorer no eval framework ships, then proved my own minimality metric rewards the wrong behavior on real patches.

Day 2: LLM-as-judge validation: the number I let get worse

I validated my eval tool’s own LLM judge against human labels, watched a bigger honest set drive the number toward chance, and kept the worse number out of the gate.

Day 3: Validation corpus: the failure mode no free model would produce

I validated my scorers on real captured agent failures, and shipped the one failure mode I couldn’t capture as a documented gap instead of faking a trace.

Day 4: Adversarial re-checks: the blind spot they inherit

My tool passed every test and an adversarial re-check, and was still broken from a clean shell, because the re-check ran in the same primed environment the tests did.

Day 5: Zero dependencies: what it cost to keep the tree empty

I held clinescope to zero runtime dependencies through statistics, an LLM call, and a PyPI release, and the last decision was to stop building.

Day 6: Integrating Clinescope to VS Code extension

I pointed clinescope at a real VS Code extension session.