<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
  <title>Singular State</title>
  <link>https:&#x2F;&#x2F;singular-state.com</link>
  <description>Advanced technical expertise for feasibility, prototyping and modernization. We build the working pilot and ramp up your team to take it forward.</description>
  <language>en-US</language>
  <atom:link href="https://singular-state.com/topics/evaluation/rss.xml" rel="self" type="application/rss+xml"/>
  <item>
    <title>Four reconstructions, one result: the defects land where no check reaches</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;experiments&#x2F;four-experiments-in-ai-assisted-reconstruction&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;experiments&#x2F;four-experiments-in-ai-assisted-reconstruction&#x2F;</guid>
    <pubDate>Wed, 09 Sep 2026 00:00:00 +0000</pubDate>
    <description>A scanned NASA specification, a novel, the memory of a product, and a sealed algorithm. What four controlled rebuilds establish about where AI-assisted reconstruction of legacy systems actually fails.</description>
  </item>
  <item>
    <title>Passing the tests is not understanding the system: a sealed-key measurement</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;experiments&#x2F;grading-a-reconstruction-against-a-sealed-answer-key&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;experiments&#x2F;grading-a-reconstruction-against-a-sealed-answer-key&#x2F;</guid>
    <pubDate>Wed, 09 Sep 2026 00:00:00 +0000</pubDate>
    <description>A compressor reconstructed from behaviour alone, graded against an answer key sealed until the work was frozen. The decoder scored 100 percent and the encoder 8.3 percent on the same format.</description>
  </item>
  <item>
    <title>Rebuilding the Space Shuttle&#x27;s flight language from a 1980 scan</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;experiments&#x2F;hal-s-reconstruction-from-a-degraded-spec&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;experiments&#x2F;hal-s-reconstruction-from-a-degraded-spec&#x2F;</guid>
    <pubDate>Wed, 09 Sep 2026 00:00:00 +0000</pubDate>
    <description>HAL&#x2F;S runs again: twelve preserved Shuttle-era programs, verified value-for-value against an independent interpreter. The load-bearing human work was refusing a majority vote between OCR engines.</description>
  </item>
  <item>
    <title>Turning a novel into a specification: the Nautilus under real physics</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;experiments&#x2F;verne-nautilus-simulator-narrative-as-specification&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;experiments&#x2F;verne-nautilus-simulator-narrative-as-specification&#x2F;</guid>
    <pubDate>Wed, 09 Sep 2026 00:00:00 +0000</pubDate>
    <description>Verne&#x27;s submarine crushes at 340 metres against a narrated 16,000. Extracting a falsifiable spec from fiction, and what it takes to trust numbers an AI produced when no external oracle exists.</description>
  </item>
  <item>
    <title>Acceptance criteria for a system that does not repeat itself</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;acceptance-criteria-non-deterministic-systems&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;acceptance-criteria-non-deterministic-systems&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>A fixed seed makes a best effort, not a guarantee. Acceptance shifts from exact output to declared task properties, a predeclared sampling rule, and uncertainty measured at the unit actually sampled.</description>
  </item>
  <item>
    <title>The benefit decays before the transfer completes</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;benefit-decays-before-transfer&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;benefit-decays-before-transfer&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>Month-one and month-nine measurements are different quantities. A gate reading the early number transfers a system whose case has already expired, and time released is capacity rather than cash.</description>
  </item>
  <item>
    <title>Developers were wrong about their own speed, and the measurement problem generalises</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;developers-wrong-about-their-own-speed&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;developers-wrong-about-their-own-speed&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>Forecast 24 percent faster, felt 20 percent faster, measured 19 percent slower. The finding is about self-reported productivity as an evidence class, and it disqualifies most of what pilots collect.</description>
  </item>
  <item>
    <title>The evaluation set is the asset that survives the kill</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;evaluation-set-survives-the-kill&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;evaluation-set-survives-the-kill&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>Killing a prototype destroys the system and preserves the encoded definition of correct behaviour. Salvage is a forward comparison, not a rescue justified by what was already spent.</description>
  </item>
  <item>
    <title>Falsifiable success criteria</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;falsifiable-success-criteria&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;falsifiable-success-criteria&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>A success criterion is falsifiable when a stated observation would end the project. Most criteria have two states and need three, because inconclusive is the result that actually occurs.</description>
  </item>
  <item>
    <title>Golden, adversarial, and regression sets</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;golden-adversarial-regression-sets&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;golden-adversarial-regression-sets&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>Three purposes, not three folders. One case can carry all three tags and still be one case family, and three passing tags are not three independent successes.</description>
  </item>
  <item>
    <title>LLM-as-judge, and what calibrating it against humans actually requires</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;llm-as-judge-calibration&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;llm-as-judge-calibration&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>A judge is a measurement component with order sensitivity and no established authority. Swap the presentation order, and a judge that picks the first answer both times has produced two conflicting verdicts.</description>
  </item>
  <item>
    <title>Masked data, real conclusions: the fidelity gap in your evidence</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;masked-data-real-conclusions&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;masked-data-real-conclusions&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>Every de-identification transform removes signal the model was going to use. What a pilot on treated data can and cannot establish, stated as a rule rather than a caveat.</description>
  </item>
  <item>
    <title>Parallel prototyping changes selection, not construction</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;parallel-prototyping-changes-selection&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;parallel-prototyping-changes-selection&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>When three attempts cost what one used to, the gain is in comparison. The scarce resource becomes the ability to choose between working options, which is an evaluation problem.</description>
  </item>
  <item>
    <title>Your pilot cannot detect the effect you are looking for</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;pilot-cannot-detect-the-effect&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;pilot-cannot-detect-the-effect&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>Cohorts are sized for access, not for power. Where the minimum detectable effect exceeds the claimed benefit, the pilot cannot produce the evidence its business case assumes.</description>
  </item>
  <item>
    <title>Retrieval, fine-tuning, prompting: which problem each one solves</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;retrieval-fine-tuning-prompting&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;retrieval-fine-tuning-prompting&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>Three intervention points, not three exclusive strategies — the original RAG paper already combined two of them. Diagnose the failure before choosing the technique.</description>
  </item>
  <item>
    <title>Shadow mode</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;shadow-mode&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;shadow-mode&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>Running a candidate against live inputs with its outputs withheld. The word does not establish the boundary — shared caches, writable destinations and quotas escape it unless someone checked.</description>
  </item>
  <item>
    <title>Synthetic, masked, sampled, production: what each data strategy costs you in evidence</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;synthetic-masked-sampled-production-data&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;synthetic-masked-sampled-production-data&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>Not four rungs of one ladder. Three are construction choices that overlap and one is an operating context, and synthetic does not mean anonymous.</description>
  </item>
  <item>
    <title>The evaluation harness is the deliverable</title>
    <link>https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;the-evaluation-harness-is-the-deliverable&#x2F;</link>
    <guid isPermaLink="true">https:&#x2F;&#x2F;singular-state.com&#x2F;articles&#x2F;the-evaluation-harness-is-the-deliverable&#x2F;</guid>
    <pubDate>Mon, 07 Sep 2026 00:00:00 +0000</pubDate>
    <description>The system can be rebuilt; the encoded definition of working cannot. Ship the harness first, and test it by making a deliberately wrong answer fail.</description>
  </item>
</channel>
</rss>
