Result statuses

The strict distinction between PASS, FAIL, and ERROR, and why it matters.

A RestorSignal result is never reduced to a vague "it worked" or "it didn't work." Every individual control receives one of four strict statuses, defined by the RS-RVP-01 reference framework.

PASS

The expected condition is demonstrated.

FAIL

The control ran correctly and demonstrates that the expected condition is not satisfied. A FAIL is a clean result: the test worked, and it is telling you something is wrong.

ERROR

The control could not correctly determine the expected state — for example because of a timeout, a test infrastructure failure, an unavailable dependency, an execution exception, or an inconsistent environment. An ERROR says nothing about the actual state of what was being checked: it only signals that the test itself could not reach a conclusion.

NOT_APPLICABLE

The control does not apply in the context of this test, and this status is explicitly allowed by the protocol in question. NOT_APPLICABLE cannot be used to sidestep an inconvenient control: its use is governed by the protocol's own definition.

From control to level

These four statuses apply to an individual control. A level (L1 through L5) groups several controls together and itself receives a status among PASS, FAIL, INDETERMINATE, NOT_EVALUATED, following a strict roll-up rule:

• a mandatory (required) control at PASS contributes positively to the level;
• a mandatory control at FAIL puts the entire level at FAIL;
• a mandatory control at ERROR puts the entire level at INDETERMINATE;
• a mandatory control at NOT_APPLICABLE is only allowed if the protocol explicitly provides for it.

A level is only positively obtained if 100% of its mandatory controls are PASS. A single ERROR on a mandatory control is enough to prevent the level from being positively obtained, even if every other control is PASS: the result becomes INDETERMINATE, never a partial success presented as achieved.

ERROR is never equivalent to PASS. An INDETERMINATE result must never be absorbed into a global summary that gives the impression everything went fine. If RestorSignal could not determine whether a mandatory control was satisfied, that uncertainty stays visible in the result — it is neither hidden nor rounded up to a success.