Modeling Guide
Knowledge Guides /How to Verify MBSE Validation Rules Without Mistaking Zero Findings for Success
Last updated: · New Horizon Digital Outsourcing Team
A model with no findings may satisfy the rules, or its rules may be disabled, its elements excluded, or its checks failing to run. Deliver validation rules with their target elements, activation manifest, and examples with known expected results. Report passes, exclusions, skipped checks, and execution failures separately.
Establish What Each Rule Actually Covers
- Define a stable identifier, version, applicable element types, scope, and activation conditions for each rule. Importing a rule package does not establish that it is active in the current project; record the suite actually used.
- Identify language constraints, organizational conventions, and project requirements by source. Keep customer rules out of a shared rule library until their applicability is reviewed, and explain severity assignments rather than relying solely on tool defaults.
Check the Checker with Known Outcomes
- Prepare a minimal model that should pass and another that should produce a finding, recording the specific difference that triggers it. Add an out-of-scope element to confirm that the rule does not misclassify inapplicable elements.
- Confirm the expected outcomes manually before running the checks, then compare element identifiers, finding locations, and explanations. A matching finding count is insufficient: findings may refer to the wrong elements, and execution errors cannot count as passes.
Make Unexecuted Checks Visible
- Report planned, executed, and unexecuted rules with reasons for each status. Distinguish missing dependencies, expression parsing failures, empty scopes, and approved exemptions instead of presenting them all as no findings.
- For exemptions, record affected elements, approval rationale, and review conditions. Reassess them when the model or rule version changes. Present execution completeness alongside model findings so reviewers can judge the reach of the conclusion.
Deliver Evidence for Rule Upgrades
- After a rule changes, compare results on the same examples and explain new, removed, or reclassified findings. If expected outcomes change, document the engineering reason before updating the examples rather than changing answers merely to obtain passes.
- Deliver the rule package, scope, example models, execution environment, and change record so the recipient can repeat a check. Passing these examples supports conclusions about the covered checking logic, not about model physics or product performance.