Simulation Guide
Knowledge Guides /How do you assess simulation credibility, and what evidence is needed for verification and validation?
Last updated: · New Horizon Digital Outsourcing Team
Credible simulation conclusions must be traceable to input sources, model assumptions, solver settings, and comparison evidence. Whether code or models implement the intended behavior is a different question from whether the model adequately supports a real-world decision. Realistic visuals, smooth curves, or a single successful run cannot individually establish credibility.
Define which decision the simulation supports first
- Specify the options to compare, metrics of interest, and applicable conditions, such as peak temperature, task completion time, or coverage intervals. Metrics need definitions, units, and statistical windows; high accuracy and fast operation are insufficient descriptions.
- Record simplified factors and unmodeled boundaries. If a model is intended only to rank options, do not treat that ranking as a guarantee of the final product's absolute performance.
Check implementation and numerical settings
- Use small, interpretable cases to check directions, dimensions, conservation relationships, and boundary behavior, and perform appropriate sensitivity checks for mesh size, step size, or solver tolerances. Retain raw logs when anomalies occur, rather than only selected attractive curves.
- For stochastic simulations, record random seeds, sample counts, and statistical definitions. Repeat runs or check independent conditions supporting the same conclusion to avoid presenting one favorable result as overall performance.
Can a project proceed without experimental data?
- Models for exploration and comparison can be built first, but their validation level must be stated. Analytical solutions, public benchmarks, or independent models can provide cross-checks; this evidence does not replace experimental calibration under actual operating conditions.
- Set reasonable ranges for key uncertain parameters and analyze how results vary with them. Project reports should identify which missing data most affects decisions, helping plan subsequent tests and sampling.
Recommended acceptance package
- Include input inventories, models and versions, run configurations, benchmark cases, error or discrepancy analyses, and the limits of applicability of conclusions. Recipients should be able to rerun key results, rather than merely play screen recordings.
- Retain a risk list and recommendations for further validation of unverified elements. Contracts should distinguish delivery of simulation projects from achievement of physical-equipment performance targets, preventing different understandings of what is being accepted.