Simulation Guide
Knowledge Guides /How many runs does a stochastic simulation need, and how should uncertainty be reported?
Last updated: · New Horizon Digital Outsourcing Team
There is no fixed run count suitable for every stochastic simulation project. Start with pilot runs, examine variation in the metrics of interest and tail events, then set the budget according to acceptable uncertainty. Reporting one run or a single average can hide failure probabilities and extreme outcomes.
Distinguish random inputs from unknown parameters
- Model runtime random disturbances separately from parameters uncertain because information is lacking. Distribution shapes, ranges, and correlations between variables need a basis; do not default to independent uniform distributions for convenience. For inputs without evidence, use scenario comparisons to explain their impact.
How should batch runs be organized?
- Record seeds, configuration identifiers, model versions, and exit statuses. Use matched conditions where possible when comparing options, and check sample independence. Include failed runs in the statistical rules; simply deleting anomalous samples may bias both averages and success rates.
How can sample sufficiency be assessed?
- Observe whether means, percentiles, and target-event frequencies stabilize, and provide corresponding interval estimates. Rare events may require far more samples than routine metrics. If the budget is exhausted without supporting a conclusion, state that samples are insufficient rather than claiming an unobserved event cannot occur.
What should delivery include?
- Retain raw batch records, statistical scripts, filtering rules, and chart-generation methods. Reports should present sample counts, anomaly counts, metric definitions, and applicability conditions together. The project team should be able to trace any aggregate figure to its runs, rather than receiving only an average curve.