Why software validation is important

Ever thought of the numerical models used in background animation when streaming your favourite movie series? Or, checking the weather forecast? In 1981, the pilot of the Columbia Space Shuttle also didn’t during the maiden flight. He just was lucky.

Today, most technical applications are developed using numerical simulation methods. Depending on the application, efficiency improvement, cost and development time reduction are the main drivers. In Health and Safety Engineering, the accuracy of the models is crucial to assess the real potential impact of an incident. It directly helps the Health and Safety engineer to adapt his strategy and thereby increase safety.

Especially with the increase in computational power during the last decades, numerical modelling is flying high, and cross-fertilization between domains continuously improves the models. But how to ensure the models correctly rebuilt the reality of interest? This is of particular relevance in Health and Safety, where the absolute numbers of effect distances and risk matter more than, for example, visual effects.

This is where the validation process comes into play. Numerical models simplify reality; usually, they only model the relevant physical processes. Different mathematical models are chosen and implemented in a software code as a computer model depending on the problem. A verification process assesses the software implementation to ensure the correct coding and elimination of errors.

The importance of validation processes and validation hierarchies

For the user, the simulations have to match the reality as close as possible. A validation process comparing experimental data with the simulations helps realize this ambition. Usually, a single comparison cannot validate a complete system model but rather a series of data, including all relevant physical effects and data from different components.

Therefore, a validation hierarchy should be considered, including the following:

• The complete system, such as dispersion or explosion at storage tanks.
• The subsystems and components, such as light, heavy or neutral dispersion of released material.
• Single physical effects, such as heat transfer into the plume or phase change.

For the whole hierarchy, dedicated experimental data has to be compared with the simulation results to validate the numerical model. In the end, it comes down to chopping the problem into pieces and validating all relevant physical effects.

Within the validation hierarchy, suited experimental data is considered, ranging from small-scale testing in a lab over medium-scale testing in a wind tunnel to full-scale testing on an experimental site. Multiple sets of experimental data are mandatory for a reliable validation procedure since every experiment targets specific features, has data gathering limitations due to size and is influenced by external factors, such as changing weather conditions. In addition, the uncertainty of the experimental data set, such as time-varying conditions during the experiment, has to be addressed.

The numerical and experimental data results are usually compared by statistical methods for all available data points. By doing this, you can achieve a rapid overview of the model performance over many data points and different experiments. The data is usually plotted in a scatter plot with experimental and numerical data (see figure below).

Especially with new or not commonly used materials or new applications, available experimental data sets might not be sufficient. Suppose you use the models outside the validated regime. In that case, consider that the model will produce an extrapolated result from the validated regime and might neglect physical features.

Continuous model improvement

At Gexcon, we continuously adapt and improve our models based on new situations, requirements and datasets. When we develop models for a certain application or range of materials in the first instance, we upgrade for new materials, applications or regulatory requirements. Apart from an extensive experimental database, we also apply higher sophisticated models, such as Computational Fluid Dynamics (CFD) to improve the standard models.

In addition, we have access to the Gexcon experimental facilities ranging from small- to large-scale testing to perform tailor-made experiments.

We also collaborate worldwide in large model validation campaigns such as the Jack Rabbit Program of the US Department of Homeland Security. These collaborations ensure thoroughly validated numerical models of Gexcon, which form the essential tools for the Health and Safety Engineer daily. Together we make the world a safer place.

Validating numerical models

Still wondering why the pilot of the Columbia Space Shuttle was lucky?

The numerical models used for the design and the tests in the wind tunnels did not include real gas effects. The body flap deflection angles ensuring a stable flight were almost double the simulated ones, just below the safety factor of 2 imposed by the design engineer. 

Today, real gas effects are well known and play a role in Health and Safety Engineering, but new technologies are still operating at the limits. The numerical models used have to be validated for that. As a rule of thumb, at least within a factor of two.