Machine Learning in Civil Engineering - Surrogate Models

A quick introduction from Seán

Welcome to the second tutorial in our Machine Learning in Civil Engineering series. In this tutorial, Ehsan Es'haghi we'll explore the predictive power and computational efficiency of Surrogate Models. If you haven't done so already, check out the first tutorial in this series on sensitivity analysis and layout optimisation. That said, you can still read this tutorial as a standalone piece.

Traditional numerical analysis methods like Finite Element Analysis (FEA) require solving large systems of equations, which becomes computationally expensive for large-scale problems. Here, we introduce the fundamental concept of a surrogate model, an approximation technique that reduces computational time by predicting outcomes without needing to solve the equations directly.

📂 To follow along, make sure to download the Jupyter Notebook (linked above) to run locally as you read through the tutorial.

Dr Seán Carroll CEng MIEI

Founder of EngineeringSkills.com

1. Introduction

In the previous tutorial in this series on sensitivity analysis and layout optimisation, we examined an efficient method for computing the derivative of compliance with respect to element sizes. Despite using advanced sensitivity analysis techniques like the adjoint method, we still face the challenge of solving $N$ equations with $N$ unknowns in each iteration, which has $O(N^3)$ computational complexity.

As the number of nodes in the structure increases, this can become impractical. Additionally, while global compliance is a popular objective due to its simplicity, it is often impractical for real-world problems.

Moreover, optimising a structural design might involve relocating nodes, changing joint types, and other modifications. Taking derivatives of even a straightforward objective like global compliance with respect to these variables is highly complex.

To address these challenges, we will explore a machine learning approach based on developing a so-called surrogate model to estimate solutions to this system of equations more efficiently.

2. Formal Problem Definition

Imagine you are presented with a static load vector $F$ and a structure characterised by a stiffness matrix $K$. Calculating nodal displacements within a structure, involves solving $N$ equations for $N$ unknowns, where $N$ represents the product of the degrees of freedom, with the matrix $K$ having dimensions $N \times N$.

The most formidable aspect of this task is matrix inversion, $U = K^{-1}F$, a process with a computational complexity of $O(N^3)$. This implies that doubling the number of nodes results in an eightfold increase in computational operations!

Finite Element Analysis (FEA) software, which can feature thousands or even millions of nodes, utilises advanced linear algebra techniques to efficiently compute this inversion. These techniques capitalise on certain properties of stiffness matrices:

1. Most entries in a stiffness matrix are zero, making these matrices sparse.
2. Given linear elastic materials and isotropy, the stiffness matrix can be assumed to be positive definite.

These characteristics allow for the optimisation of the matrix inversion operation, significantly enhancing both computational and memory efficiency. This enables simulations of building deflection involving millions of nodes to be performed in under 15 minutes on a standard laptop.

Another approach to reduce analysis time in mesh-based FEA software is to increase the mesh size, thereby reducing the dimension of $K$ but at the cost of losing accuracy. This presents a constant trade-off between accuracy and efficiency. In this tutorial, we will explore a different approach using the burgeoning field of machine learning.

As we discussed, $K^{-1}F$ is a function that inputs matrix $K$ and vector $F$ and returns the nodal displacement vector $U$. The challenge with this function is its rapidly increasing computation time as $K$ enlarges.

Throughout this series of tutorials, we will aim to design a function with a much lower time-complexity than $O(N^3)$ while still providing an acceptable approximation. Such a function is known as a surrogate model. Here is a concise yet suitable broad definition of a surrogate model from Wikipedia which nicely articulates the concept:

A surrogate model, also known as a metamodel or a response surface, is an engineering method used when an outcome of interest cannot be easily measured or computed directly, necessitating an approximate model or a substitute for it.

This tutorial will focus on applying machine learning techniques to this problem. To begin, we'll cover some machine learning fundamentals. The next tutorial in this series will introduce Artificial Neural Networks (ANNs), particularly Graph Neural Networks (GNNs), which are well-suited for our specific challenges.

While promising, it's crucial to acknowledge that this research area is not fully mature, and surrogate models do not yet match the precision of traditional FEA methods. Our exploration will illuminate both the potential and the current limitations of machine learning in this exciting interdisciplinary domain.

In the next section, we’ll explore the development of a surrogate model by considering the behaviour of a simple truss under the action of a single point load.