Models and black boxes

Black box or no Black box

Different perspectives of models.

A model is a representation of “anything” we can imagine, where we create the representation is open to our imagination. Anything can be objects, persons, systems  but it’s actually not limited to that. Us humans create models when we imagine things, create drawings, develop equations, … There are actually two major branches of modelling, black box and analytical models. In this article I will dive just a bit deeper into the subject.




Since humans got the capability of imagination, we were then able to represent mentally objects. Imagining a ball is very similar to creating a sphere in CAD. The big difference is the level of detail our imagination can hold and our ability to retain it with details. Since this could be a problem to most of us, our ancestors decided that they needed to represent their model in a medium that could be shared and detail could be added, and thus wall paintings in caverns started to appear. These drawings were one of the first models that humans created on a different medium than their mind, or on the ground. The level of detail could be higher and sharing the model easier and it could last longer. From there after humans created an endless number of models with the aim of representing reality to solve problems, share feelings and ideas. A model can be as detailed as possible or as needed and can be used as something to represent another object, to perform predictions of the future or even any point in time. However in most cases being a representation it will mean that some sort of simplification is a necessity, and thus the art of modelling is also the art of simplifying. A model needs only to serve it’s purpose for instance in some cases the geometry of a ball in detail or its behaviour when bouncing or the two if necessary, but there will be always some sort of simplification.

Hand drawing a ball involves capturing 3D information and translating its relevant features into a 2D space, doing so doesn’t actually require explaining what a circle/sphere is or any math involved, we take information measuring what we see and represent in another medium what was seen as a ball. This is called the black box modelling approach where we don’t care what’s inside just what we can actually measure/see. A good example is when we create a regression function based on data, measuring inputs and outputs allows us to build a model.

Modelling a ball

Ball model

On the other hand if we just use math and physics relations to develop for example in structural problems the equations of motion, we get a prediction of how a structure behaves. Assumptions will play an important role on how complex the equations will be. The Finite Element Methods or other similar ways is just a more fancy and systematic way of assembling those equations on complex geometries and solving the resultant differential equations.

Both approaches actually require us to solve a differential equation, however the source for building the model is different when you use an AR (Auto Regression) method which is based on data, then when we use the FEM (Finite Element Method), which is based on equations.



Neither cases are prefect. When it is difficult to explain a certain system or object through physics and Math a data approach is useful. Humans use senses to feed our brains with an enormous dataset that is gathered through years, artificial intelligence is a black box approach. Machine learning, is just one of the approaches that is data based, it is very good in image identification problems, data set analysis, and much more.


ARX (Auto Regressive Exogenous), ARMA (Auto Regressive Moving Average), ARIMA (Auto Regressive Moving Average), NARX (Nonlinear Auto Regression Exegenous) approaches allow us to represent a system behaviour through its output behaviour. They are all auto regression (AR) algorithms that predict the model, based on past data. 

These methods are not confined mechanical, thermal, … systems but even finance. Any situation when we have an output measurement in time at a specific rate and it’s input we have the necessary information to build an auto regressive method.

An important part of these methods are that the input needs to be relevant to the system measure output. This might seem easy but not every time we know the relevant inputs.

But how this works? Let us see an example of a mechanical system:

A basic model (mass - spring damper)

A basic model (mass – spring damper)

Measuring for instance the position of the mass relative to a coordinate system we can generate a model which captures the dynamics of the system. X will be the response of the system when an outer force excites the system. System identification is a set of tools that allow us to learn the model properties and their internal dynamical components (mass, spring and damper).

System Identification is broadly used in control systems theory and AR methods typically fit the following control systems:

Control System

Control System Feedback Loop

The system is what we want to learn with System identification. Having a prediction of the output position and feeding it to a control algorithm to predict a certain correction to achieve a defined goal.

ARX is based on linear regressions and can be seen in polynomial form:

ARX Symbolic representation

ARX Symbolic representation

A(q-1)y(k) = B(q-1)u(k)+e(k)

This model can describe any arbitrary linear relationship between inputs and outputs. Noise however enters in the model in a very specific way.

The explicit recursive representation can be seen bellow:

y(k) = - a1y(k-1)-...-anay(k-na)+b1u(k-1)+...+bnbu(k-nb)+e(k)

From here we will get linear equations that are necessary to discover the model parameters. It’s not the purpose of the article to describe in detail the process of developing and solving the model but to introduce the method and how it fits overall in system identification.


We however today use CAD (Computer Aided Design) and FEM (Finite Element Method),  to develop products and understand they’re physics. The main take on these methods is that they use math and physics for their formulation and are limited to our capacity to generate a complete mathematical representation of the object or system we want to study. This can be difficult when multi-physics is present, for different solvers need to be coupled in order to get a solution.

A CAD software uses math to describe the geometry. An equation can provide us all the points that make up a specific shape, take for instance a sphere and and the equation that describes it:

(x-x0)2 + (y-y0)2+(z-z0)2=r2

 The bodies in motion on a specific reference frame are described by the Kinematics equations that help model object motion (this is very useful in robotics). Here a set of equations define the position of the “End Effector” or commonly called the gripper. Either by direct kinematics where we define all angles to get the end effector position or by inverse kinematics where we know the location the end effector should be and calculate the specific joint angles to reach that objective. 

End effector kinematics

End effector kinematics

FEM based models are simply a numerical approach to solving complex statics, dynamics, thermal, electric, electro-magnetic, fluidic, and other problems, a more detailed explanation can be seen bellow for linear static structural analysis:

In black box system we discover the model from data, which is mainly the case when we don’t have for instance on mechanical systems the equations of motion. Deep learning actually can be used for system identification as deep nets as feedforward and cascade forward  nets are nonlinear ARX (NARX) models.

Actually auto regressive models can be used in finance to predict market behaviour.

A good and deep overview of system identification and reduced order models can be seen bellow. 

Black box systems have evolved so much in the past decades, where will this leads us in the future I don’t know but I am sure it will takes to new flights. Data can now be acquired more easily at an affordable cost, which is the pillar for these methods…  



Black box modelling methods are useful tools when we have extensive datasets, or the problems are very complex to describe through math and physics in general. Since data acquisition through sensors is much affordable, compared to the past, machine learning algorithms make sense for model prediction. Non black box models like Finite elements are growing in usage also, as today you can access the solver power for free, and chip costs are evermore low with increased computational power. There are CAD development software bundles with basic structural and thermo electric analysis that are free giving anyone the possibility to conduct advanced FEM study at home with a home computer, something that was not imaginable a decade before. It is interesting to note that System Identification and FEM based modelling also can fit together on applications where data points on the data set are difficult to gather from sensors i.e. I am however sure that the story isn’t just FEM or AR alone and that in the future the combination will yield better tools to transform data and equations into information that helps solve very difficult problems.