Trade-offs Between Analytical Modelling and Simulation
This module has discussed both analytical modelling and simulation of problems that we may encounter. Both methods allow us to draw conclusions from a set of given assumptions. We want to know when to use which, and the advantages and disadvantages of each approach.
Recall that an analytical method is one where we find solutions via mathematical techniques, by manipulating mathematical expressions. This gives rise to dynamical systems models, and some stochastic models too. We use game theoretic arguments and optimisation to arrive at a solution. By contrast, simulation is the process of implementing some computer simulation and measuring the observables in the course of the simulation.
When to Use Analytical
The main time to use an analytical model is whenever it is possible. When writing findings, we need to use a concise language to describe the problem, and lay the paper out in a way such that a third party can verify the introduced arguments.
Analytical approaches are generally widely established, so we can draw from a wide body of knowledge on how to lay these papers out. Analytical approaches also allow us to get general results for any combination of parameters. However, we need to be wary of the tractability of the solutions, and be aware that the published theory so far may not be at the right level for us to use for our modelling questions. In general, analytical solutions are a trade off of intellectual investment against the importance of a general solution.
Analytical approaches work especially well for linear and nonlinear systems, and if the average, or long term behaviours of the system are what we are interested in. Analytical models also allow us to see the qualitative behaviours of a system that we can abstract from the detail described in the model.
Tractability
This is an important consideration in solving a model analytically. Sometimes it is possible to give the system mathematically, but solving it exactly may be too difficult to do. The general idea is to model the system as best as possible analytically, then once we can go no further, turn to numerical approximation techniques, such as numerical integration.
This approach is widely thought of as better than starting with just simulation, because some of the advantages of the mathematical techniques can be harvested, and all is not lost to a simulation to start with.
When to Simulate
Simulations capture the system state where mathematical or analytical models of the system are ineffective or intractable. Simulations can also model nonlinear relationships much better, and are good when we want to see the low level logistics of the system. Simulations can also be more useful when we want to make the model of the system more complicated and doing so would make it hard to solve analytically, but the additions to the model of the system would make it intractable.
Simulation can be run for specific combinations of parameter values, but cannot provide truly general solutions. We therefore have to do many runs of a model with different parameters to see approximate results. It may also be difficult to formalize a model that can communicate the dynamics of the system succinctly to others.
Problems
Simulations suffer from lots of problems that don't exist if the model is solved analytically. Simulations do not often provide robust results of the model, and to overcome this, we need to perform more runs of the model with varying parameters.
Sometimes models are also unstable, and will yield different results each run, known as the stochasticity of the system. This can be somewhat alleviated through repetition or averages of many runs of the system with the same parameters.
Furthermore, simulations suffer from the 'opaque model problem', meaning that we want to keep the model as simple as possible, and rely on pre-existing approaches where possible. Models should be given with clear and concise descriptions where possible.
How to Model a System
The process of modelling a system should be iterative. We start with an initial conceptual model of the system, and begin to explore simplifications to the model that are analytically tractable. Once this is done, we can develop simulations of these cases, and compare and validate these simulations with mathematical results.
Following this, we can begin to extend the simulations, and look at the discrepancies between the theory and the results we see. We can further explore the simulations to see if any analytical simplifications of the system are possible.
Simulation should only be used where we can't deduce the result with straightforward maths.
Example: Benefits of Mutual Grooming
Here, we look at a case study of how one would model mutual grooming, where we have a population that groom others for the benefit of the other at a cost to themselves. This is a form of a 'donor-recipient' game, where we have the following constraints:
- \(b > c\), the benefit of grooming is greater than the cost of grooming
- \(c\) is the cost of grooming
- \(C\) is a strategy to groom others at a cost to myself
- \(D\) is a strategy to not groom others.
We can construct two matrices that comprise tuples of the donor benefit against the recipient benefit, with both the donor and the recipient either picking strategy \(C\) (column/row 1) or strategy \(D\) (colum/row 2):
Missing bit about r and the second matrix
Conceptual Model
Population of individuals \(N\), that meet occasionally. They either groom \(C\) or don't groom \(D\). They get benefits from their actions. Individuals may then follow strategies of other individuals who are mod successful.
We can first show that individuals meet random other individuals from the population. This approximates a mean field model. We can write down a set of equations for the spread of strategies within the population.
We see two fixed points, one unstable, and one stable. The result of this model is that grooming is only viable if everyone does it. This result is fragile because it assumes that everyone grooms at the start and nobody ever has any other thoughts.
Agent-Based Simulation
We can now develop an agent-based simulation of this model. If the populations are large enough and the model is implemented correctly, then the simulation results match the analytical results.
In reality, quite a lot of individuals in the population groom others, and so the model needs to be extended. It is possible that the conceptual model is wrong as monkeys remember who groomed them in the past. We can revise our model to introduce a memory length, with the grooming behaviour dependent on the memory. If the memory length is 1, then we can derive some strategies:
- For the opponent last time with \((C,D)\), we can do \((C,C)\), which is an all cost of grooming
- For the opponent last time with \((C,D)\), we can do \((C,D)\), which is a tit-for-tat model of grooming
- For the opponent last time with \((C,D)\), we can do \((D,C)\), which is an anti tit-for-tat model of grooming
- For the opponent last time with \((C,D)\), we can do \((D,D)\), such that we do not groom irrespective of the last step.
This again gives a mean field description of the model. The tit-for-tat strategy offers some cooperation, but this is fragile.
Adding Large Memories
If we expand the memory size from 1 to \(l\), then there is a huge explosion of the strategy space, and we need to consider the size of the payoff matrices. We cannot figure out analytically what is going on with the model. We see that cooperation can be sustained over long periods of time if the memory length is large enough, and this is likely because we have conditional punishment and forgiveness within the strategy space.
Spatial Modelling
The predictions for the conceptual model are wrong. We didn't consider that monkeys don't interact with all other monkeys, but only a set of their neighbours in space.
Robustness
Robustness of the results can be shown. For simulation based models, we want to consider the noise in the update of the strategy, the synchronicity of the updating, delays in updating, and heterogeneity in various forms.