| Reference: | François M. Hemez, Scott W. Doebling, Yakov Ben-Haim |
| Abstract | An alternative to the theory of probability is applied to the problem of assessing the robustness, to parametric sources of uncertainty, of the correlation between measurements and computer simulations. The uncertainty analysis relies on the theory of information-gap, which models the clustering of uncertain events in families of nested sets instead of assuming a probability structure. The system investigated is the propagation of a transient impact through a layer of hyper-elastic material. The two sources of non-linearity are (1) the softening of the constitutive law representing the hyper-elastic material and (2) the contact dynamics at the interface between metallic and crushable materials. Information-gap models of uncertainty are developed to represent uncertainty in the knowledge of the model’s parameters and in the form of the model itself. Although computationally expensive, it is demonstrated that information-gap reasoning can greatly enhance our understanding of a system when the theory of probability cannot be applied due to insufficient information. |
| Scores | TUIGF:100% SNHNSNDN:100% GIGO:100% |
This is an important paper in the annals of info-gap decision theory. It is important for two reasons.
First, take note of the opening statement in the abstract. This statement asserts that the paper deploys no less than "An alternative to the theory of probability". One imagines therefore that a bold proposition such as this would attract the attention of many risk-analysis scholars who would no doubt be more than curious to learn what theory have the authors worked out that can serve as "An alternative to the theory of probability".
The answer of course is that the theory that is being advanced here as an alternative to probability theory is none other than info-gap decision theory. So, for the benefit of all those readers who are not familiar with my critique of info-gap decision theory, it is important to make the following point crystal clear.
Info-gap decision theory cannot, by any stretch of the imagination, be contemplated as an alternative to other theories, even if, for the sake of argument, we allow that other theories (eg. probability theory) lack the appropriate means for handling a problem requiring "assessing the robustness, to uncertainty in model parameters".
The reason that info-gap decision theory cannot even be contemplated for this purpose is that it is a fundamentally flawed theory that is utterly unsuitable for the treatment of severe uncertainty. Indeed, info-gap decision theory's methodology for the management of severe uncertainty is the precise antithesis of what such a methodology ought to be.
The theoretical and technical arguments explaining this claim are discussed in detail on other pages on this site (e.g. FAQs about info-gap decision theory and Myths and Facts about info-gap decision theory). Hence, for our purposes here this brief summary will suffice:
To so much as suggest that info-gap decision theory can offer an alternative to the theory of probability is in the very least misleading, because:
- To begin with, what really counts in info-gap decision theory, namely info-gap's robustness model, is no more than a simple instance of Wald's famous maximin model (circa 1940).
- This simple instance is known universally as Radius of Stability model (circa 1960).
- This "alternative to the theory of probability" amounts to selecting the worst relevant outcome.
- The trouble is, however, that info-gap's application of this model is fundamentally flawed because it prescribes using it as a medium for the modeling, analysis, and management of severe uncertainty. In other words info-gap's central proposition is to use a Radius of Stability model (a thoroughly unsuitable model for the task) to manage an uncertainty that is characterized by a poor estimate, a vast (indeed unbounded) uncertainty space, and a likelihood-free quantification of the uncertainty).
This renders info-gap decision theory a classic example of a reinvented (square) wheel.
The second point to note is that, in sharp contrast to the many (including recently made) statements, denying outright that info-gap's robustness analysis is a worst-case analysis, this paper unambiguously and directly contends that this is precisely what info-gap's robustness analysis in fact is: a worst-case analysis.
Since this depiction of info-gap is repeated in three papers that Ben-Haim co-authored with the first author (Review 23, Review 24, Review 25), one suspects that this (correct) depiction of info-gap's robustness analysis is due to the insistence of the first author to call a spade a spade. Elaborating any further on this point would be pure speculation.
The important point is that in this paper info-gap's robustness analysis is unmistakably described by the Father of info-gap decision theory as a worst-case analysis.
So on page 9 we read (color is mine):
The decision-making procedure is conceptually illustrated in Figure 2. Figure 2-a shows that, at each horizon of uncertainty ak, an optimization problem must be solved that provides the worst possible performance R*(ak) within the set U(u0;ak):Focusing on the worst possible test-analysis correlation metric R*(ak) at each level of uncertainty ak provides an assessment of the adverse effect of uncertainty on performance. Decisions -- or, in our case, models -- that feature a high robustness a* are preferred because they minimize the potentially adverse effect of uncertainty.
R*(ak) = max R(q;u) (6) u∈U(u0;ak)
And on page 10 we read (color is mine):
For the application, the optimization searches for the model that yields the worst possible test-analysis correlation metric R(q;u) at each uncertainty level.
The horizontal axis of Figure 3 is enttled (color is mine) "Worst Possible Test-Analysis Error".
But, in sharp contrast to these assertions, in other publications Ben-Haim vehemently, (and erroneously) insists that info-gap's robustness analysis is not a worst-case analysis. Thus, in the second edition of the book on info-gap decision theory we read (color is mine):
Optimization of the robustness in eq. (3.172) is emphatically not a worst-case analysis. In classical worst-case min-max analysis the decision maker minimizes the impact of the maximally damaging case. But an info-gap model of uncertainty is an unbounded family of nested sets: U(α,û), for all α≥0. Consequently, there usually is no worst case: any adverse occurrence is less damaging than some other more extreme event occurring at a larger value of α. What eqs. (3.169) and (3.172) express is the greatest level of uncertainty consistent with satisficing to level rc. When the decision maker chooses the action q to maximize α(q,rc), what is maximized is the immunity to an unbounded ambient uncertainty.The closest this comes to "min-maxing" is that the action is chosen so that "bad" events (causing reward R* less than rc) occur as "far away" as possible (beyond a maximized value of α).
Ben-Haim (2006, p. 101)
And in his most recent (2010) book we read:
Info-gap theory is not a worst-case analysis. While there may be a worst case, one cannot know what it is and one should not base one's policy upon guesses of what it might be. Info-gap theory is related to robust-control and min-max methods, but nonetheless different from them. The strategy advocated here is not the amelioration of purportedly worst cases.
Ben-Haim (2010, p. 9)
The question of course is why? How is it that in the early 2000s, info-gap's robustness analysis was a worst-case analysis, whereas in 2010 it is not?
One possible explanation for this glaring contradition is the possible chain reaction, or domino effect that will ensue. An acknowledgment that info-gap's robustness analysis is a worst-case analysis will immediately pull the rug out from under info-gap decision theory as a whole.
Because, this would immediately imply that info-gap's robustness model is a Maximin model, which would further imply that info-gap decision theory is no more than a reinvention of the wheel which would then give rise to the following question: in this case, what is the point of info-gap decision theory in the first place?!
Of course, there are many other publications (in the info-gap literature) propounding that info-gap's robustness analysis is not a worst-case analysis.
So to repeat, this paper is important because it gives a vivid illustration of the main ingredients of info-gap decision theory: self-contradcition, unfounded assertions, exaggerations, and ... plain errors.
Is it surprising then that it is elementary to debunk it formally and rigorously.
The worst-case saga
The question is then: how can there possibly be any ground to so much as suggest that info-gap's robustness analysis is not a worst-case analysis given that, as demonstrated by the Maximin Theorem, it is a simple maximin model?
A possible explanation may be the difficulty to appreciate the distinction between:
- A global worst-case analysis,
- A local worst-case analysis.
The difference is this:
- A Global worst-case analysis seeks the worst case over the entire uncertainty space under consideration.
- A Local worst-case analysis seeks the worst case over a given subset of the uncertainty space under consideration.
In the case of info-gap decision theory, the worst-case analysis is local: it is conducted over the neighborhood around the estimate specified by the horizon of uncertainty α. Thus, for each value of α a worst-case analysis is conducted over a "ball" of radius α centered at the estimate.
The picture is this:
| Info-gap's Local Worst-Case Analysis |
 |
| max {α ≥ 0: r* ≤ r(q,p),∀p∈B(α,p*)} |
| The rectangle represents the parameter (uncertainty) space, P. The shaded area represents the values of the parameter p for which the system satisfies a given performance requirement, namely r* ≤ r(q,p). The center of the circles, p*, represents a given estimate of the true value of the parameter p. B(α,p*) denotes a ball of radius α around p*.
For α to be admissible, the worst value of p in B(α,p*) must satisfy the performance requirement r* ≤ r(q,p). |
So:
Info-gap's robustness analysis is not a global worst-case analysis as it is not conducted over the entire parameter space P. It is a local robustness (worst-case) analysis that is conducted only over the "balls" centered at the estimate. To be precise, a worst-case analysis is conducted on each "ball" separately, one "ball" at a time.
For the record:
The Maximin model representing info-gap's robustness model (see the proof) performs exactly the same thing that info-gap's robustness model does: a local worst-case analysis on each "ball".
Hence, the repeated claims that info-gap's robustness analysis is not a worst-case analysis are grossly misleading. Their aim apparently is to cling to the position that info-gap's robustness model is not a Maximin model. This position is, however, groundless, because it is elementary to prove formally and rigorously that info-gap's robustness model is a simple instance of Wald's Maximin model (see proof).
To sum it all up then, the basic facts are these:
- Info-gap's robustness model is a simple instance of Wald's maximin model (see proof).
- Info-gap's robustness model is a simple instance of the Radius of Stability model (see proof).
- Info-gap's robustness analysis is a local worst-case analysis (conducted separetaly on each "ball" centered at the estimate).
And just in case, ... info-gap decision theory is not an alternative to the theory of probability.
