Reviews of publications on Info-Gap decision theory

Review # 23(Posted: October 25, 2010)

Reference: François M. Hemez , Yakov Ben-Haim, Scott Cogan

Information-gap robustness for the test-analysis and optimization of a nonlinear transient simulation

Proceedings of9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization,4-6 September 2002, Atlanta, Georgia (PDF file)Abstract An alternative to the theory of probability is applied to the problem of assessing the robustness of test-analysis correlation to parametric sources of uncertainty. The analysis technique is based on the theory of information-gap, which models the clustering of uncertain events in families of nested sets instead of assuming a probability structure. The system investigated is the propagation of a transient impact through a layer of hyper-elastic material. The two sources of non-linearity are the softening of the constitutive law implemented to model the hyper-elastic material and contact dynamics at the interface between metallic and crushable materials. The robustness of test-analysis correlation to sources of parametric variability is first studied to identify the parameters of the model that significantly influence the agreement between measurements and predictions. Calibration under non-probabilistic uncertainty is then illustrated. Finally, two information-gap models of uncertainty are embedded to represent uncertainty not only in the knowledge of the model’s parameters but also in the form of the model itself. Although computationally expensive, it is demonstrated that the information-gap reasoning can greatly enhance our understanding of a moderately complex system when the theory of probability cannot be applied due to insufficient information. Scores TUIGF:100%

SNHNSNDN:100%

GIGO:100%

This is an important paper in the annals of info-gap decision theory. It is important for two reasons.

First, take note of the opening statement in the abstract. This statement asserts that the paper deploys no less than "An alternative to the theory of probability". One imagines therefore that a bold proposition such as this would attract the attention of many risk-analysis scholars who would no doubt be more than curious to learn what theory have the authors worked out that can serve as "An alternative to the theory of probability".

The answer of course is that the theory that is being advanced here as an alternative to probability theory is none other than info-gap decision theory. So, for the benefit of all those readers who are not familiar with my critique of info-gap decision theory, it is important to make the following point crystal clear.

Info-gap decision theory cannot, by any stretch of the imagination, be contemplated as an alternative to other theories, even if, for the sake of argument, we allow that other theories (eg. probability theory) lack the appropriate means for handling a problem requiring "assessing the robustness, to uncertainty in model parameters".The reason that info-gap decision theory cannot even be contemplated for this purpose is that it is a fundamentally flawed theory that is utterly unsuitable for the treatment of severe uncertainty. Indeed, info-gap decision theory's methodology for the management of severe uncertainty is the precise antithesis of what such a methodology ought to be.

The theoretical and technical arguments explaining this claim are discussed in detail on other pages on this site (e.g. FAQs about info-gap decision theory and Myths and Facts about info-gap decision theory). Hence, for our purposes here this brief summary will suffice:

To so much as suggest that info-gap decision theory can offer an alternative to the theory of probability is in the very least misleading, because:

- To begin with, what really counts in info-gap decision theory, namely info-gap's robustness model, is no more than a simple instance of Wald's famous maximin model (circa 1940).

- This simple instance is known universally as Radius of Stability model (circa 1960).

- This "alternative to the theory of probability" amounts to selecting the
worstrelevant outcome.

- The trouble is, however, that info-gap's application of this model is fundamentally flawed because it prescribes using it as a medium for the modeling, analysis, and management of
severe uncertainty. In other words info-gap's central proposition is to use a Radius of Stability model (a thoroughly unsuitable model for the task) to manage an uncertainty that is characterized by a poor estimate, a vast (indeed unbounded) uncertainty space, and a likelihood-free quantification of the uncertainty).This renders info-gap decision theory a classic example of a reinvented (square) wheel.

The second point to note is that, in sharp contrast to the many (including recently made) statements, denying outright that info-gap's robustness analysis is a

worst-caseanalysis, this paper unambiguously and directly contends that this is precisely what info-gap's robustness analysis in fact is: aworst-caseanalysis.Since this depiction of info-gap is repeated in three papers that Ben-Haim co-authored with the first author (Review 23, Review 24, Review 25), one suspects that this (correct) depiction of info-gap's robustness analysis is due to the insistence of the first author to call a spade a spade. Elaborating any further on this point would be pure speculation.

The important point is that in this paper info-gap's robustness analysis is unmistakably described by the Father of info-gap decision theory as a

worst-case analysis.In any case, on page 8 we read (color is mine):

Figure 6-1 (a) shows that, at each uncertainty level a_{k}, an optimization problem must be solved that provides the worst possible performance R^{*}(a_{k}) given the information-gap uncertainty bound U(u_{0};a_{k}):

R ^{*}(a_{k}) =max R(q;u) (6.1) u∈U(u _{0};a_{k})and

The information-gap robustness analysis is summarized in Figure 6-2 for the impact experiment. The objective is to assess the robustness of test analysis correlation to input parameter uncertainty. Therefore, theadverseaspect of uncertainty is investigated by answering the question: "By how much does uncertainty deteriorate the original correlation?"The caption of Figure 6-2 is "

Worst-case info-gap robustness".And on page 9 we read (color is mine):

The optimization searches for the worst correlation at each uncertainty level.But, in sharp contrast to these assertions, in other publications Ben-Haim vehemently, (and erroneously) insists that info-gap's robustness analysis is

nota worst-case analysis. Thus, in the second edition of the book on info-gap decision theory we read (color is mine):Optimization of the robustness in eq. (3.172) is emphaticallynota worst-case analysis. In classical worst-case min-max analysis the decision maker minimizes the impact of the maximally damaging case. But an info-gap model of uncertainty is an unbounded family of nested sets: U(α,û), for all α≥0. Consequently, there usually is no worst case: any adverse occurrence is less damaging than some other more extreme event occurring at a larger value of α. What eqs. (3.169) and (3.172) express is the greatest level of uncertainty consistent with satisficing to level r_{c}. When the decision maker chooses the action q to maximize α(q,r_{c}), what is maximized is the immunity to an unbounded ambient uncertainty.The closest this comes to "min-maxing" is that the action is chosen so that "bad" events (causing reward R_{*}less than r_{c}) occur as "far away" as possible (beyond a maximized value of α).

Ben-Haim (2006, p. 101)And in his most recent (2010) book we read:

Info-gap theory is not a worst-case analysis. While there may be a worst case, one cannot know what it is and one should not base one's policy upon guesses of what it might be. Info-gap theory is related to robust-control and min-max methods, but nonetheless different from them. The strategy advocated here isnotthe amelioration of purportedly worst cases.

Ben-Haim (2010, p. 9)

The question of course is why? How is it that in the early 2000s, info-gap's robustness analysis was a

worst-case analysis, whereas in 2010 it is not?One possible explanation for this glaring contradition is the possible chain reaction, or domino effect that will ensue. An acknowledgment that info-gap's robustness analysis is a worst-case analysis will immediately pull the rug out from under info-gap decision theory as a whole.

Because, this would immediately imply that info-gap's robustness model is a Maximin model, which would further imply that info-gap decision theory is no more than a reinvention of the wheel which would then give rise to the following question: in this case, what is the point of info-gap decision theory in the first place?!

Of course, there are many other publications (in the info-gap literature) propounding that info-gap's robustness analysis is not a worst-case analysis.

So to repeat, this paper is important because it gives a vivid illustration of the main ingredients of info-gap decision theory:

self-contradcition, unfounded assertions, exaggerations, and ... plain errors.Is it surprising then that it is elementary to debunk it formally and rigorously.

## The worst-case saga

The question is then: how can there possibly be any ground to so much as suggest that info-gap's robustness analysis is not a worst-case analysis given that, as demonstrated by the Maximin Theorem, it is a simple maximin model?

A possible explanation may be the difficulty to appreciate the distinction between:

and

A globalworst-case analysis,

A localworst-case analysis.The difference is this:

- A
Globalworst-case analysis seeks the worst case over theentireuncertainty space under consideration.

- A
Localworst-case analysis seeks the worst case over agiven subsetof the uncertainty space under consideration.In the case of info-gap decision theory, the worst-case analysis is

local: it is conducted over the neighborhood around the estimate specified by the horizon of uncertainty α. Thus, for each value of α a worst-case analysis is conducted over a "ball" of radius α centered at the estimate.The picture is this:

Info-gap's Local Worst-Case Analysismax {α ≥ 0: r* ≤ r(q,p),∀p∈B(α,p*)} The rectangle represents the parameter (uncertainty) space, P. The shaded area represents the values of the parameter p for which the system satisfies a given performance requirement, namely r* ≤ r(q,p). The center of the circles, p*, represents a given estimate of the true value of the parameter p. B(α,p*) denotes a ball of radius α around p*.

For α to be admissible, the

worst valueof p in B(α,p*) must satisfy the performance requirement r* ≤ r(q,p).So:

Info-gap's robustness analysis is

not a globalworst-case analysis as it is not conducted over the entire parameter space P. It is alocalrobustness (worst-case) analysis that is conductedonlyover the "balls" centered at the estimate. To be precise, a worst-case analysis is conducted on each "ball" separately, one "ball" at a time.For the record:

The Maximin model representing info-gap's robustness model (see the proof) performs exactly the same thing that info-gap's robustness model does: alocalworst-case analysis on each "ball".Hence, the repeated claims that info-gap's robustness analysis is not a worst-case analysis are grossly misleading. Their aim apparently is to cling to the position that info-gap's robustness model is not a Maximin model. This position is, however, groundless, because it is elementary to prove formally and rigorously that info-gap's robustness model is a simple instance of Wald's Maximin model (see proof).

To sum it all up then, the basic facts are these:

- Info-gap's robustness model is a simple instance of Wald's maximin model (see proof).

- Info-gap's robustness model is a simple instance of the Radius of Stability model (see proof).

- Info-gap's robustness analysis is a local worst-case analysis (conducted separetaly on each "ball" centered at the estimate).
And just in case, ... info-gap decision theory is not an alternative to the theory of probability.

- Ben-Haim (2001, 2006): Info-Gap Decision Theory: decisions under severe uncertainty.

- Regan et al (2005): Robust decision-making under severe uncertainty for conservation management.

- Moilanen et al (2006): Planning for robust reserve networks using uncertainty analysis.

- Burgman (2008): Shakespeare, Wald and decision making under severe uncertainty.

- Ben-Haim and Demertzis (2008): Confidence in monetary policy.

- Hall and Harvey (2009): Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.

- Ben-Haim (2009): Info-gap forecasting and the advantage of sub-optimal models.

- Yokomizo et al (2009): Managing the impact of invasive species: the value of knowing the density-impact curve.

- Davidovitch et al (2009): Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island.

- Ben-Haim et al (2009): Do we know how to set decision thresholds for diabetes?

- Beresford and Thompson (2009): An info-gap approach to managing portfolios of assets with uncertain returns

- Ben-Haim, Dacso, Carrasco, and Rajan (2009): Heterogeneous uncertainties in cholesterol management

- Rout, Thompson, and McCarthy (2009): Robust decisions for declaring eradication of invasive species

- Ben-Haim (2010): Info-Gap Economics: An Operational Introduction

- Hine and Hall (2010): Information gap analysis of flood model uncertainties and regional frequency analysis

- Ben-Haim (2010): Interpreting Null Results from Measurements with Uncertain Correlations: An Info-Gap Approach

- Wintle et al. (2010): Allocating monitoring effort in the face of unknown unknowns

- Moffitt et al. (2010): Securing the Border from Invasives: Robust Inspections under Severe Uncertainty

- Yemshanov et al. (2010): Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest

- Davidovitch and Ben-Haim (2010): Robust satisficing voting: why are uncertain voters biased towards sincerity?

- Schwartz et al. (2010): What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making

- Arkadeb Ghosal et al. (2010): Computing Robustness of FlexRay Schedules to Uncertainties in Design Parameters

- Hemez et al. (2002): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

- Hemez et al. (2003): Applying information-gap reasoning to the predictive accuracy assessment of transient dynamics simulations

- Hemez, F.M. and Ben-Haim, Y. (2004): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

- Ben-Haim, Y. (2007): Frequently asked questions about info-gap decision theory

- Sprenger, J. (2011): The Precautionary Approach and the Role of Scientists in Environmental Decision-Making

- Sprenger, J. (2011): Precaution with the Precautionary Principle: How does it help in making decisions

- Hall et al. (2011): Robust climate policies under uncertainty: A comparison of Info--Gap and RDM methods

- Ben-Haim and Cogan (2011) : Linear bounds on an uncertain non-linear oscillator: an info-gap approach

- Van der Burg and Tyre (2011) : Integrating info-gap decision theory with robust population management: a case study using the Mountain Plover

- Hildebrandt and Knoke (2011) : Investment decisions under uncertainty --- A methodological review on forest science studies.

- Wintle et al. (2011) : Ecological-economic optimization of biodiversity conservation under climate change.

- Ranger et al. (2011) : Adaptation in the UK: a decision-making process.

## Recent Articles, Working Papers, Notes

Also, see my complete list of articles

Moshe's new book! - Sniedovich, M. (2012) Fooled by local robustness,
Risk Analysis, in press.

- Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty,
International Transactions in Operational Research, in press.

- Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis,
Applications of Mathematics,56(5), 499-509.

- Sniedovich, M. (2011) Dynamic programming: introductory concepts, in
Wiley Encyclopedia of Operations Research and Management Science(EORMS), Wiley.

- Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem,
OR Spectrum, 33(4), 815-929, 2011.

- Sniedovich, M. (2011)
Dynamic Programming: Foundations and Principles, Second Edition,Taylor & Francis.

- Sniedovich, M. (2010) A bird's view of Info-Gap decision theory,
Journal of Risk Finance, 11(3), 268-283.

- Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.

- Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds),
.Safety, Reliability and Risk Analysis: Theory, Methods and Applications,pp. 2071-2079, Taylor and Francis Group, London.

- Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)

- Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.),
11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation,DIPTEM, Genova, 89-94.

- Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)

- Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)

- Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08

This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )

- Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory,
Decision Point,24, 10.

- Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty,
Decision Point,22, 8-9.

- Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!,
Journal of Risk Finance,9(3), 287-291.

- Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)

In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.

- Sniedovich. M. (2008) The Mighty Maximin! (PDF File)

This paper is dedicated to the modeling aspects of Maximin and robust optimization.

- Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty,
Decision Making in Manufacturing and Services,1-2, 111-136. (PDF File) .

- Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)

In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.

- My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)
This is an exciting development!

- Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.

- Ben-Haim is not familiar with the fundamental concept
point estimate. He does not realize that a function can be a point estimate of another function.So when you read my papers make sure that you do not misinterpret the notion

point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!

- A formal proof that
Info-GapisWald's Maximin Principlein disguise. (December 31, 2006)

This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)

It shows thatInfo-Gapis not a new theory but rather a simple instance ofWald's famous Maximin Principledating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context ofGame Theory(1928).

- A proof that
Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)

This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .

It shows that becauseInfo-Gapdeploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.

- A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)

This is a very short article entitled The GAP in Info-Gap (PDF File) .

It is a math-free version of the paper above. Read it if you are allergic to math.

- A long essay entitled
What's Wrong with Info-Gap? An Operations Research Perspective(PDF File) (December 31, 2006).

This is a paper that I presented at theASOR Recent Advances in Operations Research(PDF File) mini-conference (December 1, 2006, Melbourne, Australia).## Recent Lectures, Seminars, Presentations

If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.

Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.

- ASOR Recent Advances, 2011, Melbourne, Australia, November 16 2011. Presentation: The Power of the (peer-reviewed) Word. (PDF file).

- Alex Rubinov Memorial Lecture The Art, Science, and Joy of (mathematical) Decision-Making, November 7, 2011, The University of Ballarat. (PDF file).

- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Invited tutorial, ALIO/INFORMS Conference, Buenos Aires, Argentina, July 6-9, 2010).

- A Critique of Info-Gap Decision theory: From Voodoo Decision-Making to Voodoo Economics(PDF File) .

(Recent Advances in OR, RMIT, Melbourne, Australia, November 25, 2009)

- Robust decision-making in the face of severe uncertainty(PDF File) .

(GRIPS, Tokyo, Japan, October 16, 2009)

- Decision-making in the face of severe uncertainty(PDF File) .

(KORDS'09 Conference, Vilnius, Lithuania, September 30 -- OCtober 3, 2009)

- Modeling robustness against severe uncertainty (PDF File) .

(SOR'09 Conference, Nova Gorica, Slovenia, September 23-25, 2009)

- How do you recognize a Voodoo decision theory?(PDF File) .

(School of Mathematical and Geospatial Sciences, RMIT, June 26, 2009).

- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, Info-Gaps, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Department of Econometrics and Business Statistics, Monash University, May 8, 2009).

- The Rise and Rise of Voodoo Decision Theory.

ASOR Recent Advances, Deakin University, November 26, 2008. This presentation was based on the pages on my website (voodoo.moshe-online.com).

- Responsible Decision-Making in the face of Severe Uncertainty (PDF File) .

(Singapore Management University, Singapore, September 29, 2008)

- A Critique of Info-Gap's Robustness Model (PDF File) .

(ESREL/SRA 2008 Conference, Valencia, Spain, September 22-25, 2008)

- Robust Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Technion, Haifa, Israel, September 15, 2008)

- The Art and Science of Robust Decision-Making (PDF File) .

(AIRO 2008 Conference, Ischia, Italy, September 8-11, 2008 )

- The Fundamental Flaws in Info-Gap Decision Theory (PDF File) .

(CSIRO, Canberra, July 9, 2008 )

- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .

(OR Conference, ADFA, Canberra, July 7-8, 2008 )

- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .

(University of Sydney Seminar, May 16, 2008 )

- Decision-Making Under Severe Uncertainty: An Australian, Operational Research Perspective (PDF File) .

(ASOR National Conference, Melbourne, December 3-5, 2007 )

- A Critique of Info-Gap (PDF File) .

(SRA 2007 Conference, Hobart, August 20, 2007)

- What exactly is wrong with Info-Gap? A Decision Theoretic Perspective (PDF File) .

(MS Colloquium, University of Melbourne, August 1, 2007)

- A Formal Look at Info-Gap Theory (PDF File) .

(ORSUM Seminar , University of Melbourne, May 21, 2007)

The Art and Science of Decision-Making Under Severe Uncertainty(PDF File) .

(ACERA seminar, University of Melbourne, May 4, 2007)

- What exactly is Info-Gap? An OR perspective. (PDF File)

ASOR Recent Advances in Operations Researchmini-conference (December 1, 2006, Melbourne, Australia).

Disclaimer:This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.