The Spin Stops Here!
Decision-Making Under Severe Uncertainty  
Faqs | Help | @ | Contact | home  
voodoostan info-gap decision theory info-gap economics severe uncertainty mighty maximin robust decisions responsible decisions


Reviews of publications on Info-Gap decision theory

Review # 5 (Posted: April 20, 2009; Last update: May 17, 2009)

Reference: Ben-Haim, Y. and Demertzis, M.
Confidence in monetary policy.
DNB Working Paper No. 192 (PDF file)
December, 2008
Abstract In situations of relative calm and certainty, policy makers have confidence in the mechanisms at work and feel capable of attaining precise and ambitious results. As the environment becomes less and less certain, policy makers are confronted with the fact that there is a trade-off between the quality of a certain outcome and the confidence (robustness) with which it can be attained. Added to that, in the presence of knightian uncertainty, confidence itself can no longer be represented in probabilistic terms (because probabilities are unknown). We adopt the technique of Info-Gap Robust Satisficing to first define confidence under Knightian uncertainty, and second quantify the trade-off between quality and robustness explicitly. We apply this to a standard monetary policy example and provide Central Banks with a framework to rank policies in a way that will allow them to pick the one that either maximizes confidence given an acceptable level of performance, or alternatively, optimize performance for a given level of confidence.
ScoresTUIGF:100%
SNHNSNDN:5000%
GIGO:100%


This is a working paper of De Netherlandshe Bank. It is on my list due to its extremely high SNHNSNDN score. I shall therefore concentrate only on this aspect of the paper.

The objective of this review is twofold. Firstly, my aim is to make it clear that the authors' distinction between what they term "minimax strateries" and the "info-gap robust satisficing" strategy, that culminates in their proposition to adopt the "info-gap robust satisficing" strategy as a framework for formulating monetary policy, stems from a grossly erroneous, hence thoroughly misleading conception of the Maximin/Minimax paradigm.

Secondly, to show that the explanation that purportedly identifies "fundamental distinctions" between the so called "minimax strateries" and the "info-gap robust satisficing" strategy amounts to no more than hollow rhetoric and pure spin.

For the benefit of readers who are new to my Web-site, I ought to point out that this grossly erroneous distinction is of a piece with the broader thesis that right from the start proclaimed Info-Gap as new and distinctly different as follows (emphais is mine):

Info-gap decision theory is radically different from all current theories of decision under uncertainty. The difference originates in the modeling of uncertainty as an information gap rather than as a probability. The need for info-gap modeling and management of uncertainty arises in dealing with severe lack of information and highly unstructured uncertainty.

Ben-Haim (2006, p. xii)

and
In this book we concentrate on the fairly new concept of information-gap uncertainty, whose differences from more classical approaches to uncertainty are real and deep. Despite the power of classical decision theories, in many areas such as engineering, economics, management, medicine and public policy, a need has arisen for a different format for decisions based on severely uncertain evidence.

Ben-Haim (2006, p. 11)

So, as you can see, Info-Gap decision theory is being promoted as nothing short of a "revolutionary" theory indeed, a breakthrough in decision theory.

It is important therefore to be clear that not only is it the case that Info-Gap's robustness model is neither new nor radically different from classical models, but that it is in fact a special case of the most famous paradigm in classical decision theory for the treatment of severe uncertainty.

In other words, to pull the rug out from under the proposition that the justification for using Info-gap's robustness model is due to the "fundamental distictions" between this model and Minimax/Maximin, it is important to expose the profound misrepresentation, in this paper, of the relation between Info-Gap decision theory and Wald's Maximin model.

To begin with, I ought to point out that Ben-Haim — the creator of Info-Gap decision theory — is fully aware of my criticism of Info-Gap decision theory, notably the short theorem proving that Info-Gap's robustness model is in fact a simple instance of the Maximin model. Ben-Haim does not dispute the validity of this theorem.

And yet, Ben-Haim and Demertzis do not hesitate to make the following claims (2008, p. 17, emphasis is mine):

Info-gap robust-satisficing is motivated by the same perception of uncertainty which motivates the min-max class of strategies: lack of reliable probability distributions and the potential for severe and extreme events. We will see that the robust-satisficing decision will sometimes coincide with a min-max decision. On the other hand we will identify some fundamental distinctions between the min-max and the robust-satisficing strategies and we will see that they do not always lead to the same decision.

In other words, without taking issue with, or rebutting, or explicitly commenting on the theorem which decisively proves that Info-Gap's robustness model is a run-of-the mill Maximin model, the authors philosophize at length on the distinctions between Info-Gap's robustness model and the so called 'min-max' strategies. Worse, in what seems like a maneovour aimed at ducking their scholarly duty to take on the theorem directly, the authors refer to 'min-max' strategies rather than to Maximin strategies. Worse yet, without bothering to give a formal definition to the 'min-max' model that the authors presumably compare to Info-Gap's robustness model, they go on about the purportedly different results yielded by the purportedly different strategies.

I shall not address these points here, as a detailed analysis of the misconceptions pervading Ben-Haim's understanding of the relationship between Info-Gap decision theory and the Maximin paradigm can be found in a number of my articles (eg. Sniedovich (2007), Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model) as well as in Faqs about Info-Gap Decision Theory and in WIKIPEDIA.

But to give you an idea of how profound these misconceptions are, consider the first purported fundamental distinction that the authors identify between Info-Gap and 'min-max' (2008, p. 17):

First of all, if a worst case or maximal uncertainty is unknown, then the min-max strategy cannot be implemented. That is, the min-max approach requires a specific piece of knowledge about the real world: "What is the greatest possible error of the analyst's model?". This is an ontological question: relating to the state of the real world. In contrast, the robust-satisficing strategy does not require knowledge of the greatest possible error of the analyst's model. The robust-satisficing strategy centers on the vulnerability of the analyst's knowledge by asking: "How wrong can the analyst be, and the decision still yields acceptable outcomes?" The answer to this question reveals nothing about how wrong the analyst in fact is. The answer to this question is the info-gap robustness function, while the true maximal error may or may not exceed the info-gap robust satisficing. This is an epistemic question, relating to the analyst's knowledge, positing nothing about how good that knowledge actually is. The epistemic question relates to the analyst's knowledge, while the ontological question relates to the relation between that knowledge and the state of the world. In summary, knowledge of a worst case is necessary for the min-max approach, but not necessary for the robust-satisficing approach.

The authors are badly in error regarding the relation of these two facts:

It is important to note that, the second fact in no way implies that the domains on which Maximin/Minimax models operate are required to be bounded. For example, consider the following classic Minimax model, exhibiting one of the most famous saddle points on Planet Earth:

 p := min  max  x2 - y2
x∈ℜy∈ℜ

= real line.

The optimal solution to this simple Minimax problem is the saddle point (x,y) = (0,0), yielding p=0. Note that the objective function here, namely the function f=f(x,y) defined by f(x,y)=x2 - y2, is unbounded on ℜ2.

But more than this, the unbounded vs. bounded horizons issue which presumably accounts for the difference between the two models is totally spurious. Indeed, insofar as Info-Gap's model is concerned, this point is utterly irrelevant. This is so because, insofar as Info-gap's model is concerned, it matters not in the slightest whether the horizon is bounded or unbounded. For, the robustness sought by Info-Gap's model is dictated by one consideration alone: the performance requirement, namely a constraint. Therefore, only two 'cases' need to be considered here:

In other words, Info-Gap's robustness model cares not one iota about the degree, the level, the point, or what have you, at which the constraint is violated. Insofar as Info-gap's robustness model is concerned, all violations of the constraints are 'worst-cases'. So, irrespective of whether the complete region of uncertainty is unbounded, there always is a worst-case.

In short, the authors are wrong — very wrong — in their assessment of what constitutes a 'worst-case' in a Maximin/Minimax formulation of Info-Gap's robustness model.

And to impress on the reader the heights to which rhetoric/spin is taken in this paper, here is an extended quote from the discussion on the relationship between Info-Gap robustness and Maximin — in fact 'min-max'. Clearly implicit in this dissertation is the fact that at least one of the authors (ie. Ben-Haim) is fully aware of the theorem that decisively shows that Info-Gap's robustness model can indeed be formulated as a simple Maximin model. But, in what seems a clear effort to avoid "direct contact" with the Maximin theorm the authors talk about "min-max" (one suspects rather than Maximin) as though "min-max" has got nothing to do with Maximin. What is the point of discoursing at length about differences between the two models without first of all proving this theorem to be false. But no attempt whatsoever is made to dispute the validity of this theorem. Instead, this is what you read in the paper:

4.2 Min-Max, Robust Control, and Robust-Satisficing

We take a brief intermezzo to compare the robust satisficing strategy with a class of alternatives. The term 'min-max', 'robust control' and 'worst-case' refer to a collection of decision strategies which attempt to ameliorate a maximally adverse outcome. This can of course be formulated in a variety of ways. In one way or another, whether explicitly or implicitly, a greatest level of uncertainty or a worst possible outcome is posited. Then a strategy is sought which maximally diminishes the impact of this outcome.

Info-gap robust-satisficing is motivated by the same perception of uncertainty which motivates the min-max class of strategies: lack of reliable probability distributions and the potential for severe and extreme events. We will see that the robust-satisficing decision will sometimes coincide with a min-max decision. On the other hand we will identify some fundamental distinctions between the min-max and the robust-satisficing strategies and we will see that they do not always lead to the same decision.

First of all, if a worst case or maximal uncertainty is unknown, then the min-max strategy cannot be implemented. That is, the min-max approach requires a specific piece of knowledge about the real world: "What is the greatest possible error of the analyst's model?". This is an ontological question: relating to the state of the real world. In contrast, the robust-satisficing strategy does not require knowledge of the greatest possible error of the analyst's model. The robust-satisficing strategy centers on the vulnerability of the analyst's knowledge by asking: "How wrong can the analyst be, and the decision still yields acceptable outcomes?" The answer to this question reveals nothing about how wrong the analyst in fact is. The answer to this question is the info-gap robustness function, while the true maximal error may or may not exceed the info-gap robust satisficing. This is an epistemic question, relating to the analyst's knowledge, positing nothing about how good that knowledge actually is. The epistemic question relates to the analyst's knowledge, while the ontological question relates to the relation between that knowledge and the state of the world. In summary, knowledge of a worst case is necessary for the min-max approach, but not necessary for the robust-satisficing approach.

The second consideration is that the min-max approaches depend on what tends to be the least reliable part of our knowledge about the uncertainty. Under Knightian uncertainty we do not know the probability distribution of the uncertain entities. We may be unsure what are typical occurrences, and the systematics of extreme events are even less clear. Nonetheless the min-max decision hinges on ameliorating what is supposed to be a worst case. This supposition may be substantially wrong, so the min-max strategy may be mis-directed.

A third point of comparison is that min-max aims to ameliorate a worst case, without worrying about whether an adequate or required outcome is achieved. This strategy is motivated by severe uncertainty which suggests that catastrophic outcomes are possible, in conjunction with a precautionary attitude which stresses preventing disaster. The robust-satisficing strategy acknowledges unbounded uncertainty, but also incorporates the outcome requirements of the analyst. The choice between the two strategies — min-max and robust-satisficing — hinges on the priorities and preferences of the analyst.

The fourth distinction between the min-max and robust-satisficing approaches is that they need not lead to the same decision, even starting with the same information.

Ben-Haim and Demertzis (2008, p. 17)

What this misguided thesis proves is not that Info-Gap's robustness model is not a Maximin model. What it does prove, though, is the authors' obvious misconceptions about the modeling aspects of the Maximin/Minimax paradigm, their misapprehension as to how the Maximin/Minimax paradigm is modeled, and so on. All this bars them from grasping the full extent of the affinity between Info-Gap's robustness model and Wald's Maximin model:

Maximin Theorem:

Info-Gap's Robustness model Corresponding instance of Wald's Maximin model
max {α ≥ 0: r(d,u) ≤ r* , ∀u∈U(α,û)}     ≡    
max min f(d,u,α,û)
  α ≥ 0     u∈U(α,û)  

where f(d,u,α,û) = α if r(d,u) ≤ r*; and f(d,u,α,û) = -∞, otherwise.

Proof of the Maximin Theorem:

Instance of Wald's Maximin Model Equivalent Math Programming formulation
max min f(d,u,α,û)
  α ≥ 0     u∈U(α,û)  
    ≡    
max { v: v ≤ f(d,u,α,û), ∀ u∈U(α,û) }
  α ≥ 0  
v ∈ ℜ
 
    ≡    
max { α: α ≤ f(d,u,α,û), ∀ u∈U(α,û) }
  α ≥ 0  
 
    ≡    
max   { α: α ≤ f(d,u,α,û), ∀ u∈U(α,û) }
 
    ≡    
max   { α: r(d,u) ≤ r*, ∀ u∈U(α,û) }    
Info-Gap's Robustness Model

So, the bottom line is this: Info-Gap's robustness model is a simple instance of Wald's famous Maximin model. And no amount of rhetoric/spin can change it. This simple instance of Wald's famous Maximin model always yields the same decision(s) that are yielded by Info-Gap's robustness model.

Note that The Maximin Theorem is constructive: it sets out a simple recipe for constructing the instance of the generic Maximin model that represents Info-Gap's robustness model.

No amount of rhetoric/spin can change this bottom line.

The conceptual and technical mistakes that led Ben-Haim astray in this matter are discussed in detail in the article Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model and in FAQ # 20.

Remark:

For the record, I should point out that Ben-Haim is fully aware of the existence of this theorem, what is more, that he does not dispute its validity. The situation is similar with respect to the Invariance Theorem.

It is therefore most regrettable, indeed inexcusable, that Ben-Haim has chosen to waltz around these theorems by means of spurious explanations to thereby extend already existing errors rather than admit to mistakes.

It will be interesting to see how long will Ben-Haim pursue this deliberate strategy of avoiding to deal with theorems that invalidate his repeated claims and pronouncements regarding Info-Gap's unique role and place in decision theory.

Other Reviews

  1. Ben-Haim (2001, 2006): Info-Gap Decision Theory: decisions under severe uncertainty.

  2. Regan et al (2005): Robust decision-making under severe uncertainty for conservation management.

  3. Moilanen et al (2006): Planning for robust reserve networks using uncertainty analysis.

  4. Burgman (2008): Shakespeare, Wald and decision making under severe uncertainty.

  5. Ben-Haim and Demertzis (2008): Confidence in monetary policy.

  6. Hall and Harvey (2009): Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.

  7. Ben-Haim (2009): Info-gap forecasting and the advantage of sub-optimal models.

  8. Yokomizo et al (2009): Managing the impact of invasive species: the value of knowing the density-impact curve.

  9. Davidovitch et al (2009): Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island.

  10. Ben-Haim et al (2009): Do we know how to set decision thresholds for diabetes?

  11. Beresford and Thompson (2009): An info-gap approach to managing portfolios of assets with uncertain returns

  12. Ben-Haim, Dacso, Carrasco, and Rajan (2009): Heterogeneous uncertainties in cholesterol management

  13. Rout, Thompson, and McCarthy (2009): Robust decisions for declaring eradication of invasive species

  14. Ben-Haim (2010): Info-Gap Economics: An Operational Introduction

  15. Hine and Hall (2010): Information gap analysis of flood model uncertainties and regional frequency analysis

  16. Ben-Haim (2010): Interpreting Null Results from Measurements with Uncertain Correlations: An Info-Gap Approach

  17. Wintle et al. (2010): Allocating monitoring effort in the face of unknown unknowns

  18. Moffitt et al. (2010): Securing the Border from Invasives: Robust Inspections under Severe Uncertainty

  19. Yemshanov et al. (2010): Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest

  20. Davidovitch and Ben-Haim (2010): Robust satisficing voting: why are uncertain voters biased towards sincerity?

  21. Schwartz et al. (2010): What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making

  22. Arkadeb Ghosal et al. (2010): Computing Robustness of FlexRay Schedules to Uncertainties in Design Parameters

  23. Hemez et al. (2002): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

  24. Hemez et al. (2003): Applying information-gap reasoning to the predictive accuracy assessment of transient dynamics simulations

  25. Hemez, F.M. and Ben-Haim, Y. (2004): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

  26. Ben-Haim, Y. (2007): Frequently asked questions about info-gap decision theory

  27. Sprenger, J. (2011): The Precautionary Approach and the Role of Scientists in Environmental Decision-Making

  28. Sprenger, J. (2011): Precaution with the Precautionary Principle: How does it help in making decisions

  29. Hall et al. (2011): Robust climate policies under uncertainty: A comparison of Info-­-Gap and RDM methods

  30. Ben-Haim and Cogan (2011) : Linear bounds on an uncertain non-linear oscillator: an info-gap approach

  31. Van der Burg and Tyre (2011) : Integrating info-gap decision theory with robust population management: a case study using the Mountain Plover

  32. Hildebrandt and Knoke (2011) : Investment decisions under uncertainty --- A methodological review on forest science studies.

  33. Wintle et al. (2011) : Ecological-economic optimization of biodiversity conservation under climate change.

  34. Ranger et al. (2011) : Adaptation in the UK: a decision-making process.


Recent Articles, Working Papers, Notes

Also, see my complete list of articles
    Moshe's new book!
  • Sniedovich, M. (2012) Fooled by local robustness, Risk Analysis, in press.

  • Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty, International Transactions in Operational Research, in press.

  • Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis, Applications of Mathematics, 56(5), 499-509.

  • Sniedovich, M. (2011) Dynamic programming: introductory concepts, in Wiley Encyclopedia of Operations Research and Management Science (EORMS), Wiley.

  • Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem, OR Spectrum, 33(4), 815-929, 2011.

  • Sniedovich, M. (2011) Dynamic Programming: Foundations and Principles, Second Edition, Taylor & Francis.

  • Sniedovich, M. (2010) A bird's view of Info-Gap decision theory, Journal of Risk Finance, 11(3), 268-283.

  • Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.

  • Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds), Safety, Reliability and Risk Analysis: Theory, Methods and Applications, pp. 2071-2079, Taylor and Francis Group, London.
  • .
  • Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)

  • Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.), 11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation, DIPTEM, Genova, 89-94.

  • Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)

  • Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)

  • Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08
    This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )

  • Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory, Decision Point, 24, 10.

  • Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty, Decision Point, 22, 8-9.

  • Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!, Journal of Risk Finance, 9(3), 287-291.

  • Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)
    In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.

  • Sniedovich. M. (2008) The Mighty Maximin! (PDF File)
    This paper is dedicated to the modeling aspects of Maximin and robust optimization.

  • Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty, Decision Making in Manufacturing and Services, 1-2, 111-136. (PDF File) .

  • Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)
    In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.

  • My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)

    This is an exciting development!

    • Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.

    • Ben-Haim is not familiar with the fundamental concept point estimate. He does not realize that a function can be a point estimate of another function.

      So when you read my papers make sure that you do not misinterpret the notion point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.

      Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!


  • A formal proof that Info-Gap is Wald's Maximin Principle in disguise. (December 31, 2006)
    This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)
    It shows that Info-Gap is not a new theory but rather a simple instance of Wald's famous Maximin Principle dating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context of Game Theory (1928).

  • A proof that Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)
    This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .
    It shows that because Info-Gap deploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.

  • A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)
    This is a very short article entitled The GAP in Info-Gap (PDF File) .
    It is a math-free version of the paper above. Read it if you are allergic to math.

  • A long essay entitled What's Wrong with Info-Gap? An Operations Research Perspective (PDF File) (December 31, 2006).
    This is a paper that I presented at the ASOR Recent Advances in Operations Research (PDF File) mini-conference (December 1, 2006, Melbourne, Australia).

Recent Lectures, Seminars, Presentations

If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.

Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.


Disclaimer: This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.


Last modified: [an error occurred while processing this directive]