The Spin Stops Here!
Decision-Making Under Severe Uncertainty  
Faqs | Help | @ | Contact | home  
voodoostan info-gap decision theory info-gap economics severe uncertainty mighty maximin robust decisions responsible decisions

Reviews of publications on Info-Gap decision theory

Review # 6 (Posted: April 5, 2009; Last update:April 14, 2009)

Reference: Hall, J. and Harvey, H.
Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.
Eighth International Conference on Hydroinformatics
January 12-16, 2009, Concepcion, Chile.
(PDF file)
Abstract Flood risk analysis is subject to uncertainties, often severe, which have the potential to undermine engineering decisions. This is particularly true in strategic planning, which requires appraisal over long periods of time. Traditional economic appraisal techniques largely ignore this uncertainty, preferring to use a precise measure of performance, which affords the possibility of unambiguously ranking options in order of preference. In this paper we describe an experimental application of information-gap theory, or info-gap for short to a flood risk management decision. Info-gap is a quantified non-probabilistic theory of robustness. It provides a means of examining the sensitivity of a decision to uncertainty. Rather than simply presenting a range of possible values of performance, info-gap explores how this range grows as uncertainty increases. This allows considerably greater opportunity for insight into the behaviour of our model of option performance. The information generated may be of use in improving the model, refining the options, or justifying the selection of one option over the others in the absence of an unambiguous rank order. Secondly, we demonstrate the possibility of exploring the value of waiting until improved knowledge becomes available by constructing options that explicitly model this possibility.
Acknowledgement The work described in this publication was supported by the European Community's Sixth Framework Programme through the grant to the budget of the Integrated Project FLOODsite, Contract GOCE-CT-2004-505420. This paper reflects the authors' views and not those of the European Community. Neither the European Community nor any member of the FLOODsite Consortium is liable for any use of the information in this paper.

I do not know for a fact that this is a refereed paper. Still, I consider it important for this project because, to the best of my knowledge, this is the first paper by Info-Gap proponents in which some sort of an attempt is made to deal with the fact that Info-Gap's robustness model engages in a ... voodoo analysis. I suspect that this attempt at rectifying the model is due to my criticism of Info-Gap — which, I am confident, is known to the authors.

So what exactly is it that I find so interesting in this paper?

Although this paper is afflicted with TUIGF, of particular interest is the following remarkable statement made immediately after the description of Info-Gap's regions of uncertainty, the horizon of uncertainty α and the estimate û of the parameter of interest (page 2, emphasis is mine):

An assumption remains that values of u become increasingly unlikely as they diverge from û.

In other words, this assumption indicates that the estimate û is the most likely value of the parameter u and that the likelihood of u decreases as u deviates from û. The inference clearly seem to be that this assumption is introduced to justify Info-Gap's fixation with the estimate as the focus of the robustness analysis.

Particularly noteworthy in the phrasing of this assumption is the word remains. What exactly are we to make of remain? Does it mean that in the context of Info-Gap, which boasts of being a non-probabilistic likelihood-free theory, this assumption was all along the case and it thus remains? If so how does it square with the "official" Info-Gap decision theory? Or, is this a "new" assumption? One that the authors decided to append to the "official" theory? If it is a newly added assumption then surely this must be made clear. Whatever the case, the authors must explain how this assumption tallies with the claim that the uncertainty under consideration is severe?

Three comments on this remarkable assumption:

In any event, despite its failure, this attempt at dealing with the fundamental flaw in Info-Gap's uncertainty model attests to the fact that some Info-Gap scholars begin to realize that -- in the absence of such assumptions -- Info-Gap decision theory cannot escape being a manifestly voodoo decision theory.

Remark: Attempts of this kind are doomed to fail simply because it is untenable to justify a local robustness analysis in the neighborhood of the estimate û and at the same time assume that the estimate is a poor indication of the true value of u and is likely to be substantially wrong. Info-Gap scholars would thus be well advised to consult the literature on Robust Optimization to see how severe uncertainty is modeled and managed without a priori localizing the analysis around a poor estimate. I also suggest that they remind themselves of the GIGO Axiom, and of course, the second challenge cited on the list of top challenges for making robust decision (emphasis is mine):

2. Managing uncertainty
Decisions depend on the best estimates of past performance, assessments of the current situation and visions into the future. Where the past performance may be known, the current is clouded by its immediacy and the future is a best guess. The robustness of any decision and the risk incurred in making that decision is only as good as the estimates on which it is based. Making estimation even more challenging, virtually all estimates that affect decisions are uncertain. Uncertainty can not be eliminated, but it can be managed.

Top Ten Challenges for Making Robust Decisions
The Decision Expert Newsletter™ -- Volume 1; Issue 2

But it is important to take note that one cannot postulate that the estimate û is so good that it is sufficient to conduct the robustness analysis in its neighborhood, and at the same time contend that the uncertainty in the true value of u is severe.

For the record, I should also point out that the article maintains the SNHNSNDN attitude towards the Maximin/Info-Gap connection. It continues to propagate the myth that Info-Gap decision theory is unique in its non-probabilistic, likelihood-free approach to the quantification and managing of uncertainty. Not only does this article fail to make the slightest mention of the famous Maximin paradigm, it takes what seems to be a rather evasive stance in this regard as born out by the assertion (page 2):

" ... All of these approaches rely upon some normalised measure being applied over the space of possibilities. Info-gap theory, but [sic] contrast does not employ normalised measure at all, so does not fit anywhere within the information-theoretic hierarchy of theories of uncertainty developed by Klir [9]. Rather, uncertainty is represented by a family of nested sets bounding the variation of system behaviour about some nominal value û. ..."

Why do the authors make do with the reference to Klir's article in the journal Reliability Engineering and Systems Safety? Why don't they refer to the theories developed by the founders of modern decision theory in the 1950s? In particular, why don't the authors refer to standard textbooks in decision theory, operations research, and robust optimization, where Wald's Maximin paradigm (circa 1940) is portrayed as the primary paradigm for decision-making under severe uncertainty? And why don't the authors refer to Sniedovich's (2007) proof that Info-Gap's robustness model is in fact a simple Maximin model?

How long can senior Info-Gap scholars keep their heads buried in the sand with regard to the Maximin/Info-Gap connection?

My view on this is rather cynical.

I believe that there is considerable pressure these days on academics to portray the methods that they use/develop as new and revolutionary. This is especially the case in applications for research grants. Indeed, what is the likelihood of your being awarded a grant if you state clearly in your grant application that your research will be based on an old mainstream theory that is described in undergraduate textbooks?

Relevant FAQs: 1-81, especially 13-19, 37, 78.

Other Reviews

  1. Ben-Haim (2001, 2006): Info-Gap Decision Theory: decisions under severe uncertainty.

  2. Regan et al (2005): Robust decision-making under severe uncertainty for conservation management.

  3. Moilanen et al (2006): Planning for robust reserve networks using uncertainty analysis.

  4. Burgman (2008): Shakespeare, Wald and decision making under severe uncertainty.

  5. Ben-Haim and Demertzis (2008): Confidence in monetary policy.

  6. Hall and Harvey (2009): Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.

  7. Ben-Haim (2009): Info-gap forecasting and the advantage of sub-optimal models.

  8. Yokomizo et al (2009): Managing the impact of invasive species: the value of knowing the density-impact curve.

  9. Davidovitch et al (2009): Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island.

  10. Ben-Haim et al (2009): Do we know how to set decision thresholds for diabetes?

  11. Beresford and Thompson (2009): An info-gap approach to managing portfolios of assets with uncertain returns

  12. Ben-Haim, Dacso, Carrasco, and Rajan (2009): Heterogeneous uncertainties in cholesterol management

  13. Rout, Thompson, and McCarthy (2009): Robust decisions for declaring eradication of invasive species

  14. Ben-Haim (2010): Info-Gap Economics: An Operational Introduction

  15. Hine and Hall (2010): Information gap analysis of flood model uncertainties and regional frequency analysis

  16. Ben-Haim (2010): Interpreting Null Results from Measurements with Uncertain Correlations: An Info-Gap Approach

  17. Wintle et al. (2010): Allocating monitoring effort in the face of unknown unknowns

  18. Moffitt et al. (2010): Securing the Border from Invasives: Robust Inspections under Severe Uncertainty

  19. Yemshanov et al. (2010): Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest

  20. Davidovitch and Ben-Haim (2010): Robust satisficing voting: why are uncertain voters biased towards sincerity?

  21. Schwartz et al. (2010): What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making

  22. Arkadeb Ghosal et al. (2010): Computing Robustness of FlexRay Schedules to Uncertainties in Design Parameters

  23. Hemez et al. (2002): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

  24. Hemez et al. (2003): Applying information-gap reasoning to the predictive accuracy assessment of transient dynamics simulations

  25. Hemez, F.M. and Ben-Haim, Y. (2004): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

  26. Ben-Haim, Y. (2007): Frequently asked questions about info-gap decision theory

  27. Sprenger, J. (2011): The Precautionary Approach and the Role of Scientists in Environmental Decision-Making

  28. Sprenger, J. (2011): Precaution with the Precautionary Principle: How does it help in making decisions

  29. Hall et al. (2011): Robust climate policies under uncertainty: A comparison of Info-­-Gap and RDM methods

  30. Ben-Haim and Cogan (2011) : Linear bounds on an uncertain non-linear oscillator: an info-gap approach

  31. Van der Burg and Tyre (2011) : Integrating info-gap decision theory with robust population management: a case study using the Mountain Plover

  32. Hildebrandt and Knoke (2011) : Investment decisions under uncertainty --- A methodological review on forest science studies.

  33. Wintle et al. (2011) : Ecological-economic optimization of biodiversity conservation under climate change.

  34. Ranger et al. (2011) : Adaptation in the UK: a decision-making process.

Recent Articles, Working Papers, Notes

Also, see my complete list of articles
    Moshe's new book!
  • Sniedovich, M. (2012) Fooled by local robustness, Risk Analysis, in press.

  • Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty, International Transactions in Operational Research, in press.

  • Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis, Applications of Mathematics, 56(5), 499-509.

  • Sniedovich, M. (2011) Dynamic programming: introductory concepts, in Wiley Encyclopedia of Operations Research and Management Science (EORMS), Wiley.

  • Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem, OR Spectrum, 33(4), 815-929, 2011.

  • Sniedovich, M. (2011) Dynamic Programming: Foundations and Principles, Second Edition, Taylor & Francis.

  • Sniedovich, M. (2010) A bird's view of Info-Gap decision theory, Journal of Risk Finance, 11(3), 268-283.

  • Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.

  • Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds), Safety, Reliability and Risk Analysis: Theory, Methods and Applications, pp. 2071-2079, Taylor and Francis Group, London.
  • .
  • Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)

  • Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.), 11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation, DIPTEM, Genova, 89-94.

  • Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)

  • Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)

  • Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08
    This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )

  • Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory, Decision Point, 24, 10.

  • Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty, Decision Point, 22, 8-9.

  • Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!, Journal of Risk Finance, 9(3), 287-291.

  • Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)
    In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.

  • Sniedovich. M. (2008) The Mighty Maximin! (PDF File)
    This paper is dedicated to the modeling aspects of Maximin and robust optimization.

  • Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty, Decision Making in Manufacturing and Services, 1-2, 111-136. (PDF File) .

  • Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)
    In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.

  • My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)

    This is an exciting development!

    • Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.

    • Ben-Haim is not familiar with the fundamental concept point estimate. He does not realize that a function can be a point estimate of another function.

      So when you read my papers make sure that you do not misinterpret the notion point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.

      Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!

  • A formal proof that Info-Gap is Wald's Maximin Principle in disguise. (December 31, 2006)
    This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)
    It shows that Info-Gap is not a new theory but rather a simple instance of Wald's famous Maximin Principle dating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context of Game Theory (1928).

  • A proof that Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)
    This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .
    It shows that because Info-Gap deploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.

  • A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)
    This is a very short article entitled The GAP in Info-Gap (PDF File) .
    It is a math-free version of the paper above. Read it if you are allergic to math.

  • A long essay entitled What's Wrong with Info-Gap? An Operations Research Perspective (PDF File) (December 31, 2006).
    This is a paper that I presented at the ASOR Recent Advances in Operations Research (PDF File) mini-conference (December 1, 2006, Melbourne, Australia).

Recent Lectures, Seminars, Presentations

If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.

Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.

Disclaimer: This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.

Last modified: