Reviews of publications on Info-Gap decision theory
Review # 6 (Posted: April 5, 2009; Last update:April 14, 2009)
Reference: Hall, J. and Harvey, H.
Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.
Eighth International Conference on Hydroinformatics
January 12-16, 2009, Concepcion, Chile.
Abstract Flood risk analysis is subject to uncertainties, often severe, which have the potential to undermine engineering decisions. This is particularly true in strategic planning, which requires appraisal over long periods of time. Traditional economic appraisal techniques largely ignore this uncertainty, preferring to use a precise measure of performance, which affords the possibility of unambiguously ranking options in order of preference. In this paper we describe an experimental application of information-gap theory, or info-gap for short to a flood risk management decision. Info-gap is a quantified non-probabilistic theory of robustness. It provides a means of examining the sensitivity of a decision to uncertainty. Rather than simply presenting a range of possible values of performance, info-gap explores how this range grows as uncertainty increases. This allows considerably greater opportunity for insight into the behaviour of our model of option performance. The information generated may be of use in improving the model, refining the options, or justifying the selection of one option over the others in the absence of an unambiguous rank order. Secondly, we demonstrate the possibility of exploring the value of waiting until improved knowledge becomes available by constructing options that explicitly model this possibility. Acknowledgement The work described in this publication was supported by the European Community's Sixth Framework Programme through the grant to the budget of the Integrated Project FLOODsite, Contract GOCE-CT-2004-505420. This paper reflects the authors' views and not those of the European Community. Neither the European Community nor any member of the FLOODsite Consortium is liable for any use of the information in this paper. Scores TUIGF:90%
I do not know for a fact that this is a refereed paper. Still, I consider it important for this project because, to the best of my knowledge, this is the first paper by Info-Gap proponents in which some sort of an attempt is made to deal with the fact that Info-Gap's robustness model engages in a ... voodoo analysis. I suspect that this attempt at rectifying the model is due to my criticism of Info-Gap — which, I am confident, is known to the authors.
So what exactly is it that I find so interesting in this paper?
Although this paper is afflicted with TUIGF, of particular interest is the following remarkable statement made immediately after the description of Info-Gap's regions of uncertainty, the horizon of uncertainty α and the estimate û of the parameter of interest (page 2, emphasis is mine):An assumption remains that values of u become increasingly unlikely as they diverge from û.
In other words, this assumption indicates that the estimate û is the most likely value of the parameter u and that the likelihood of u decreases as u deviates from û. The inference clearly seem to be that this assumption is introduced to justify Info-Gap's fixation with the estimate as the focus of the robustness analysis.
Particularly noteworthy in the phrasing of this assumption is the word remains. What exactly are we to make of remain? Does it mean that in the context of Info-Gap, which boasts of being a non-probabilistic likelihood-free theory, this assumption was all along the case and it thus remains? If so how does it square with the "official" Info-Gap decision theory? Or, is this a "new" assumption? One that the authors decided to append to the "official" theory? If it is a newly added assumption then surely this must be made clear. Whatever the case, the authors must explain how this assumption tallies with the claim that the uncertainty under consideration is severe?
Three comments on this remarkable assumption:
- This assumption is not an integral part of info-Gap's generic model of uncertainty. The fact is that this assumption represents a radical departure from the "official" Info-Gap decision theory. Note that the founder of Info-Gap goes to great lengths to emphasize that this theory is not only probability-free, but also likelihood-free (emphasis is mine):Furthermore, this assumption is not postulated in previous Info-Gap articles by Jim Hall, for instance the paper discussed in my Responsible Decision-Making page.
In info-gap set models of uncertainty we concentrate on cluster-thinking rather than on recurrence or likelihood. Given a particular quantum of information, we ask: what is the cloud of possibilities consistent with this information? How does this cloud shrink, expand and shift as our information changes? What is the gap between what is known and what could be known. We have no recurrence information, and we can make no heuristic or lexical judgments of likelihood.Ben-Haim (2006, p. 18)
Info-Gap Decision Theory: Decisions Under Severe uncertainty
- This assumption contradicts the authors' initial assumption that the uncertainty under consideration is severe. For, how can we possibly treat the uncertainty as "severe" and at the same time assume that values of u become increasingly unlikely as they diverge from û? Furthermore, how does this assumption sit with Ben-Haim's (2006, pp. 280-281) characterization of severe uncertainty (2006, pp. 280-281) where it is posited that under severe uncertainty the estimate û "... is a poor indication of the true value ..." and " ... is likely to be substantially wrong ...".
In fact, in his FAQs about Info-Gap Ben-Haim (2007) states in so many words that under severe uncertainty the estimate û can be a wild guess. Are we to understand then that the above assumption departs from Ben-haim's position about the nature of the estimate û to contend that it is not a wild guess? Of course this assumption is unable to invalidate the fact that under severe uncertainty the estimate is a wild guess. Still, let us strike compromise and assume that here the estimate is only a mildly wild guess.
- This assumption does not even come close to correcting the basic flaw in Info-Gap's uncertainty model. The reason is very simple: it does not even remotely give the slightest guarantee that the true value of the parameter u is very likely to be in the immediate neighborhood of the estimate û. But what is more, it is easy to construct examples where this assumption is satisfied, yet the true value of u is very unlikely to be in the neighborhood of the estimate û. The implication is then that this assumption comes to naught as it fails to meet the central difficulty besetting decision-making under severe uncertainty. The fact that the estimate û is a poor indication of the true value of u and is likely to be substantially wrong.
In any event, despite its failure, this attempt at dealing with the fundamental flaw in Info-Gap's uncertainty model attests to the fact that some Info-Gap scholars begin to realize that -- in the absence of such assumptions -- Info-Gap decision theory cannot escape being a manifestly voodoo decision theory.Remark: Attempts of this kind are doomed to fail simply because it is untenable to justify a local robustness analysis in the neighborhood of the estimate û and at the same time assume that the estimate is a poor indication of the true value of u and is likely to be substantially wrong. Info-Gap scholars would thus be well advised to consult the literature on Robust Optimization to see how severe uncertainty is modeled and managed without a priori localizing the analysis around a poor estimate. I also suggest that they remind themselves of the GIGO Axiom, and of course, the second challenge cited on the list of top challenges for making robust decision (emphasis is mine):
2. Managing uncertainty
Decisions depend on the best estimates of past performance, assessments of the current situation and visions into the future. Where the past performance may be known, the current is clouded by its immediacy and the future is a best guess. The robustness of any decision and the risk incurred in making that decision is only as good as the estimates on which it is based. Making estimation even more challenging, virtually all estimates that affect decisions are uncertain. Uncertainty can not be eliminated, but it can be managed.Top Ten Challenges for Making Robust Decisions
The Decision Expert Newsletter™ -- Volume 1; Issue 2
But it is important to take note that one cannot postulate that the estimate û is so good that it is sufficient to conduct the robustness analysis in its neighborhood, and at the same time contend that the uncertainty in the true value of u is severe.
For the record, I should also point out that the article maintains the SNHNSNDN attitude towards the Maximin/Info-Gap connection. It continues to propagate the myth that Info-Gap decision theory is unique in its non-probabilistic, likelihood-free approach to the quantification and managing of uncertainty. Not only does this article fail to make the slightest mention of the famous Maximin paradigm, it takes what seems to be a rather evasive stance in this regard as born out by the assertion (page 2):" ... All of these approaches rely upon some normalised measure being applied over the space of possibilities. Info-gap theory, but [sic] contrast does not employ normalised measure at all, so does not fit anywhere within the information-theoretic hierarchy of theories of uncertainty developed by Klir . Rather, uncertainty is represented by a family of nested sets bounding the variation of system behaviour about some nominal value û. ..."
Why do the authors make do with the reference to Klir's article in the journal Reliability Engineering and Systems Safety? Why don't they refer to the theories developed by the founders of modern decision theory in the 1950s? In particular, why don't the authors refer to standard textbooks in decision theory, operations research, and robust optimization, where Wald's Maximin paradigm (circa 1940) is portrayed as the primary paradigm for decision-making under severe uncertainty? And why don't the authors refer to Sniedovich's (2007) proof that Info-Gap's robustness model is in fact a simple Maximin model?
How long can senior Info-Gap scholars keep their heads buried in the sand with regard to the Maximin/Info-Gap connection?
My view on this is rather cynical.
I believe that there is considerable pressure these days on academics to portray the methods that they use/develop as new and revolutionary. This is especially the case in applications for research grants. Indeed, what is the likelihood of your being awarded a grant if you state clearly in your grant application that your research will be based on an old mainstream theory that is described in undergraduate textbooks?
Relevant FAQs: 1-81, especially 13-19, 37, 78.
- Ben-Haim (2001, 2006): Info-Gap Decision Theory: decisions under severe uncertainty.
- Regan et al (2005): Robust decision-making under severe uncertainty for conservation management.
- Moilanen et al (2006): Planning for robust reserve networks using uncertainty analysis.
- Burgman (2008): Shakespeare, Wald and decision making under severe uncertainty.
- Ben-Haim and Demertzis (2008): Confidence in monetary policy.
- Hall and Harvey (2009): Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.
- Ben-Haim (2009): Info-gap forecasting and the advantage of sub-optimal models.
- Yokomizo et al (2009): Managing the impact of invasive species: the value of knowing the density-impact curve.
- Davidovitch et al (2009): Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island.
- Ben-Haim et al (2009): Do we know how to set decision thresholds for diabetes?
- Beresford and Thompson (2009): An info-gap approach to managing portfolios of assets with uncertain returns
- Ben-Haim, Dacso, Carrasco, and Rajan (2009): Heterogeneous uncertainties in cholesterol management
- Rout, Thompson, and McCarthy (2009): Robust decisions for declaring eradication of invasive species
- Ben-Haim (2010): Info-Gap Economics: An Operational Introduction
- Hine and Hall (2010): Information gap analysis of flood model uncertainties and regional frequency analysis
- Ben-Haim (2010): Interpreting Null Results from Measurements with Uncertain Correlations: An Info-Gap Approach
- Wintle et al. (2010): Allocating monitoring effort in the face of unknown unknowns
- Moffitt et al. (2010): Securing the Border from Invasives: Robust Inspections under Severe Uncertainty
- Yemshanov et al. (2010): Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest
- Davidovitch and Ben-Haim (2010): Robust satisficing voting: why are uncertain voters biased towards sincerity?
- Schwartz et al. (2010): What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making
- Arkadeb Ghosal et al. (2010): Computing Robustness of FlexRay Schedules to Uncertainties in Design Parameters
- Hemez et al. (2002): Info-gap robustness for the correlation of tests and simulations of a non-linear transient
- Hemez et al. (2003): Applying information-gap reasoning to the predictive accuracy assessment of transient dynamics simulations
- Hemez, F.M. and Ben-Haim, Y. (2004): Info-gap robustness for the correlation of tests and simulations of a non-linear transient
- Ben-Haim, Y. (2007): Frequently asked questions about info-gap decision theory
- Sprenger, J. (2011): The Precautionary Approach and the Role of Scientists in Environmental Decision-Making
- Sprenger, J. (2011): Precaution with the Precautionary Principle: How does it help in making decisions
- Hall et al. (2011): Robust climate policies under uncertainty: A comparison of Info--Gap and RDM methods
- Ben-Haim and Cogan (2011) : Linear bounds on an uncertain non-linear oscillator: an info-gap approach
- Van der Burg and Tyre (2011) : Integrating info-gap decision theory with robust population management: a case study using the Mountain Plover
- Hildebrandt and Knoke (2011) : Investment decisions under uncertainty --- A methodological review on forest science studies.
- Wintle et al. (2011) : Ecological-economic optimization of biodiversity conservation under climate change.
- Ranger et al. (2011) : Adaptation in the UK: a decision-making process.
Also, see my complete list of articles
Moshe's new book!
- Sniedovich, M. (2012) Fooled by local robustness, Risk Analysis, in press.
- Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty, International Transactions in Operational Research, in press.
- Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis, Applications of Mathematics, 56(5), 499-509.
- Sniedovich, M. (2011) Dynamic programming: introductory concepts, in Wiley Encyclopedia of Operations Research and Management Science (EORMS), Wiley.
- Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem, OR Spectrum, 33(4), 815-929, 2011.
- Sniedovich, M. (2011) Dynamic Programming: Foundations and Principles, Second Edition, Taylor & Francis.
- Sniedovich, M. (2010) A bird's view of Info-Gap decision theory, Journal of Risk Finance, 11(3), 268-283.
- Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.
- Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds), Safety, Reliability and Risk Analysis: Theory, Methods and Applications, pp. 2071-2079, Taylor and Francis Group, London..
- Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)
- Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.), 11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation, DIPTEM, Genova, 89-94.
- Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)
- Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)
- Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08
This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )
- Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory, Decision Point, 24, 10.
- Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty, Decision Point, 22, 8-9.
- Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!, Journal of Risk Finance, 9(3), 287-291.
- Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)
In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.
- Sniedovich. M. (2008) The Mighty Maximin! (PDF File)
This paper is dedicated to the modeling aspects of Maximin and robust optimization.
- Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty, Decision Making in Manufacturing and Services, 1-2, 111-136. (PDF File) .
- Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)
In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.
- My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)
This is an exciting development!
- Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.
- Ben-Haim is not familiar with the fundamental concept point estimate. He does not realize that a function can be a point estimate of another function.
So when you read my papers make sure that you do not misinterpret the notion point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.
Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!
- A formal proof that Info-Gap is Wald's Maximin Principle in disguise. (December 31, 2006)
This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)
It shows that Info-Gap is not a new theory but rather a simple instance of Wald's famous Maximin Principle dating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context of Game Theory (1928).
- A proof that Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)
This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .
It shows that because Info-Gap deploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.
- A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)
This is a very short article entitled The GAP in Info-Gap (PDF File) .
It is a math-free version of the paper above. Read it if you are allergic to math.
- A long essay entitled What's Wrong with Info-Gap? An Operations Research Perspective (PDF File) (December 31, 2006).
This is a paper that I presented at the ASOR Recent Advances in Operations Research (PDF File) mini-conference (December 1, 2006, Melbourne, Australia).
If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.
Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.
ASOR Recent Advances, 2011, Melbourne, Australia, November 16 2011. Presentation: The Power of the (peer-reviewed) Word. (PDF file).
- Alex Rubinov Memorial Lecture The Art, Science, and Joy of (mathematical) Decision-Making, November 7, 2011, The University of Ballarat. (PDF file).
- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .
(Invited tutorial, ALIO/INFORMS Conference, Buenos Aires, Argentina, July 6-9, 2010).
- A Critique of Info-Gap Decision theory: From Voodoo Decision-Making to Voodoo Economics(PDF File) .
(Recent Advances in OR, RMIT, Melbourne, Australia, November 25, 2009)
- Robust decision-making in the face of severe uncertainty(PDF File) .
(GRIPS, Tokyo, Japan, October 16, 2009)
- Decision-making in the face of severe uncertainty(PDF File) .
(KORDS'09 Conference, Vilnius, Lithuania, September 30 -- OCtober 3, 2009)
- Modeling robustness against severe uncertainty (PDF File) .
(SOR'09 Conference, Nova Gorica, Slovenia, September 23-25, 2009)
- How do you recognize a Voodoo decision theory?(PDF File) .
(School of Mathematical and Geospatial Sciences, RMIT, June 26, 2009).
- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, Info-Gaps, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .
(Department of Econometrics and Business Statistics, Monash University, May 8, 2009).
- The Rise and Rise of Voodoo Decision Theory.
ASOR Recent Advances, Deakin University, November 26, 2008. This presentation was based on the pages on my website (voodoo.moshe-online.com).
- Responsible Decision-Making in the face of Severe Uncertainty (PDF File) .
(Singapore Management University, Singapore, September 29, 2008)
- A Critique of Info-Gap's Robustness Model (PDF File) .
(ESREL/SRA 2008 Conference, Valencia, Spain, September 22-25, 2008)
- Robust Decision-Making in the Face of Severe Uncertainty (PDF File) .
(Technion, Haifa, Israel, September 15, 2008)
- The Art and Science of Robust Decision-Making (PDF File) .
(AIRO 2008 Conference, Ischia, Italy, September 8-11, 2008 )
- The Fundamental Flaws in Info-Gap Decision Theory (PDF File) .
(CSIRO, Canberra, July 9, 2008 )
- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .
(OR Conference, ADFA, Canberra, July 7-8, 2008 )
- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .
(University of Sydney Seminar, May 16, 2008 )
- Decision-Making Under Severe Uncertainty: An Australian, Operational Research Perspective (PDF File) .
(ASOR National Conference, Melbourne, December 3-5, 2007 )
- A Critique of Info-Gap (PDF File) .
(SRA 2007 Conference, Hobart, August 20, 2007)
- What exactly is wrong with Info-Gap? A Decision Theoretic Perspective (PDF File) .
(MS Colloquium, University of Melbourne, August 1, 2007)
- A Formal Look at Info-Gap Theory (PDF File) .
(ORSUM Seminar , University of Melbourne, May 21, 2007)
- The Art and Science of Decision-Making Under Severe Uncertainty (PDF File) .
(ACERA seminar, University of Melbourne, May 4, 2007)
- What exactly is Info-Gap? An OR perspective. (PDF File)
ASOR Recent Advances in Operations Research mini-conference (December 1, 2006, Melbourne, Australia).
Disclaimer: This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.
Last modified: [an error occurred while processing this directive]