The Spin Stops Here! Decision-Making Under Severe Uncertainty	Faqs | Help | @ | Contact |
voodoostan info-gap decision theory info-gap economics severe uncertainty mighty maximin robust decisions responsible decisions

---

Reviews of publications on Info-Gap decision theory

---

Review # 4 (Posted: April 5, 2009; Last update: April 14, 2009)

Reference:	Burgman, M. Shakespeare, Wald and decision making under severe uncertainty. Decision Point (23), p. 10 October, 2008
Scores	TUIGF:100% SNHNSNDN:0% GIGO:120%

---

This is Burgman's response to my article "From Shakespeare to Wald: modelling worst-case analysis in the face of severe uncertainty" (Decision Point 22, 2008, pp. 8-9)

I had already responded to Burgman's (2008) paper elsewhere on my web site, see

Info-Gap decision theory and the small applied world of environmental decision-making

PDF version                   HTML version

But for your convenience I mention it briefly here. You can regard this as a "review" of Burgman's (2008) article.

For an idea as to the kind of objections that Burgman (2008) raises to my criticism of Info-Gap decision theory, consider this.

Burgman's (2008) seeks to portray my criticism of Info-Gap decision theory as being no more than semantic. For example, he criticizes me for misrepresenting the decision problems in the conservation biology and applied ecology literature. He points out that in my paper I formulate this questions as follows:

How wrong can I be, yet get an acceptable level of performance?

He then goes on to claim

But this question wasn't asked in the first place. Rather, the question asked in various applications of Info-gap has been the following: How wrong can this model be, without jeopardizing an acceptable level of performance?

So, I had to remind him that my formulation is based on that in Halpern et al (2006, p. 2), a formulation of the question that reads as follows:

How wrong can one be and still get an acceptable result?

I also had to remind him that this is not the question that is in fact addressed by Info-Gap's robustness model. Rather, Info-Gap's robustness model addresses the following question

How much can I deviate from a given estimate so that the performance requirement is satisfied throughout the region of uncertainty (around the estimate) which is contained within this deviation?

The point is that my criticism of Info-Gap raises the ire of its scholars/users because it shows this theory for what it is: a voodoo decision theory par excellence. Specifically, my unrelenting hammering of the obvious that, measured as it is in the neighborhood of the point estimate û, Info-Gap's robustness is local. Thus, in the context of severe uncertainty, where the estimate is assumed to be a wild guess, or at best a mildly wild guess, Info-Gap's analysis amounts to a voodoo analysis.

It is important to read my response to Burgman's (2008) comments because these give a good indication of how deep-rooted are the misconception about what Info-Gap decision theory promises on one hand and what it actually delivers on the other.

The high GIGO score is for the assertion that my definition of "severe uncertainty" is " ... too narrow to be useful ...". As indicated in my response,

• I do not have a definition for "severe uncertainty".
• The working assumption that I use to describe the meaning of this term in the context of Info-Gap decision theory is that introduced by the Father of Info-Gap: the estimate can be a wild guess, it is a poor indication of the true value of the parameter of interest and is likely to be substantially wrong.

The low SNHNSNDN score is in recognition of Burgman's (2008) reference to my claim that Info-Gap's robustness model is a Maximin model.

Other Reviews

1. Ben-Haim (2001, 2006): Info-Gap Decision Theory: decisions under severe uncertainty.


2. Regan et al (2005): Robust decision-making under severe uncertainty for conservation management.


3. Moilanen et al (2006): Planning for robust reserve networks using uncertainty analysis.


4. Burgman (2008): Shakespeare, Wald and decision making under severe uncertainty.


5. Ben-Haim and Demertzis (2008): Confidence in monetary policy.


6. Hall and Harvey (2009): Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.


7. Ben-Haim (2009): Info-gap forecasting and the advantage of sub-optimal models.


8. Yokomizo et al (2009): Managing the impact of invasive species: the value of knowing the density-impact curve.


9. Davidovitch et al (2009): Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island.


10. Ben-Haim et al (2009): Do we know how to set decision thresholds for diabetes?


11. Beresford and Thompson (2009): An info-gap approach to managing portfolios of assets with uncertain returns


12. Ben-Haim, Dacso, Carrasco, and Rajan (2009): Heterogeneous uncertainties in cholesterol management


13. Rout, Thompson, and McCarthy (2009): Robust decisions for declaring eradication of invasive species


14. Ben-Haim (2010): Info-Gap Economics: An Operational Introduction


15. Hine and Hall (2010): Information gap analysis of flood model uncertainties and regional frequency analysis


16. Ben-Haim (2010): Interpreting Null Results from Measurements with Uncertain Correlations: An Info-Gap Approach


17. Wintle et al. (2010): Allocating monitoring effort in the face of unknown unknowns


18. Moffitt et al. (2010): Securing the Border from Invasives: Robust Inspections under Severe Uncertainty


19. Yemshanov et al. (2010): Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest


20. Davidovitch and Ben-Haim (2010): Robust satisficing voting: why are uncertain voters biased towards sincerity?


21. Schwartz et al. (2010): What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making


22. Arkadeb Ghosal et al. (2010): Computing Robustness of FlexRay Schedules to Uncertainties in Design Parameters


23. Hemez et al. (2002): Info-gap robustness for the correlation of tests and simulations of a non-linear transient


24. Hemez et al. (2003): Applying information-gap reasoning to the predictive accuracy assessment of transient dynamics simulations


25. Hemez, F.M. and Ben-Haim, Y. (2004): Info-gap robustness for the correlation of tests and simulations of a non-linear transient


26. Ben-Haim, Y. (2007): Frequently asked questions about info-gap decision theory


27. Sprenger, J. (2011): The Precautionary Approach and the Role of Scientists in Environmental Decision-Making


28. Sprenger, J. (2011): Precaution with the Precautionary Principle: How does it help in making decisions


29. Hall et al. (2011): Robust climate policies under uncertainty: A comparison of Info-­-Gap and RDM methods


30. Ben-Haim and Cogan (2011) : Linear bounds on an uncertain non-linear oscillator: an info-gap approach


31. Van der Burg and Tyre (2011) : Integrating info-gap decision theory with robust population management: a case study using the Mountain Plover


32. Hildebrandt and Knoke (2011) : Investment decisions under uncertainty --- A methodological review on forest science studies.


33. Wintle et al. (2011) : Ecological-economic optimization of biodiversity conservation under climate change.


34. Ranger et al. (2011) : Adaptation in the UK: a decision-making process.

Recent Articles, Working Papers, Notes

Also, see my complete list of articles

Moshe's new book!

• Sniedovich, M. (2012) Fooled by local robustness, Risk Analysis, in press.


• Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty, International Transactions in Operational Research, in press.


• Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis, Applications of Mathematics, 56(5), 499-509.


• Sniedovich, M. (2011) Dynamic programming: introductory concepts, in Wiley Encyclopedia of Operations Research and Management Science (EORMS), Wiley.


• Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem, OR Spectrum, 33(4), 815-929, 2011.


• Sniedovich, M. (2011) Dynamic Programming: Foundations and Principles, Second Edition, Taylor & Francis.


• Sniedovich, M. (2010) A bird's view of Info-Gap decision theory, Journal of Risk Finance, 11(3), 268-283.


• Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.


• Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds), Safety, Reliability and Risk Analysis: Theory, Methods and Applications, pp. 2071-2079, Taylor and Francis Group, London.
.

• Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)


• Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.), 11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation, DIPTEM, Genova, 89-94.


• Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)


• Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)


• Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08
  This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )


• Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory, Decision Point, 24, 10.


• Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty, Decision Point, 22, 8-9.


• Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!, Journal of Risk Finance, 9(3), 287-291.


• Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)
  In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.


• Sniedovich. M. (2008) The Mighty Maximin! (PDF File)
  This paper is dedicated to the modeling aspects of Maximin and robust optimization.


• Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty, Decision Making in Manufacturing and Services, 1-2, 111-136. (PDF File) .


• Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)
  In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.


• My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)

  This is an exciting development!

  • Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.
  
  
  • Ben-Haim is not familiar with the fundamental concept point estimate. He does not realize that a function can be a point estimate of another function.

    So when you read my papers make sure that you do not misinterpret the notion point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.

    Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!



• A formal proof that Info-Gap is Wald's Maximin Principle in disguise. (December 31, 2006)
  This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)
  It shows that Info-Gap is not a new theory but rather a simple instance of Wald's famous Maximin Principle dating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context of Game Theory (1928).


• A proof that Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)
  This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .
  It shows that because Info-Gap deploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.


• A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)
  This is a very short article entitled The GAP in Info-Gap (PDF File) .
  It is a math-free version of the paper above. Read it if you are allergic to math.


• A long essay entitled What's Wrong with Info-Gap? An Operations Research Perspective (PDF File) (December 31, 2006).
  This is a paper that I presented at the ASOR Recent Advances in Operations Research (PDF File) mini-conference (December 1, 2006, Melbourne, Australia).

Recent Lectures, Seminars, Presentations

If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.

Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.

ASOR Recent Advances, 2011, Melbourne, Australia, November 16 2011. Presentation: The Power of the (peer-reviewed) Word. (PDF file).

• Alex Rubinov Memorial Lecture The Art, Science, and Joy of (mathematical) Decision-Making, November 7, 2011, The University of Ballarat. (PDF file).


• Black Swans, Modern Nostradamuses, Voodoo Decision Theories, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .
  (Invited tutorial, ALIO/INFORMS Conference, Buenos Aires, Argentina, July 6-9, 2010).


• A Critique of Info-Gap Decision theory: From Voodoo Decision-Making to Voodoo Economics(PDF File) .
  (Recent Advances in OR, RMIT, Melbourne, Australia, November 25, 2009)


• Robust decision-making in the face of severe uncertainty(PDF File) .
  (GRIPS, Tokyo, Japan, October 16, 2009)


• Decision-making in the face of severe uncertainty(PDF File) .
  (KORDS'09 Conference, Vilnius, Lithuania, September 30 -- OCtober 3, 2009)


• Modeling robustness against severe uncertainty (PDF File) .
  (SOR'09 Conference, Nova Gorica, Slovenia, September 23-25, 2009)


• How do you recognize a Voodoo decision theory?(PDF File) .
  (School of Mathematical and Geospatial Sciences, RMIT, June 26, 2009).


• Black Swans, Modern Nostradamuses, Voodoo Decision Theories, Info-Gaps, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .
  (Department of Econometrics and Business Statistics, Monash University, May 8, 2009).


• The Rise and Rise of Voodoo Decision Theory.
  ASOR Recent Advances, Deakin University, November 26, 2008. This presentation was based on the pages on my website (voodoo.moshe-online.com).


• Responsible Decision-Making in the face of Severe Uncertainty (PDF File) .
  (Singapore Management University, Singapore, September 29, 2008)


• A Critique of Info-Gap's Robustness Model (PDF File) .
  (ESREL/SRA 2008 Conference, Valencia, Spain, September 22-25, 2008)


• Robust Decision-Making in the Face of Severe Uncertainty (PDF File) .
  (Technion, Haifa, Israel, September 15, 2008)


• The Art and Science of Robust Decision-Making (PDF File) .
  (AIRO 2008 Conference, Ischia, Italy, September 8-11, 2008 )


• The Fundamental Flaws in Info-Gap Decision Theory (PDF File) .
  (CSIRO, Canberra, July 9, 2008 )


• Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .
  (OR Conference, ADFA, Canberra, July 7-8, 2008 )


• Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .
  (University of Sydney Seminar, May 16, 2008 )


• Decision-Making Under Severe Uncertainty: An Australian, Operational Research Perspective (PDF File) .
  (ASOR National Conference, Melbourne, December 3-5, 2007 )


• A Critique of Info-Gap (PDF File) .
  (SRA 2007 Conference, Hobart, August 20, 2007)


• What exactly is wrong with Info-Gap? A Decision Theoretic Perspective (PDF File) .
  (MS Colloquium, University of Melbourne, August 1, 2007)


• A Formal Look at Info-Gap Theory (PDF File) .
  (ORSUM Seminar , University of Melbourne, May 21, 2007)


• The Art and Science of Decision-Making Under Severe Uncertainty (PDF File) .
  (ACERA seminar, University of Melbourne, May 4, 2007)


• What exactly is Info-Gap? An OR perspective. (PDF File)
  ASOR Recent Advances in Operations Research mini-conference (December 1, 2006, Melbourne, Australia).

---

Disclaimer: This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.

Last modified: