Reviews of publications on Info-Gap decision theoryLast modified: Friday, 30-Dec-2011 13:03:02 MST

On this page I comment on publications dealing with Info-Gap decision theory. Although this commentary is intended primarily for the benefit of the authors and reviewers of these publications, it clearly lends support to and justifies the rationale behind my Info-Gap Campaign.

Requests for reviews of Info-Gap publications that are not currently on the list will be considered.

Moshe's new book!

Table of contents:

- New Arrivals
- Reviews
- Overview
- Selection criteria
- How to read the reviews
- Reviews
- Factland
- Articles, papers, notes
- Lectures, seminars
## New Arrivals

The following is a list of new info-gap publications. If you want to be on the list, send me the details of your info-gap publication.

- Riegels, N., Jensen, R., Bensasson, L., Banou, S., Møller, F. and Bauer-Gottwein, P. (2011)

Estimating resource costs of compliance with EU WFD ecological status requirements at the river basin scale

Journal of Hydrology,396:197-214.

- Keith J. Beven and Ruth E. Alcock (2011).

Modelling everything everywhere: a new approach to decision-making for water management under uncertainty.

Freshwater Biology,in press 2011.

- R . Xu , H . Tang and S . Xue (2011).

Structural robust design based on Info-Gap model.

In Jinrong Zhu (editor),Modelling and Computation in Engineering,pp. 67–71. CRC Press.

- Vanessa M. Adams and Robert L. Pressey (2011)

An info-gap model to examine the robustness of cost-efficient budget allocations

ICVRAM 2011: 1st International Conference on Vulnerability and Risk Assessment and Management, April 11-13, 2011, University of Maryland, College Park, pp. 971-979.

- Lior Davidovitch and Yakov Ben-Haim (2011)

Robust resource allocation: An info-gap approach, ICVRAM 2011: 1st International Conference on Vulnerability and Risk Assessment and Management, April 11-13, 2011, University of Maryland, College Park, pp. 988-995.

- Eve McDonald-Madden, Peter W.J. Baxter and Hugh Possibgham (2011)

Robust conservation decision-making, International Conference on Vulnerability and Risk Analysis and Management, ICVRAM 2011 and the International Symposium on Uncertainty Modeling and Analysis, ISUMA 2011, Hyattsville, MD, 11-13 April 2011, pp. 945-952.

- Robert Lempert and Nidhi Kalra (2011)

Managing Climate Risks in Developing Countries with Robust Decision Making, World Resources Report, Washington. DC. Available at: http://www.worldresourcesreport.org.

Isn't it amazing that the flow of info-gap publications continues?

Let me know of any other publications that should be on this list.

## Reviews

- Ben-Haim (2001, 2006): Info-Gap Decision Theory: decisions under severe uncertainty.

- Regan et al (2005): Robust decision-making under severe uncertainty for conservation management.

- Moilanen et al (2006): Planning for robust reserve networks using uncertainty analysis.

- Burgman (2008): Shakespeare, Wald and decision making under severe uncertainty.

- Ben-Haim and Demertzis (2008): Confidence in monetary policy.

- Hall and Harvey (2009): Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.

- Ben-Haim (2009): Info-gap forecasting and the advantage of sub-optimal models.

- Yokomizo et al (2009): Managing the impact of invasive species: the value of knowing the density-impact curve.

- Davidovitch et al (2009): Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island.

- Ben-Haim et al (2009): Do we know how to set decision thresholds for diabetes?

- Beresford and Thompson (2009): An info-gap approach to managing portfolios of assets with uncertain returns.

- Ben-Haim, Dacso, Carrasco, and Rajan (2009): Heterogeneous uncertainties in cholesterol management.

- Rout, Thompson, and McCarthy (2009): Robust decisions for declaring eradication of invasive species.

- Ben-Haim (2010): Info-Gap Economics: An Operational Introduction.

- Hine and Hall (2010): Information gap analysis of flood model uncertainties and regional frequency analysis.

- Ben-Haim (2010): Interpreting Null Results from Measurements with Uncertain Correlations: An Info-Gap Approach

- Wintle et al. (2010): Allocating monitoring effort in the face of unknown unknowns

- Moffitt et al. (2010): Securing the Border from Invasives: Robust Inspections under Severe Uncertainty

- Yemshanov et al. (2010): Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest

- Davidovitch and Ben-Haim (2010): Robust satisficing voting: why are uncertain voters biased towards sincerity?

- Schwartz et al. (2010): What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making

- Arkadeb Ghosal et al. (2010): Computing Robustness of FlexRay Schedules to Uncertainties in Design Parameters

- Hemez et al. (2002): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

- Hemez et al. (2003): Applying information-gap reasoning to the predictive accuracy assessment of transient dynamics simulations

- Hemez, F.M. and Ben-Haim, Y. (2004): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

- Ben-Haim, Y. (2007): Frequently asked questions about info-gap decision theory

- Sprenger, J. (2011): The Precautionary Approach and the Role of Scientists in Environmental Decision-Making

- Sprenger, J. (2011): Precaution with the Precautionary Principle: How does it help in making decisions

- Hall et al. (2011) : Robust climate policies under uncertainty: A comparison of Info-‐Gap and RDM methods

- Ben-Haim and Cogan (2011) : Linear bounds on an uncertain non-linear oscillator: an info-gap approach

- Van der Burg and Tyre (2011) : Integrating info-gap decision theory with robust population management: a case study using the Mountain Plover

- Hildebrandt and Knoke (2011) : Investment decisions under uncertainty --- A methodological review on forest science studies.

- Wintle et al. (2011) : Ecological–economic optimization of biodiversity conservation under climate change.

- Ranger et al. (2010) : Adaptation in the UK: a decision-making process.

I shall expand the list as required as this project progresses. If you can't wait, send me a note about your very special Reader-type.

Let me know of any articles/books that you think should appear on this list.

In addition to a brief commentary on the Info-Gap content of the publications, I also provide

TUIGF,SNHNSNDNandGIGOscores (%) (see below).

## Overview

Over the past two years I have been asked, on several occasions, the following intriguing question:

Given that the flaws in Info-Gap decision theory are so obvious and grave, how do you explain the large number of publications on Info-Gap, particularly the significant number of articles published by refereed journals?My usual answer to this question is as follows:

I'll be happy to discuss this matter with you over a cup of coffee (skinny latte, please!)I do not plan to deviate from this answer here.

However, I believe that the reviews that I provide here will give you a good idea of what I may tell you over a cup of coffee so, try to read between the lines. And, for a fuller appreciation of one of the central points that I am making in this collection of reviews, read the discussion on fog, spin and rhetoric. Because, as you will see, whatever Info-Gap decision theory lacks in substance and rigor, is made up for, and covered up, by heavy fog, spin, and rhetoric.

I use the acronym

TUIGFas shorthand for "The usual Info-Gap flaws". These flaws are discussed in detail in my compilation of FAQs about Info-Gap so there is no need to elaborate on them in these reviews. On the other hand, they will be used as benchmarks.I also use the acronym

SNHNSNDNas shorthand for "see nothing, hear nothing, say nothing, do nothing". This refers to authors who, for various reasons, choose to ignore my criticism of Info-Gap decision theory, particularly my theorems proving that Info-Gap's robustness model is a Maximin model (circa 1940) and Info-Gap's opportuneness model is a Minimin model (circa 1952). For your convenience and enjoyment these beauties are displayed at the bottom of the page.So, recall that Info-Gap's two main fundamental flaws are:

- It misrepresents the facts about its place and role in decision theory:

Info-Gap decision theory is hailed as new, revolutionary and radically different from all current theories for decisions under uncertainty. But the fact of the matter is that its robustness model is just a simple Maximin model (circa 1940) and its opportuneness model is just a simple Minimin model (circa 1952). The situation is further exacerbated by Ben-Haim's repeated futile attempts to show that Info-Gap's robustness model is not a Maximin model.The proof is straightforward: see the Maximin Theorem at the bottom of the page.

- It fails to tackle the severity of the uncertainty that it professes to manage:

Info-Gap is hailed as a methodology that is designed specifically for decision-making undersevereuncertainty. Yet, a superficial examination of its robustness and opportuneness models immediately reveals that not only do these models fail to tackle the severity of the uncertainty, they ignore the severity of the uncertainty altogether. More specifically, these models arelocalin that they explore, in the first instance, theimmediate neighborhoodof a given estimate. Since under conditions ofsevereuncertainty the estimate is apoorindication of the true value of u and is likely to besubstantially wrong, these models are in violation of the Garbage In -- Garbage Out (GIGO) Axiom. This is the reason for my proclaiming Info-Gap decision theory a classic example of voodoo decision-making.See the No Man's Land and Invariance Theorems at the bottom of the page.

The question therefore arises: how is it that so many Info-Gap papers have been published in refereed journals? Surely, the flaws are so obvious that any referee with a basic knowledge of decision-making under severe uncertainty should have been able to identify them.

Indeed, even more basic is the point that Info-Gap scholars who should know better continue to publish Info-Gap papers, repeating the same unsubstantiated claims, again and again and again.

The objective of my reviews is to show these publications for what they are.

## Selection criteria

I do not have strict criteria for the selection of papers for review. For obvious reasons I prefer the more recent papers that were published in refereed journals. But I shall consider other papers, including working papers -- if they shed new light on on the failings of Info-Gap decision theory and its applications.

So the plan is to start with recently (2009) published papers, especially papers that attempt to deal with my criticism.

If you have come across a publication that you think ought to be reviewed, please send me a copy of it and I shall have a look at it to see if it is suitable for this project.

## How to read the reviews

I realize that the potential readership of my reviews may be diverse with various interests and degrees of knowledge of decision theory in general and Info-Gap decision theory in particular.

My aim is to address my reviews, in the first instance, to the authors of the articles, the referees who reviewed them, the editors of the journals who accepted the papers for publication, and the bodies that provided the funds to support the work on which the articles are based. So, please read the reviews with this point in mind.

- My reviews are based on the (unrealistic) assumption that you have read the original publications.

- If you are conversant with classical decision theory you should be able to make sense of my reviews even if you have not read the publications themselves.

- Recall that according to Info-Gap decision theory, the robustness of decision d is defined as follows:
α(d,û) = max {α≥0: r(d,u) ≥ r*, ∀u∈U(α,û)}

where:

- d = decision;

- u = parameter of interest whose true value is subject severe uncertainty;

- û = estimate of the true value of u;

- α = size of the region of uncertainty;

- U(α,û) = uncertainty region of size α centered at û

- r* = critical level of performance;

- r(d,u) = level of performance attained by decision d if the true value of the parameter of interest is u;

- The constraint r(d,u) ≥ r* is referred to as the "performance requirement".
Let

Udenote thecompleteregion of uncertainty. Roughly, this is a set that contains U(α,û) for all α≥0.Any assessment of Info-Gap's contribution to the state of the art in decision theory must be made against these claims by Ben-Haim (2001, 2006) — the Father of Info-Gap:

Info-gap decision theory isandradically differentfromallcurrent theories of decision under uncertainty. The difference originates in the modeling of uncertainty as an information gap rather than as a probability. The need for info-gap modeling and management of uncertainty arises in dealing withsevere lack of informationandhighly unstructureduncertainty.Ben-Haim (2006, p. xii)

In this book we concentrate on the fairlynewconcept of information-gap uncertainty, whosedifferencesfrom more classical approaches to uncertainty arereal and deep. Despite the power of classical decision theories, in many areas such as engineering, economics, management, medicine and public policy, a need has arisen for adifferent formatfor decisions based onseverely uncertainevidence.Ben-Haim (2006, p. 11)

So my reviews address primarily the following two questions:

- Is Info-Gap radically different from classical decision theories?

- Does Info-Gap tackles the severity of the uncertainty?
As indicated above, the fact of the matter is that Info-Gap decision theory fails on both counts. So, it should come as no surprise that my criticism of Info-Gap decision theory is very harsh indeed.

The following mini-guide may be useful to some readers.

Reader type Tips Author Please, read the relevant FAQs cited in my reviews before you send me an angry letter. I'd be most happy to discuss your paper with you. Referee If you were not vigilant enough, or not sufficiently au fait with the literature on decision-making under severe uncertainty, make sure that next time you are better prepared before you proceed to referee an Info-Gap paper. In any event, take note that Info-Gap's fundamental flaws cannot be smoothed over by fancy rhetoric. Editor Make sure that you send Info-Gap papers to qualified referees. Please, do not send me Info-Gap papers for review. Info-Gap aficionado Relax! You can ignore my criticism. This is in fact what Info-Gap scholars -- with some exceptions -- have done so far. Inactive Info-Gap Critic Wake up! Join my campaign. PhD Student Indeed, not everything that gets published in refereed journals is "correct". Also, PhD supervisors are not always well-versed in the subject matter and can fall victim to the adverse effects of spin and rhetoric. Therefore, study carefully the claims that are made in the Info-Gap literature. Grantor of research funds Read carefully the research proposals. If a proposal is too good to be true, then rest assured that it is. In the case of decision-making under severe uncertainty, watch out for voodoo decision theories that are in violations of the GIGO Axiom, but are dressed up by spin. Accidental tourist It would be a good idea to read the first 20 items in FAQs about Info-Gap decision theory before reading the reviews.

This contribution is dedicated to the Info-Gap people at Wikipedia. They were searching for a formal proof that .... |

Fact 1: Info-Gap is a simple instance of Wald's Maximin model [1945].

Fact 2: Info-Gap does not deal with severe uncertainty: it simply ignores it.

## Recent Articles, Working Papers, Notes

Also, see my complete list of articles

Moshe's new book! - Sniedovich, M. (2012) Fooled by local robustness,
Risk Analysis, in press.

- Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty,
International Transactions in Operational Research, in press.

- Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis,
Applications of Mathematics,56(5), 499-509.

- Sniedovich, M. (2011) Dynamic programming: introductory concepts, in
Wiley Encyclopedia of Operations Research and Management Science(EORMS), Wiley.

- Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem,
OR Spectrum, 33(4), 815-929, 2011.

- Sniedovich, M. (2011)
Dynamic Programming: Foundations and Principles, Second Edition,Taylor & Francis.

- Sniedovich, M. (2010) A bird's view of Info-Gap decision theory,
Journal of Risk Finance, 11(3), 268-283.

- Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.

- Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds),
.Safety, Reliability and Risk Analysis: Theory, Methods and Applications,pp. 2071-2079, Taylor and Francis Group, London.

- Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)

- Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.),
11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation,DIPTEM, Genova, 89-94.

- Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)

- Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)

- Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08

This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )

- Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory,
Decision Point,24, 10.

- Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty,
Decision Point,22, 8-9.

- Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!,
Journal of Risk Finance,9(3), 287-291.

- Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)

In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.

- Sniedovich. M. (2008) The Mighty Maximin! (PDF File)

This paper is dedicated to the modeling aspects of Maximin and robust optimization.

- Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty,
Decision Making in Manufacturing and Services,1-2, 111-136. (PDF File) .

- Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)

In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.

- My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)
This is an exciting development!

- Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.

- Ben-Haim is not familiar with the fundamental concept
point estimate. He does not realize that a function can be a point estimate of another function.So when you read my papers make sure that you do not misinterpret the notion

point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!

- A formal proof that
Info-GapisWald's Maximin Principlein disguise. (December 31, 2006)

This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)

It shows thatInfo-Gapis not a new theory but rather a simple instance ofWald's famous Maximin Principledating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context ofGame Theory(1928).

- A proof that
Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)

This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .

It shows that becauseInfo-Gapdeploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.

- A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)

This is a very short article entitled The GAP in Info-Gap (PDF File) .

It is a math-free version of the paper above. Read it if you are allergic to math.

- A long essay entitled
What's Wrong with Info-Gap? An Operations Research Perspective(PDF File) (December 31, 2006).

This is a paper that I presented at theASOR Recent Advances in Operations Research(PDF File) mini-conference (December 1, 2006, Melbourne, Australia).## Recent Lectures, Seminars, Presentations

If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.

Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.

- ASOR Recent Advances, 2011, Melbourne, Australia, November 16 2011. Presentation: The Power of the (peer-reviewed) Word. (PDF file).

- Alex Rubinov Memorial Lecture The Art, Science, and Joy of (mathematical) Decision-Making, November 7, 2011, The University of Ballarat. (PDF file).

- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Invited tutorial, ALIO/INFORMS Conference, Buenos Aires, Argentina, July 6-9, 2010).

- A Critique of Info-Gap Decision theory: From Voodoo Decision-Making to Voodoo Economics(PDF File) .

(Recent Advances in OR, RMIT, Melbourne, Australia, November 25, 2009)

- Robust decision-making in the face of severe uncertainty(PDF File) .

(GRIPS, Tokyo, Japan, October 16, 2009)

- Decision-making in the face of severe uncertainty(PDF File) .

(KORDS'09 Conference, Vilnius, Lithuania, September 30 -- OCtober 3, 2009)

- Modeling robustness against severe uncertainty (PDF File) .

(SOR'09 Conference, Nova Gorica, Slovenia, September 23-25, 2009)

- How do you recognize a Voodoo decision theory?(PDF File) .

(School of Mathematical and Geospatial Sciences, RMIT, June 26, 2009).

- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, Info-Gaps, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Department of Econometrics and Business Statistics, Monash University, May 8, 2009).

- The Rise and Rise of Voodoo Decision Theory.

ASOR Recent Advances, Deakin University, November 26, 2008. This presentation was based on the pages on my website (voodoo.moshe-online.com).

- Responsible Decision-Making in the face of Severe Uncertainty (PDF File) .

(Singapore Management University, Singapore, September 29, 2008)

- A Critique of Info-Gap's Robustness Model (PDF File) .

(ESREL/SRA 2008 Conference, Valencia, Spain, September 22-25, 2008)

- Robust Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Technion, Haifa, Israel, September 15, 2008)

- The Art and Science of Robust Decision-Making (PDF File) .

(AIRO 2008 Conference, Ischia, Italy, September 8-11, 2008 )

- The Fundamental Flaws in Info-Gap Decision Theory (PDF File) .

(CSIRO, Canberra, July 9, 2008 )

- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .

(OR Conference, ADFA, Canberra, July 7-8, 2008 )

- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .

(University of Sydney Seminar, May 16, 2008 )

- Decision-Making Under Severe Uncertainty: An Australian, Operational Research Perspective (PDF File) .

(ASOR National Conference, Melbourne, December 3-5, 2007 )

- A Critique of Info-Gap (PDF File) .

(SRA 2007 Conference, Hobart, August 20, 2007)

- What exactly is wrong with Info-Gap? A Decision Theoretic Perspective (PDF File) .

(MS Colloquium, University of Melbourne, August 1, 2007)

- A Formal Look at Info-Gap Theory (PDF File) .

(ORSUM Seminar , University of Melbourne, May 21, 2007)

The Art and Science of Decision-Making Under Severe Uncertainty(PDF File) .

(ACERA seminar, University of Melbourne, May 4, 2007)

- What exactly is Info-Gap? An OR perspective. (PDF File)

ASOR Recent Advances in Operations Researchmini-conference (December 1, 2006, Melbourne, Australia).

Disclaimer:This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.