The Spin Stops Here!
Decision-Making Under Severe Uncertainty  
Faqs | Help | @ | Contact | home  
voodoostan info-gap decision theory info-gap economics severe uncertainty mighty maximin robust decisions responsible decisions

You can't be a prophet in your own land!

Last modified:

Over the past four years -- as part of my Info-Gap Campaign to contain the spread of Info-Gap decision theory in Australia -- I exposed the serious fundamental flaws afflicting info-gap decision theory. I gave numerous lectures on this subject in Australia and overseas and have written a number of articles on this topic.

After my lectures, people often ask me: since Info-Gap decision theory's failings are so obvious and so detrimental to it, how is it that those who use it don't see these flaws? How can they possibly continue to use it, let alone promote it?

As I explain in my lectures, I prefer to discuss these questions over a cup of coffee, so I will not deviate from this rule on this page either except ... to repeat what I have been arguing all along.

My experience has shown that people tend to fall for the misleading rhetoric in which info-gap decision theory is packaged. Info-Gap decision theory's real forte is a battery of buzzwords (Info-Gap, Knightian uncertainty, etc.) and a web of empty misleading rhetoric that is spun around these buzzwords. This seems to captivate those who are not conversant with decision theory, operations research, optimization theory -- especially robust optimization -- control theory, and related areas. My point is that to be able to properly assess this theory and to form a correct view of it, it is necessary to go beyond the verbal depictions of Info-Gap decision theory that one might encounter in the Info-Gap literature, or in a presentation thereof, and to examine closely its prescription for the treatment of severe uncertainty. Only a careful evaluation of Info-Gap's approach to severe uncertainty -- which requires more than a fleeting familiarity with the topics of risk and uncertainty -- reveals that there is hardly any connection between the rhetoric describing Info-Gap decision theory and its capabilities and the facts attesting to what this theory actually is and does. In a word, the rhetoric and the spin camouflage the truth about Info-Gap decision theory.

Add to this the fact that instead of addressing Info-Gap's basic failings, Info-Gap users and proponents waltz around them by introducing ... more of the same spin and empty rhetoric, and the picture is complete.

So, on this page I point out yet again -- especially for the benefit of those who are not familiar with all the facts -- that despite it being fundamentally flawed, Info-Gap is being promoted vigorously in Australia (see my Second Call for a Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia), as attested for instance, by the numerous articles/reports written by Australian scholars on this theory (see my reviews of some of these publications).


The Seminar at Monash

Consider the following short piece of typical Info-Gap Spin (emphasis is mine):

Info-gap theory is a method for analysis, planning, decision and design under Knightian uncertainty. The future may differ from the past, so our models may err in ways we cannot know. Our data may lack evidence about surprises: catastrophes or windfalls. Our scientific and technical understanding may be incomplete. These are info-gaps: incomplete understanding of the system being managed. Info-gap theory provides decision-support tools for modelling and managing severe uncertainty. Info-gap theory has been applied to many disciplines, including economics, engineering, biological conservation, medicine, homeland security and so on. After outlining the info-gap methodology, we explore applications to three very distinct economic policy problems: financial value at risk with uncertain quantiles, management of climate change, and forecasting an unstable process with incomplete data.

This paragraph is the abstract of a forthcoming seminar:

Info-Gap Economics: An Overview of Policy Analysis

15 September 2010
Dept. of Economics, Monash University
Melbourne, Australia

And here is a si,ilar abstract of a seminar at ACERA on August 20, 2010:

Info-Gap Theory in Ecology: New Directions for Modelling, Inference and Planning

Info-gap theory is a method for analysis, planning, decision and design under uncertainty. The future may differ from the past, so our models may err in ways we cannot know. Our data may lack evidence about surprises: catastrophes or windfalls. Our scientific and technical understanding may be incomplete. These are info-gaps: incomplete understanding of the system being managed. Info-gap theory provides decision-support tools for modelling and managing severe uncertainty. We discuss some applications of info-gap theory to modelling, statistical inference, and planning.

The objective of this short note is to expose, yet again, the Info-Gap rhetoric in these and similar paragraphs in the Info-Gap literature for what it is: spin, fog and empty rhetoric. This statement, like many other similar statements in the Info-Gap literature covers up a number of highly embarrassing facts about Info-Gap decision theory, including:

These are just a few of the fundamental questions that are avoided/evaded in Ben-Haim's seminars/books/articles.

So, I shall confine this discussion to the following two simple questions:

Question # 1:

How is it that there is not even the remotest suggestion, in the above abstract, that Info-Gap's robustness model (the centerpiece of Info-Gap decision theory) is in fact a simple instance of one of the most important paradigms for the treatment of severe uncertainty. In other words, how is it that the true relationship between the following two simple mathematical models is concealed to thus give the false impression that Info-Gap's robustness model is a distinct model of decision-making under severe uncertainty?

Info-Gap's Robustness Model (2001)   Wald's Maximin Model (circa 1940)
max {α ≥ 0: r(d,u) ≤ r* , ∀u∈U(α,û)}
max min     f(x,s)
  x∈X   s∈S(x)

The simple and straightforward answer to this question has been available to the public since the end of 2006 and it reads as follows:

Answer # 1:

Relation between info-gap's robustness model and Wald's famous Maximin model.

Contrary to the repeated claims in the info-gap literature that info-gap decision theory is a unique, novel, distinct, revolutionary theory, that is radically different from all current theories for decision under uncertainty, the truth is that info-gap's robustness model is a simple instance of Wald's Maximin model (1939, 1945, 1950).

Question # 2:

How can Info-Gap robustness model, which prescribes a local analysis in the neighborhood of a poor estimate that is likely to be substantially wrong , possibly be contemplated, let alone be proposed, as a tool for handling "surprises and catastrophes" under Knightian uncertainty?

The answer to this question is equally simple and straightforward:

Answer # 2:

Info-Gap's robustness model is not just a simple instance of Wald's Maximin model. This instance of Wald's Maximin model, which is known universally as Radius of Stability model, was designed specifically for the treatment of robustness against small perturbations in a given nominal value of a parameter of interest. In the case of info-gap decision theory, this means that as a Radius of Stability model all that Info-Gap's robustness model is capable of doing is to seek decisions that are robust in the neighborhood of the poor estimate (a wild guess) which is the fulcrum of its robustness analysis.

The implication is therefore clear. Info-Gap's robustness model for the treatment of severe uncertainty -- where the uncertainty space is vast and the estimate of the true value of the parameter of interest is of poor quality -- is as a matter of principle unsuitable for the modeling, analysis, management of systems, problems etc. that are subject to severe uncertainty.

More on this can be found here.

So, the proposition that this model is a suitable tool (and a new one at that) for the economic analysis of problems that are subject to severe (Knightian) uncertainty, swmonstrates a complete lack of understanding of the difficulties involved in the modeling and analysis of decision problems that are subject to severe uncertainty and a total lack of familiarity with the relevant literature.

In short, the Info-Gap economics theory proposed by Ben-Haim is a new type of Voodoo Economics.

As for the application of info-gap to economics, engineering, biological conservation, medicine, homeland security and so on. Have a look at my reviews of articles discussing info-gap's applications in these areas.

Second Call for a Reassessment

I take this opportunity to repeat my old Call for a Reassessment of Info-Gap decision theory in Australia.

Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia
PDF version                   HTML version

I should point out that in this old Call I refer primarily to the Info-Gap activities in AEDA. So it is only appropriate that in this renewed call I should also mention ACERA's extensive engagement with Info-Gap decision theory over the past three years.

The basic myths and facts about Info-Gap decision theory are fully documented and well understood and it is important that persons who promote this theory in Australia be familiar with them.

At the end of 2010 I shall post a "formal" 2nd Call.

Progress Report

    For the benefit of some of my colleagues, who wonder why my campaign is so long, I prepared a short Official Mobile Debunker of info-gap decision theory. Reading it you'll understand that the task of debunking info-gap decision theory is very easy and straightforward.
  2. A comprehensive critique of Info-Gap decision theory can be found in the recently published article
    Sniedovich, M. (2010) A bird's view of Info-Gap decision theory, Journal of Risk Finance, 11(3), 268-283.

    It discusses in detail the fundamental flaws in info-gap decision theory and the erroneous/misleading statements made in the literature about this theory.

    Info-gap scholars who remain unconvinced that my criticism is valid are encouraged to read this article.

  3. At long last, after more than four years, I managed to convince two local Info-Gap scholars that info-gap's robustness model is ... a Maximin model! (emphasis is mine):
    The main problem with CAPM and related models is that they are based on expected future returns on assets that in principle are unknown and subject to considerable uncertainty. In such situations we are dealing with "true uncertainty" in the sense of Knight (1921) who was the first to distinguish between "risk" based on known probability distributions and true uncertainty when the underlying statistical distributions are unknown. Knight's ideas have been further developed by several authors over the years and in particular by Ben-Haim (2006) who has developed a quantitative formulation known as information-gap decision theory. This theory has recently been shown by Sniedovich (2008) to be formally equivalent to Wald's maximin model in classical decision theory (French, 1988).

    Bryan Beresford-Smith and Colin J. Thompson
    An info-gap approach to managing portfolios of assets with uncertain returns
    Journal of Risk Finance 10(3), 277-287, 2009.

    Still, it must be pointed out that this statement is incorrect because what I proved is not that info-gap's robustness model is equivalent to Wald's Maximin model. Far from it! What I proved is that info-gap's robustness model is a simple instance of Wald's Maximin model. All the same, contrary to Ben-Haim, these authors at least acknowledge info-gap's close kinship to the Maximin. Obviously, the immediate implication of this fact is that -- contrary to Ben-Haim's repeated claims -- Info-gap's robustness model does not offer a distinct, much less a new approach, to the modeling and management of severe uncertainty.

    You may wish to read my review of this article.

    And for the record I should also point out that my first refereed article demonstrating the Maximin connection is from 2007.

  4. Two Info-Gap scholars from the UK now apparently concur with my claim that info-gap's non-probabilistic and likelihood-free robustness model is utterly unsuitable for the treatment of severe uncertainty. In an apparent attempt to amend this serious fault in info-gap decision theory, they impose the following interesting assumption on info-gap's decision model (emphasis is mine):
    An assumption remains that values of u become increasingly unlikely as they diverge from û.

    Hall, J. and Harvey, H.
    Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.
    Eighth International Conference on Hydroinformatics
    January 12-16, 2009, Concepcion, Chile.

    On the face of it this seems to be a move in the right direction, as it presumably aims to justify Info-gap's robustness analysis fixing on û, except that ... in so doing this assumption demolishes info-gap's claim to fame, namely its claim to being a non-probabilistic, likelihood-free decision theory!

    You may wish to read my review of this article.

  5. At long last, three local info-gap scholars now concur with my claim that info-gap's robustness model cannot handle severe uncertainty in situations where the estimate is likely to be substantially wrong.
    Although info-gap theory is relevant for many management problems, two components must be carefully selected: the nominal estimate of the uncertain parameter, and the model of uncertainty in that parameter. If the nominal estimate is radically different from the unknown true parameter value, then the horizon of uncertainty around the nominal estimate may not encompass the true value, even at low performance requirements.

    Tracy M. Rout, Colin J. Thompson, and Michael A. McCarthy
    Robust decisions for declaring eradication of invasive species
    Journal of Applied Ecology 46, 782–786, 2009.

    You may wish to read my review of this article.

  6. The following paragraph is a quote from a UK Government commissioned report.

    More recently, Info-Gap approaches that purport to be non-probabilistic in nature developed by Ben-Haim (2006) have been applied to flood risk management by Hall and Harvey (2009). Sniedovich (2007) is critical of such approaches as they adopt a single description of the future and assume alternative futures become increasingly unlikely as they diverge from this initial description. The method therefore assumes that the most likely future system state is known a priori. Given that the system state is subject to severe uncertainty, an approach that relies on this assumption as its basis appears paradoxical, and this is strongly questioned by Sniedovich (2007).

    Mervyn Bramley, Ben Gouldby, Anthony Hurford, Jaap-Jeroen Flikweert
    Marta Roca Collell, Paul Sayers, Jonathan Simm, Michael Wallis
    Delivering Benefits Through Evidence
    PAMS (Performance-based Asset Management System)
    Phase 2 Outcome Summary Report (PDF File)
    Project: SC040018/R1
    Environment Agency -- December 2009
    Department for Environment Food and Rural Affairs
  7. The diplomatic language of the report cannot veil the obvious fact: Info-Gap decision theory is a voodoo theory!

  8. I acknowledge with appreciation David Fox's public statements regarding the status of info-gap decision theory in Australia. For example, consider this short quote from a note that David posted on Andrew Gelman's website (April 30, 2009 8:30 PM):

    A rather fierce debate has been taking place among academics in our departments of Botany and Mathematics and Statistics about the use of a 'new' form of decision-making under extreme uncertainty. It is called Info-Gap (short for information gap) Theory and owes its existence to Prof. Yakov Ben-Haim at Technion in Israel (Ben-Haim 2006). Yakov is well known to the aforementioned academics -- he visits here regularly and has done a remarkably good job at 'selling' his product -- to the extent that some staff and students in our Botany department and The Australian Centre of Excellence in Risk Analysis ( have enthusiastically (and some would say, blindly) embraced this 'new' paradigm for decision-making under extreme uncertainty. I must plead mea culpa, having been swept up in the initial enthusiasm and published a couple of papers which use info-gap. However, I have a growing unease that IG is not 'new' but in fact a variant of existing methodologies."

    You may also wish to read David's short article on info-gap decision theory in Decision Point (24, pp. 10--11, 2008).

    To the best of my knowledge David is the only person in Australia (other than I) who has thus far expressed his "reservation" about info-gap decision theory in public. For the record, I should stress that there are senior academics in Australia who are very critical of info-gap decision theory, but who, for various reasons, have not expressed their views on this matter in public (yet?).

What's Next?

Two of the three main goals of my campaign have been accomplished:

Still ....

It is of the utmost importance that all those info-gap scholars who now accept my criticism of info-gap decision theory come forward and state in clear and unambiguous language the message emerging from the following:

In other words, the time has come to set the record straight on info-gap decision theory especially for the benefit of info-gap users (in areas such as applied ecology and conservation biology) who are not conversant with decision theory and robust optimization, and who may therefore not be in a position to establish on their own the truth behind the plethora of misleading statements in the info-gap literature about the nature and capabilities of info-gap decision theory.

This is doubly important in view of the new situation in the info-gap literature where some info-gap scholars, who although on the face of it continue to endorse this theory (as a method that is particularly well-suited for the management of problems say in applied ecology and conservation biology), now accept facts about info-gap decision theory that are in stark contradiction to the positions stated by the founder of this theory.

Because, as indicated above, while Ben-Haim continues to insist that info-gap's robustness model is not a Maximin model, some info-gap scholars now concede that info-gap's robustness model is a Maximin model. And while Ben-Haim insists that info-gap decision theory is particularly suitable for situation where the uncertainty is severe and the estimate is a wild guess, some info-gap scholars indicate, albeit cryptically, that info-gap decision theory is unsuitable for such situations.

So we will have to wait and see ....

  Latest News  
  Supplementary Material  

Recent Articles, Working Papers, Notes

Also, see my complete list of articles
    Moshe's new book!
  • Sniedovich, M. (2012) Fooled by local robustness, Risk Analysis, in press.

  • Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty, International Transactions in Operational Research, in press.

  • Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis, Applications of Mathematics, 56(5), 499-509.

  • Sniedovich, M. (2011) Dynamic programming: introductory concepts, in Wiley Encyclopedia of Operations Research and Management Science (EORMS), Wiley.

  • Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem, OR Spectrum, 33(4), 815-929, 2011.

  • Sniedovich, M. (2011) Dynamic Programming: Foundations and Principles, Second Edition, Taylor & Francis.

  • Sniedovich, M. (2010) A bird's view of Info-Gap decision theory, Journal of Risk Finance, 11(3), 268-283.

  • Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.

  • Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds), Safety, Reliability and Risk Analysis: Theory, Methods and Applications, pp. 2071-2079, Taylor and Francis Group, London.
  • .
  • Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)

  • Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.), 11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation, DIPTEM, Genova, 89-94.

  • Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)

  • Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)

  • Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08
    This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )

  • Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory, Decision Point, 24, 10.

  • Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty, Decision Point, 22, 8-9.

  • Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!, Journal of Risk Finance, 9(3), 287-291.

  • Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)
    In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.

  • Sniedovich. M. (2008) The Mighty Maximin! (PDF File)
    This paper is dedicated to the modeling aspects of Maximin and robust optimization.

  • Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty, Decision Making in Manufacturing and Services, 1-2, 111-136. (PDF File) .

  • Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)
    In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.

  • My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)

    This is an exciting development!

    • Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.

    • Ben-Haim is not familiar with the fundamental concept point estimate. He does not realize that a function can be a point estimate of another function.

      So when you read my papers make sure that you do not misinterpret the notion point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.

      Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!

  • A formal proof that Info-Gap is Wald's Maximin Principle in disguise. (December 31, 2006)
    This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)
    It shows that Info-Gap is not a new theory but rather a simple instance of Wald's famous Maximin Principle dating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context of Game Theory (1928).

  • A proof that Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)
    This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .
    It shows that because Info-Gap deploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.

  • A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)
    This is a very short article entitled The GAP in Info-Gap (PDF File) .
    It is a math-free version of the paper above. Read it if you are allergic to math.

  • A long essay entitled What's Wrong with Info-Gap? An Operations Research Perspective (PDF File) (December 31, 2006).
    This is a paper that I presented at the ASOR Recent Advances in Operations Research (PDF File) mini-conference (December 1, 2006, Melbourne, Australia).

Recent Lectures, Seminars, Presentations

If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.

Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.

Disclaimer: This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.

Last modified: