The Spin Stops Here!
Decision-Making Under Severe Uncertainty  
Faqs | Help | @ | Contact | home  
voodoostan info-gap decision theory info-gap economics severe uncertainty mighty maximin robust decisions responsible decisions


Reviews of publications on Info-Gap decision theory

Review # 27 (Posted: April 9, 2011)

Reference:

Jan Sprenger
The Precautionary Approach and the Role of Scientists in Environmental Decision-Making
http://www.laeuferpaar.de/Papers/PSA_Symposium_Paper_v3.pdf (January 9, 2011).
Presented at the Philosophy of Science Association (PSA) 2010 Conference, November 4–6, 2010, Montréal, Quebec, Canada.

Opening paragraph The role of scientists in advising public policy has recently been the subject of vivid discussion, particularly with respect to the value-ladenness of the scientists' epistemic analysis. The typical example for discussing this question is the problem of accepting vs. rejecting a particular hypothesis of interest (Rudner 1953; Levi 1960): setting thresholds for the probability of false positives and false negatives requires considerations about the impact and consequences of wrong decisions. An allegedly objective, impartial analysis is thus inevitably shaped a scientist's personal view on the significance of various kinds of damage (Douglas 2009, ch. 5). A similar case can be made for inferences in a Bayesian framework, where the choice of a Bayes estimator of a quantity of interest -- the estimator that minimizes the expected risk, based on one’s posterior distribution -- does not only depend on that posterior distribution, but also on the context-dependent loss function that has been adopted. In that sense, scientists who advise a Government agency on the values of a quantity of interest or the truth of a certain hypothesis go beyond a purely epistemic analysis, and become implicit policy-makers themselves.
Scores TUIGF:30%
SNHNSNDN:90%
GIGO:10%

I decided to take up this paper for review because it refers to ...... my criticism of info-gap decision theory. However, before I can get down to the review itself, I want to call the reader's attention to two points.

First, I want to point out that the paper under review here has a "Philosophy of Science" bent.

Second, I want to make it clear that as I have no way of knowing to what extent the author of this paper is familiar with the details of my criticism of info-gap decision theory, I restrict my review to those statements in the paper that refer to my critique of info-gap decision theory. However, as would be expected, in so doing I end commenting on the author's characterization of the theory.

I then expand the discussion somewhat, addressing issues related to statements made in the abstracts of two other presentations by the same author/speaker (Jan Sprenger) in 2010.

Let us begin then by considering this statement:

On the other hand, this simplicity of info-gap is also a weakness. On various occasions (e.g., Sniedovich (2008)), the info-gap approach has been criticized for relying on an inferential procedure on a specific estimate, generated from a specific model.
Sprenger (2011, p. 9)

This is clearly a case of mistaken identity.

I can assure the reader that in Sniedovich (2008) I do not criticize info-gap decision theory. What is more, I most definitely do not discuss in this article the criticism attributed to me in Sprenger (2011, p. 9). So, for the record, I must first give the full bibliographic details of my 2008 article cited by Sprenger (2011, p. 9):

Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!, Journal of Risk Finance, 9(3), 287-291.

It is important to take note that, as can be gathered from the title of this article, in it I discuss the modeling aspects of Wald's famous Maximin model. So, the immediate implication that the discussion in this article has for info-gap decision theory is that in it I prove, formally and rigorously, that info-gap's robustness model is in fact a simple instance of Wald's maximin model.

Hence, by referring to Sniedovich (2008), Sprenger (2011) should have pointed out that ... Sniedovich (2008) provides solid proof that info-gap's robustness model is in fact a simple Maximin model!

It is most unfortunate that he does not give this vital piece of information about the theory that he refers to here, namely info-gap decision theory. Because, in the lack of this centrally important piece of information, from the word go, his readers get an inaccurate description of the theory.

More on the Maximin connection below.

To give the reader a clear picture of Sprenger's (2011) assessment of info-gap decision theory, I begin by quoting in full these statements (emphasis added).

VirtuesWeaknesses
This approach has a variety of virtues. By means of the horizon of uncertainty, robustness considerations are directly built into the entire model. Info-gap may therefore be called a "truly precautionary approach to management and conservation" (Halpern et al. 2006). Moreover, it can be applied without using complicated model averaging procedures: we can directly plug-in our best scientific model (the most comprehensive one, the most successful one, the one most closely related to scientific theory, etc.) and evaluate the estimates based on that model. At least from a practical point of view, that is a great simplification.
Sprenger (2011, pp. 8-9)
On the other hand, this simplicity of info-gap is also a weakness. On various occasions (e.g., Sniedovich (2008)), the info-gap approach has been criticized for relying on an inferential procedure on a specific estimate, generated from a specific model. What justification do we have for basing our robustness analysis on a single model estimate? This model may be highly biased, as argued in the introduction. Much depends on whether our best estimate is reasonable at all. This presumption does not seem to do proper justice to a situation of radical uncertainty that we often encounter in practice. Similarly, the choice of an appropriate metric such as the fractional deviation in (1) is likely to affect the outcome. These non-trivial decisions are unfortunately often passed over in silence, even in standard literature on the topic (Ben-Haim 2006). It is also unclear whether it makes sense to apply the approach when we face non-linear dynamics, e.g. in climate science, where the regions of acceptance for any actions can be ill-behaved, and the robustness functions of the available options can be misleading.
Sprenger (2011, p. 9)

To set the record straight I call the reader's attention to the fact that these "Weaknesses" are discussed in great detail in Sniedovich (2007, 2009, 2010, 2011) and in many working papers and discussions on my website (see list below). So, to form a correct view of my criticism of info-gap decision theory, I urge readers to read or at least browse through my writings on this subject. I submit that those who will take this trouble are likely to find a considerable amount of, if you will, enlightening discussion on the various aspects of decision-making in the face of uncertainty.

Turning now to Sprenger's (2011) assessment of info-gap decision theory. The first point to note is this.

How in the world, can one possibly give a meaningful, indeed a correct assessment of a theory, without addressing it from the fundamental standpoint of the objective for which this theory was invented?

This of course is a rhetorical question because this is what sets the tone of Sprenger's (2011) assessment of info-gap decision theory. It is therefore important to stress that Sprenger's (2011) assessment does not point out, let alone does it discuss, the most important fact in all of this, which is that info-gap decision theory (on the testimony of its founder) was invented to serve only as a theory for decision under severe uncertainty. Thus, not only does Sprenger (2011) not enlighten the reader on the type of uncertainty that info-gap decision theory is supposed to deal with --- which, to repeat is made clear by the primary texts on this topic, namely Ben-Haim (2001, 2006, 2010). He does not even give a clue to the fact that Ben-Haim argues strongly for instance, in his 2007 compilation of FAQs on info-gap decision theory, that this is the sole purpose of info-gap decision theory. In this compilation, he clearly argues that one does not need info-gap decision theory for other purposes ( cases where the uncertainty is not severe).

Second,

How in the world, can one possibly give a correct assessment of a theory, one that would truly benefit the readers, without addressing it from the fundamental standpoint of the theory's relation and contribution to the state of the art?

This is another rhetorical question, which equally sets the tone of Sprenger's (2011) assessment of info-gap decision theory.

It is important, therefore, to point out that it is most unfortunate that Sprenger's (2011) assessment is silent on the vitally important fact that info-gap's robustness model is in fact a simple instance of a well-established model known universally as the Radius of Stability model (circa 1960), itself a simple instance of, the much more famous, Wald's Maximin model (circa 1940).

Readers, therefore, have no clue that this theory is in fact, a reinvention of an old wheel, but alas a square one, because it prescribes a misapplication of this model outside the model's intended domain of application.

And this brings me to Sprenger's (2011) reference to my criticism of info-gap decision theory.

Reading Sprenger (2011), I have the impression that he did not read my criticism of info-gap decision theory very carefully, if at all. For, what is conspicuously missing from Sprenger's (2011) assessment of info-gap decision theory is the whole point of my criticism, which is that info-gap decision theory is being hailed (by its proponents) as a theory that was devised specifically for decision making under severe uncertainty. Hence, that it is singularly suitable for situations where the uncertainty space is vast (can be unbounded), the estimate is poor, and the uncertainty is quantified by a likelihood-free model.

This, so the claims go, makes info-gap decision theory a reliable tool for handling surprises, rare events, catastrophes, shocks, etc. It is even suggested that this theory is suitable for the treatment of "Black Swans" and "Unknown Unknowns" (see Review 17).

So, the first point I want to make in my commentary on Sprenger's (2011) reference to my criticism of info-gap is that it is not only a case of info-gap proponents being silent on the fundamental flaws of info-gap decision theory, as one might have gathered from Sprenger (2011). It is more a case of its proponents vigorously advancing info-gap decision theory as a method that is particularly suitable for robust decision-making under severe uncertainty. My criticism simply points out the obvious, namely that info-gap decision theory is clearly the wrong method for this purpose. And this is precisely the reason for my labeling this proposition a prescription for voodoo decision making par excellence. This is the punch-line of my argument!

There is not the slightest hint in Sprenger (2011) that this is the case.

In fact, it is even unclear whether Sprenger (2011) is fully aware of the fact that the literature on info-gap decision theory (Ben-Haim 2001, 2006, 2010) goes out of its way to depict it as a theory that is specifically designed for severe uncertainty that is manifested in:

It is important to point out, therefore, given Sprenger (2011) assessment of the "virtues" of a "local analysis", that according to its inventor (Ben-Haim (2001, 2006, 2010)), info-gap decision theory was not designed to seek local robustness. Rather, it was designed to seek decisions that are robust against severe uncertainty of the type described above. Indeed, it is repeatedly stated in the info-gap literature (I hasten to add, erroneously and without any foundation), that info-gap decision theory selects decisions that yield the widest range of acceptable outcomes (e.g. see Review 21). And in Ben-Haim (2007) it is stated categorically (I again hasten to add, erroneously and without any foundation), that info-gap decision theory is not a theory of local robustness:

Thus an info-gap model is not a “local analysis of risk” since the family of sets expands, usually boundlessly, as the unknown horizon of uncertainty, α, grows.
Ben-Haim (2007, p. 1)

This being the case, the "great simplification" that Sprenger (2011) ascribes to info-gap's approach actually reflects a fundamental flaw, not a virtue. Because, in the case of info-gap decision theory, this "great simplification" is a manifestation of the fact that info-gap decision theory does not deal with the severity of the uncertainty that it is supposed to manage -- it simply takes no notice of it. That is, "the great simplification" in the case of info-gap decision theory in fact comes down to a prescription for robust decision under severe uncertainty which instructs the following: perform a robustness analysis in the vicinity of a wild guess and call it a day!

Clearly, this "simplicity" does not speak in info-gap's favor, neither practically nor theoretically.

If anything, Spenger's (2011) explanation of the great simplification (see quote above) brings out my claim that info-gap decision theory does the precise opposite of what a decision theory for severe uncertainty ought to do. And this is what earns this theory the title: voodoo decision theory par excellence.

Consider next this statement, which is a continuation of the one cited above:

Whether these objections can, in a concrete case, be answered satisfactorily is a question that only scientists can decide. They are best equipped to evaluate the soundness of a peculiar decision-theoretic approach against the epistemic situation we are in, and the likely sources of error and bias.
Sprenger (2011, pp. 9-10)

Sprenger (2011) is reminded that I have demonstrated on numerous occasions that even senior info-gap scholars are unaware of the limitations of the theory. Hence, it is naive in the extreme to assume, as Sprenger (2011) does, that we can take it for granted that info-gap scientists in the areas of applied ecology and conservation biology are "best equipped to evaluate the soundness of a peculiar decision-theoretic approach against the epistemic situation we are in".

I note this point because, a significant number of the reviews posted on this website deal with articles written by info-gap scholars in areas such as applied ecology, conservation biology, environmental management, and so on.

A case in point is the proposition by senior info-gap scholars in the area of applied ecology and conservation biology arguing that info-gap decision theory is a suitable tool for dealing with "Unknown Unknowns" and "Black Swans"! (see Review 17).

The Maximin connection

In the discussion on the different understandings of the Precautionary Principle, Sprenger (2011) states the following:

These understandings can be mutually compatible, that is, it is possible to adopt two or more of the suggested readings of the PP. The first interpretation is clearly the most concrete, demanding and controversial. It seems to suggest something like a maximin rule, namely to adopt the option that ensures, if nature is malevolent, the least evil.
Sprenger (2011, p. 4)

It is important to point out that there is not a clue in Sprenger (2011) that info-gap's robustness model is in fact a Maximin model (see proof here). This is puzzling given that this fact is proved formally in Sniedovich (2008), the very article referred to by Springer (2011, p. 9) in his discussion on the weaknesses of info-gap decision theory.

This fact is of the utmost importance because Sprenger's (2011) entire analysis is oblivious to the fundamental difference between local and global robustness, hence to the fundamentally different ramifications that a local robustness analysis has as opposed to those of a global robustness analysis.

So, the upshot is that because he fails to point out that info-gap's robustness model is in fact a special case of the Maximin model --- one that can perform only a local robustness analysis (it is a radius of stability model), readers have no clue of the true meaning of the Precautionary approach that info-gap is credited with by Sprenger's (2011). If one is going to describe it as a Precautionary Principle, one must point out that as the special case of the Maximin model known as the Radius of Stability model, info-gap's robustness model performs a "local worst case analysis" which renders its analysis precautionary only with regard to the vicinity of an estimate that can be substantially wrong!

This omission is particularly surprising in view of the two abstracts discussed below.

In sum:

Info-gap's robustness model is a simple Radius of Stability model, itself a simple instance of Wald's Maximin model, the foremost model in decision theory for the treatment of severe uncertainty. It is therefore very odd that Sprenger (2011) contrasts an expected utility analysis with info-gap's robust satisficing model, without pointing out to the reader that this model is a simple instance of Wald's famous Maximin model that is known universally as Radius of Stability model.

And finally this:

Whereas most approaches, such as bounding probability densities, or working with probability intervals rather than precise numbers, specify the amount of uncertainty in advance, info-gap applies a sort of reverse engineering by deliberately leaving open the amount of uncertainty that we could possibly encounter (Halpern et al. 2006).
Sprenger (2011, p. 10)

The fact that in the framework of info-gap decision theory the horizon of uncertainty (α) is unbounded does not mean that "... info-gap applies a sort of reverse engineering by deliberately leaving open the amount of uncertainty that we could possibly encounter ..." . The "amount of uncertainty that we could possibly encounter" is specified by the uncertainty space of the model, which must be stipulated by the user. The nested uncertainty sets U(α,û), α≥0, are specified accordingly.

The reason that info-gap users/scholars seem to be impressed with the fact that "...the amount of uncertainty that we could possibly encounter ..." can be left open (namely, that an info-gap model allows the uncertainty space to be unbounded), is simply due to a fundamental misconception of the impact of the size of the uncertainty space on the results of info-gap's analysis. The simple fact is, of course, that the robustness analysis conducted by info-gap's robustness model is by definition local. This means that, methodologically speaking, it matters not in the slightest how large the uncertainty space actually is. The results are totally independent of the size of the uncertainty space. I coined the term No Man's Land to describe this bizarre characteristic of info-gap's robustness analysis, to explain the reason for my labeling info-gap decision theory a avoodoo decision theory par excellence (see Sniedovich 2010, 2011).

So again, what to info-gap proponents (e.g. Halpern et al. 2006; Ben-Haim (2001, 2006, 2010) appears as a great virtue, not only turns out to be the precise opposite, but proves risible. Because, what this comes down to is that we can leave "the amount of uncertainty that we could possibly encounter ..." as open as we want, and this won't make one jot of difference to the results of our info-gap analysis. Because, the analysis is dictated only by the model's mode of operation, which as a radius of stability model, a priori seeks the smallest destabilizing deviation from the estimate, in total disregard to the "amount of uncertainty that we could possibly encounter". In other words, the fact that "...the amount of uncertainty that we could possibly encounter ..." can be left open, in the end comes to naught!

And this bizarre characteristic of info-gap's analysis brings out the point that I make above, namely: how totally unsuitable a tool it is for the treatment of severe uncertainty. Because, for all the fuss about info-gap's uncertainty space being vast, indeed unbounded, hence singularly suitable for the treatment of severe uncertainty, when it comes to the crunch, the fact that the robustness analysis is inherently local means that it does not even begin to address the severe uncertainty, which according to info-gap's basic assumptions, is manifested in the vast uncertainty space.

So much then for the "reverse engineering" applied by info-gap!

Breaking news (April 22, 2011)

In the recent issue of Decision Point (48), Jan Sprenger tells us about his 2011 article.
It is interesting that he has chosen not to mention his criticism of info-gap decision theory. He thus gives the (very) wrong impression that info-gap decision theory maximizes the chance of an acceptable outcome under severe uncertainty.

I just wonder how long the info-gap saga will continue unabated!

Within a week or so I shall update this review to reflect on Sprenger's Decision Point article. I may also be able to update the review itself based on a new version of the paper.


I take this opportunity to comment briefly on two abstracts written by Jan Sprenger in 2010. Since the full papers/presentations associated with these abstracts are not available, my comments are based solely on the abstracts.

With this in mind, consider this (color is added):

‘Philosophy of Science in a Forest’ (PSF2010)
Internationale Schol voor Wijsbegeerte, Leusden
Friday afternoon 14 May -- Saturday 15 May 2010
Jan Sprenger
Environmental Modeling and Decision-Making: Local vs. Global Approaches
http://www.nvwf.nl/Philosophy_of_Science_in_a_Forest/Programme_files/abstracts_contributed_papers_PSF_2010.pdf

Abstract

Climate scientists, conservation ecologists and econometricians have to make robust decisions under conditions of severe, fundamental uncertainty. Specifying all that is needed for a full decision-theoretic analysis (e.g. degrees of belief expressed as probabilistic judgments) is very often a matter of pure guesswork. This makes such an analysis appear overly subjective, and hard to implement in a practical problem. In particular, high degrees of uncertainty raise the question of whether it is admissible to take a particular mathematical model, computer simulation or numerical estimate as a point of departure (while accounting for the possibility that the starting point might be wrong). Such decision-theoretic approaches fall under the label of local decision theory. This is opposed to global decision theory, such as expected utility maximization or the maximin principle, which is based on the worst possible consequences of a certain action (Savage 1972). There we require a full probabilistic analysis to calculate a chances/risks tradeoff, and to choose the most efficient action.

Against a local approach speaks that in the environmental sciences, we often know that all models are missing the truth (Stainforth et al., 2007), due to the highly non-linear and often chaotic nature of the studied processes. Hence, even the most sophisticated and complicated models have to be regarded with great caution. Working with a particular estimate grants a privilege to the model used to generate it, although that model is, most of the time, not better and sometimes even worse than its competitors. Hence, privileging a particular model seems to lack a sound foundation and to lead to biased inferences and decisions. We feign a robustness that does not exist; instead we should select an approach where our nearly complete ignorance is more honestly represented.

This contribution examines the virtues and vices of local approaches in decision theory and points out that in spite of the aforementioned problems, they are often without alternative, and can have substantial advantages as well. The three arguments made in favor of local approaches concern (i) the difference between aiming at an optimal and aiming at an acceptable outcome (in other words, optimizing vs. satisficing) (ii) an analysis of the price of the robustness of a decision vis-a-vis its efficiency and (iii) the reference function of particular models and estimates. A reference scenario is a typical, though highly uncertain scenario where parameters can be varied for better studying and understanding the dynamics of the studied processes (e.g. global warming), and the consequences of our decisions (e.g. certain abatement and mitigation strategies).

Summing up, the paper investigates to what extent local approaches differ from, and can be defended against critiques from the globalist camp (Sniedovich 2008), and the implications of that debate for responsible decision-making in the environmental sciences.

References

Ben-Haim, Yakov (2006): Info-Gap Decision Theory: Decisions Under Severe Uncertainty. Second edition. San Diego: Academic Press.

Regan, H.M., Y. Ben-Haim, B. Langford, W.G. Wilson, P. Lundberg, S.J. Andelman, and M.A. Burgman (2005): Robust decision making under severe uncertainty for conservation management". Ecological Applications 15: 1471-1477.

Savage, Leonard J. (1972): The Foundations of Statistics. New York: Wiley and Sons.

Sniedovich, Moshe (2008): Wald's Maximin model: a treasure in disguise!, Journal of Risk Finance 9: 287-291.

Stainforth, D. A., M. R. Allen, E. R. Tredger and L. A. Smith (2007): Confidence, uncertainty and decision-support relevance in climate predictions, Philosophical Transactions of the Royal Society A 365, 2145-2161.

My comments on this abstract:

So, having dismissed the points that supposedly count in favor of a local robustness analysis, what emerges then is that, according to Sprenger (2011), the sole virtue that he would be able to ascribe to a local robustness analysis is that ... it is very simple.

But, as I point out above, in the case of info-gap decision theory, the purported "simplicity" is a manifestation of the fact that the local robustness analysis that it prescribes effectively instructs doing the following: ignore the severity of the uncertainty, conduct an analysis around a wild guess and call it a day! This done, you have completed a "simple" analysis whose objective is to seek out decisions that are robust against severe uncertainty!

Can this "simple" approach have any merit, methodologically speaking, as an approach for robust decision against severe uncertainty?!


Now consider this (color added):

Modeling in the Social and Behavioral Sciences I, IHPST, Paris.
May 22, 2010, 17:30
Jan Sprenger
Environmental Modeling and Decision-Making: Local vs. Global Approaches
http://www-ihpst.univ-paris1.fr/operations/colloque.php?id_colloque=23

Abstract

Climate scientists, conservation ecologists and econometricians have to make robust decisions under conditions of severe, fundamental uncertainty. Specifying all that is needed for a full decision-theoretic analysis (e.g. degrees of belief expressed as probabilistic judgments) is very often a matter of pure guesswork. This makes such an analysis appear overly subjective, and hard to implement in a practical problem. In particular, high degrees of uncertainty raise the question of whether it is admissible to take a particular mathematical model, computer simulation or numerical estimate as a point of departure (while accounting for the possibility that the starting point might be wrong).

Such decision-theoretic approaches fall under the label of local decision theory. This is opposed to global decision theory, such as expected utility maximization or the maximin principle, which is based on the worst possible consequences of a certain action. There we require a full probabilistic analysis to calculate a chances/risks tradeoff, and to choose the most efficient action.

Against a local approach speaks that in the environmental sciences, we often know that all models are missing the truth, due to the highly non-linear and often chaotic nature of the studied processes. Hence, even the most sophisticated and complicated models have to be regarded with great caution. Working with a particular estimate grants a privilege to the model used to generate it, although that model is, most of the time, not better and sometimes even worse than its competitors. Privileging a particular model seems to lack a sound foundation and to lead to biased inferences and decisions. We feign a robustness that does not exist; instead we should select an approach where our nearly complete ignorance is more honestly represented.

This contribution examines the virtues and vices of local approaches in decision theory and points out that in spite of the aforementioned problems, they are often without alternative, and can have substantial advantages as well.

My comments on this:


Summary and conclusions

To the best of my knowledge, the short paragraph in Sprenger (2011, p. 9) summarizing the weaknesses of info-gap decision theory, is the first "serious discussion" in the literature on the weaknesses of info-gap decision theory that I identified and documented some time ago (2006).

However, it is too early to tell whether the message, this time delivered by another person, will reach its destination.

It is unfortunate, however, that Sprenger (2010, 2010a, 2011) has not read carefully my critique of info-gap decision theory. I suggest and he read the articles listed at the bottom of the page, especially the following three, to appreciate what my criticism of info-gap decision theory is all about:

Sprenger (2010, 2010a, 2011) will also benefit from reading the reviews of other info-gap publications in this directory.

 

Other Reviews

  1. Ben-Haim (2001, 2006): Info-Gap Decision Theory: decisions under severe uncertainty.

  2. Regan et al (2005): Robust decision-making under severe uncertainty for conservation management.

  3. Moilanen et al (2006): Planning for robust reserve networks using uncertainty analysis.

  4. Burgman (2008): Shakespeare, Wald and decision making under severe uncertainty.

  5. Ben-Haim and Demertzis (2008): Confidence in monetary policy.

  6. Hall and Harvey (2009): Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.

  7. Ben-Haim (2009): Info-gap forecasting and the advantage of sub-optimal models.

  8. Yokomizo et al (2009): Managing the impact of invasive species: the value of knowing the density-impact curve.

  9. Davidovitch et al (2009): Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island.

  10. Ben-Haim et al (2009): Do we know how to set decision thresholds for diabetes?

  11. Beresford and Thompson (2009): An info-gap approach to managing portfolios of assets with uncertain returns

  12. Ben-Haim, Dacso, Carrasco, and Rajan (2009): Heterogeneous uncertainties in cholesterol management

  13. Rout, Thompson, and McCarthy (2009): Robust decisions for declaring eradication of invasive species

  14. Ben-Haim (2010): Info-Gap Economics: An Operational Introduction

  15. Hine and Hall (2010): Information gap analysis of flood model uncertainties and regional frequency analysis

  16. Ben-Haim (2010): Interpreting Null Results from Measurements with Uncertain Correlations: An Info-Gap Approach

  17. Wintle et al. (2010): Allocating monitoring effort in the face of unknown unknowns

  18. Moffitt et al. (2010): Securing the Border from Invasives: Robust Inspections under Severe Uncertainty

  19. Yemshanov et al. (2010): Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest

  20. Davidovitch and Ben-Haim (2010): Robust satisficing voting: why are uncertain voters biased towards sincerity?

  21. Schwartz et al. (2010): What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making

  22. Arkadeb Ghosal et al. (2010): Computing Robustness of FlexRay Schedules to Uncertainties in Design Parameters

  23. Hemez et al. (2002): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

  24. Hemez et al. (2003): Applying information-gap reasoning to the predictive accuracy assessment of transient dynamics simulations

  25. Hemez, F.M. and Ben-Haim, Y. (2004): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

  26. Ben-Haim, Y. (2007): Frequently asked questions about info-gap decision theory

  27. Sprenger, J. (2011): The Precautionary Approach and the Role of Scientists in Environmental Decision-Making

  28. Sprenger, J. (2011): Precaution with the Precautionary Principle: How does it help in making decisions

  29. Hall et al. (2011): Robust climate policies under uncertainty: A comparison of Info-­-Gap and RDM methods

  30. Ben-Haim and Cogan (2011) : Linear bounds on an uncertain non-linear oscillator: an info-gap approach

  31. Van der Burg and Tyre (2011) : Integrating info-gap decision theory with robust population management: a case study using the Mountain Plover

  32. Hildebrandt and Knoke (2011) : Investment decisions under uncertainty --- A methodological review on forest science studies.

  33. Wintle et al. (2011) : Ecological-economic optimization of biodiversity conservation under climate change.

  34. Ranger et al. (2011) : Adaptation in the UK: a decision-making process.

Recent Articles, Working Papers, Notes

Also, see my complete list of articles
    Moshe's new book!
  • Sniedovich, M. (2012) Fooled by local robustness, Risk Analysis, in press.

  • Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty, International Transactions in Operational Research, in press.

  • Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis, Applications of Mathematics, 56(5), 499-509.

  • Sniedovich, M. (2011) Dynamic programming: introductory concepts, in Wiley Encyclopedia of Operations Research and Management Science (EORMS), Wiley.

  • Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem, OR Spectrum, 33(4), 815-929, 2011.

  • Sniedovich, M. (2011) Dynamic Programming: Foundations and Principles, Second Edition, Taylor & Francis.

  • Sniedovich, M. (2010) A bird's view of Info-Gap decision theory, Journal of Risk Finance, 11(3), 268-283.

  • Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.

  • Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds), Safety, Reliability and Risk Analysis: Theory, Methods and Applications, pp. 2071-2079, Taylor and Francis Group, London.
  • .
  • Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)

  • Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.), 11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation, DIPTEM, Genova, 89-94.

  • Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)

  • Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)

  • Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08
    This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )

  • Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory, Decision Point, 24, 10.

  • Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty, Decision Point, 22, 8-9.

  • Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!, Journal of Risk Finance, 9(3), 287-291.

  • Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)
    In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.

  • Sniedovich. M. (2008) The Mighty Maximin! (PDF File)
    This paper is dedicated to the modeling aspects of Maximin and robust optimization.

  • Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty, Decision Making in Manufacturing and Services, 1-2, 111-136. (PDF File) .

  • Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)
    In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.

  • My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)

    This is an exciting development!

    • Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.

    • Ben-Haim is not familiar with the fundamental concept point estimate. He does not realize that a function can be a point estimate of another function.

      So when you read my papers make sure that you do not misinterpret the notion point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.

      Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!


  • A formal proof that Info-Gap is Wald's Maximin Principle in disguise. (December 31, 2006)
    This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)
    It shows that Info-Gap is not a new theory but rather a simple instance of Wald's famous Maximin Principle dating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context of Game Theory (1928).

  • A proof that Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)
    This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .
    It shows that because Info-Gap deploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.

  • A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)
    This is a very short article entitled The GAP in Info-Gap (PDF File) .
    It is a math-free version of the paper above. Read it if you are allergic to math.

  • A long essay entitled What's Wrong with Info-Gap? An Operations Research Perspective (PDF File) (December 31, 2006).
    This is a paper that I presented at the ASOR Recent Advances in Operations Research (PDF File) mini-conference (December 1, 2006, Melbourne, Australia).

Recent Lectures, Seminars, Presentations

If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.

Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.


Disclaimer: This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.


Last modified: [an error occurred while processing this directive]