The Spin Stops Here!
Decision-Making Under Severe Uncertainty  
Faqs | Help | @ | Contact | home  
voodoostan info-gap decision theory info-gap economics severe uncertainty mighty maximin robust decisions responsible decisions


Reviews of publications on Info-Gap decision theory

Review # 28 (Posted: May 9, 2011)

Reference:

Jan Sprenger
Precaution with the Precautionary Principle: How does it help in making decisions
Decision Point (48), p. 7, April 2011. (PDF file of the issue.)

Opening section

According to the Wikipedia the Precautionary Principle states that if an action or policy has a suspected risk of causing harm to the public or the environment then, in the absence of scientific consensus, the burden of proof falls on those taking the action. You’ll hear it cited all the time; but what does it mean for our decision making?

Unfortunately, how it works is vaguely formulated. The Rio Declaration (1992) states that in the face of environmental hazards, “lack of full scientific certainty shall not be used as a reason for postponing cost-effective measures.” Other wordings point out that measures must be taken to avoid “scientifically plausible but uncertain” and “morally unacceptable” harm (World Commission on the Ethics of Scientific Knowledge and Technology, 2005).

None of these definitions bother to explain what “scientifically plausible” or “lack of certainty” means. As a consequence, the Precautionary Principle doesn’t easily translate into practice. So where’s the value in a decision framework? Let’s consider some of the possible implications. These derive from three possible interpretations.

Scores TUIGF:100%
SNHNSNDN:200%
GIGO:101%

Introduction

This is a very short (1 page) article, henceforth ARTICLE, citing two references:

So, before I turn to my critique of this ARTICLE I want to explain the reasons for my decision to take up such a short piece for review. Furthermore, I want to clarify why I engage in such a detailed critique that ends being much longer than the ARTICLE itself.

The first point I want to make is that despite its brevity, it is extremely important to give a detailed review of this ARTICLE because it is important to point out that even publications such as this, that make only a "fleeting" yet "positive" reference to info-gap decision theory, perpetuate this fundamentally flawed theory in the applied ecology, conservation biology, and environmental management literatures.

I suspect that those who read this ARTICLE may argue that it has no info-gap content. Or, that it has no obvious connection to info-gap decision theory, as the ARTICLE's main topic of discussion is The Precautionary Principle, henceforth PRINCIPLE. Because, for one thing, the ARTICLE makes no direct reference to info-gap decision theory, at least not by name.

My reply to this is that for all that, the info-gap connection in this ARTICLE, is very tangible indeed. This connection is established in an important way via the reference to Regan et al. (2005) (see Review), which is cited in the ARTICLE, as an illustration of the use of the "robust satisficing" approach. The point of course is that "robust satisficing" is treated in this ARTICLE as a direct implications of the PRINCIPLE. And as all info-gap scholars are well aware, Regan et al. (2005) is a peer-reviewed article that in the info-gap literature counts as a seminal paper, because it was the first to discuss an application of info-gap decision theory to a conservation problem.

My point is then that despite its brevity and its presumably "indirect" reference to info-gap decision theory, this ARTICLE is very much of a piece with the wider info-gap literature. Hence, my critique.

Furthermore, I want to show through my critique of this ARTICLE that the huge difficulties besetting the treatment of severe uncertainty are fundamental, meaning that they cannot be overcome by such means as ... rhetoric and alchemy.

In a nutshell, my basic argument is that irrespective of its length, a publication can be packed with statements, claims etc. that require criticism, sometimes even censure.

My review is thus divided into three parts. The first part gives a bird's view of the main issues under discussion in this review, which, needless to say, are info-gap decision theory oriented. The second discusses in detail the section of the ARTICLE that deals with the "robust satisficing" approach to severe uncertainty. The third, gives a summary of the main points.

A bird's view

To set the stage, I want to remind the reader of the distinction made in decision theory between "risk" and "uncertainty":

So under "risk", the robustness of a system is usually defined as the probability that the system will not fail. Hence, in this case a typical robustness model will seek decisions that minimize the probability of failure, or equivalently maximize the probability of "success".

But the whole point is that, under "uncertainty", this definition of robustness, hence such robustness models, are meaningless.

So, how about these alternatives for "robustness" seeking strategies, which presumably "steer clear" of a probabilistic definition, hence a probabilistic quanification, of uncertainty:

Strictly speaking, under uncertainty, let alone "severe uncertainty", these approaches to robustness are meaningless because the ruling assumption is that notions such as "likelihood", "chance", "belief", and so on, are ill-defined in this setting. The more severe the uncertainty, the more problematic these notions are.

This fact should be well known to info-gap scholars because the Father of info-gap decision theory is crystal clear about it:

However, unlike in a probabilistic analysis, r has no connotation of likelihood. We have no rigorous basis for evaluating how likely failure may be; we simply lack the information, and to make a judgment would be deceptive and could be dangerous. There may definitely be a likelihood of failure associated with any given radial tolerance. However, the available information does not allow one to assess this likelihood with any reasonable accuracy.

Ben-Haim (1994, p. 152)
Convex models of uncertainty: applications and implications
Erkenntnis, 4, 139-156.

Since the horizon of uncertainty is unknown and unbounded, there is no worst case. Since no measure functions of probability (or plausibility, or belief, etc.) are specified by an info-gap model, the analyst cannot calculate statistical expectations and cannot probabilistically insure against the unknown contingencies identified in the info-gap model.

Ben-Haim Y. and Jeske, K. (2003, p. 12)
,Bias in Financial Markets: Robust Satisficing with Info Gaps
FRB of Atlanta Working Paper No. 2003-35.
Available at SSRN: http://ssrn.com/abstract=487585

Uncertainty is the potential for deviation of an actual realization from its normative form. Neither norm nor any specific potential realization is uncertain; it is the potential for deviation of one from the other which is info-gap uncertainty.
The spatial analogy for info-gap uncertainty demonstrates that we need no concept of chance, frequency of recurrence, likelihood, plausibility or belief in order to speak of uncertainty.

Ben Haim (2006, p. 22)

The bottom line is then that a major difficulty in the treatment of severe uncertainty is that notions such as "likelihood", "chance" and "belief" that in ordinary usage seem to be intuitively meaningful, in an uncertainty, not to mention severe uncertainty setting, have no meaning. This means of course that one cannot use these concepts as one would in ordinary usage. Indeed, as indicated by Ben-Haim, ... this is deceptive and can be dangerous.

I need hardly point out that this "ban", is not set in stone. It only means that the use of such terms in a severe uncertainty setting requires that the user give a precise definition of what these terms signify ... under these conditions. From a modeling point of view, this requires that the uncertainty model include formal definitions of these terms, a recipe for quantifying and computing them, etc.

In the absence of such definitions, an approach that is based on a rhetorical device such as "maximize the chance of an acceptable outcome" amounts to proposing the practice of alchemy: the creation of the "chance of an acceptable outcome" out of thin air.

Example

Consider the following seemingly simple task:

Task: Determine the "chance" that 3 + 5u ≤ 20 given that

To simplify matters, consider the much simpler question associated with this task:

What is the "chance", or "likelihood", or "belief", that the true value of u is in the interval I=[3,6]?

The difficulty, of course, is that formulating meaningful definitions of intuitive terms such as "chance", "likelihood", "belief" and so on, under conditions of severe uncertainty, is easier said than done.

And, in the context of info-gap decision theory, Ben-Haim exorts that " ... to make a judgment would be deceptive and could be dangerous. ..."

Back to the ARTICLE.

The ARTICLE claims that the third interpretation of the PRINCIPLE requires a "robust satisficing" approach to severe uncertainty (emphasis added):

" ... The Precautionary Principle is supposed to apply under conditions of severe uncertainty, and to maintain a high level of environmental protection. Given these objectives, the principle should be understood as imposing a robust satisficing approach to environmental management. That is, our decision model should aim at maximizing the chance of an acceptable outcome, and our conclusions should be robust against potential errors in the underlying scientific model. So, rather than estimating precise payoffs, this approach would be more about classifying outcomes into acceptable vs. unacceptable or manageable vs. unmanageable. ... "

But to repeat, the task of maximizing the chance of an acceptable outcome, under severe uncertainty, is meaningless. Because, the requirement "maximize the chance of an acceptable outcome," under severe uncertainty, is undefined. So, equating the "robust satisficing" approach to severe uncertainty with: maximizing the chance of an acceptable outcome --- as indeed proposed in the ARICLE --- amounts to proposing that the PRINCIPLE imposes an approach to a meaningless task.

And, for the record, I should point out that the assertion that the PRINCIPLE imposes a "robust satisficing" approach to environmental management is no more than an unsubstantiated proposition made by the author of the ARTICLE. Indeed, the discussion in Sprenger (2011) (see Review) clearly indicates that this proposition is based on the author's personal view of the PRINCIPLE. It does not have the status of a proposition arrived at on grounds of a careful analysis much less a formal proof. Therefore, the text "... Given these objectives, the principle should be understood as imposing ... " should be read: "... I suggest that, given these objectives, the principle be understood as imposing ...".

But more than this, the reference in the ARTICLE to Regan et al. (2005) (see Review) as an example of an analysis that uses the proposed "robust satisficing" approach, as this approach is described here by the author, is grossly misleading. This is so because the analysis in Regan et al. (2005) (see Review) is an info-gap robustness analysis. And as an info-gap robustness analysis it does not -- by definition -- hence cannot, aim at maximizing the chance of an acceptable outcome.

What is surprising in all this is that the author of the ARTICLE is well aware that info-gap's robustness analysis is an inherently local approach to robustness (see Review 27), which means that he ought to have realized that it does not/cannot aim to maximize the "chance" of an acceptable outcome, however meaningfully the term "chance" is defined/interpreted.

In other words, the author should have realized that as the model used in Regan et al. (2005) (see Review) is an info-gap robustness model, it does not aim to, much less can it, maximize the "chance" of an acceptable outcome. Hence, the approach put forward in Regan et al. (2005) (see Review) cannot be claimed to be an exponent of the "robust-satisficing" approach --- as this approach is described by the author in the ARTICLE.

To complete the picture I should point out that like all info-gap's robustness models, the model in Regan et al. (2005) (see Review) seeks to --- and this is all it can by definition do --- maximize the size of the immediate neighborhood around a (poor) estimate over which the outcome is acceptable.

In sum, for all the author's demonstrated awareness of what he calls info-gap's "weaknesses" in Sprenger (2011) (see Review), this assertion in the ARTICLE is of a piece with the wider info-gap literature which is utterly unaware of these "weaknesses".

Relevant details

The ARTICLE presents three interpretations of the PRINCIPLE. I address only the third interpretation, namely: "Interpretation 3: Specifying particular types of decision-making."
  1. The first thing to note is that the particular --- and only --- type of decision-making approach discussed under this heading is what the ARTICLE terms "robust satisficing". In fact, it contends that this specific type is dictated/implied by the PRINCIPLE:
    " ... The Precautionary Principle is supposed to apply under conditions of severe uncertainty, and to maintain a high level of environmental protection. Given these objectives, the principle should be understood as imposing a robust satisficing approach to environmental management. ..."
    But as I point out above, the ARTICLE does not indicate that this is a very subjective interpretation of the PRINCIPLE. The fact is that there is no solid evidence that this conclusion/interpretation is shared by the majority, or a large sector, of experts in the field. There are no signs (references?) that this interpretation represents the relevant literature. In other words, the claim that "this principle should be understood as" is just a suggestion put forward unsupported. But more than this, no such claims are made in the full paper (Sprenger 2011) (see Review) which is more detailed on this topic.

    I should therefore point out that the "type of decision-making" --- to use the author's term --- that is by far the most frequently invoked, appealed to etc. in the literatures dealing with the application of the PRINCIPLE is Wald's Maximin paradigm: do the best you can, assuming that nature is playing against you. Yet, not a single clue is given to this fact in the ARTICLE.


  2. Next, I want to enlarge somewhat on what I said above, on the meaning and content given to the term "robust satisficing" in this context. So, here is how the ARTICLE elaborates the term "robust satisficing":
    " ... That is, our decision model should aim at maximizing the chance of an acceptable outcome, and our conclusions should be robust against potential errors in the underlying scientific model. So, rather than estimating precise payoffs, this approach would be more about classifying outcomes into acceptable vs. unacceptable or manageable vs. unmanageable. ..."

    In other words, according to the ARTICLE, a "robust satisficing" model is a decision model which decrees that the best alternative is that which maximizes the chance of an acceptable outcome.

    But, as I say above, not only is such a model (given the conditions of uncertainty let alone severe uncertainty) a contradiction in terms. Such a model cannot even be contemplated.

    Because, to repeat:

    Models of severe uncertainty are by definition, non-probabilistic and likelihood-free. Hence, the term "chance" has no meaning in the context of such models. Therefore, a decision-model where the uncertainty is defined by means of a non-probabilistic and likelihood-free uncertainty model, cannot possibly aim at maximizing the "chance" of anything.

    In fact, the author of the ARTICLE seems to be aware of this fact. Because, in Sprenger (2011, p.2) (see Review) we read:

    " ... It is important to recognize that the Precautionary Principle is not designed for situations where we can quantify the risk associated with a certain action, e.g. by assigning a probability of a harmful outcome: the scope is a scientifically plausible, but highly uncertain environmental hazard where we lack a sensible way to quantify the risk involved. Indeed, in environmental decision-making, theoretical understanding is very often scarce, models may be biased in various directions, and systems are complex to an extent that we can hardly assess and quantify hazards that might occur (Halpern et al. 2006; Tebaldi et al. 2005; Tebaldi and Knutti 2007). ..."

    So, if the PRINCIPLE itself is not designed for situations where the uncertainty can be quantified by probabilistic models, how can an approach dictated by this very PRINCIPLE aim at maximizing the "chance" of an event?

    The answer is clear so that there is no need to repeat it.


  3. And more on the info-gap connection:

    As I note above, to illustrate the employment of the proposed "robust satisficing" approach, the ARTICLE points out the following:

    " ... Using this approach to the problem of species conservation, we would evaluate each option -- translocation, new reserve, captive breeding etc -- in terms of the amount of uncertainty which our models permits for achieving an acceptable result (Regan et al. 2005). ..."

    The implication of this statement is clear. As an exponent of the "robust satisficing" approach, the model used by Regan et al. (2005) (see Review) does precisely what according to the author a "robust satisficing" approach does: it maximizes the "chance" of an acceptable outcome.

    But, the point is, as I indicate above:

    • The model discussed in Regan et al. (2005) cannot possibly serve as an illustration of the "robust satisficing" approach, as this approach is described by the author. Because, the notion of "chance" is totally absent from the model in Regan et al. (2005) (see Review). This is so because, the robustness model in Regan et al. (2005) (see Review) is an info-gap robustness model. And as an info-gap robustness model it is a non-probabilistic, likelihood-free model. This means that it does not specify any notion of "chance" the implication being that it cannot seek decisions that maximize the chance of an acceptable outcome. And to reiterate, the Father of info-gap decision theory himself warns against the use of terms such as "chance", "belief" and "likelihood" (and its derivatives) in this context, stressing that this is deceptive and might even be dangerous, for instance in this passage that I quote above,
      " ... However, unlike in a probabilistic analysis, r has no connotation of likelihood. We have no rigorous basis for evaluating how likely failure may be; we simply lack the information, and to make a judgment would be deceptive and could be dangerous. There may definitely be a likelihood of failure associated with any given radial tolerance. However, the available information does not allow one to assess this likelihood with any reasonable accuracy. ..."
      Ben-Haim (1994, p. 152)
      Convex models of uncertainty: applications and implications
      Erkenntnis, 4, 139-156.

    • But more than this. Had the author taken full note of the fact that the robustness model in Regan et al. (2005) (see Review) is an info-gap robustness model he would have realized that this model is the wrong model for "environmental management under severe uncertainty". Because, an info-gap robustness model is by definition a model of local robustness. As such, it seeks decisions that are robust against small perturbations in the nominal value (estimate) of the parameter of interest. Which means of course that it lacks the capabilities to search for decisions that are robust to severe uncertainty. The implication being that it cannot even be contemplated as a model for (robust) "environmental management under severe uncertainty". The author is well aware of the "weaknesses" of models of this type. For, in Sprenger (2011, p. 9) (see Review) we read:
      " ... What justification do we have for basing our robustness analysis on a single model estimate? This model may be highly biased, as argued in the introduction. Much depends on whether our best estimate is reasonable at all. This presumption does not seem to do proper justice to a situation of radical uncertainty that we often encounter in practice. Similarly, the choice of an appropriate metric such as the fractional deviation in (1) is likely to affect the outcome. These non-trivial decisions are unfortunately often passed over in silence, even in standard literature on the topic (Ben-Haim 2006). It is also unclear whether it makes sense to apply the approach when we face non-linear dynamics, e.g. in climate science, where the regions of acceptance for any actions can be ill-behaved, and the robustness functions of the available options can be misleading. ..."

  4. Perspective

    I want to emphasize again that this ARTICLE maintains that the "robust satisficing" approach --- as described by the author -- is mandated by the PRINCIPLE. That is:

    " ... The Precautionary Principle is supposed to apply under conditions of severe uncertainty, and to maintain a high level of environmental protection. Given these objectives, the principle should be understood as imposing a robust satisficing approach to environmental management. ..."

    But the fact is that not only is this assertion totally unsubstantiated, it is at odds with the characterization of the PRINCIPLE in Sprenger (2011, p. 5, bold-face decoration added):

    " ... It goes beyond the scope of this paper to provide a neat taxonomy of the rules that exemplify the PP, but I would like to contend that these rules should satisfy two properties: first, they should focus on performance thresholds rather than the search for an optimal outcome1, second, they should be robust to error in the underlying scientific model. ..."

    In other words, in Sprenger (2011) (see Review) the author only contends that the PRINCIPLE yield decision-making paradigms that satisfy the two properties mentioned above.

    But the point is that this reading of the PRINCIPLE does not imply that the decision-making paradigm is necessarily the one described in the ARTICLE, namely, the "robust satisficing" approach. Recall that the latter --- according to the ARTICLE --- aims at "maximizing the chance of acceptable outcomes".

    The bottom line is then that the assertion in the ARTICLE that the PRINCIPLE should be understood as requiring that "our decision model should aim at maximizing the chance of an acceptable outcome," is without any foundation.

    It is also important to set the record straight on the ARTICLE's assessment of Regan et al.'s (2005) (see Review) "robust satisficing" model:

    " ... This is a remarkable departure from an expected utility model where any decision is based on averaging the utility of fortunate and disastrous, likely and improbable scenarios. The utility model is firmly entrenched in economics and decision theory, but it’s inappropriate for many environmental management problems. ..."

    The point of course is that this supposed "remarkable departure" in fact goes back more than 50 years. More specifically, far from being a "remarkable departure" the info-gap's robustness model proposed in Regan et al.'s (2005) (see Review) is in fact a Radius Stability model (circa 1960) itself a simple instance of Wald's famous Maximin model (1939).

    So, while it is no doubt true that utility maximization models are used extensively in economics and decision theory, the fact of the matter is that other robustness models, prescribing "the satisficing" of constraints are just as prevalent in these and many other fields, including environmental management.

    But more than this, it should be noted that the author of the ARTICLE is well aware that the distinction between "satisficing" and "optimizing" is a hollow one in that it is a distinction that has to do with style not substance. Indeed, the proposed "robust satisficing" approach is subsumed as a special (degenerate) case by "robust optimization". Consider then this comment in (Sprenger 2011, footnote 1, p.5):

    " ... 1As the reader may have noticed, this distinction resembles the well-known optimizing/satisficing dichotomy, but that terminology is slightly misleading since a satisficing rule can be represented as an optimizing rule as well. ..."

    The point of course is that this applies in full to the generic term "robust satisficing" that the author uses in this ARTICLE to designate a type of approach to robust decision making. "Robust satisficing" is no more and no less than a special (degenerate) case of "robust optimizing".

    I should add that, in the info-gap jargon, the term "robust satisficing" serves to designate info-gap's robustness model. Furthermore, the position of info-gap scholars is that this ("robust satisficing") model renders the info-gap methodology unique. Because, --- so the argument goes --- unlike other methodologies that seek to "optimize", the info-gap methodology pursues "robust satisficing".

    But, the fact of the matter is of course that the robustness problem that info-gap's robustness model seeks to solve is a robust optimization problem par excellence. This problem requires the maximization of the size of the "safe deviation" (uniformlly in all "directions") from the (poor) estimate, subject to a performance constraint.

    Yet, there is no reference in the ARTICLE to the area of robust optimization.

Remarks

    To conclude I want to comment on my assertion --- in the introduction --- that: "the huge difficulties besetting the treatment of severe uncertainty are fundamental, meaning that they cannot be overcome by such means as ... rhetoric and alchemy."

    The reason that --- to my mind --- such a comment is required is that I often make assertions to this effect in my ongoing critique of info-gap publications, as for instance in my review of the article entitled "Allocating monitoring effort in the face of unknown unknowns" where I state the following:

    " ... A problem involving genuine "unknown unknowns" cannot be solved by mere rhetoric. No amount of rhetoric will transform "unknown unknowns" into "known unknowns". .."

    My objective in making this comment is twofold:

  1. First, I want to point out that some info-gap scholars regard such statements as "too harsh", particularly my use of the term "rhetoric" to describe the discussions in info-gap publications. Some claim that this "style" and "language" distract from the valid and legitimate points that I make about the flaws in info-gap decision theory and the technical errors afflicting many info-gap publications.

    So, the point I want to make clear in this regard is that the terminology I use in my criticism of info-gap decision theory is chosen very carefully, especially my use of the term "rhetoric". My aim in using this term is to bring to the attention of readers of info-gap publications that the narrative (read: rhetoric) in these publications, which describes verbally what this theory is and does, has got nothing in common with the hard facts about the theory. That is, I want to make it clear that the discourse about this theory, in these publications, has got nothing in common with info-gap's robustness model's mode of operation, its capabilities, hence the results it yields --- as these are determined by the model's mathematical definition hence its properties.

    This remark applies to the ARTICLE under review here, especially to statements describing Regan et al.'s (2005) (see Review) "robust satisficing" model as a "remarkable departure", and to the attribution of the "maximization of the chance of an acceptable outcome" to this model. To repeat, these statements have got nothing in common with this model's mode of operation, capabilities and results as these are determined by the model's mathematical definition, hence properties.


  2. Secondly, I want to point out that I am fully aware that only so much can be said in a one-page article of a monthly magazine such as Decision Point, where the ARTTICLE was published. My point is then that taking this constraint into consideration, it is doubly important that assertions and propositions made in such concise articles be valid and substantiated.

Summary

The issues addressed by the Precautionary Principle in the context of decision making in the face of severe uncertainty are extremely difficult and challenging. They can be addressed from different, even conflicting, points of view, depending on one's attitude towards severe uncertainty. But it is important to realize that these difficult issues cannot be resolved by ... Rhetoric.

Thus, as I explain above, in cases where probabilistic models are used to quantify the uncertainty in the true value of a parameter of interest, the robustness of a system can be, and often is, defined as the probability that the system will not fail.

The difficulty posed by severe uncertainty is that however meaningfull this definition might be in the framework of a probabilistically defined/quantified uncertainty, it is completely meaningless in the framework of decision theories that are based on non-probabilsitic, likelihood-free models of uncertainty.

Hence, it must be appreciated that this difficulty cannot be met by means of "words games". For example, by substituting the supposedly "intuitively obvious" term "chance" for the term "probability". In the framework of a non-probabilsitic, likeilood-free model of uncertainty, the prescription

maximize the chance of an acceptable outcome

is utterly meaningless, unless the intuitive term "chance" is suitably defined in this framework.

But, as indicated above, precisely because it is so difficult to formulate a meaningful definition of supposedly "intuitively compelling" terms such as "chance" and "likely" that the treatment and management of severe uncertainty is such an enormous task.

Another point I wanted to make is the following.

I wanted to illustrate that even short articles such as this are instrumental in sustaining and perpetuating info-gap decision theory despite it being a fundamentally flawed theory.

For, reading this ARTICLE one is led to believe that as an exponent of the "robust satisficing" approach --- as this approach is sketched in the ARTICLE --- Regan et al. (2005) (see Review) is an illustration of the PRINCIPLE's working. But, as anyone even vaguely familiar with Regan et al. (2005) would know, the "robust satisficing" approach discussed in Regan et al. (2005) is in fact the approach prescribed by info-gap decision theory. The implication is then that reading this ARTICLE one is lead to conclude that the approach prescribed by info-gap decision theory, is an illustration of the PRINCIPLE's meaning and content.

And this proposition, as I emphasize above via a reference to the above quote from Sprenger (2011, p. 9) (see Review) --- is put forward, despite the author's recognition that info-gap decision theory is riddled with easily identified weaknesses which render it unsuitable for the task!

Indeed, any proper evaluation of the model proposed in Regan et al. (2005) (see Review) should have immediately revealed that, as an info-gap robustness model, it is utterly unsuitable for the treatment of severe uncertainty. Hence, that it is utterly unsuitable as an illustration of the PRINCIPLE's working, which, as Sprenger (2011) (see Review) assumes, applies in situations that are subject to severe uncertainty.

So, the upshot of all this is that a discussion that set out to shed light on how to understand the precautionary approach to environmental management under severe uncertainty, as prescribed by the Precautionary Principle, ends giving a reading of this PRINCIPLE describing it as prescribing an utterly incautious approach.

And to top it all off, the article ends promoting a decision theory that is fundamentally flawed.

Other Reviews

  1. Ben-Haim (2001, 2006): Info-Gap Decision Theory: decisions under severe uncertainty.

  2. Regan et al (2005): Robust decision-making under severe uncertainty for conservation management.

  3. Moilanen et al (2006): Planning for robust reserve networks using uncertainty analysis.

  4. Burgman (2008): Shakespeare, Wald and decision making under severe uncertainty.

  5. Ben-Haim and Demertzis (2008): Confidence in monetary policy.

  6. Hall and Harvey (2009): Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.

  7. Ben-Haim (2009): Info-gap forecasting and the advantage of sub-optimal models.

  8. Yokomizo et al (2009): Managing the impact of invasive species: the value of knowing the density-impact curve.

  9. Davidovitch et al (2009): Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island.

  10. Ben-Haim et al (2009): Do we know how to set decision thresholds for diabetes?

  11. Beresford and Thompson (2009): An info-gap approach to managing portfolios of assets with uncertain returns

  12. Ben-Haim, Dacso, Carrasco, and Rajan (2009): Heterogeneous uncertainties in cholesterol management

  13. Rout, Thompson, and McCarthy (2009): Robust decisions for declaring eradication of invasive species

  14. Ben-Haim (2010): Info-Gap Economics: An Operational Introduction

  15. Hine and Hall (2010): Information gap analysis of flood model uncertainties and regional frequency analysis

  16. Ben-Haim (2010): Interpreting Null Results from Measurements with Uncertain Correlations: An Info-Gap Approach

  17. Wintle et al. (2010): Allocating monitoring effort in the face of unknown unknowns

  18. Moffitt et al. (2010): Securing the Border from Invasives: Robust Inspections under Severe Uncertainty

  19. Yemshanov et al. (2010): Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest

  20. Davidovitch and Ben-Haim (2010): Robust satisficing voting: why are uncertain voters biased towards sincerity?

  21. Schwartz et al. (2010): What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making

  22. Arkadeb Ghosal et al. (2010): Computing Robustness of FlexRay Schedules to Uncertainties in Design Parameters

  23. Hemez et al. (2002): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

  24. Hemez et al. (2003): Applying information-gap reasoning to the predictive accuracy assessment of transient dynamics simulations

  25. Hemez, F.M. and Ben-Haim, Y. (2004): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

  26. Ben-Haim, Y. (2007): Frequently asked questions about info-gap decision theory

  27. Sprenger, J. (2011): The Precautionary Approach and the Role of Scientists in Environmental Decision-Making

  28. Sprenger, J. (2011): Precaution with the Precautionary Principle: How does it help in making decisions

  29. Hall et al. (2011): Robust climate policies under uncertainty: A comparison of Info-­-Gap and RDM methods

  30. Ben-Haim and Cogan (2011) : Linear bounds on an uncertain non-linear oscillator: an info-gap approach

  31. Van der Burg and Tyre (2011) : Integrating info-gap decision theory with robust population management: a case study using the Mountain Plover

  32. Hildebrandt and Knoke (2011) : Investment decisions under uncertainty --- A methodological review on forest science studies.

  33. Wintle et al. (2011) : Ecological-economic optimization of biodiversity conservation under climate change.

  34. Ranger et al. (2011) : Adaptation in the UK: a decision-making process.

Recent Articles, Working Papers, Notes

Also, see my complete list of articles
    Moshe's new book!
  • Sniedovich, M. (2012) Fooled by local robustness, Risk Analysis, in press.

  • Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty, International Transactions in Operational Research, in press.

  • Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis, Applications of Mathematics, 56(5), 499-509.

  • Sniedovich, M. (2011) Dynamic programming: introductory concepts, in Wiley Encyclopedia of Operations Research and Management Science (EORMS), Wiley.

  • Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem, OR Spectrum, 33(4), 815-929, 2011.

  • Sniedovich, M. (2011) Dynamic Programming: Foundations and Principles, Second Edition, Taylor & Francis.

  • Sniedovich, M. (2010) A bird's view of Info-Gap decision theory, Journal of Risk Finance, 11(3), 268-283.

  • Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.

  • Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds), Safety, Reliability and Risk Analysis: Theory, Methods and Applications, pp. 2071-2079, Taylor and Francis Group, London.
  • .
  • Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)

  • Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.), 11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation, DIPTEM, Genova, 89-94.

  • Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)

  • Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)

  • Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08
    This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )

  • Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory, Decision Point, 24, 10.

  • Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty, Decision Point, 22, 8-9.

  • Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!, Journal of Risk Finance, 9(3), 287-291.

  • Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)
    In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.

  • Sniedovich. M. (2008) The Mighty Maximin! (PDF File)
    This paper is dedicated to the modeling aspects of Maximin and robust optimization.

  • Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty, Decision Making in Manufacturing and Services, 1-2, 111-136. (PDF File) .

  • Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)
    In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.

  • My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)

    This is an exciting development!

    • Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.

    • Ben-Haim is not familiar with the fundamental concept point estimate. He does not realize that a function can be a point estimate of another function.

      So when you read my papers make sure that you do not misinterpret the notion point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.

      Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!


  • A formal proof that Info-Gap is Wald's Maximin Principle in disguise. (December 31, 2006)
    This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)
    It shows that Info-Gap is not a new theory but rather a simple instance of Wald's famous Maximin Principle dating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context of Game Theory (1928).

  • A proof that Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)
    This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .
    It shows that because Info-Gap deploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.

  • A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)
    This is a very short article entitled The GAP in Info-Gap (PDF File) .
    It is a math-free version of the paper above. Read it if you are allergic to math.

  • A long essay entitled What's Wrong with Info-Gap? An Operations Research Perspective (PDF File) (December 31, 2006).
    This is a paper that I presented at the ASOR Recent Advances in Operations Research (PDF File) mini-conference (December 1, 2006, Melbourne, Australia).

Recent Lectures, Seminars, Presentations

If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.

Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.


Disclaimer: This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.


Last modified: [an error occurred while processing this directive]