Reviews of publications on Info-Gap decision theory

Review # 27(Posted: April 9, 2011)

Reference: Jan Sprenger

The Precautionary Approach and the Role of Scientists in Environmental Decision-Making

http://www.laeuferpaar.de/Papers/PSA_Symposium_Paper_v3.pdf (January 9, 2011).

Presented at the Philosophy of Science Association (PSA) 2010 Conference, November 4–6, 2010, Montréal, Quebec, Canada.Opening paragraph The role of scientists in advising public policy has recently been the subject of vivid discussion, particularly with respect to the value-ladenness of the scientists' epistemic analysis. The typical example for discussing this question is the problem of accepting vs. rejecting a particular hypothesis of interest (Rudner 1953; Levi 1960): setting thresholds for the probability of false positives and false negatives requires considerations about the impact and consequences of wrong decisions. An allegedly objective, impartial analysis is thus inevitably shaped a scientist's personal view on the significance of various kinds of damage (Douglas 2009, ch. 5). A similar case can be made for inferences in a Bayesian framework, where the choice of a Bayes estimator of a quantity of interest -- the estimator that minimizes the expected risk, based on one’s posterior distribution -- does not only depend on that posterior distribution, but also on the context-dependent loss function that has been adopted. In that sense, scientists who advise a Government agency on the values of a quantity of interest or the truth of a certain hypothesis go beyond a purely epistemic analysis, and become implicit policy-makers themselves. Scores TUIGF:30%

SNHNSNDN:90%

GIGO:10%

I decided to take up this paper for review because it refers to ...... my criticism of info-gap decision theory. However, before I can get down to the review itself, I want to call the reader's attention to two points.

First, I want to point out that the paper under review here has a "Philosophy of Science" bent.

Second, I want to make it clear that as I have no way of knowing to what extent the author of this paper is familiar with the details of my criticism of info-gap decision theory, I restrict my review to those statements in the paper that refer to my critique of info-gap decision theory. However, as would be expected, in so doing I end commenting on the author's characterization of the theory.

I then expand the discussion somewhat, addressing issues related to statements made in the abstracts of two other presentations by the same author/speaker (Jan Sprenger) in 2010.

Let us begin then by considering this statement:

On the other hand, this simplicity of info-gap is also a weakness. On various occasions (e.g., Sniedovich (2008)), the info-gap approach has been criticized for relying on an inferential procedure on a specific estimate, generated from a specific model.Sprenger (2011, p. 9)This is clearly a case of

mistaken identity.I can assure the reader that in Sniedovich (2008) I do not criticize info-gap decision theory. What is more, I most definitely do not discuss in this article the criticism attributed to me in Sprenger (2011, p. 9). So, for the record, I must first give the full bibliographic details of my 2008 article cited by Sprenger (2011, p. 9):

Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!,Journal of Risk Finance,9(3), 287-291.It is important to take note that, as can be gathered from the title of this article, in it I discuss the modeling aspects of Wald's famous Maximin model. So, the immediate implication that the discussion in this article has for info-gap decision theory is that in it I prove, formally and rigorously, that info-gap's robustness model is in fact a simple instance of Wald's maximin model.

Hence, by referring to Sniedovich (2008), Sprenger (2011) should have pointed out that ... Sniedovich (2008) provides solid proof that info-gap's robustness model is in fact a simple Maximin model!

It is most unfortunate that he does not give this vital piece of information about the theory that he refers to here, namely info-gap decision theory. Because, in the lack of this centrally important piece of information, from the word go, his readers get an inaccurate description of the theory.

More on the Maximin connection below.

To give the reader a clear picture of Sprenger's (2011) assessment of info-gap decision theory, I begin by quoting in full these statements (emphasis added).

Virtues Weaknesses This approach has a variety of virtues. By means of the horizon of uncertainty, robustness considerations are directly built into the entire model. Info-gap may therefore be called a "truly precautionary approach to management and conservation" (Halpern et al. 2006). Moreover, it can be applied without using complicated model averaging procedures: we can directly plug-in our best scientific model (the most comprehensive one, the most successful one, the one most closely related to scientific theory, etc.) and evaluate the estimates based on that model. At least from a practical point of view, that is a great simplification.

Sprenger (2011, pp. 8-9)On the other hand, this simplicity of info-gap is also a weakness. On various occasions (e.g., Sniedovich (2008)), the info-gap approach has been criticized for relying on an inferential procedure on a specific estimate, generated from a specific model. What justification do we have for basing our robustness analysis on a single model estimate? This model may be highly biased, as argued in the introduction. Much depends on whether our best estimate is reasonable at all. This presumptiondoes not seem to do proper justice to a situation of radical uncertainty that we often encounter in practice. Similarly, the choice of an appropriate metric such as the fractional deviation in (1) is likely to affect the outcome. These non-trivial decisions are unfortunately often passed over in silence, even in standard literature on the topic (Ben-Haim 2006). It is also unclear whether it makes sense to apply the approach when we face non-linear dynamics, e.g. in climate science, where the regions of acceptance for any actions can be ill-behaved, and the robustness functions of the available optionscan be misleading.

Sprenger (2011, p. 9)To set the record straight I call the reader's attention to the fact that these "Weaknesses" are discussed in great detail in Sniedovich (2007, 2009, 2010, 2011) and in many working papers and discussions on my website (see list below). So, to form a correct view of my criticism of info-gap decision theory, I urge readers to read or at least browse through my writings on this subject. I submit that those who will take this trouble are likely to find a considerable amount of, if you will, enlightening discussion on the various aspects of decision-making in the face of uncertainty.

Turning now to Sprenger's (2011) assessment of info-gap decision theory. The first point to note is this.

How in the world, can one possibly give a meaningful, indeed a correct assessment of a theory, without addressing it from the fundamental standpoint of the objective for which this theory was invented?This of course is a rhetorical question because this is what sets the tone of Sprenger's (2011) assessment of info-gap decision theory. It is therefore important to stress that Sprenger's (2011) assessment does not point out, let alone does it discuss, the most important fact in all of this, which is that info-gap decision theory (on the testimony of its founder) was invented

to serve only as a theory for decision under severe uncertainty. Thus, not only does Sprenger (2011) not enlighten the reader on thetype of uncertaintythat info-gap decision theoryis supposed to deal with--- which, to repeat is made clear by the primary texts on this topic, namely Ben-Haim (2001, 2006, 2010). He does not even give a clue to the fact that Ben-Haim argues strongly for instance, in his 2007 compilation of FAQs on info-gap decision theory, that this is thesole purposeof info-gap decision theory. In this compilation, he clearly argues that one does not need info-gap decision theory for other purposes ( cases where the uncertainty is not severe).Second,

How in the world, can one possibly give a correct assessment of a theory, one that would truly benefit the readers, without addressing it from the fundamental standpoint of the theory's relation and contribution to thestate of the art?This is another rhetorical question, which equally sets the tone of Sprenger's (2011) assessment of info-gap decision theory.

It is important, therefore, to point out that it is most unfortunate that Sprenger's (2011) assessment is silent on the vitally important fact that info-gap's robustness model is in fact a simple instance of a well-established model known universally as the Radius of Stability model (circa 1960), itself a simple instance of, the much more famous, Wald's Maximin model (circa 1940).

Readers, therefore, have no clue that this theory is in fact, a

reinvention of an old wheel, but alasa square one, because it prescribes a misapplication of this model outside the model's intended domain of application.And this brings me to Sprenger's (2011) reference to my criticism of info-gap decision theory.

Reading Sprenger (2011), I have the impression that he did not read my criticism of info-gap decision theory very carefully, if at all. For, what is conspicuously missing from Sprenger's (2011) assessment of info-gap decision theory is the whole point of my criticism, which is that info-gap decision theory is being hailed (by its proponents) as a theory that was devised

specificallyfor decision making undersevere uncertainty. Hence, that it is singularly suitable for situations where the uncertainty space is vast (can be unbounded), the estimate is poor, and the uncertainty is quantified by a likelihood-free model.This, so the claims go, makes info-gap decision theory a reliable tool for handling surprises, rare events, catastrophes, shocks, etc. It is even suggested that this theory is suitable for the treatment of "Black Swans" and "Unknown Unknowns" (see Review 17).

So, the first point I want to make in my commentary on Sprenger's (2011) reference to my criticism of info-gap is that it is not only a case of info-gap proponents being silent on the fundamental flaws of info-gap decision theory, as one might have gathered from Sprenger (2011). It is more a case of its proponents vigorously advancing info-gap decision theory as a method that is

particularly suitablefor robust decision-making undersevere uncertainty. My criticism simply points out the obvious, namely that info-gap decision theory is clearly thewrong methodfor this purpose. And this is precisely the reason for my labeling this proposition a prescription for voodoo decision making par excellence. This is the punch-line of my argument!There is not the slightest hint in Sprenger (2011) that this is the case.

In fact, it is even unclear whether Sprenger (2011) is fully aware of the fact that the literature on info-gap decision theory (Ben-Haim 2001, 2006, 2010) goes out of its way to depict it as a theory that is

specifically designedforsevereuncertainty that is manifested in:

- A vast uncertainty space that is often unbounded.

- A poor estimate that is likely to be substantially wrong.

- A likelihood-free quantification of uncertainty
It is important to point out, therefore, given Sprenger (2011) assessment of the "virtues" of a "local analysis", that according to its inventor (Ben-Haim (2001, 2006, 2010)), info-gap decision theory was

notdesigned to seeklocalrobustness. Rather, it was designed to seek decisions that are robust against severe uncertainty ofthe type described above. Indeed, it is repeatedly stated in the info-gap literature (I hasten to add, erroneously and without any foundation), that info-gap decision theory selects decisions that yield the widest range of acceptable outcomes (e.g. see Review 21). And in Ben-Haim (2007) it is stated categorically (I again hasten to add, erroneously and without any foundation), that info-gap decision theory isnota theory of local robustness:Thus an info-gap model is not a “local analysis of risk” since the family of sets expands, usually boundlessly, as the unknown horizon of uncertainty, α, grows.

Ben-Haim (2007, p. 1)This being the case, the "great simplification" that Sprenger (2011) ascribes to info-gap's approach actually reflects a fundamental flaw, not a virtue. Because, in the case of info-gap decision theory, this "great simplification" is a manifestation of the fact that info-gap decision theory does not deal with the severity of the uncertainty that it is supposed to manage -- it simply takes no notice of it. That is, "the great simplification" in the case of info-gap decision theory in fact comes down to a prescription for robust decision under severe uncertainty which instructs the following: perform a robustness analysis in the vicinity of a wild guess and call it a day!

Clearly, this "simplicity" does not speak in info-gap's favor, neither practically nor theoretically.

If anything, Spenger's (2011) explanation of the

great simplification(see quote above) brings out my claim that info-gap decision theorydoes the precise oppositeof what a decision theory for severe uncertainty ought to do. And this is what earns this theory the title: voodoo decision theory par excellence.Consider next this statement, which is a continuation of the one cited above:

Whether these objections can, in a concrete case, be answered satisfactorily is a question that only scientists can decide. They are best equipped to evaluate the soundness of a peculiar decision-theoretic approach against the epistemic situation we are in, and the likely sources of error and bias.

Sprenger (2011, pp. 9-10)Sprenger (2011) is reminded that I have demonstrated on numerous occasions that even senior info-gap scholars are unaware of the limitations of the theory. Hence, it is naive in the extreme to assume, as Sprenger (2011) does, that we can take it for granted that info-gap scientists in the areas of applied ecology and conservation biology are "best equipped to evaluate the soundness of a peculiar decision-theoretic approach against the epistemic situation we are in".

I note this point because, a significant number of the reviews posted on this website deal with articles written by info-gap scholars in areas such as applied ecology, conservation biology, environmental management, and so on.

A case in point is the proposition by senior info-gap scholars in the area of applied ecology and conservation biology arguing that info-gap decision theory is a suitable tool for dealing with "Unknown Unknowns" and "Black Swans"! (see Review 17).

The Maximin connectionIn the discussion on the different understandings of the Precautionary Principle, Sprenger (2011) states the following:

These understandings can be mutually compatible, that is, it is possible to adopt two or more of the suggested readings of the PP. The first interpretation is clearly the most concrete, demanding and controversial. It seems to suggest something like amaximin rule, namely to adopt the option that ensures, if nature is malevolent, the least evil.

Sprenger (2011, p. 4)It is important to point out that there is not a clue in Sprenger (2011) that info-gap's robustness model is in fact a Maximin model (see proof here). This is puzzling given that this fact is proved formally in Sniedovich (2008), the very article referred to by Springer (2011, p. 9) in his discussion on the weaknesses of info-gap decision theory.

This fact is of the utmost importance because Sprenger's (2011) entire analysis is oblivious to the fundamental difference between

localandglobalrobustness, hence to the fundamentally different ramifications that alocal robustness analysishas as opposed to those of aglobal robustness analysis.So, the upshot is that because he fails to point out that info-gap's robustness model is in fact a special case of the Maximin model --- one that can perform

only a local robustness analysis(it is a radius of stability model), readers have no clue of the true meaning of the Precautionary approach that info-gap is credited with by Sprenger's (2011). If one is going to describe it as a Precautionary Principle, one must point out that as the special case of the Maximin model known as the Radius of Stability model, info-gap's robustness model performs a "local worst case analysis" which renders its analysis precautionary only with regard to the vicinity of an estimate that can be substantially wrong!This omission is particularly surprising in view of the two abstracts discussed below.

In sum:Info-gap's robustness model is a simple Radius of Stability model, itself a simple instance of Wald's Maximin model, the foremost model in decision theory for the treatment of severe uncertainty. It is therefore very odd that Sprenger (2011) contrasts an expected utility analysis with info-gap's robust satisficing model, without pointing out to the reader that this model is a simple instance of Wald's famous Maximin model that is known universally as Radius of Stability model.

And finally this:

Whereas most approaches, such as bounding probability densities, or working with probability intervals rather than precise numbers, specify the amount of uncertainty in advance, info-gap applies a sort of reverse engineering by deliberately leaving open the amount of uncertainty that we could possibly encounter (Halpern et al. 2006).

Sprenger (2011, p. 10)The fact that in the framework of info-gap decision theory the horizon of uncertainty (α) is unbounded does not mean that "... info-gap applies a sort of reverse engineering by deliberately leaving open the amount of uncertainty that we could possibly encounter ..." . The "amount of uncertainty that we could possibly encounter" is specified by the uncertainty space of the model, which must be stipulated by the user. The nested uncertainty sets U(α,û), α≥0, are specified accordingly.

The reason that info-gap users/scholars seem to be impressed with the fact that "...the amount of uncertainty that we could possibly encounter ..." can be left open (namely, that an info-gap model allows the uncertainty space to be unbounded), is simply due to a fundamental misconception of the impact of the size of the uncertainty space on the results of info-gap's analysis. The simple fact is, of course, that the robustness analysis conducted by info-gap's robustness model is

by definition local. This means that, methodologically speaking,it matters not in the slightesthow large the uncertainty space actually is. The results are totallyindependent ofthe size of the uncertainty space. I coined the termNo Man's Landto describe this bizarre characteristic of info-gap's robustness analysis, to explain the reason for my labeling info-gap decision theory a avoodoo decision theory par excellence (see Sniedovich 2010, 2011).So again, what to info-gap proponents (e.g. Halpern et al. 2006; Ben-Haim (2001, 2006, 2010) appears as a great virtue, not only turns out to be the precise opposite, but proves risible. Because, what this comes down to is that we can leave "the amount of uncertainty that we could possibly encounter ..." as open as we want, and this won't make one jot of difference to the results of our info-gap analysis. Because, the analysis is dictated only by the model's mode of operation, which as a radius of stability model, a priori seeks the

smallest destabilizing deviation from the estimate, in total disregard tothe "amount of uncertainty that we could possibly encounter". In other words, the fact that "...the amount of uncertainty that we could possibly encounter ..." can be left open, in the end comes to naught!And this bizarre characteristic of info-gap's analysis brings out the point that I make above, namely: how totally unsuitable a tool it is for the treatment of severe uncertainty. Because, for all the fuss about info-gap's uncertainty space being vast, indeed unbounded, hence singularly suitable for the treatment of severe uncertainty, when it comes to the crunch, the fact that the robustness analysis is

inherently localmeans that it does not even begin to address the severe uncertainty, which according to info-gap's basic assumptions, is manifested in the vast uncertainty space.So much then for the "reverse engineering" applied by info-gap!

Breaking news (April 22, 2011)

In the recent issue of Decision Point (48), Jan Sprenger tells us about his 2011 article.

It is interesting that he has chosen not to mention his criticism of info-gap decision theory. He thus gives the (very) wrong impression that info-gap decision theory maximizes the chance of an acceptable outcome under severe uncertainty.I just wonder how long the info-gap saga will continue unabated!

Within a week or so I shall update this review to reflect on Sprenger's Decision Point article. I may also be able to update the review itself based on a new version of the paper.

I take this opportunity to comment briefly on two abstracts written by Jan Sprenger in 2010. Since the full papers/presentations associated with these abstracts are not available, my comments are based solely on the abstracts.

With this in mind, consider this (color is added):

‘Philosophy of Science in a Forest’ (PSF2010)

Internationale Schol voor Wijsbegeerte, Leusden

Friday afternoon 14 May -- Saturday 15 May 2010

Jan Sprenger

Environmental Modeling and Decision-Making: Local vs. Global Approaches

http://www.nvwf.nl/Philosophy_of_Science_in_a_Forest/Programme_files/abstracts_contributed_papers_PSF_2010.pdf AbstractClimate scientists, conservation ecologists and econometricians have to make robust decisions under conditions of severe, fundamental uncertainty. Specifying all that is needed for a full decision-theoretic analysis (e.g. degrees of belief expressed as probabilistic judgments) is very often a matter of pure guesswork. This makes such an analysis appear overly subjective, and hard to implement in a practical problem. In particular, high degrees of uncertainty raise the question of whether it is admissible to take a particular mathematical model, computer simulation or numerical estimate as a point of departure (while accounting for the possibility that the starting point might be wrong). Such decision-theoretic approaches fall under the label of local decision theory. This is opposed to global decision theory, such as expected utility maximization or the maximin principle, which is based on the worst possible consequences of a certain action (Savage 1972). There we require a full probabilistic analysis to calculate a chances/risks tradeoff, and to choose the most efficient action.

Against a local approach speaks that in the environmental sciences, we often know that all models are missing the truth (Stainforth et al., 2007), due to the highly non-linear and often chaotic nature of the studied processes. Hence, even the most sophisticated and complicated models have to be regarded with great caution. Working with a particular estimate grants a privilege to the model used to generate it, although that model is, most of the time, not better and sometimes even worse than its competitors. Hence, privileging a particular model seems to lack a sound foundation and to lead to biased inferences and decisions. We feign a robustness that does not exist; instead we should select an approach where our nearly complete ignorance is more honestly represented.

This contribution examines the virtues and vices of local approaches in decision theory and points out that in spite of the aforementioned problems, they are often without alternative, and can have substantial advantages as well. The three arguments made in favor of local approaches concern (i) the difference between aiming at an optimal and aiming at an acceptable outcome (in other words, optimizing vs. satisficing) (ii) an analysis of the price of the robustness of a decision vis-a-vis its efficiency and (iii) the reference function of particular models and estimates. A reference scenario is a typical, though highly uncertain scenario where parameters can be varied for better studying and understanding the dynamics of the studied processes (e.g. global warming), and the consequences of our decisions (e.g. certain abatement and mitigation strategies).

Summing up, the paper investigates to what extent local approaches differ from, and can be defended against critiques from the globalist camp (Sniedovich 2008), and the implications of that debate for responsible decision-making in the environmental sciences.

References Ben-Haim, Yakov (2006): Info-Gap Decision Theory: Decisions Under Severe Uncertainty. Second edition. San Diego: Academic Press. Regan, H.M., Y. Ben-Haim, B. Langford, W.G. Wilson, P. Lundberg, S.J. Andelman, and M.A. Burgman (2005): Robust decision making under severe uncertainty for conservation management". Ecological Applications 15: 1471-1477. Savage, Leonard J. (1972): The Foundations of Statistics. New York: Wiley and Sons. Sniedovich, Moshe (2008): Wald's Maximin model: a treasure in disguise!, Journal of Risk Finance 9: 287-291. Stainforth, D. A., M. R. Allen, E. R. Tredger and L. A. Smith (2007): Confidence, uncertainty and decision-support relevance in climate predictions, Philosophical Transactions of the Royal Society A 365, 2145-2161.My comments on this abstract:

- It is totally unclear on what grounds does Sprenger (2010) conclude that my arguments in Sniedovich (2008) place me in a so-called "globalist camp".

- For to begin with, as I indicate above, the reference to Sniedovich (2008) is a case of a
mistaken identity. In Sniedovich (2008) I do not criticize info-gap decision theory, much less do I argue either for or against a local or a global analysis.

- If this description of my position is aimed to contend that I am arguing against a so-called "localist approach" to robustness, then this claim is of course utterly unfounded.

- As indicated above, in all my writings on this subject I explain clearly that there are situations where a local analysis is required and there are situations where a global analysis is required. Therefore, to present my position (as expressed in my writings on this subject) as categorically arguing for a global approach, hence against a local approach, is absurd. I doubt that anybody with the vaguest idea of the fundamental difference between local and global robustness would even dream of proposing such an outlandish idea.

- It is important to point out then that there is no such thing as a "globalist camp" as opposed to a "localist camp". There are methods that can be used only for a local analysis because, by definition, this is their mode of operation. And there are methods that can be used for a global analysis, because by definition, their mode of operation enables a global analysis. This means, of course, that my argument is not made as a member of some "globalist camp". I simply argue the obvious: it is utterly objectionable to employ a local robustness analysis when a global analysis is required.

- Hence, my criticism of info-gap decision theory is based precisely on this obvious fact. I argue that (1) this theory is based on a total lack of awareness of the fundamental distinction between local and global robustness, so that (2) it prescribes conducting a local analysis in situations that require a global analysis.

- I am also calling attention to the reference in this abstract to the "optimizing vs. satisficing" issue. The reason that I do this is to make it abundantly clear that the so-called difference between "optimizing and "satisficing" has got nothing whatsoever to do with the difference between local and global robustness. Furthermore, as I explained on numerous occasions, the "optimizing vs. satisficing" debate is without any merit because any "satisficing problem" can be formulated as an (equivalent) "optimization problem". I should add, tough, that Jan Sprenger discovered this fact for himself sometime between May 15, 2010 and January 9, 2011. For in Sprenger's own words:
As the reader may have noticed, this distinction resembles the well-known optimizing/satisficing dichotomy, but that terminology is slightly misleading since a satisficing rule can be represented as an optimizing rule as well.Sprenger (2011, footnote no. 1, p. 5)The reason that I nevertheless take up this point is to correct the groundless, plainly false claim by Sprenger (2010) that "optimizing vs. satisficing" speaks in favor of a local approach. The suggestion seems to be that given the (purported) difference between "optimizing" and "satisficing", a local approach has the merit of allowing "aiming at an acceptable outcome" rather than "aiming at an optimal outcome". This proposition is of course downright false because based on what I say above, even if assuming for a moment the (spurious) distinction between "satisficing" and "optimizing", this is immaterial to the type of analysis that you conduct, be it local or global.

- The other two points, namely (ii) and (iii) mentioned in the abstract, that presumably speak in favor of a local analysis, are also unwarranted, because they also have got nothing to do with the distinction between local and global robustness analyses. Still, I want to point out with regard to (iii) that if it is so desired, or required, then reference points, nominal points, estimates, etc. can be easily incorporated in a global robustness analysis. In fact, this is done routinely in the framework of the (global) expected utility approach where more "weight" can be given to certain points (e.g. estimate).
So, having dismissed the points that supposedly count in favor of a local robustness analysis, what emerges then is that, according to Sprenger (2011), the sole virtue that he would be able to ascribe to a local robustness analysis is that ... it is very simple.

But, as I point out above, in the case of info-gap decision theory, the purported "simplicity" is a manifestation of the fact that the local robustness analysis that it prescribes effectively instructs doing the following: ignore the severity of the uncertainty, conduct an analysis around a wild guess and call it a day! This done, you have completed a "simple" analysis whose objective is to seek out decisions that are robust against severe uncertainty!

Can this "simple" approach have any merit, methodologically speaking, as an approach for robust decision against

severeuncertainty?!

Now consider this (color added):

Modeling in the Social and Behavioral Sciences I, IHPST, Paris.

May 22, 2010, 17:30

Jan Sprenger

Environmental Modeling and Decision-Making: Local vs. Global Approaches

http://www-ihpst.univ-paris1.fr/operations/colloque.php?id_colloque=23 AbstractClimate scientists, conservation ecologists and econometricians have to make robust decisions under conditions of severe, fundamental uncertainty. Specifying all that is needed for a full decision-theoretic analysis (e.g. degrees of belief expressed as probabilistic judgments) is very often a matter of pure guesswork. This makes such an analysis appear overly subjective, and hard to implement in a practical problem. In particular, high degrees of uncertainty raise the question of whether it is admissible to take a particular mathematical model, computer simulation or numerical estimate as a point of departure (while accounting for the possibility that the starting point might be wrong).

Such decision-theoretic approaches fall under the label of local decision theory. This is opposed to global decision theory, such as expected utility maximization or the maximin principle, which is based on the worst possible consequences of a certain action. There we require a full probabilistic analysis to calculate a chances/risks tradeoff, and to choose the most efficient action.

Against a local approach speaks that in the environmental sciences, we often know that all models are missing the truth, due to the highly non-linear and often chaotic nature of the studied processes. Hence, even the most sophisticated and complicated models have to be regarded with great caution. Working with a particular estimate grants a privilege to the model used to generate it, although that model is, most of the time, not better and sometimes even worse than its competitors. Privileging a particular model seems to lack a sound foundation and to lead to biased inferences and decisions. We feign a robustness that does not exist; instead we should select an approach where our nearly complete ignorance is more honestly represented.

This contribution examines the virtues and vices of local approaches in decision theory and points out that in spite of the aforementioned problems, they are often without alternative, and can have substantial advantages as well.

My comments on this:

- First, the identification of the Maximin principle with a global decision theory. Does Sprenger (2010a) suggest that the Maximin model cannot be employed to formulate a local robustness analysis ?!?!

- Furthermore, if Maximin is identified only with a global decision theory, does Sprenger (2010a) argue that info-gap's robustness model (which he identifies with a local decision theory) is not a Maximin model?

- Sprenger (2010a) must also explain and justify the last statement (here printed in red) in his abstract. In brief.
As I indicated on numerous occasions, there are situations where a local robustness analysis is required and there are situations where a global analysis is required. For example, in my recent critique of info-gap decision theory (see Sniedovich 2010, 2011), I point out that info-gap's robustness model is a Radius of Stability model and that this model has been used extensively in many fields for the treatment of small perturbations of a nominal value of a parameter of interest. Namely, I indicate clearly that the Radius of Stability model is used routinely for the purpose of determining

localrobustness/stability.Pursuant to this, I argue that this model is utterly unsuitable for the treatment of severe uncertainty of the type addressed by info-gap decision theory. That is, I argue that the local analysis prescribed by such a model is utterly unsuitable for dealing with problems subject to an uncertainty that is manifested in a vast uncertainty space, a poor estimate and a non-probabilistic likelihood-free quantification of uncertainty.

Are we to conclude then from the last statement in this abstract that Sprenger (2010a) is suggesting that a local approach, such as info-gap decision theory, is -- warts and all -- "without alternative" for decision making under severe uncertainty, where the severity of the uncertainty is of this type?

I urge Jan Sprenger to clarify this point.

## Summary and conclusions

To the best of my knowledge, the short paragraph in Sprenger (2011, p. 9) summarizing the weaknesses of info-gap decision theory, is the first "serious discussion" in the literature on the weaknesses of info-gap decision theory that I identified and documented some time ago (2006).

However, it is too early to tell whether the message, this time delivered by another person, will reach its destination.

It is unfortunate, however, that Sprenger (2010, 2010a, 2011) has not read carefully my critique of info-gap decision theory. I suggest and he read the articles listed at the bottom of the page, especially the following three, to appreciate what my criticism of info-gap decision theory is all about:

- Sniedovich, M., (2011) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty,
International Transactions in Operational Research, in press.

- Sniedovich, M., (2010) A bird's view of Info-Gap decision theory,
Journal of Risk Finance, 11(3), 268-283.

- Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty,
Decision Making in Manufacturing and Services,1-2, 111-136. (PDF File).Sprenger (2010, 2010a, 2011) will also benefit from reading the reviews of other info-gap publications in this directory.

- Ben-Haim (2001, 2006): Info-Gap Decision Theory: decisions under severe uncertainty.

- Regan et al (2005): Robust decision-making under severe uncertainty for conservation management.

- Moilanen et al (2006): Planning for robust reserve networks using uncertainty analysis.

- Burgman (2008): Shakespeare, Wald and decision making under severe uncertainty.

- Ben-Haim and Demertzis (2008): Confidence in monetary policy.

- Hall and Harvey (2009): Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.

- Ben-Haim (2009): Info-gap forecasting and the advantage of sub-optimal models.

- Yokomizo et al (2009): Managing the impact of invasive species: the value of knowing the density-impact curve.

- Davidovitch et al (2009): Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island.

- Ben-Haim et al (2009): Do we know how to set decision thresholds for diabetes?

- Beresford and Thompson (2009): An info-gap approach to managing portfolios of assets with uncertain returns

- Ben-Haim, Dacso, Carrasco, and Rajan (2009): Heterogeneous uncertainties in cholesterol management

- Rout, Thompson, and McCarthy (2009): Robust decisions for declaring eradication of invasive species

- Ben-Haim (2010): Info-Gap Economics: An Operational Introduction

- Hine and Hall (2010): Information gap analysis of flood model uncertainties and regional frequency analysis

- Ben-Haim (2010): Interpreting Null Results from Measurements with Uncertain Correlations: An Info-Gap Approach

- Wintle et al. (2010): Allocating monitoring effort in the face of unknown unknowns

- Moffitt et al. (2010): Securing the Border from Invasives: Robust Inspections under Severe Uncertainty

- Yemshanov et al. (2010): Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest

- Davidovitch and Ben-Haim (2010): Robust satisficing voting: why are uncertain voters biased towards sincerity?

- Schwartz et al. (2010): What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making

- Arkadeb Ghosal et al. (2010): Computing Robustness of FlexRay Schedules to Uncertainties in Design Parameters

- Hemez et al. (2002): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

- Hemez et al. (2003): Applying information-gap reasoning to the predictive accuracy assessment of transient dynamics simulations

- Hemez, F.M. and Ben-Haim, Y. (2004): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

- Ben-Haim, Y. (2007): Frequently asked questions about info-gap decision theory

- Sprenger, J. (2011): The Precautionary Approach and the Role of Scientists in Environmental Decision-Making

- Sprenger, J. (2011): Precaution with the Precautionary Principle: How does it help in making decisions

- Hall et al. (2011): Robust climate policies under uncertainty: A comparison of Info--Gap and RDM methods

- Ben-Haim and Cogan (2011) : Linear bounds on an uncertain non-linear oscillator: an info-gap approach

- Van der Burg and Tyre (2011) : Integrating info-gap decision theory with robust population management: a case study using the Mountain Plover

- Hildebrandt and Knoke (2011) : Investment decisions under uncertainty --- A methodological review on forest science studies.

- Wintle et al. (2011) : Ecological-economic optimization of biodiversity conservation under climate change.

- Ranger et al. (2011) : Adaptation in the UK: a decision-making process.

## Recent Articles, Working Papers, Notes

Also, see my complete list of articles

Moshe's new book! - Sniedovich, M. (2012) Fooled by local robustness,
Risk Analysis, in press.

- Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty,
International Transactions in Operational Research, in press.

- Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis,
Applications of Mathematics,56(5), 499-509.

- Sniedovich, M. (2011) Dynamic programming: introductory concepts, in
Wiley Encyclopedia of Operations Research and Management Science(EORMS), Wiley.

- Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem,
OR Spectrum, 33(4), 815-929, 2011.

- Sniedovich, M. (2011)
Dynamic Programming: Foundations and Principles, Second Edition,Taylor & Francis.

- Sniedovich, M. (2010) A bird's view of Info-Gap decision theory,
Journal of Risk Finance, 11(3), 268-283.

- Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.

- Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds),
.Safety, Reliability and Risk Analysis: Theory, Methods and Applications,pp. 2071-2079, Taylor and Francis Group, London.

- Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)

- Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.),
11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation,DIPTEM, Genova, 89-94.

- Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)

- Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)

- Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08

This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )

- Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory,
Decision Point,24, 10.

- Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty,
Decision Point,22, 8-9.

- Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!,
Journal of Risk Finance,9(3), 287-291.

- Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)

In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.

- Sniedovich. M. (2008) The Mighty Maximin! (PDF File)

This paper is dedicated to the modeling aspects of Maximin and robust optimization.

- Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty,
Decision Making in Manufacturing and Services,1-2, 111-136. (PDF File) .

- Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)

In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.

- My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)
This is an exciting development!

- Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.

- Ben-Haim is not familiar with the fundamental concept
point estimate. He does not realize that a function can be a point estimate of another function.So when you read my papers make sure that you do not misinterpret the notion

point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!

- A formal proof that
Info-GapisWald's Maximin Principlein disguise. (December 31, 2006)

This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)

It shows thatInfo-Gapis not a new theory but rather a simple instance ofWald's famous Maximin Principledating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context ofGame Theory(1928).

- A proof that
Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)

This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .

It shows that becauseInfo-Gapdeploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.

- A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)

This is a very short article entitled The GAP in Info-Gap (PDF File) .

It is a math-free version of the paper above. Read it if you are allergic to math.

- A long essay entitled
What's Wrong with Info-Gap? An Operations Research Perspective(PDF File) (December 31, 2006).

This is a paper that I presented at theASOR Recent Advances in Operations Research(PDF File) mini-conference (December 1, 2006, Melbourne, Australia).## Recent Lectures, Seminars, Presentations

If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.

Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.

- ASOR Recent Advances, 2011, Melbourne, Australia, November 16 2011. Presentation: The Power of the (peer-reviewed) Word. (PDF file).

- Alex Rubinov Memorial Lecture The Art, Science, and Joy of (mathematical) Decision-Making, November 7, 2011, The University of Ballarat. (PDF file).

- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Invited tutorial, ALIO/INFORMS Conference, Buenos Aires, Argentina, July 6-9, 2010).

- A Critique of Info-Gap Decision theory: From Voodoo Decision-Making to Voodoo Economics(PDF File) .

(Recent Advances in OR, RMIT, Melbourne, Australia, November 25, 2009)

- Robust decision-making in the face of severe uncertainty(PDF File) .

(GRIPS, Tokyo, Japan, October 16, 2009)

- Decision-making in the face of severe uncertainty(PDF File) .

(KORDS'09 Conference, Vilnius, Lithuania, September 30 -- OCtober 3, 2009)

- Modeling robustness against severe uncertainty (PDF File) .

(SOR'09 Conference, Nova Gorica, Slovenia, September 23-25, 2009)

- How do you recognize a Voodoo decision theory?(PDF File) .

(School of Mathematical and Geospatial Sciences, RMIT, June 26, 2009).

- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, Info-Gaps, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Department of Econometrics and Business Statistics, Monash University, May 8, 2009).

- The Rise and Rise of Voodoo Decision Theory.

ASOR Recent Advances, Deakin University, November 26, 2008. This presentation was based on the pages on my website (voodoo.moshe-online.com).

- Responsible Decision-Making in the face of Severe Uncertainty (PDF File) .

(Singapore Management University, Singapore, September 29, 2008)

- A Critique of Info-Gap's Robustness Model (PDF File) .

(ESREL/SRA 2008 Conference, Valencia, Spain, September 22-25, 2008)

- Robust Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Technion, Haifa, Israel, September 15, 2008)

- The Art and Science of Robust Decision-Making (PDF File) .

(AIRO 2008 Conference, Ischia, Italy, September 8-11, 2008 )

- The Fundamental Flaws in Info-Gap Decision Theory (PDF File) .

(CSIRO, Canberra, July 9, 2008 )

- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .

(OR Conference, ADFA, Canberra, July 7-8, 2008 )

- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .

(University of Sydney Seminar, May 16, 2008 )

- Decision-Making Under Severe Uncertainty: An Australian, Operational Research Perspective (PDF File) .

(ASOR National Conference, Melbourne, December 3-5, 2007 )

- A Critique of Info-Gap (PDF File) .

(SRA 2007 Conference, Hobart, August 20, 2007)

- What exactly is wrong with Info-Gap? A Decision Theoretic Perspective (PDF File) .

(MS Colloquium, University of Melbourne, August 1, 2007)

- A Formal Look at Info-Gap Theory (PDF File) .

(ORSUM Seminar , University of Melbourne, May 21, 2007)

The Art and Science of Decision-Making Under Severe Uncertainty(PDF File) .

(ACERA seminar, University of Melbourne, May 4, 2007)

- What exactly is Info-Gap? An OR perspective. (PDF File)

ASOR Recent Advances in Operations Researchmini-conference (December 1, 2006, Melbourne, Australia).

Disclaimer:This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.

Last modified: [an error occurred while processing this directive]