Reviews of publications on Info-Gap decision theory

Review # 12(Posted: July 10, 2009; Last update:August 13, 2009)

Reference: Heterogeneous uncertainties in cholesterol management

Yakov Ben-Haim, Clifford C. Dacso, Jonathon Carrasco, Nithin Rajan

International Journal of Approximate Reasoning

50, 1046–1065, 2009Abstract Physicians use clinical guidelines to inform judgment about therapy. Clinical guidelines do not address three important uncertainties: (1) uncertain relevance of tested populations to the individual patient, (2) the patient’s uncertain preferences among possible outcomes, and (3) uncertain subjective and financial costs of intervention. Unreliable probabilistic information is available for some of these uncertainties; no probabilities are available for others. The uncertainties are in the values of parameters and in the shapes of functions. We explore the usefulness of info-gap decision theory in patient-physician decision making in managing cholesterol level using clinical guidelines. Info-gap models of uncertainty provide versatile tools for quantifying diverse uncertainties. Info-gap theory provides two decision functions for evaluating alternative therapies. The robustness function assesses the confidence—in light of uncertainties—in attaining acceptable outcomes. The opportuneness function assesses the potential for better-than-anticipated outcomes. Both functions assist in forming preferences among alternatives. Hypothetical case studies demonstrate that decisions using the guidelines and based on best estimates of the expected utility are sometimes, but not always, consistent with robustness and opportuneness analyses. The info-gap analysis provides guidance when judgment suggests that a deviation from the guidelines would be productive. Finally, analysis of uncertainty can help resolve ambiguous situations. Acknowledgement The authors gratefully acknowledge the support of the Abramson Center for the Future of Health. The authors are indebted to comments by Scott Ferson, Malka Gorfine, Matthias Troffaes and Miriam Zacksenhouse. Scores TUIGF:100%

SNHNSNDN:6000%

GIGO:100%

## Overview

This is a typical Info-Gap paper: it is full of the usual errors, misconceptions, and obfuscations that by now have become part and parcel of the Info-Gap enterprise.

Since 2006, I have repeatedly pointed out these two embarrassing facts about Info-Gap decision theory:

Fact 1: Info-Gap's robustness model is a simple instance of Wald's famous Maximin model (circa 1940).

Fact 2: Info-Gap decision theory is a classic example of a voodoo decision theory.The reason that the truth about Fact 1 is embarrassing is simple. Since its introduction in the late 1990s, Info-Gap decision theory has been acclaimed as distinct, novel, revolutionary, and radically different from all current theories for decision under severe uncertainty.

Yet, my Maximin Theorem shows this claim for what it is: a pure myth!

The reason that the truth here is embarrassing is not merely that Info-Gap's robustness model is a reinvention of the wheel. Indeed, the truth here is hugely embarrassing because Info-Gap's robustness model is in fact a simple instance of none other than the most famous model in classical decision theory for the treatment of severe uncertainty: Wald's Maximin model. In other words, Info-Gap's robustness model is a simple instance of a model that, since the 1950s, has become the bread and butter approach to the management of severe uncertainty in classical decision theory and robust optimization.

The truth about Fact 2 is embarrassing because Info-Gap decision theory is hailed as a theory that is particularly suitable for the treatment of severe uncertainty

Yet, my Invariance Theorem shows this claim for what it is: a pure myth!

It shows that Info-Gap decision theory in fact constitutes the precise

antithesisof what a theory for the treatment ofsevereuncertainty ought to be. In other words, it shows that Info-Gap decision theory "deals" withsevereuncertainty by simply ...ignoring the severityof the uncertainty altogether. This fact renders Info-Gap decision theory a voodoo decision theory par excellence.Details concerning these and other myths about Info-Gap decision theory can be found in my discussion on Myths and Facts about Info-Gap and in FAQs about Info-Gap.

So, you may well wonder: given that Info-Gap decision theory is so gravely flawed, and that its flaws are so detrimental to it, how is it that the paper under review here was accepted for publication in a refereed journal?

The answer is very simple: whatever Info-Gap decision theory lacks in substance and rigor, is made up for, and covered up, by heavy fog, spin, and rhetoric.

Indeed, to be able to pin down the truth about this theory, I had to cut through some mighty fog, spin, and rhetoric.

## Some Basic Facts

The good news is that now that I have managed to accomplish this, the fundamental flaws in Info-Gap decision theory are clearly there for all to see. For example, to see that Info-Gap decision theory does not tackle the

severityof the uncertainty at all, but simplyignoresit, you need not even examine the Invariance Theorem. A quick look at this simple picture reveals most of the story:

No Man's Land û No Man's Land

<-------------- Complete region of uncertainty under consideration --------------> where û denotes the estimate of the parameter of interest, the black area represents the complete region of uncertainty under consideration, the red area around û represents the region of uncertainty that actually affects the results generated by Info-Gap's robustness analysis, and the vast

No Man's Landrepresents that part of the complete region of uncertainty that has no impact whatsoever on the results generated by Info-Gap's robustness model.Recalling that under conditions of

severeuncertainty the estimate û is awild guess, apoor indicationof the true value of the parameter of interest and is likely to besubstantially wrong, it is immediately clear that Info-Gap decision theory violates the universally accepted Garbage In Garbage Out (GIGO) Axiom. This crucial fact renders this theory a voodoo decision theory par excellence.

Example 1: On q' and q''.Consider the case shown in the following picture:

where q' and q'' are two decisions, R

_{c}=6 is the critical reward level associated with the performance constraint R(q,u) ≤ R_{c}, and u denotes the parameter of interest whose true value is subject to severe uncertainty. Note that the estimate of the true value of u is û = 0, and that the complete region of uncertainty isU=(-∞,∞).The performance requirement R(q,u) ≤ R

_{c}means that we prefer decisions whose R(q,u) values are small — ideally below the critical level R_{c}= 6 — overU.It is assumed that R(q'',u) continues its (linear) ascent in both directions and that R(q',u) continues its (quadratic) descent in both directions. The colored (red and blue) areas at the bottom of the picture show the regions of

Uwhere the performance requirement is satisfied by these two decisions.It is clear that q' dominates q'' over most of the uncertainty space

U=(-∞,∞). In fact, q'' violates the performance requirement on most ofU, whereas q' satisfices the performance requirement on most ofU.But this does not prevent Info-Gap decision theory to deem q'' to be more robust than q'.

So, there are no too ways about it: q' is far more robust than q'' with respect to the given performance requirement R(q,u) ≤ R

_{c}relative to the complete uncertainty spaceU. Yet, Info-Gap's peculiar (local) definition of robustness deems q'' to be more robust.

You are cordially invited to a guided tour of Info-Gap robustness analysis including an interactive animation and an explanation of the reasons why Info-Gap decision theory regards q'' to be more robust than q'. In so doing, Info-Gap decision theory puts itself at loggerheads with the universally accepted Garbage In — Garbage Out (GIGO) Axiom and the well known dictum that the results of an analysis can be only as good as the estimate on which they are based. So, the point is that in the case of Info-Gap decision theory the results yielded by its analysis can be no better than ... wild guesses.

To suggest otherwise would amount to suggesting that Info-Gap decision theory possesses mysterious (magical) powers that enable it to translate an analysis around a wild guess into a reliable methodology for the treatment of severe uncertainty.

Conventional Science Info-Gap Decision Theory

wild guess -----> Model -----> wild guess

wild guess -----> Model -----> reliable

robust decisionIf this is not a classic example of a voodoo decision theory what is? And isn't it also modern alchemy?

The trouble of course is that this simple fact is, as a rule, enveloped in the Info-Gap publications by fog, spin, and rhetoric. The article under review here is no exception. If anything, the fog, spin, and rhetoric have reached here unprecedented heights. Therefore, to give a full account of this phenomenon will take volumes.

But, to give you an idea, I shall illustrate it in action in connection with the Maximin issue. That is, I shall show how the simple fact that Info-Gap's robustness model is actually a simple instance of Wald's famous Maximin model is covered up by vintage Info-Gap fog, spin, and rhetoric.

The story goes like this.

In 2007 I advised users/promoters of Info-Gap decision theory that — contrary to repeated claims in the Info-Gap literature — not only is Info-Gap's robustness model neither new nor radically "different" from classical decision theory models, it is in fact a simple instance of Wald's famous Maximin model (circa 1940).

To substantiate this claim, I provided a detailed proof.

For our limited purposes here, it will be convenient to summarize this result as follows:

Info-Gap's Robustness model Corresponding instance of Wald's Maximin model max {α ≥ 0: r(d,u) ≤ r* , ∀u∈U(α,û)} ≡

max min f(d,α,u) α ≥ 0 u∈U(α,û) where f(d,α,u) = α if r(d,u) ≤ r*; and f(d,α,u) = -∞, otherwise.

Note that this implies that

α(û) := max max {α≥0: r(d,u) ≤ r*, ∀u∈U(α,û)} d∈D ≡

α(û):= max min f(d,α,u) d∈D u∈U(α,û) α≥0

Proof of the Maximin Theorem:

Instance of Wald's Maximin Model Equivalent Math Programming formulation

max min f(d,α,u) α ≥ 0 u∈U(α,û) ≡

max { v: v ≤ f(d,α,u), ∀ u∈U(α,û) } α ≥ 0 v ∈ ℜ

≡

max { α: α ≤ f(d,α,u), ∀ u∈U(α,û) } α ≥ 0

≡

max { α: α ≤ f(d,α,u) , ∀ u∈U(α,û) }

≡

max { α: r(d,u) ≤ r*, ∀ u∈U(α,û) } Info-Gap's Robustness Model The bottom line is there for all to see: Info-Gap's robustness model is a simple instance of Wald's famous Maximin model, namely

Wald's Maximin Model z*:= max min g(x,s) x∈X s∈S(x) More on this model can be found in the discussion on the Maximin model.

This simple instance of Wald's famous Maximin model always yields the same results as those yielded by Info-Gap's robustness model. No amount of empty rhetoric and spin can change this fact.

Note that the Maximin Theorem is constructive: it sets out a simple recipe for constructing that instance of the generic Maximin model that represents Info-Gap's robustness model.

No amount of fancy rhetoric or spin or fog can change this bottom line.

So far so good.

Given this proof, the question of course is: how can anyone possibly argue that Info-Gap's robustness model is

nota Maximin model?In other words, the proof of the Maximin Theorem is so simple, indeed straightforward, so how can one possibly prove the opposite?

Answer: faulty reasoning supported by plenty of fog, spin and rhetoric.

## Faulty Reasoning

For the benefit of readers who are not familiar with Info-Gap decision theory, let me begin the illustration with a simple example that requires no knowledge of Info-Gap.

How would you prove formally that a perfectly healthy, seven year old kangaroo from Western Australia, call her Jill, is

nota marsupial?Of course, if your reasoning is sound and you are using conventional scientific methods of biological identification/classification it is impossible to do this.

Because, as it is a well-established fact that kangaroos are marsupials, and Jill being a healthy kangaroo, it follows that Jill is a marsupial.

But employing the type of faulty reasoning that underlies Info-Gap decision theory, showing that Jill is

nota marsupial is a trifle.Because, all one needs to do to this end is to find a certified ... koala. Yes, trust me, a ... certified koala,

nota kangaroo.

So, to simplify the discussion assume that we found a certified koala, call him Jack.

We now have a certified koala called Jack, and a kangaroo called Jill.

The rest, according to Info-Gap's reasoning, is a trivial task. To wit:

A proof that Jill the kangaroo is nota marsupial

- Clearly, Jack is a marsupial.

- Clearly, Jill is different from Jack.

- Therefore, clearly Jill is
nota marsupial.Indeed, employing this kind of reasoning almost anything can be proved. For instance, consider this gravity-defying trick:

Example 3: On p and q.A proof that q(x) = 1 + x + x ^{2}isnota polynomial

- Clearly, p(x):= x
^{4}is a polynomial.

- Clearly, q is different from p.

- Therefore, clearly q(x)=1 + x + x
^{2}isnota polynomial.Let me now show you how — based on this kind of reasoning — Ben-Haim argues that Info-Gap's robustness models, namely

Info-Gap's robustness model

α(û) = max max {α≥0: r(d,u) ≤ r*, ∀u∈U(α,û)} d∈D is not a Maximin/Minimax model. Namely that it is not an instance of

Wald's generic Maximin Model z*:= max min g(x,s) x∈X s∈S(x) In line with the reasoning illustrated above, the first thing to do is to come up with a Maximin/Minimax model that is

notequivalent to Info-Gap's robustness model.This is an easy task because there are infinitely many such models. The following is the Minimax model selected by Ben-Haim:

Ben-Haim's Minimax Model α(û;α'):= min max r(d,u) d∈D u∈U(α',û) where α'>0 is some given real number.

Note that this is a Minimax, rather than a Maximin, model!

Obviously, there is a value of α' such that the optimal decision generated by Ben-Haim's model for this value of α' is optimal with respect to the Info-Gap model. The difficulty is that this specific value of α is not known in advance.

Thus, it is very easy to show that, in general, this specific Minimax model is not equivalent to Info-Gap's robustness model.

Ben-Haim uses this trivial argument as "proof" that Info-Gap's robustness model is not a Minimax/Maximin model.

This, however, is not approximate reasoning: this is

faultyreasoning.All that Ben-Haim's "proof" shows is that the ad hoc, inappropriate, Minimax model that he created for this exercise is not equivalent to Info-Gap's robustness model. But, this does not show that Info-Gap's robustness model is not an instance of Wald's Maximin model. Indeed, as shown by the Maximin Theorem, there is definitely an instance of Wald's Maximin model that is equivalent to Info-Gap's robustness model.

In short, the picture is this:

Info-Gap's robustness model

α(û) = max max {α≥0: r(d,u) ≤ r*, ∀u∈U(α,û)} d∈D Ben-Haim's Incorrect Minimax model Correct Maximin model

ρ(û;α'):= min max r(d,u) d∈D u∈U(α',û)

α(û):= max min α·(r(u,d) ◊ r*) d∈D u∈U(α,û) α≥0 Note: α' is a pre-specified positive number. Note: a◊b =1 iff a≤b; a◊b =0 otherwise. Note that in contrast to Ben-Haim's incorrect Minimax model, the correct Maximin model generates, as an integral part of the maximization operation, the specific value of α that is required by Ben-Haim's incorrect Minimax model, namely α'=α(û).

Of course, had Ben-Haim's "proof" been shown for what it is, namely had it been formulated as shown above, then even the referees of journals specializing in approximate reasoning would no doubt have identified the flaw in the reasoning underlying the "proof".

But the trouble is that Ben-Haim — the Founder of Info-Gap decision theory — and his co-authors, set out to "make the case" for the indefensible claim that Info-Gap's robustness model is not a Maximin and/or Minimax model with the aid of an avalanche of empty rhetoric and spin that are unprecedented even by Info-Gap standards.

## Fog, Spin and Rhetoric

Let us see then how thick the fog, how extensive the spin, and how high the rhetoric can be, in a situation where the question under consideration is the relationship between two simple mathematical models:

Wald's generic Maximin Model z*:= max min g(x,s) x∈X s∈S(x)

Info-Gap's robustness model

α(û) = max max {α≥0: r(d,u) ≤ r*, ∀u∈U(α,û)} d∈D Consider the following dissertation, quoted from the paper under review, while all the while keeping in mind that the Maximin Theorem is staring at us from the pages of refereed journals, from WIKIPEDIA and from many articles posted on the web:

Min–max decisions.The min–max approach identifies a set of possible contingencies or models or relevant functions and seeks the decision which minimizes the worst (maximal) loss on this set. (The max-min approach maximizes the lowest (minimal) gain when considering benefits rather than losses.) This concept is implicit in many robust Bayesian realizations, and many of our comments there apply here as well.Epistemically,the info-gap and min–max approaches are similar in representing uncertainty without measure functions, though the min-max approach requires the choice of a specific set. Thehybridizationof a min-max with an info-gap approach is often attractive, as discussed in connection with Bayesian methods. Indeed, Wald’s work in the early 1940s on min-max considers sets of uncertain probability distributions [51].

Behavior.We will discuss two concepts: the observational equivalence of min-max with info-gap robust-satisficing, and the behavioral difference of these methods [52].

Observational equivalence:Suppose a robust-satisficing decision maker must choose between two options, D_{1}and D_{2}, and requires an outcome no worse than L_{c}in Fig. 14. This leads the robust-satisficer to choose decision D_{1}, which is more robust than D_{2}at this requirement. An observer can describe this behavior by supposing the decision maker to be an min-maxer who believes that the horizon of uncertainty is α_{1}, because at this level of uncertainty the maximum potential loss from D_{1}is less than from D_{2}. Conversely, a min-maxing decision maker who believes that α_{1}is the true horizon of uncertainty would likewise choose D_{1}over D_{2}. An observer could describe this by supposing the decision maker is a robust-satisficer whose requirement is L_{c}. In short, either strategy can be used to describe observed behavior by ascribing particular beliefs to the decision maker. In other words, the modelling of decision-behavior under uncertainty is under-determined in choosing between robust-satisficing and min-maxing.

Fig. 14. Crossing robustness curves, illustrating the observational equivalence and behavioral difference between min-maxing and robust-satisficing.

Behavioral difference:Suppose a min-maxing decision maker believes that the horizon of uncertainty can be as large as α_{2}in Fig. 14, but no larger. The min-maxer will prefer D_{2}, whose loss can be as large as L_{m}, but less than the maximum potential loss of D_{1}. Suppose a robust-satisficing decision maker can accept a loss as large as L_{c}, but no larger. This robust-satisficing decision maker will prefer D_{1}over D_{2}since D_{1}can tolerate greater uncertainty for achieving this requirement. The robust-satisficer will choose D_{1}over D_{2}even if the min-maxer has convinced the robust-satisficer that α_{2}is the true horizon of uncertainty. In short, when the robustness curves for two decisions cross one another, a min-maxer and robust-satisficer may disagree about the decision, depending on their beliefs and requirements.In conclusion, the

observational equivalencebetween min-maxing and robust-satisficing means that modellers can use either strategy to describe observed behavior of decision makers. In contrast, the behavioral difference means that actual decision makers will not necessarily be indifferent between these strategies, and will choose a strategy according to their beliefs and aspirations.The concepts of observational equivalence and behavioral difference have been noted before, in different terms. Walley writes [53, p. 10]:

Every [Dempster-Shafer] belief function can be represented as a lower envelope of a set of probability measures. This is merely a mathematical representation, however; it is misleading and unnecessary to regard a belief function as a lower bound for some unknown probability measure. In the same way, every coherent lower prevision can be represented as a lower envelope of a set of linear previsions, but this is no reason to regard the lower prevision as a model for partial information about an unknown linear prevision.The observational equivalence of min-maxing and robust-satisficing asserts that either can be used as a mathematical representation of the other. The behavioral difference between these methods asserts that real decision makers with specific beliefs and requirements need not be indifferent between these methods.

Yakov Ben-Haim, Clifford C. Dacso, Jonathon Carrasco, Nithin Rajan (2009, pp. 1061-1062)

Note: Fig. 14 is mine. It is very similar to the original.As clearly indicated by the Maximin Theorem, the authors' conclusion regarding the relationship between Info-Gap's robustness model and Wald's Maximin model is out and out false!

More specifically, the authors' statement

In conclusion, theobservational equivalencebetween min-maxing and robust-satisficing means that modellers can use either strategy to describe observed behavior of decision makers. In contrast, the behavioral difference means that actual decision makers will not necessarily be indifferent between these strategies, and will choose a strategy according to their beliefs and aspirations.is grossly misleading.

Keep in mind that the min-max model that this assertion refers to is the ill-considered, hence inappropriate, min-max model which, as I pointed out already, Ben-Haim always deploys to "make the case" for the alleged dissimilarity between Info-Gap and Maximin:

Ben-Haim's ill-considered Minimax Model α(û;α'):= min max r(d,u) d∈D u∈U(α',û) This ill-conceived min-max model is discussed in detail in the article Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model and in FAQ # 20.

The authors' statement does

notapply to

Wald's generic Maximin Model z*:= max min g(x,s) x∈X s∈S(x) Indeed, the Maximin Theorem assures us that Info-Gap's robustness model is a simple instance of this model.

For this reason the authors' statement is as misleading and as hair-raising as the statement

q(x) = 1 + x + x ^{2}isnota polynomialThe point is this.

To so much as attempt a comparison between Info-Gap’s robustness model and Wald's Maximin model, in a manner suggesting that the two models are equals, is already grossly misleading, because such a comparison puts a general prototype model on the same footing with one of its countless instances.

Wald's Maximin model, as anyone in the business of decision theory would no doubt know, is an all-embracing, extremely flexible, hence powerful paradigm. This means of course that it has the inherent ability to subsume infinitely many instances. And as indicated by the Maximin Theorem, Info-Gap’s robustness model is one of these instances. So, as one of the countless instances of Wald's generic Maximin model, Info-Gap's robustness model will (surprise, surprise!) be different from other instances. Hence, as one of the instances of Wald's generic Maximin model, Info-Gap's robustness model is different from the ad hoc model formulated by Ben-Haim, which is equally an instance of Wald's generic Maximin model.

This fact merely indicates the obvious, namely that Wald's generic Maximin model is incomparably more general and powerful than Info-Gap's robustness model.

All one needs to do to prove formally that Info-Gap's robustness model is an instance of Wald's Maximin model is to show that at

least oneof the infinitely many instances of Wald's generic Maximin model is equivalent to Info-Gap's robustness model.Instead, Ben-Haim, prefers to engage in empty rhetoric. That is, rather than go for the legitimate, perfectly suitable instance that is staring at him from the Maximin Theorem, Ben-Haim always picks his ill-considered instance of Wald's Maximin that is not equivalent to Info-Gap's robustness model, to "make a case" for an alleged difference” between Info-Gap's robustness model and Maximin.

All this means then is that the authors' conclusion applies only to the ill-considered Minimax model that they formulate in this paper. It does not apply to the simple instance of Wald's generic Maximin model that is specified by the Maximin Theorem. Indeed, this instance of Wald's Maximin model always yields the same results as those yielded by Info-Gap's robustness model. Therefore, the difference between these two models, namely Info-Gap's robustness model and the Maximin model specified by the Maximin Theorem, has to do with style not with substance. And as a consequence, the decision makers' choice between the two models has absolutely nothing to do with their beliefs and aspirations. Both are Maximin models and both yield the same results.

And so, the authors erroneous thesis does not prove that Info-Gap's robustness model is not a Maximin model. All it proves is the authors' obvious misconceptions about the modeling aspects of the Maximin/Minimax paradigm, their misapprehension as to how the Maximin/Minimax paradigm handles constraints, and so on. All this bars them from grasping the full extent of the affinity between Info-Gap's robustness model and Wald's Maximin model.

Note that the Maximin Theorem is constructive: it sets out a simple recipe for constructing the instance of the generic Maximin model that represents Info-Gap's robustness model.

No amount of fog/rhetoric/spin can change this bottom line.

As indicated above, the conceptual and technical mistakes that led Ben-Haim astray on this matter are discussed in detail in the article Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model and in FAQ # 20.The trouble of course is that the avalanche of empty rhetoric and spin hides all this from view. What is more, it hides from view some fundamental technical errors.

## Elementary Technical Matters

Unfortunately, even in this department the authors are on extremely shaky grounds.

For instance, consider the statement

Epistemically,the info-gap and min–max approaches are similar in representing uncertainty without measure functions, though the min-max approach requires the choice of a specific set.How did the authors come up with this manifestly erroneous idea? Where did they get it from?

Consider the generic Maximin model

Wald's Maximin Model z*:= max min g(x,s) x∈X s∈S(x) Here the uncertainty space is associated with the inner (min) player and is denoted by S(x), where x denotes the decision selected by the outer (max) player. So, as it is there for all to see, the uncertainty space of the inner (min) player is not fixed at all: it is allowed to depend on the choice made by the outer (max) player. But what is more, nothing here "requires the choice of a specific set".

As a matter of fact, the uncertainty sets (S(x): x∈ X) are extremely flexible, certainly far more so than the uncertainty sets (U(α,û): α>0) deployed by Info-Gap decision theory. In particular, the sets (S(x): x∈ X) are not required to have any specific structure at all thus giving the modeller great freedom in their formulation. In contrast, the sets (U(α,û): α>0) deployed by Info-Gap decision theory are required to be

nested(increasing with α).Of course, this is precisely what makes Info-Gap's robustness and opportunessness models

local, hence utterly unsuitable for decision-making undersevereuncertainty. Thenestingproperty of the uncertainty regions (U(α,û): α>0) prevents Info-Gap’s robustness and opportunessess models from exploring globally the complete uncertainty space.More importantly, perhaps, Info-Gap decision theory assumes that the uncertainty regions [U(α,û): α>0] are

notdependent on the decisions made by the decision-maker. In contrast, the Maximin model allows the sets [S(x): x∈ X] to depend on the decisions made by the decision maker.To see how these constricting requirements of Info-Gap's uncertainty model greatly limit its capabilities, observe that the Maximin formulation of Info-Gap's uncertainty model is based on the following setup

x = (d,α)

S(d,α) = U(α,û)

recalling that the sets [U(α,û): α≥0] are nested (increasing with α).

The point to note here is that in this setting, the flexibility allowed by the Maximin model cannot be (indeed, is not) fully utilized in that S(d,α) does not depend on d, and furthermore, the sets (S(d,α): α≥0) are

nested(increasing with α).By the same token, the objective function of the Maximin model is much more general than the objective function of Info-Gap's robustness model. That is, the Maximin model allows its objective function g to be any real-valued function of x and s, whereas in the framework of Info-Gap's robustness model the objective is to maximize the value of α. Thus, as indicated above, the Maximin setup for Info-Gap's robustness model is

x = (d,α); s=u

g(d,α,u) = α·(r(u,d) ◊ r*)observing that g(d,α,u) is equal to either α or 0.

In sum, the Maximin paradigm provides a modeling medium that is incomparably more general and powerful than the one offered by Info-Gap's robustness model. Is it surprising then that Info-Gap's robustness model is a simple instance of Wald's generic Maximin model?

## Conclusions

For the benefit of readers who are not familiar with Info-Gap decision theory I need to point out that the dissertation in this paper, on the alleged differences and similarities between Info-Gap’s robustness model and the Maximin model, must be placed in its proper context. Observe then that the discussion in this paper comes in the wake of a long list of grossly misleading and unsubstantiated assertions on Info-Gap's role and place in decision theory and its capabilities to deal with severe uncertainty.

For instance, consider this (empahsis is mine):

Info-gap decision theory isandradically differentfromallcurrent theories of decision under uncertainty. The difference originates in the modeling of uncertainty as an information gap rather than as a probability. The need for info-gap modeling and management of uncertainty arises in dealing with severe lack of information and highly unstructured uncertainty.Ben-Haim (2006, p. xii)

In this book we concentrate on the fairlyandnewconcept of information-gap uncertainty, whosedifferencesfrom more classical approaches to uncertainty arereal and deep. Despite the power of classical decision theories, in many areas such as engineering, economics, management, medicine and public policy, a need has arisen for adifferentformat for decisions based onseverely uncertainevidence.Ben-Haim (2006, p. 11)

Optimization of the robustness in eq. (3.172) is emphaticallynota worst-case analysis. In classical worst-case min-max analysis the decision maker minimizes the impact of the maximally damaging case. But an info-gap model of uncertainty is an unbounded family of nested sets: U(α,û), for all α≥0. Consequently, there usually is no worst case: any adverse occurrence is less damaging than some other more extreme event occurring at a larger value of α. What eqs. (3.169) and (3.172) express is the greatest level of uncertainty consistent with satisficing to level r_{c}. When the decision maker chooses the action q to maximize α(q,r_{c}), what is maximized is theimmunity to an unbounded ambient uncertainty.The closest this comes to "min-maxing"is that the action is chosen so that "bad" events (causing reward R_{*}less than r_{c}) occur as "far away" as possible (beyond a maximized value of α).

Ben-Haim (2006, p. 101 )Or consider the opening paragraph of a paper posted on the web site FloodRiskNet in the UK since November 2007:

Making Responsible Decisions (When it Seems that You Can't)

Engineering Design and Strategic Planning Under Severe UncertaintyWhat happens when the uncertainties facing a decision maker are so severe that the assumptions in conventional methods based on probabilistic decision analysis are untenable? Jim Hall and Yakov Ben-Haim describe how the challenges of really severe uncertainties in domains as diverse as climate change, protection against terrorism and financial markets are stimulating the development of quantified theories of robust decision making.

These assertions can easily give the impression that Jim Hall and Yakov Ben-Haim have made a colossal breakthrough in decision theory. Indeed that they have managed to devise a methodology for responsible decision-making in the face of severe uncertainty that is capable of the most incredible feats.

The truth of course is summed up by the following facts:

Fact 1: Info-Gap's robustness model is a simple instance of Wald's famous Maximin model (circa 1940).

Fact 2: Info-Gap decision theory is a classic example of a voodoo decision theory.The paper under review here is further testimony to the fact that Ben-Haim and his followers have no qualms to continue promulgating the same old myths about Info-Gap decision theory. Their method for dealing with the challenges exposing these myths for what they are is to ... intensify the fog, spin, and rhetoric.

## Remarks

For the record, I should point out that Ben-Haim is fully aware of the existence of the Maximin Theorem, what is more, that he does not dispute its validity. The same is true about the Invariance Theorem.

Regrettably, Ben-Haim has chosen to waltz around these theorems by means of spurious explanations whose only effect is to extend already existing errors.

It will be interesting to see how long will Ben-Haim and his followers pursue this deliberate strategy of avoiding to deal with theorems that invalidate Ben-Haim's repeated pronouncements on Info-Gap's unique role and place in decision theory.

Ben-Haim is also fully aware of the very rich literature on Robust Optimization and the kinship between Info-Gap's robustness model and run-of-the-mill robust optimization models. Yet, in the list of 56 references cited in the paper under review not one reference is made to this body of knowledge.

The good news is that there are signs that, in response to my criticism of Info-Gap decision theory, some of Info-Gap's adherents begin to question the validity of Ben-Haim's judgments on Info-Gap decision theory. For instance, as indicated in Review 11, Bryan Beresford-Smith and Colin J. Thompson (2009) now concede that Info-Gap's robustness model is a Maximin model.

## Other Reviews

- Ben-Haim (2001, 2006): Info-Gap Decision Theory: decisions under severe uncertainty.

- Regan et al (2005): Robust decision-making under severe uncertainty for conservation management.

- Moilanen et al (2006): Planning for robust reserve networks using uncertainty analysis.

- Burgman (2008): Shakespeare, Wald and decision making under severe uncertainty.

- Ben-Haim and Demertzis (2008): Confidence in monetary policy.

- Hall and Harvey (2009): Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.

- Ben-Haim (2009): Info-gap forecasting and the advantage of sub-optimal models.

- Yokomizo et al (2009): Managing the impact of invasive species: the value of knowing the density-impact curve.

- Davidovitch et al (2009): Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island.

- Ben-Haim et al (2009): Do we know how to set decision thresholds for diabetes?

- Beresford and Thompson (2009): An info-gap approach to managing portfolios of assets with uncertain returns

- Ben-Haim, Dacso, Carrasco, and Rajan (2009): Heterogeneous uncertainties in cholesterol management

- Rout, Thompson, and McCarthy (2009): Robust decisions for declaring eradication of invasive species

- Ben-Haim (2010): Info-Gap Economics: An Operational Introduction

- Hine and Hall (2010): Information gap analysis of flood model uncertainties and regional frequency analysis

- Ben-Haim (2010): Interpreting Null Results from Measurements with Uncertain Correlations: An Info-Gap Approach

- Wintle et al. (2010): Allocating monitoring effort in the face of unknown unknowns

- Moffitt et al. (2010): Securing the Border from Invasives: Robust Inspections under Severe Uncertainty

- Yemshanov et al. (2010): Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest

- Davidovitch and Ben-Haim (2010): Robust satisficing voting: why are uncertain voters biased towards sincerity?

- Schwartz et al. (2010): What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making

- Arkadeb Ghosal et al. (2010): Computing Robustness of FlexRay Schedules to Uncertainties in Design Parameters

- Hemez et al. (2002): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

- Hemez et al. (2003): Applying information-gap reasoning to the predictive accuracy assessment of transient dynamics simulations

- Hemez, F.M. and Ben-Haim, Y. (2004): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

- Ben-Haim, Y. (2007): Frequently asked questions about info-gap decision theory

- Sprenger, J. (2011): The Precautionary Approach and the Role of Scientists in Environmental Decision-Making

- Sprenger, J. (2011): Precaution with the Precautionary Principle: How does it help in making decisions

- Hall et al. (2011): Robust climate policies under uncertainty: A comparison of Info-Gap and RDM methods

- Ben-Haim and Cogan (2011) : Linear bounds on an uncertain non-linear oscillator: an info-gap approach

- Van der Burg and Tyre (2011) : Integrating info-gap decision theory with robust population management: a case study using the Mountain Plover

- Hildebrandt and Knoke (2011) : Investment decisions under uncertainty --- A methodological review on forest science studies.

- Wintle et al. (2011) : Ecological-economic optimization of biodiversity conservation under climate change.

- Ranger et al. (2011) : Adaptation in the UK: a decision-making process.

## Recent Articles, Working Papers, Notes

Also, see my complete list of articles

Moshe's new book! - Sniedovich, M. (2012) Fooled by local robustness,
Risk Analysis, in press.

- Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty,
International Transactions in Operational Research, in press.

- Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis,
Applications of Mathematics,56(5), 499-509.

- Sniedovich, M. (2011) Dynamic programming: introductory concepts, in
Wiley Encyclopedia of Operations Research and Management Science(EORMS), Wiley.

- Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem,
OR Spectrum, 33(4), 815-929, 2011.

- Sniedovich, M. (2011)
Dynamic Programming: Foundations and Principles, Second Edition,Taylor & Francis.

- Sniedovich, M. (2010) A bird's view of Info-Gap decision theory,
Journal of Risk Finance, 11(3), 268-283.

- Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.

- Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds),
.Safety, Reliability and Risk Analysis: Theory, Methods and Applications,pp. 2071-2079, Taylor and Francis Group, London.

- Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)

- Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.),
11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation,DIPTEM, Genova, 89-94.

- Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)

- Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)

- Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08

This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )

- Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory,
Decision Point,24, 10.

- Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty,
Decision Point,22, 8-9.

- Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!,
Journal of Risk Finance,9(3), 287-291.

- Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)

In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.

- Sniedovich. M. (2008) The Mighty Maximin! (PDF File)

This paper is dedicated to the modeling aspects of Maximin and robust optimization.

- Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty,
Decision Making in Manufacturing and Services,1-2, 111-136. (PDF File) .

- Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)

In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.

- My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)
This is an exciting development!

- Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.

- Ben-Haim is not familiar with the fundamental concept
point estimate. He does not realize that a function can be a point estimate of another function.So when you read my papers make sure that you do not misinterpret the notion

point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!

- A formal proof that
Info-GapisWald's Maximin Principlein disguise. (December 31, 2006)

This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)

It shows thatInfo-Gapis not a new theory but rather a simple instance ofWald's famous Maximin Principledating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context ofGame Theory(1928).

- A proof that
Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)

This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .

It shows that becauseInfo-Gapdeploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.

- A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)

This is a very short article entitled The GAP in Info-Gap (PDF File) .

It is a math-free version of the paper above. Read it if you are allergic to math.

- A long essay entitled
What's Wrong with Info-Gap? An Operations Research Perspective(PDF File) (December 31, 2006).

This is a paper that I presented at theASOR Recent Advances in Operations Research(PDF File) mini-conference (December 1, 2006, Melbourne, Australia).## Recent Lectures, Seminars, Presentations

If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.

Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.

- ASOR Recent Advances, 2011, Melbourne, Australia, November 16 2011. Presentation: The Power of the (peer-reviewed) Word. (PDF file).

- Alex Rubinov Memorial Lecture The Art, Science, and Joy of (mathematical) Decision-Making, November 7, 2011, The University of Ballarat. (PDF file).

- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Invited tutorial, ALIO/INFORMS Conference, Buenos Aires, Argentina, July 6-9, 2010).

- A Critique of Info-Gap Decision theory: From Voodoo Decision-Making to Voodoo Economics(PDF File) .

(Recent Advances in OR, RMIT, Melbourne, Australia, November 25, 2009)

- Robust decision-making in the face of severe uncertainty(PDF File) .

(GRIPS, Tokyo, Japan, October 16, 2009)

- Decision-making in the face of severe uncertainty(PDF File) .

(KORDS'09 Conference, Vilnius, Lithuania, September 30 -- OCtober 3, 2009)

- Modeling robustness against severe uncertainty (PDF File) .

(SOR'09 Conference, Nova Gorica, Slovenia, September 23-25, 2009)

- How do you recognize a Voodoo decision theory?(PDF File) .

(School of Mathematical and Geospatial Sciences, RMIT, June 26, 2009).

- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, Info-Gaps, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Department of Econometrics and Business Statistics, Monash University, May 8, 2009).

- The Rise and Rise of Voodoo Decision Theory.

ASOR Recent Advances, Deakin University, November 26, 2008. This presentation was based on the pages on my website (voodoo.moshe-online.com).

- Responsible Decision-Making in the face of Severe Uncertainty (PDF File) .

(Singapore Management University, Singapore, September 29, 2008)

- A Critique of Info-Gap's Robustness Model (PDF File) .

(ESREL/SRA 2008 Conference, Valencia, Spain, September 22-25, 2008)

- Robust Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Technion, Haifa, Israel, September 15, 2008)

- The Art and Science of Robust Decision-Making (PDF File) .

(AIRO 2008 Conference, Ischia, Italy, September 8-11, 2008 )

- The Fundamental Flaws in Info-Gap Decision Theory (PDF File) .

(CSIRO, Canberra, July 9, 2008 )

- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .

(OR Conference, ADFA, Canberra, July 7-8, 2008 )

- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .

(University of Sydney Seminar, May 16, 2008 )

- Decision-Making Under Severe Uncertainty: An Australian, Operational Research Perspective (PDF File) .

(ASOR National Conference, Melbourne, December 3-5, 2007 )

- A Critique of Info-Gap (PDF File) .

(SRA 2007 Conference, Hobart, August 20, 2007)

- What exactly is wrong with Info-Gap? A Decision Theoretic Perspective (PDF File) .

(MS Colloquium, University of Melbourne, August 1, 2007)

- A Formal Look at Info-Gap Theory (PDF File) .

(ORSUM Seminar , University of Melbourne, May 21, 2007)

The Art and Science of Decision-Making Under Severe Uncertainty(PDF File) .

(ACERA seminar, University of Melbourne, May 4, 2007)

- What exactly is Info-Gap? An OR perspective. (PDF File)

ASOR Recent Advances in Operations Researchmini-conference (December 1, 2006, Melbourne, Australia).

Disclaimer:This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.