Reviews of publications on Info-Gap decision theory

Review # 21(Posted: October 10, 2010)

Reference: Barry Schwartz, Yakov Ben-Haim, and Cliff Dacso

What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making

The Journal for the Theory of Social Behaviour,in press, 2010.Abstract Most decisions in life involve ambiguity, where probabilities can not be meaningfully specified, as much as they involve probabilistic uncertainty. In such conditions, the aspiration to utility maximization may be self-deceptive. We propose "robust satisficing" as an alternative to utility maximizing as the normative standard for rational decision making in such circumstances. Instead of seeking to maximize the expected value, or utility, of a decision outcome, robust satisficing aims to maximize the robustness to uncertainty of a satisfactory outcome. That is, robust satisficing asks, "what is a 'good enough' outcome," and then seeks the option that will produce such an outcome under the widest set of circumstances. We explore the conditions under which robust satisficing is a more appropriate norm for decision making than utility maximizing. Scores TUIGF:100%

SNHNSNDN:3786%

GIGO:100%

RemarkThis review is based on a pre-print of the article downloaded from

http://www.technion.ac.il/yakov/aaa-schwartz.doc on September 26, 2010.

Once it is published, I shall revisit it to see whether this review requires any modification.

## Overview

This is an extremely interesting article.

It's main interest lies of course in the proposition to substitute "utility maximization" with Info-gap's "robust satisficing". But, it is interesting for other reasons as well, foremost of which is its rhetoric. The authors have manage to raise info-gap's customary rhetoric -- which is quite significant in its own right -- to new and unprecedented heights, by bolstering it with an extra doze of rhetoric that is taken from the long-running but pointless indeed counter-productive debate on the topic of: satisficing vs optimizing .

The article's main proposition is then to use info-gap decision theory as the foundation for a new normative standard for decision making under severe uncertainty that is centered on info-gap decision theory's so called

robust satisficingfeature.As I have already debunked info-gap decision theory, I could have dismissed the proposed new normative theory simply on the grounds that it is based on a debunked theory.

I could have, but I shall not make do with this argument.

Because, it is important to bring to the readers' attention that in addition to the "old" errors and misconceptions that are already entrenched in the info-gap literature, this article puts forward new errors and misconceptions that must be addressed and debunked as well.

For example, this article puts forth what, to the best of my knowledge, is a new myth about info-gap decision theory, which asserts the following:

New Myth

Info-gap decision theory selects the decision that generates thewidest rangeof satisfactory outcomes.So, before we get started, it is important first of all to set the record straight on this claim:

Old Fact

Info-gap's robustness model is a model oflocalrobustness. Therefore, by definition, itdoes not, much less is it able to,seek decisions with the widest range of satisfactory outcomes.Allit can do is seek decisions that are robust againstsmall perturbationsin a given nominal value of the parameter of interest.The immediate implication of this fact is of course that the authors are basing their purported new theory on a capability that info-gap decision theory

by definition does not possess. This is sufficient grounds to dismiss the purported new theory out of hand.However, for the sake of argument, suppose that the authors would counter as follows:

Granting that info-gap's robustness model does not select a decision with the widest range of satisfactory outcomes, as indeed required by the proposed new theory,

still, ...

this does not undermine the proposed new

Robust Satisficingstandard itself, because the standard can be backed up by some other model that does select a decision with the wideset range of satisfactory outcomes.To which I say:

In this case, the proposed "new"

Robust Satisficingstandard is utterlyredundant. Because, we already have at our disposal a methodology forrobust decision making in the face of severe uncertaintyproposing a model thatdoes indeed dowhat info-gap's robustness model is incapable of doing. Namely, it selects a decision with the widest range of satisfactory outcomes. This methodology has been available at least since 1972!!!The question is then: what is the point/merit of the proposed "new" standard, especially in view of it being based on a fundamentally flawed theory?

As you will see, it will take a relatively lengthy argument to make this obvious point clear ...

My intention is to come down hard on the info-gap foundation of the proposed "new" standard. But, I shall also have to address the "satisficing vs optimizing" facet of the proposed standard. This, however, I shall do very briefly, indeed reluctantly, because as my brief comments on this question will attest, I consider the continuing discussion on this question pointless.

So, given the rather lengthy commentary that follows this introduction, it is important that the readers have a clear picture of the authors' thesis, and the arguments that rebut them.

## The Debunkee

The authors propose a normative standard that they call

Robust Satisficing, to guide decision-making in the face of radical uncertainty. According to the authors,

- The proposed Robust Satisficing approach to radical uncertainty is
new.

- It furnishes a standard for identifying decisions that
maximize the range of satisfactory outcomes.

Minimax is a kind of cousinto the proposed Robust Satisficing approach.## The Debunker

To refute the purported new approach it is sufficient to show that

- The proposed "Robust Satisficing" approach to radical uncertainty is
notnew. This type of approach goes back to at least the late 1960s, early 1970s.

- The model that is at the center of the proposed approach,
does not indeed cannotpursue decisions that maximize the range of satisfactory outcomes.Allit can do is seek decisions that are robust in theneighborhood of a poor estimateof the true value of the parameter of interest.

- The Maximin model is
nota cousin of the proposed Robust Satisficing model. It is in fact its Grandmother (circa 1940) -- the Mother being the Radius of Stability model (circa 1960).For the benefit of readers who are not at home with the topic of "decision-making under severe uncertainty", the list of facts that pulls the rug out from under the proposed approach is as follows:

- The idea to regard the best decision (under conditions of severe uncertainty) as that which maximizes the range of satisfactory outcomes is
notnew, it had been proposed a long time ago (e.g. Rosenhead et al. 1972).

- The proposed robustness model is a simple version of a model known universally as Radius of Stability (circa 1960).

- The proposed robustness model is a simple instance of Wald's famous Maximin model (circa 1940).

- The proposed robustness model is
notdesigned to seek out decisions that maximizes the range of satisfactory outcomes.

- The proposed robustness model is designed to handle
small perturbationsin a nominal value of the parameter of interest.

- The proposed robustness model is therefore
the wrong modelfor decision making in the face ofradicaluncertainty.

If you are familiar with my critique of info-gap decision theory you may perhaps assume that the ensuing review will be no more than a repetition of what I have been arguing all along ...

I therefore suggest that you continue reading because you are likely to learn more about info-gap decision theory's potential for leading astray ...

Still, for the benefit of readers who do not intend to read the remaining sections of the review, here is a summary of the main points of the discussion.

- The proposition to use
robust-satisficingas a "new" normative standard to guide sound decision-making in the face of radical uncertainty, in lieu ofutility maximizationwhich, the authors consider to be the prevailing approach in this area, suggest that the authors are unaware of the progress that has been made, over the past 40 years, in the study of "decision-making".

- The authors' oblivion to the state of the art in this area is evidenced in the absence of all reference to the thriving area of Robust Optimization or to the area of
Decisions with Multiple Objectivesor to that ofMultiple Criteria Decision-Makingor toPareto Optimizationwhich, it is important to note, bears directly on their proposition.

- This total oblivion to the state of the art puts the authors squarely on the path to reinventing the wheel, and a square one at that.

- Because, as anyone even mildly conversant with robust decision-making under severe uncertainty can immediately see, the article's high aspirations to provide no less than a new "normative standard for rational decision making" under conditions of "radical uncertainty", in fact take us back to models of the early 1970s, namely to models predating new developments in the areas of robust decision-making and multiple objective decision-making.

- The authors' approach to dealing with radical uncertainty is essentially in line with info-gap decision theory's prescription for managing radical uncertainty, which -- as I have been arguing -- is in contravention of the Garbage In -- Garbage Out (GIGO) and similar maxims, and which therefore places it squarely in the voodoo theories camp.

- The authors thus continue to perpetuate the myth (entrenched in the info-gap literature) that info-gap's robustness model is particularly suitable for handling situations subject to severe (radical) uncertainty where the uncertainty space is vast, the estimate is poor, and the uncertainty model is likelihood-free.

- The authors also continue to misrepresent the facts about the true relation between info-gap's robustness model and the Maximin model.
These statements are explained and corroborated in the Appendix. For a quick check, simply go to the short mobile debunker of info-gap decision theory.

## Search/Selection Committee

Reading the article under review, the question that immediately springs to mind is this:

Let us assume that it is indeed a good idea to challenge the hegemony of what the authors claim is the prevailing approach to dealing with severe uncertainty, namely "utility maximization", and to replace it with a new "maximize the robustness of a given satisfaction level" approach.

I hasten to add that I am neither for or against, or whatever. I merely assume, for the purposes of this discussion, that this proposition is on the agenda.

In this case, assume next that we establish a

Search/Selection Committeeto find the most suitable robust satisficing theory to do the job.Because, why settle for less?

To do this, the Committee would surely have to read the literature on robust decision-making in the face of radical uncertainty so as to be in a position to select the most appropriate robustness theory for dealing with radical uncertainty.

It would also have to read the literature on multiple objective decision-making so as to be able to select the most appropriate theory capable of dealing with a problem of two conflicting objectives:

robustnessvssatisfaction level.Because, why settle for less?

In particular, why should it adopt a debunked theory for this purpose?!?!

Clearly, what I am driving at is that as the article under review was published in a peer-reviewed journal, it must be treated as a scholarly work. So, the main issues that it should have addressed are these:

- What
robust-satisficingmethods are available for the management of radical uncertainty?

- What
multiple objective decision-makingmethods are available for handling problems of conflicts between two objectives?

- What are the capabilities and limitations of these methods?

- What method(s) is(are) most suitable for the objectives stipulated in the article?
This is what one would have expected from a scholarly analysis of the authors' basic proposition, and this is what the paper's referees should have demanded that the authors do.

But this is not what the article does.

What is particularly striking about the article is its total disregard of the state of the art in robust decision-making, as evidenced in its grossly misleading, uncritical, assessment of info-gap decision theory as the "selected" method.

So, obviously, my review will have to deal with a host of methodological and technical issues. I plan to do this in stages, and I expect that the complete review will be ready by the end of the year (2010).

Stay tuned ...!

## Where is Robustness Optimization?

The main tenor of this article article is that info-gap decision theory is the only available theory of robust decision-making in the face of uncertainty. I discuss this issue in detail in the section entitled State of the art.

At this preliminary stage, I just want to point out that robust decision-making in the face of severe uncertainty is a well-established area of research and application. Particularly relevant to this article is the the thriving field of Robust Optimization which is concerned with precisely the type of robustness issues that are discussed in the article.

Yet, not one single reference is made in the article to this fact!

As I explain in the section State of the art, this is not an isolated case. The info-gap literature systematically, and one suspects deliberately, avoids any mention of the area of Robust Optimization.

So:

For the benefit of readers who are not at home with the topic of optimization, notably with that of Robust optimization, it is important to point out the following.

The authors' proposition to substitute "utility maximization" with "robust satisficing" not only takes no account of developments in the field of Robust Optimization that bear directly on this proposition. It is based in fact on a profound misrepresentation of the whole concept of optimization, hence of what an optimization problem is all about. Because the othors’ underlying proposition is based on the gospel of "satisficing is better than optimizing", their basic position is then the following. "Optimization" only seeks to maximize an objective -- which is not really all that good -- whereas, satisficing makes do with less than maximizing ... but in the end accomplishes more: the satisfaction of a specified outcome which is a much better thing.

Note then that an optimization problem poses two tasks:

- The satisficing of constraints (requirements)

- Achieving the best outcome stipulated by an objective function.
For example, you may want to find the most reliable electronic device for a given job subject to budgetary and operational constraints. Or you may wish to find the most robust system that satisfices certain performance requirements.

The bottom line is that satisficing is an integral part of "optimizing" and therefore "robust-satisficing" is a special case of "robust optimizing".

Needless to say, the treatment of severe uncertainty is an issue of prime importance in Robust Optimization, because, in this context, robustness is sought against uncertainty in the true value of a parameter of the problem under consideration.

All this can be summed up as follows: a robust solution to an optimization problem is a solution that performs well (according to pre-determined performance criteria) with respect to the objective function and/or the constraints of the problem over a range of values of the parameter of interest.

Now back to the article under review.

The robustness considered in the article is described as follows (page 1):That is, robust satisficing asks, "what is a 'good enough' outcome," and then seeks the option that will produce such an outcome under the widest set of circumstances.The radical uncertainty is manifested in the lack of knowledge as to which "circumstance" will be realized. Namely, there is a radical uncertainty regarding which element of the set of possible circumstances will be realized. We thus seek an option (decision) that is most robust against this radical uncertainty.

That said, I hasten to point out that the fundamental flaws in info-gap decision theory render it utterly unsuitable for this task. More on this can be found in the mobile debunker of info-gap decision theory and in subsequent sections of this review.

My objective is then to explain how grossly misleading the above quoted description of "robust satisficing" is, given that it is based on info-gap decision theory. That is, to explain that the proposed new normative theory does not (cannot) even handle (conceptually) trivially simple tasks.

## How do we determine what is 'good enough'?

The first point that must be made abundantly clear is that when the authors say that "robust satisficing asks, 'what is a 'good enough' outcome?", the term

"asks"does not designate a method, a procedure, guidelines, or what have you, instructing or enlightening us on how to determine what is 'good enough'. Indeed, the proposed theory does not even begin to "ask" this rudimentary question, it simply posits that we begin with a fully fledged predetermined 'good enough' outcome" in hand.But the whole point is that this is precisely where our difficulties lie. Generally, in problems depicting situations of conflicts between objectives -- the "good enough outcome" and other objectives -- an answer to the question "What is a good enough outcome?" is hardly straightforward. In other words, in general, to determine what is a 'good enough outcome' we need to know how changes in the value of the 'good enough outcome' affect the values of other objectives.

And in the case under consideration, we cannot decide what is a 'good enough outcome' without knowing the value of the associated robustness against radical uncertainty, because the higher the value of the 'good enough outcome', the lower its robustness against radical uncertainty.

So, how are we going to decide what is the appropriate balance or tradeoff between the 'good enough outcome' and robustness to radical uncertainty?

The proposed new theory does not even bother to address this fundamental issue.

Its underlying assumption is that the

valueof the 'good enough outcome' isgivenand all that is left to do is to determine the robustness of thisgivenvalue of the 'good enough outcome' to radical uncertainty.## Where is the estimate?

As the proposed "robust satisficing" normative standard is anchored in info-gap decision theory, it is important to be clear on the following point:

The centerpiece of info-gap decision theory is a robustness model and the fulcrum of this robustness model is anestimateof the true value of the parameter of interest. It is the true value of this parameter that is subject to radical uncertainty. This means that info-gap's robustness analysis revolves around thisestimate. There is no info-gap analysis without anestimatebeing at the center of this analysis.And yet, this important fact is not only not discussed in the article, it is absent from the authors' description of info-gap's robustness analysis that "... seeks the option ... " namely the (purportedly) acceptably satisfactory decision.

So to give you an idea of how the (info-gap) robustness model underlying the "robust satisficing" analysis proposed in this article should have been explained, consider the following:

Let:

- u = parameter of interest whose true value is subject to radical uncertainty.

- û = an estimate of the true value of this parameter.

- U = a set containing the true value of u and the estimate û
Note that, given the radical uncertainty, all we know about the true value of u is that it is an element of U. We shall refer to the set U as the

uncertainty spaceof the problem.Next, let:

- D = the set of decisions available to the decision maker.

- S(d) = the set of values of u that satisfy the performance requirement imposed on decision d.
Formally, S(d) is a subset of U which contains the values of u in U that are

satisfactorywith respect to decision d.Hence:

The definition of robustness according to info-gap decision theory:

The robustness of decision d is thesizeof the largest neighborhood around the estimate û, all of whose elements are satisfactory with respect to decision d.In other words, the robustness of decision d is the

sizeof the largest neighborhood around the estimate û, that is a subset of S(d).It is important to note that the "neighborhoods" around the estimate that designate the robustness of decisions are required by the axioms of info-gap decision theory to to be

nested. Think about them as "balls" centered at the estimate, so that the size of a neighborhood would be equal to theradiusof the ball representing the neighborhood.This means of course that the robustness of decision d can be stated as follows:

The robustness of decision d is theradius of the largest ballaround the estimate û that is contained in S(d).Obviously, this concept is known universally as Radius of stability (circa 1960). But, info-gap scholars are apparently unaware of this fact, insisting as they do that this is a new concept that is radically different from other measures of robustness.

Clearly then, to formulate an info-gap definition of robustness we need an

estimateof the true value of u. Without anestimateof the true value of u there is no info-gap robustness analysis. in other words, without stipulating anestimatewe cannot use info-gap decision theory.But, this 'trifle" is not even mentioned in the article.

The following example is designed then to illustrate the utter absurd that the robustness model proposed in this article gives rise to in the absence of an

estimateof the true value of u. In other words, the idea is to illustrate that, without anestimate, it impossible to use the proposed "new" normative standard even in situations where the problem is trivial.## Example

Consider the following picture where the large rectangle represents the uncertainty space U and the shaded area represents the set S(d) for some decision d.

Since S(d) covers most of U, it follows that most of the values of u in U are "satisfactory" and therefore we would have no doubts to deem decision d robust against the radical uncertainty in the true value of u. No formal analysis is required to reach this obvious conclusion.

However, as things stand, the robustness of decision d cannot be assessed/determined according to the precepts of info-gap decision theory because, as the picture illustrates, there is no point in U to represent an estimate of the true value of u.

Clearly, the radius of the largest ball contained in the shaded area would depend on the location of the center of the ball (estimate û). In other words, the value of the estimate û plays a central role in determining the robustness of decisions in info-gap decision theory.

To reiterate, given that the robustness of decision d, according to info-gap decision theory, hangs on the location of the estimate û in U, in the absence of the estimate we cannot determine the robustness of d.

This feature of info-gap's robustness model is illustrated in the next picture, where the robustness of decision d is determined for three different values of the estimate û, denoted by a, b and c.

- Case 1: û = a.

Decision d is clearly robust. The radius of the largest completely satisfactory ball is quite large relative to the size of U.

- Case 2: û = b.

Decision d clearly fragile. The radius of the largest completely satisfactory ball is relatively small compared to the size of U.

- Case 3: û = c.

Decision d is not robust at all. In fact, its robustness is equal to 0.In short, it is patently clear that according to the precepts of info-gap decision theory, the robustness of decisions is crucially sensitive to the "location" of the estimate û in the uncertainty space U.

Some readers may wonder why I go into such lengths to elaborate this obvious point. So let me explain.

What I wanted to make clear through this analysis is not only that the authors' depiction of the robustness analysis, which underlies the proposed "robust satisficing" standard, is (according to Info-gap decision theory) utterly misleading. Indeed, this analysis enables me to bring out the fundamental flaw of info-gap decision theory itself.

One of the basic characteristics of

radicaluncertainty is that the estimates (of parameters) arepoor, so much so that they can besubstantially wrong. This fact is acknowledged in the info-gap literature, where the estimate û is often called aguess.Another important point to keep in mind is that models of

radicaluncertainty arelikelihood-free. That is, there is no ground to assume that the true value of u is more/less likely to be in any one particular neighborhood of the uncertainty space U. Specifically, there is no ground to assume that the true value of u is more likely to be in the neighborhood of the estimate û rather than in the neighborhood of any other point in U.The implication is therefore clear. If we want to determine the robustness of a decision against radical uncertainty in the true value of u, there is no ground whatsoever to single out, for the robustness analysis, the neighborhood of the estimate û, or for that matter the neighborhood of any other given point in U. Hence, there is clearly no justification whatsoever to conduct the robustness analysis

only onthe neighborhood of the estimate û, or for that matter only on the neighborhood of any other given point in U.But this is precisely what info-gap decision theory does, and this is precisely what earns this theory the title voodoo decision theory.

The point is that the treatment of radical uncertainty requires models of

globalrobustness whereas info-gap decision theory treats radical uncertainty by means of a model oflocalrobustness.## Fooled by robustness

Although the term "robustness" clearly has a familiar ring to it, it may have different meanings in different contexts. Hence, it is of the utmost importance to be clear on the definition of robustness that one uses in a specific context.

The authors do not provide a formal definition of the robustness that they have in mind, but they do explain what they mean by "robust satisficing" in the context of various examples.

Alas, these descriptions are inconsistent

For example, compare these three cases:

Case Page Description in the article A 1 That is, robust satisficing asks, "what is a 'good enough' outcome," and then seeks the option that will produce such an outcome under the widest set of circumstances. B 9-10 The robust satisficer answers two questions: first, what will be a "good enough" or satisfactory outcome; and second, of the options that will produce a good enough outcome, which one will do so under the widest range of possible future states of the world. C 19 A robust satisficing decision (perhaps about pollution abatement) is one whose outcome is acceptable for the widest range of possible errors in the best estimate. Note that in Case C there is a reference to something called

best estimatewhich is conspicuously missing from Case A and Case B. Moreover, the authors do not state what thisbest estimateis. In other words, an estimate of what is it?In fact, as indicated in the preceding section, the authors keep mum on the essential fact that an

estimateof the unknown true value of the parameter of interest is the center of info-gap's robust-satisficing model; the true value of this parameter being subject to radical uncertainty.Hence, the descriptions of Case A and Case B are misleading because they are incompatible with the (info-gap) robustness model that underlies the proposed theory. This means of course -- as indicated in the preceding section -- that

the proposed new normative theory for decision-making under radical uncertainty cannot handle even a simple problem such this.## Example

Consider a case with two decisions, d' and d'' where the uncertainty space U is a rectangle, as shown below. First, suppose that there is no estimate for the true value of u.

Decision d'In the picture, the large rectangle U represents the

uncertainty spaceassociated with the problem. This is the set of all the possible values of the parameter of interest whose true value is subject to radical uncertainty. The shaded area represents values of this parameter for which decision d' satisfies the performance requirement.Since most of the area of U is shaded, decision d' is decidedly robust against the radical uncertainty in the true value of the parameter of interest.

Decision d''In the picture, the large rectangle U represents the

uncertainty spaceassociated with the problem. This is the set of all the possible values of the parameter of interest whose true value is subject to radical uncertainty. The shaded area represents values of the parameter for which decision d'' satisfies the performance requirement.Since most of the area of U is

not shaded, decision d''cannotbe deemed robust against the radical uncertainty in the true value of the parameter of interest.

Question: Which of the two decisions (d', d'') is more robust against the radical uncertainty in the true value of the parameter of interest? Answer: Sorry, Mate!To answer this question info-gap decision theory requires an estimate of the true value of u.

Info-gap decision theory cannot answer this simple question.Suppose then that in view of this situation you turn to a consulting company,

Estimates Galore, instructing them to work out an estimate of the true value of u. So, $100,000 later you receive a report fromEstimates Galoreoffering 6 different estimates, call them û^{(1)}, û^{(1)}, ..., û^{(6)}, obtained from 6 different established scientific methods.Equipped with these 6 estimates, you now ask the local info-gap expert to determine which decision, d' or d'' is the more robust, given these 6 estimates.

However, to your great surprise, you will be advised by the local info-gap expert that info-gap decision theory cannot handle 6 estimates. More precisely, the theory requires

exactly oneestimate. And so ... you will go right back to where you started ...In other words, it is important to appreciate that because the new normative theory, proposed in the article, is rooted in info-gap decision theory, it necessarily suffers from all the ills afflicting info-gap decision theory. So, although the authors keep mum on the centrality of the

estimate, note that info-gap's robustness modelconducts the robustness analysis in the neighborhood of a point estimate of the true value of the parameter of interest. Thus, the robustness of decision d is thesize of the largest neighborhood around the point estimateall of whose elements are satisfactory with respect to decision d.To examine then how the proposed new theory would address the above case, let us modify it a bit and assume that, for a bargain of $20,000 extra,

Estimates Galoreprovided you with a single point estimate, û, of the true value of the parameter of interest. In the pictures below, this estimate is represented by a black dot • in the rectangle representing the uncertainty space U associated with the problem.

The large rectangle U represents in the pictures the

Decision d'uncertainty spaceassociated with the problem, and thepoint estimateis represented by the dark black dot.

The largest neighborhood around the estimate, all of whose elements satisfy the performance constraint imposed on decision d', is represented by the blue circle. The length of the radius of this circle is the robustness of decision d'.

Decision d''The large rectangle U represents in picture the

uncertainty spaceassociated with the problem, and thepoint estimateis represented by the dark black dot.The largest neighborhood around the estimate whose elements satisfy the performance constraint imposed on decision d'' is represented by the blue circle. The length of the radius of this circle is the robustness of decision d''.

Question: Which of the two decisions (d', d'') is more robust against the radical uncertainty in the true value of the parameter of interest? Answer: According to the proposed new theory, decision d'' is much more robust than d' against the radical uncertainty in the true value of the parameter of interest. The bottom line:

The proposed new normative theory does not (cannot) seek decisions that are most robust against the radical uncertainty in the true value of the parameter of interest.It can only seek decisions that are most robust against

small perturbationsin the value of the point estimate of the true value of the parameter of interest.The following simple example illustrates how absurd the proposed new theory is.

## Example

To see why the proposed theory is utterly unsuitable for the treatment of radical uncertainty, consider the case where the uncertainty space is the

real line, namely the true value of the parameter can be any real number, so U = (−∞,∞).Now suppose that there are two decisions (options), say d' and d'', and that the estimate of the true value of u is equal to 0. Also, assume that the ranges of satisfactory values of the parameter u is the interval I' = [-1,1] for decision d' and the union of the intervals (−∞,-1], [-0.9,0.9] and [1,∞] for d'', namely let I''= (−∞,-1] ∪ [-0.9,0.9] ∪ [1,∞).

According to the proposed new theory, the winner is ... decision d'.

To appreciate the absurd in this verdict, consider the picture depicting the situation:

In this case the simple facts are as follows:

- Decision d'' performs satisfactorily over the entire unbounded uncertainty space U, except on the minute intervals [-1,-0.9] and [0.9,1].

- Decision d' performs satisfactorily only over the minute interval I'=[-1,1].

- Decision d'' performs better than decision q' over the entire uncertainty space, except on the minute intervals [-1,-0.9] and [0.9,1].

- Yet, according to the proposed theory, d' is more robust than d'' against the radical uncertainty in the true value of the parameter of interest.
No amount of rhetoric can explain away the absurd of this evaluation.

## Local vs Global robustness

It is important to keep in mind that conditions of

radical uncertaintymean that the estimate is poor and the uncertainty model is likelihood-free. This means that there are no grounds to assume that the true value of the parameter is in the neighborhood of the estimate, hence the robustness that must be sought must beglobalrather thanlocal. The implication therefore is that info-gap's robustness model is thewrongtool for the job. By definition, info-gap's robustness model seeks decisions that are robust in the locale of a given poor estimate. But, to deal properly with radical uncertainty, a robustness model must be able to seek decisions that are robust globally, namely in relation to the entire uncertainty space under consideration.The authors apparently are unaware of the distinction between local and global robustness, because they erroneously treat info-gap's robustness model as a model of global robustness. This explains their strong (but grossly erroneous) conviction that info-gap's robustness model generates decisions whose performance is satisfactory under the

widest rangeof circumstances.As shown in the preceding examples, there is

no assurancethat the proposed new theory will select a decision whose performance is satisfactory under thewidest rangeof circumstances. It therefore cannot be counted on to select a decision whose performance is satisfactory under thewidest rangeof circumstances.For instance, consider again the following case:

Clearly, the

widest rangeof satisfactory outcomes is provided by the decision associated with I''. But, according to the proposed new theory, the most robust decision is the one associated with I'.Yet, the authors repeatedly claim that this is so (emphasis is mine):

On Page 1 we read:That is, robust satisficing asks, "what is a ‘good enough’ outcome," and then seeks the option that will produce such an outcome under theOn page Page 9-10 we read :widest setof circumstances.The robust satisficer answers two questions: first, what will be a “good enough” or satisfactory outcome; and second, of the options that will produce a good enough outcome, which one will do so under theOn page 19 we read:widest rangeof possible future states of the world.A robust satisficing decision (perhaps about pollution abatement) is one whose outcome is acceptable for theOn page 21 we read:widest rangeof possible errors in the best estimate. No probability is presumed or employed.4. I want $1 million when I retire. What investment strategy will get me that million under theOn page 21 we read:widest rangeof conditions?But an investment strategy that aims to get you a million dollars under theOn page 22 we read:widest setof circumstances is likely to be very different from one that aims to maximize the current estimate of future return on investment.A business operating with an eye toward robust satisficing asks not, “How can we maximize return on investment in the coming year?” It asks, instead, “What kind of return do we want in the coming year, say, in order to compare favorably with the competition? And what strategy will get us that return under theOn page 27 we read:widest arrayof circumstances?”For an individual who recognizes the costliness of decision making, and who identifies adequate (as opposed to extreme) gains that must be attained, a satisficing approach will achieve those gains for thewidest rangeof contingencies.These claims are without any foundation, they are in fact false hence, grossly misleading.

The proposed robustness model seeks decisions that maximize the

size of the set of satisfactory outcomes in the neighborhood around a poor estimateof the parameter of interest,not the widest set of satisfactory outcomes.Now, on page 25 the authors argue that:

But there are also strategic situations in which robust satisficing makes sense (see Ben-Haim & Hippel, 2002 for a discussion of the Cuban missile crisis; and Davidovitch & Ben-Haim, in press, for a discussion of the strategic decisions of voters).There are of course situations where the ranking of decisions based on local robustness is equivalent to that of global robustness. For example, consider the opening paragraph of the article entitled

Global stability of population modelsby Cull (1981, p. 47,Bulletin of Mathematical Biology,Vol. 43, pp. 47-58.):It is well known that local and global stability are not equivalent and that it is much easier to test for local stability than for global stability (see LaSalle, 1976). The point of this paper is to show that for the usual one-dimensional population models, local and global stability are equivalent.However, it is eminently clear that info-gap decision theory does not, indeed cannot -- methodologically speaking -- provide any evidence or proof in support of its generic robustness model’s capability to produce such results. Therefore, the duty to find a way out of this impasse is left to the users of this theory, who must do this on a case-by-case basis. Thus, it is not surprising that, once made aware of this situation, some info-gap scholars have attempted to provide some sort of evidence, or proof, or logic, to justify their use of info-gap's robustness model.

I discuss this in a subsection section entitled Fixing the square wheel.

From a robust optimization perspective, models of

localrobustness are regarded as unsuitable for the treatment of severe uncertainty, especially in cases where the uncertainty space is large. For example, Ben-Tal et al. (2009, European Journal of Operational Research, 199(3), 922-935) view models that restrict the robustness analysis only to the range of "normal" values of the parameter as representing a "somewhat 'irresponsible' decision maker".I go one step further and argue that the use of a Radius of Stability model for the treatment of radical uncertainty associated with unbounded regions of uncertainty amounts to ... voodoo decision-making.

To sum it all up, the authors' claims that the proposed theory seeks decisions that provide the

widest rangeof circumstances where the decisions performs satisfactorily aregroundless. These claims are therefore grosslymisleading.It would appear that the authors were taken in by their own rhetoric ...

## Reinventing a square wheel

The proposed theory is a reinvented square wheel for the same reasons that info-gap decision theory is a reinvented square wheel.

To explain.

Info-gap decision theory is claimed to be a new theory that is radically different from all current theories of decision under uncertainty. The truth of course is that info-gap's robustness model is a simple Radius of Stability model (circa 1960). Furthermore, info-gap's robustness model is also a simple instance of Wald's famous Maximin model (circa 1940), by far the most prevalent and well-known model for robust decision-making under severe uncertainty. These facts earn it the title "a reinvented wheel".

As for being a "reinvented

squarewheel".Info-gap's

misapplicationof its robustness model, namely its prescription to use its robustness model -- which, as we saw, yieldslocalrather thanglobalrobustness -- in the management of severe uncertainty, renders it a reinvented square wheel.## Fixing the square wheel

A number of (unsuccessful) attempts have been made recently by a number of info-gap scholars, presumably in view of my criticism of info-gap decision theory, to correct this fundamental flaw in info-gap's uncertainty model.

For instance, in Hall and Harvey (2009, p. 2) we read:An assumption remains that values of u become increasingly unlikely as they diverge from û.and in Hine and Hall (2010, p. 203) we read:The main assumption is that u, albeit uncertain, will to some extent be clustered around some central estimate û in the way described by U(α,û), though the size of the cluster (the horizon of uncertainty α) is unknown. In other words, there is no known or meaningfully bounded worst case. Specification of the info gap uncertainty model U(α,û) may be based upon current observations or best future projections.A more difficult to identify "quick fix" is hidden in Rout et al. (2009, p. 785):

Thus, the method challenges us to question our belief in the nominal estimate, so that we evaluate whether differences within the horizon of uncertainty are 'plausible'. Our uncertainty should not be so severe that a reasonable nominal estimate cannot be selected.Apparently, contrary to the above, the authors of the article under review continue to hold that a local robustness analysis in the neighborhood of a poor estimate is a reliable method for the treatment of radical uncertainty.

If this is indeed the case, than the authors need reminding that the basic difficulty in decision-making under radical uncertainty is precisely the fact that such an analysis is unreliable because in this environment the

estimate is poor and unreliable. And just in case, ... they should also be reminded thatresults can be only as good as the estimates on which they are based.They should also be advised that this is the reason that the Radius of Stability model is used only in the analysis of small perturbations in a nominal value of a parameter. Hence, it cannot be used to determine the robustness of decisions against radical uncertainty.

## Where is Pareto Optimization?

The decision problem considered in the article is a typical Pareto Optimization problem: negotiating between conflicting objectives. Thus, in the case of the problem studied in the article, the task is to achieve the largest robustness possible and at the same time achieve the highest possible required satisfaction level.

This concept is used extensively in multiobjective optimization.

So from this perspective, the model under consideration is a typical multiobjective optimization model.

What is particularly objectionable about the authors' depiction of this problem is that they discuss tradeoffs between robustness and satisfaction levels, referring to a (probabilistic) multi-attribute utility analysis , but they keep mum on, what is far more pertinent to and bears directly on the problem they consider, namely Multiobjective Optimization in general, and Pareto Optimization in particular.

Go figure!

## The Maximin saga ...

On page 25 the authors claim the following:Various suggestions have been proposed for making decisions in competitive strategic games. A common one is what is called the "minimax" strategy: choose the option with which you do as well as you can if the worst happens. Though you can't specify how likely it is that the worst will happen, adopting this strategy is a kind of insurance policy against total disaster. Minimax is a kind of cousin to robust satisficing, but it is not the same. First, at least sometimes, you can't even specify what the worst possible outcome can bring. In such situations, a minimax strategy is unhelpful. Second, and more important, robust satisficing is a way to manage uncertainty, not a way to manage bad outcomes. In choosing Brown over Swarthmore, you are not insuring a tolerable outcome if the worst happens. You are acting to produce a good-enough outcome if any of a large number of things happen. There are certainly situations in which minimax strategies make sense.Before I turn to a detailed examination of specific claims in this paragraph about the so called "minimax" strategy and its relation to the so called "robust satisficing" strategy (read info-gap's robustness model), it is important to call attention to the following three facts.

Fact 1:

Not a single reference is given here to the "minimax" literature to enable the readers to see for themselves whether the authors' claims about "minimax" are sound, nor to support the various claims in this paragraph regarding the relationship between the "minimax" strategy and info-gap's robustness model.

Fact 2:

There are formal, rigorous proofs that info-gap's robustness model is a very simple instance (special case) of Wald's Maximin model (circa 1940). Thus, the authors' comments on what they call a "minimax" strategy are factually wrong and misleading. The bottom line is that whenever info-gap's robustness model is capable of handling the (local) "worst case" pertaining to "robust satisficing", so can its Maximin counterpart.Fact 3:

The second author -- The Father of info-gap decision theory -- is aware of the fact that such theorems and proofs exist.That said, let us examine some of the specific claims in this paragraph, starting with this:

A common one is what is called the "minimax" strategy: choose the option with which you do as well as you can if the worst happens.But this is an inaccurate description of the "minimax" strategy. The point is that

eachoption may haveits own worst case. Here is Rawls' (1971, p. 152) famous description of the (methodologically equivalent) Maximin strategyThe maximin rule tells us to rank alternatives by their worst possible outcomes: we are to adopt the alternative the worst outcome of which is superior to the worst outcome of the others.So, each decision may have its

ownworst outcome which means that it is erroneous to talk about a (general) worst outcome.Next, consider this pearl:

Minimax is a kind of cousin to robust satisficing, but it is not the same.This claim is doubly misleading because it gives the false impression that the robust-satisficing model proposed by the authors, that is info-gap's robustness model, and Wald's Maximin model, are on a par, but "not the same". Note then that the basic facts about the relation between these two models are as follows:

- Wald's Maximin model is incomparably more general and powerful than info-gap's robustness model.

- Wald's Maximin model is the prototype and info-gap's robustness model is a simple instance of this prototype. Namely, info-gap's robustness model is just a specific case of Wald's Maximin model.

- Info-gap's robustness model is a model of
localrobustness, whereas Wald's Maximin model offers you the flexibility to employ it either as a model of local robustness or as a model of global robustness.For formal proofs see the Mobile Debunker of Info-Gap Decision Theory.

Next, consider this claim:

First, at least sometimes, you can't even specify what the worst possible outcome can bring. In such situations, a minimax strategy is unhelpful.This claim is grossly misleading because it implies that info-gap's robustness model has an advantage on the Maximin model in that its application does not require that a worst outcome exist. The relevant facts are as follows:

- Wald's Maximin model is incomparably more general and powerful than info-gap's robustness model.

- Info-gap's robustness model is just a simple instance of Wald's Maximin model.

- So in cases where the worst case issue does not impede the application of info-gap's robustness model, the worst case issue does not impede the application of that instance of the Maximin model that represents info-gap's robustness model.
In other words, the above comparison between the (generic) Maximin model and info-gap's robustness model is wrong because it compares a supposed implementation of the prototype with an implementation of an instance of the prototype. But, a valid comparison can be made

onlybetween instances of the prototype. The inference is then that in cases where existence/knowledge of the worst case is not an issue for info-gap's robustness model, then this is also not an issue forthat particular instanceof the Maximin model that represents info-gap's robustness model.Regarding the Family tree of info-gap decision theory, the relevant facts are as follows:

The basic dates are these:

- Minimax is not the same as info-gap's robust satisficing: Indeed, it is incomparably more general and powerful than info-gap's robust satisficing.
2. Minimax is not a cousin of info-gap robust satisficing: it is its Grandmother -- its Mother being the Radius of stability model (circa 1960).

- 1939: Wald's Maximin is formulated in the context of a statistical analysis.

- 1950s: Wald's Maximin model establishes itself as one of the primary modeling/analysis tools in decision theory.

- 1960s: Radius of stability (simple case of Wald's Maximin model) appears on the scene as a model of
localstability/robustness.

- 2001: Info-gap's robustness model (Simple case of Radius of stability, hence of Maximin) appears on the scene as the central ingredient of info-gap decision theory.
These facts are there for all to see, and no amount of (misleading) rhetoric can explain them away.

In fact, a number of info-gap scholars have already conceded that info-gap's robustness model is a Maximin model (for example, see Review of Beresford-Smith and Thompson (2009) article.).

And finally consider the following cryptic statement:

Second, and more important, robust satisficing is a way to manage uncertainty, not a way to manage bad outcomes.Are the authors suggesting that Wald's Maximin model is not "a way to manage" uncertainty? If so, then the authors would do well to read introductory books on decision theory. They will discover that Wald's maximin model is the foremost model offered by classical decision theory for the management of radical uncertainty.

For example, in French (1988) the maximin model is discussed in Chapter 2:

Decisions under strict uncertainty, and where the question addressed is as follows (page 36): "How should a decision maker choose a in a situation of strict uncertainty?"In Resnik (1987) the model is introduced in Chapter 2:

Decisions under ignorance, where on page 26, the statement introducing the Maximin model is the following: " ... using ordinal utility functions we can formulate a very simple rule for making decisions under ignorance. "In Ragsdale (2004):, it appears in Chapter 15:

Decision Analysis, where in section 15.6:Nonprobabiisitc methods, we read (page 760):The decision rules we will discuss can be divided into two categories: those that assume that probabilities of occurrence can be assigned to the states of nature in a decision problem (probabilistic methods), and those that do not (nonprobabilisitc methods).And how about the entry

Robust Controlby Noah Williams (Dictionary of Economics, 2008; emphasis is mine):Robust control is an approach for confrontingmodel uncertaintyin decision making, aiming at finding decision rules which perform well across a range of alternative models.This typically leads to a minimax approach, where the robust decision rule minimizes the worst-case outcome from the possible set. This article discusses the rationale for robust decisions, the background literature in control theory, and different approaches which have been used in economics, including the most prominent approach due to Hansen and Sargent.The suggestion that Wald's Maximin model is not used in decision theory to model uncertainty is ludicrous.

I should also point out that according to the two primary text on info-gap decision theory (Ben-Haim 2001, 2006), info-gap robsut-satisficing is not just about managing uncertainty. Rather it is about the pursuit of decisions that are robust against severe uncertainty. Recall, however, that as indicated above, info-gap's robustness model is utterly unsuitable for this task.

## And the irony is ....

All the authors have to do to achieve the robustness that they seem to be interested in, namely

the widest range of satisfactory outcomes, is to use the following simple Maximin model (expressed in the MP format):robust(d):= max {size(Y): Y⊆U, u∈S(d),∀u∈Y}

where size(Y) denotes the size of set Y, namely the range of values of u contained in Y. If Y is a finite set then size(Y) could be the cardinality of Y.

And if you insist on using the classic Maximin format, here it is:

where

max min f(Y,u,d) Y⊆U u∈Y f(Y,u,d):= size(Y) , if u∈S(d)

and

f(Y,u,d):= 0 , if u∉S(d)

For the record, robustness models based on "widest range of satisfactory outcomes" date back to Starr (1963, 1966), Rosenhead et al. (1972), Rosenblat (1987), and many others. The reason that they are not used widely in practice is that it is often extremely difficult to solve the optimization problem induced by these models.

## Robust satisficing vs utility maximizing

The rhetoric in this article about the supposed superiority of satisficing over optimizing is counter productive and amounts to no more than word games.

Consider, for instance what Jan Odhnoff said about this matter a long time ago:

In my opinion there is room for both 'optimizing' and 'satisficing' models in business economics. Unfortunately, the difference between 'optimizing' and 'satisficing' is often referred to as a difference in the quality of a certain choice. It is a triviality that an optimal result in an optimization can be an unsatisfactory result in a satisficing model. The best things would therefore be to avoid a general use of these two words.

Odhnoff, Jan, (1965).

On the Techniques of Optimizing and Satisficing.

The Swedish Journal of Economics,67(1), 24-39.But what is more:

It is elementary to show that any satisficing problem can be easily transformed into an equivalent optimization problem. Hence, the distinction between "satisficing" and "optimizing" is a mater of style, not of substance. What is important is to determine

whatto optimize andwhatto satisfice.For the same reason, the distinction between robust-satisficing and utility maximization is spurious. Because, if in the case you are concerned with you define utility as "robustness" and maximize this utility, then in effect you are maximizing robustness, which is precisely what robust-satisficing does.

It is therefore interesting to note that in the 2001 version of the info-gap book (Ben-Haim (2001)), the term

robust-optimizingis used to describe what the theory does. This term was changed torobust-satisficingin the 2006 edition of the book.But, word games cannot salvage the serious flaws in info-gap decision theory. The flaws are foundational and no amount of rhetoric can fix them.

## Example

To illustrate the above points, consider one of the examples mentioned in the paper (p. 21):

4. I want $1 million when I retire. What investment strategy will get me that million under thewidest rangeof conditions?First, to be able to appreciate why this problem is treated (by the authors) as a satisficing problem, let us ignore the specific robustness requirement "under the widest range of conditions", and let us concentrate on the problem’s main task.

Then, expressing this task in terms of a "saisficing problem", yields the following problem formulation:

Satisficing Problem:

Find an investment strategy x∈X such that r(x)≤$1000,000, where X denotes the set of available investment strategies and r(x) denotes the return ($) from investment x. Let X^{*}denotes the set of all the investment strategies that satisfy this requirement.Convince yourself that the following optimization problem is equivalent to this "satisficing problem":

Optimizing Problem:max {f(x): x∈X}where f is a real-valued function on X such that f(x)=r(x) if r(x)≥1000,000 and f(x)=1000,000 if r(x)>1000,000. Let X

^{o}denote the set of (optimal) solutions to this problem.Then clearly, X

^{*}=X^{o}.Now, let us go back to the robust satisficing problem under consideration, namely

4. I want $1 million when I retire. What investment strategy will get me that million under thewidest rangeof conditions?Let u denote the uncertain parameter, U denote the parameter space, and r(x,u) denote the return from investment strategy x given u. So the "condition" is r(x,u)≥1,000,000.

Therefore, the robustness problem under consideration is as follows:

Robust satisficing problem:Find an x in X such that the set {u∈U: r(x,u)≥1,000,000} is as large as possible.

Convince yourself that this problem is equivalent to the following optimization problem:

Robust optimization problem:

max {ρ(Y): r(x,u)≥1,000,000, ∀u∈Y} x∈X,Y⊆U where ρ(Y) denotes the size of set Y.

So what is so special about "robust satisficing" that cannot be (achieved) by "robust optimizing"?

Info-gap enthusiasts take note:the above formulation of the robust optimization problem is a Maximin model (MP format), but it isnotan info-gap's robustnenss model!!!!!

Exercise:Transform the above Maximin formulation of the robust optimization problem from the MP format to the Classic format.Hint: Have a look at And the irony is ... .

## Super-Alchemy

Recall that "conventional alchemy" is the pseudoscientific predecessor of chemistry that sought a method for transmuting base metals into gold.

In this vein, seeking a theory that would transform ignorance and lack of information into "robust decisions" can be regarded an exponent of

super-alchemy.For, consider the basic facts:

- The uncertainty we are dealing with is
radical:

- The uncertainty space can be
vast(even unbounded).

- The estimate we have is
poor(can be aguess, even awild guess).

- The uncertainty model is
likelihood-free.

- We conduct an analysis in the
neighborhood of the poor point-estimate.

- We
ignorethe performance of decisions in areas of the uncertainty space that are at a distance from the estimate.And yet, against all odds, and in violation of the Garbage In -- Garbage Out (GIGO) Axiom and its relations, we obtain decisions that are robust against radical uncertainty.

Doesn't this amount to

super alchemy?## Fooled by rhetoric

Decision-making in the face of radical uncertainty is a formidable task, and developing a compressive theory for this task is even more demanding.

The authors' approach to this difficult task is to raise the level of rhetoric with the aid of words games.

That is, they take a model that was developed initially in 1996 (Ben-Haim (1996)) for the purpose of handling (small) perturbations in a given nominal value of a parameter, and elevate it -- by means of rhetoric -- to the level of a model for dealing with radical uncertainty. They then take this model, which is capable only of handling neighborhoods of a nominal value of the parameter of interest, and elevate it -- by rhetoric -- to the level of a model that (purportedly) determines the widest range of satisfactory outcomes associated with a given decision.

They then elevate this model --- known as a simple instance of the most famous model in the trade --- by means of rhetoric --- to the level of a new normative panacea for decision-making in the face of radical uncertainty.

But, the rhetoric cannot conceal the following simple facts about the proposed new theory:

- The proposed robustness model is
not new. It is a simple instance of the famous Radius of Stability model (circa 1960), and therefore an instance of the even more famous Wald's Maximin model (1940).

- The proposed robustness model does
notseek decisions with the widest range of satisfactory outcomes. It seeks decisions that maximize the size of theneighborhood around a poor estimatewhose elements are satisfactory.

- The proposed robustness model is
utterly unsuitablefor the treatment ofradicaluncertainty. The more radical the uncertainty, the more unsuitable the proposed model is.

- Robustness models that maximize the range of satisfactory outcomes were proposed
years ago(circa 1970). They are rarely used in practice because the optimization problems that they induce are usually extremely difficult to solve. So their application is restricted to highly specialized situations.

- The proposed new theory does
notaddress, let alone resolve, the difficult issues pertaining to decidingwhat constitutes a satisfactory( 'good enough') outcome.An explanation of the large font :

My "public" critique of info-gap decision theory goes back to the end of 2006. But some info-gap proponents prefer to avoid a direct reply to it. They prefer to spin their way around it.

This may have to do with the fact that I normally use normal font, so an increase in the size of the font may perhaps do the trick ....

I tried

colorbefore, but it did not work ...So ..we shall have to wait and see.

## State of the art

The article exhibits a total disrespect for the state of the art in the main topics discussed in the article.

One of the important traditions in academia is that a discussion on a topic engages with the facts that bear on it, even if these facts are not favorable to the ideas advanced in the discussion. It is equally customary to discuss both the capabilities and limitations of proposed methods, theories, procedures, etc.

But in this article, the authors do not even attempt to follow this long tradition. They portray a rosy picture of the proposed new theory completely ignoring its limitations and the critique that is directed at its various ingredients.

Consider for example the proposed robustness model, that is, info-gap's robustness model:

Formal proofs -- in peer-reviewed publications going back to 2007 -- demonstrate that this model is neither "new" nor "radically different", as claimed in the info-gap literature, but is rather a simple instance of Wald's famous Maximin.

But, there is no mention of this fact in the article.

Proofs -- in peer-reviewed publications going back to 2007 -- show that the proposed robustness model does not seek robustness against severe uncertainty, but rather against small perturbations in the nominal value of a parameter.

But, there is no mention of this fact in the article.

There are numerous peer-reviewed publications that question the validity of the thesis that "satisficing is better than optimizing".

But there is no mention of this fact in the article.

There is a thriving branch of optimization theory, called Robust Optimization that deals with robust decision-making problems of the type discussed in the article.

But, there is no mention of this fact in the article.

The upshot of all this is that the authors present as new a very simple instance of old and well established robustness models. Worse, the authors fail to advise the readers of the limitations of this model notably the fact that it deals with local rather than global robustness.

The authors thus depict a thoroughly distorted picture of the state of the art and the proposed theory.

## What Next?

It will be interesting to see how long will proponents of info-gap decision theory continue to promulgate the myth that info-gap decision theory is a distinct, novel theory that is radically different from all current theories of decisions under uncertainty.

And it will be even more interesting to see how long will peer reviewed journals continue to accept publications that conceal from the public the Maximin and Radius of stability connections.

Of particular interest is the new myth about info-gap decision theory: its ability to

maximize the range of satisfactory outcomes.Because this myth asserts a real (scientific) miracle: a very simple mathematical model of

localrobustness has the capability to determine thewidest rangeof satisfactory outcomes!And this when, all these years (at least since the 1960s), this model has been used extensively in a range of fields (numerical analysis, optimization, control theory, stability theory, economics, operations research, and so on)

onlyfor the purpose of analyzing small perturbations in the nominal value of a parameter!Imagine how much we lost as a result of so many scholars failing to identify the (miraculous) capabilities of this simple model!

But, as we say here, No Worries!!!

- Ben-Haim (2001, 2006): Info-Gap Decision Theory: decisions under severe uncertainty.

- Regan et al (2005): Robust decision-making under severe uncertainty for conservation management.

- Moilanen et al (2006): Planning for robust reserve networks using uncertainty analysis.

- Burgman (2008): Shakespeare, Wald and decision making under severe uncertainty.

- Ben-Haim and Demertzis (2008): Confidence in monetary policy.

- Hall and Harvey (2009): Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.

- Ben-Haim (2009): Info-gap forecasting and the advantage of sub-optimal models.

- Yokomizo et al (2009): Managing the impact of invasive species: the value of knowing the density-impact curve.

- Davidovitch et al (2009): Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island.

- Ben-Haim et al (2009): Do we know how to set decision thresholds for diabetes?

- Beresford and Thompson (2009): An info-gap approach to managing portfolios of assets with uncertain returns

- Ben-Haim, Dacso, Carrasco, and Rajan (2009): Heterogeneous uncertainties in cholesterol management

- Rout, Thompson, and McCarthy (2009): Robust decisions for declaring eradication of invasive species

- Ben-Haim (2010): Info-Gap Economics: An Operational Introduction

- Hine and Hall (2010): Information gap analysis of flood model uncertainties and regional frequency analysis

- Ben-Haim (2010): Interpreting Null Results from Measurements with Uncertain Correlations: An Info-Gap Approach

- Wintle et al. (2010): Allocating monitoring effort in the face of unknown unknowns

- Moffitt et al. (2010): Securing the Border from Invasives: Robust Inspections under Severe Uncertainty

- Yemshanov et al. (2010): Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest

- Davidovitch and Ben-Haim (2010): Robust satisficing voting: why are uncertain voters biased towards sincerity?

- Schwartz et al. (2010): What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making

- Arkadeb Ghosal et al. (2010): Computing Robustness of FlexRay Schedules to Uncertainties in Design Parameters

- Hemez et al. (2002): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

- Hemez et al. (2003): Applying information-gap reasoning to the predictive accuracy assessment of transient dynamics simulations

- Hemez, F.M. and Ben-Haim, Y. (2004): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

- Ben-Haim, Y. (2007): Frequently asked questions about info-gap decision theory

- Sprenger, J. (2011): The Precautionary Approach and the Role of Scientists in Environmental Decision-Making

- Sprenger, J. (2011): Precaution with the Precautionary Principle: How does it help in making decisions

- Hall et al. (2011): Robust climate policies under uncertainty: A comparison of Info--Gap and RDM methods

- Ben-Haim and Cogan (2011) : Linear bounds on an uncertain non-linear oscillator: an info-gap approach

- Van der Burg and Tyre (2011) : Integrating info-gap decision theory with robust population management: a case study using the Mountain Plover

- Hildebrandt and Knoke (2011) : Investment decisions under uncertainty --- A methodological review on forest science studies.

- Wintle et al. (2011) : Ecological-economic optimization of biodiversity conservation under climate change.

- Ranger et al. (2011) : Adaptation in the UK: a decision-making process.

## Recent Articles, Working Papers, Notes

Also, see my complete list of articles

Moshe's new book! - Sniedovich, M. (2012) Fooled by local robustness,
Risk Analysis, in press.

- Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty,
International Transactions in Operational Research, in press.

- Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis,
Applications of Mathematics,56(5), 499-509.

- Sniedovich, M. (2011) Dynamic programming: introductory concepts, in
Wiley Encyclopedia of Operations Research and Management Science(EORMS), Wiley.

- Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem,
OR Spectrum, 33(4), 815-929, 2011.

- Sniedovich, M. (2011)
Dynamic Programming: Foundations and Principles, Second Edition,Taylor & Francis.

- Sniedovich, M. (2010) A bird's view of Info-Gap decision theory,
Journal of Risk Finance, 11(3), 268-283.

- Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.

- Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds),
.Safety, Reliability and Risk Analysis: Theory, Methods and Applications,pp. 2071-2079, Taylor and Francis Group, London.

- Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)

- Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.),
11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation,DIPTEM, Genova, 89-94.

- Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)

- Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)

- Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08

This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )

- Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory,
Decision Point,24, 10.

- Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty,
Decision Point,22, 8-9.

- Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!,
Journal of Risk Finance,9(3), 287-291.

- Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)

In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.

- Sniedovich. M. (2008) The Mighty Maximin! (PDF File)

This paper is dedicated to the modeling aspects of Maximin and robust optimization.

- Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty,
Decision Making in Manufacturing and Services,1-2, 111-136. (PDF File) .

- Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)

In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.

- My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)
This is an exciting development!

- Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.

- Ben-Haim is not familiar with the fundamental concept
point estimate. He does not realize that a function can be a point estimate of another function.So when you read my papers make sure that you do not misinterpret the notion

point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!

- A formal proof that
Info-GapisWald's Maximin Principlein disguise. (December 31, 2006)

This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)

It shows thatInfo-Gapis not a new theory but rather a simple instance ofWald's famous Maximin Principledating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context ofGame Theory(1928).

- A proof that
Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)

This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .

It shows that becauseInfo-Gapdeploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.

- A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)

This is a very short article entitled The GAP in Info-Gap (PDF File) .

It is a math-free version of the paper above. Read it if you are allergic to math.

- A long essay entitled
What's Wrong with Info-Gap? An Operations Research Perspective(PDF File) (December 31, 2006).

This is a paper that I presented at theASOR Recent Advances in Operations Research(PDF File) mini-conference (December 1, 2006, Melbourne, Australia).## Recent Lectures, Seminars, Presentations

If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.

Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.

- ASOR Recent Advances, 2011, Melbourne, Australia, November 16 2011. Presentation: The Power of the (peer-reviewed) Word. (PDF file).

- Alex Rubinov Memorial Lecture The Art, Science, and Joy of (mathematical) Decision-Making, November 7, 2011, The University of Ballarat. (PDF file).

- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Invited tutorial, ALIO/INFORMS Conference, Buenos Aires, Argentina, July 6-9, 2010).

- A Critique of Info-Gap Decision theory: From Voodoo Decision-Making to Voodoo Economics(PDF File) .

(Recent Advances in OR, RMIT, Melbourne, Australia, November 25, 2009)

- Robust decision-making in the face of severe uncertainty(PDF File) .

(GRIPS, Tokyo, Japan, October 16, 2009)

- Decision-making in the face of severe uncertainty(PDF File) .

(KORDS'09 Conference, Vilnius, Lithuania, September 30 -- OCtober 3, 2009)

- Modeling robustness against severe uncertainty (PDF File) .

(SOR'09 Conference, Nova Gorica, Slovenia, September 23-25, 2009)

- How do you recognize a Voodoo decision theory?(PDF File) .

(School of Mathematical and Geospatial Sciences, RMIT, June 26, 2009).

- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, Info-Gaps, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Department of Econometrics and Business Statistics, Monash University, May 8, 2009).

- The Rise and Rise of Voodoo Decision Theory.

ASOR Recent Advances, Deakin University, November 26, 2008. This presentation was based on the pages on my website (voodoo.moshe-online.com).

- Responsible Decision-Making in the face of Severe Uncertainty (PDF File) .

(Singapore Management University, Singapore, September 29, 2008)

- A Critique of Info-Gap's Robustness Model (PDF File) .

(ESREL/SRA 2008 Conference, Valencia, Spain, September 22-25, 2008)

- Robust Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Technion, Haifa, Israel, September 15, 2008)

- The Art and Science of Robust Decision-Making (PDF File) .

(AIRO 2008 Conference, Ischia, Italy, September 8-11, 2008 )

- The Fundamental Flaws in Info-Gap Decision Theory (PDF File) .

(CSIRO, Canberra, July 9, 2008 )

- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .

(OR Conference, ADFA, Canberra, July 7-8, 2008 )

- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .

(University of Sydney Seminar, May 16, 2008 )

- Decision-Making Under Severe Uncertainty: An Australian, Operational Research Perspective (PDF File) .

(ASOR National Conference, Melbourne, December 3-5, 2007 )

- A Critique of Info-Gap (PDF File) .

(SRA 2007 Conference, Hobart, August 20, 2007)

- What exactly is wrong with Info-Gap? A Decision Theoretic Perspective (PDF File) .

(MS Colloquium, University of Melbourne, August 1, 2007)

- A Formal Look at Info-Gap Theory (PDF File) .

(ORSUM Seminar , University of Melbourne, May 21, 2007)

The Art and Science of Decision-Making Under Severe Uncertainty(PDF File) .

(ACERA seminar, University of Melbourne, May 4, 2007)

- What exactly is Info-Gap? An OR perspective. (PDF File)

ASOR Recent Advances in Operations Researchmini-conference (December 1, 2006, Melbourne, Australia).

Disclaimer:This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.