Reviews of publications on Info-Gap decision theory

Review # 34(Posted: December 10, 2011; Last update: December 13, 2011)

Reference: | Nicola Ranger, Antony Millner, Simon Dietz, Sam Fankhauser, Ana Lopez and Giovanni Ruta. Adaptation in the UK: a decision-making process with Technical Annexes by Nicola Ranger, Antony Millner, Ana Lopez, Giovanni Ruta and Alice Hardiman Policy brief, September 2010 The Grantham Research Institute on Climate Change and the Environment and the Centre for Climate Change Economics and Policy, London School of Economics and Political Science (LSE), UK Available on line at: http://www2.lse.ac.uk/GranthamInstitute/publications/Policy/briefs.aspx |

From the Introduction | Chapter III also provides complementary guidance on decision analytical approaches for appraising adaptation options and making decisions (steps 5 and 6). In particular, Chapter III and Technical Annex A provide a comprehensive, rigorous, and upto-date treatment of decisionmaking under deep uncertainty, i.e. when it is difficult or impossible to make detailed quantitative estimates of the probabilities of alternative climate conditions being realized. |

Acknowledgements | The authors wish to thank the experts that provided advice and reviews of this report. This includes: Caroline Cowen, Suraje Dessai, Bill Donovan, Mark Ellis-Jones, Polly Ericksen, Jonathan Fisher, Marcus Francis, Su-Lin Garbett-Shiels, Sam Jenkins, Michael Mullen, Tim Reeder, Howard Perry, Michael Spackman, Leonard Smith, David Stainforth, Roger Street, Adam Ward, Robert Ward and Glenn Watts. The authors also wish to thank the members of the Adaptation Sub-Committee for their support and expert advice, Lord John Krebs, Andrew Dlugolecki, Sam Fankhauser, Jim Hall, Anne Johnson, Martin Parry, Tim Palmer, Graham Wynne and Baroness Barbara Young, and the secretariat to the Committee, particularly, Sebastian Catovsky, Neil Golborne, Steve Smith and Kiran Sura. |

Scores | TUIGF: 100%SNHNSNDN: 500%GIGO: 300% |

What drew my attention to this Policy brief (henceforth Brief) was the information stated at the bottom of Table III.2 (page 44) whose caption reads as follows:

Summary characteristics of a set of standard decision methods. For a full description see Annex A. Here is a reproduction of this part of the table.

2. Method applicable when exact probabilities are not known Decision method Decision criteria Preference assumptions Information assumptions Additional requirements Reference Annex A Maximin expected utility As for expected utility As for expected utility + Extreme ambiguity aversion (act as if the worst plausible probability distribution were correct) Multiple plausible probability distributions. 6.i Smooth ambiguity model As for expected utility As above, but allows for any attitude to ambiguity. Multiple plausible probability distributions, and weights on each of these distributions 6.i Maximin Any Ordinal ranking of outcomes ^{24}No likelihood information 6.iii Minimax regret Any Cardinal ranking of outcomes ^{25}As above 6.iii Info-gap decision theory Various Does not rigorously account for preferences. Assumes satisficing ^{26}thresholds, i.e. acceptable levels of minimum performance/maximum windfall.A "best guess" model of the decision environment, and a set of models that are "close" to this best guess. A method for measuring the distance between different models (an "uncertainty model" 6.iii

^{24}Ordinal rankings tell us which of two outcomes is preferred, but not by how much.

^{25}Cardinal rankings allow us to say how much better one outcome is than another. Differences between outcomes that are ranked cardinally are meaningful.

^{26}A satisficing threshold is the value of a decision criterion at which an adaptation option is considered "good enough". See Annex A, section 6.3.

As pointed out in the table, details of these decision methods are discussed in Annex A. So, to assess the analysis presented in this Brief we need to examine Annex A. However, before we do this, I want to call attention to a number of points arising from specific claims made in the table itself and to comment on them briefly:

- The caption of the table puts
allthe listed methods on an equal footing treating them all asstandarddecision methods. My question to the authors is therefore this: does this correctly reflect the situation in the area of decision-making?

- The table does not indicate what scope of
valid applicationsdo these methods have.

- The characterization of info-gap decision theory is downright
incorrect.For obvious reasons, in this review report I focus exclusively on info-gap decision theory.

In greater detail:

The point that I want to highlight through the question that I am putting above to the authors is this. Are the authors earnestly claiming that decision theory specialists, or more generally, specialists in disciplines concerned with robust decision-making regard info-gap decision theory as a "standard decision method" in the same manner that they would regard say, Wald's Maximin model as a standard method?! Surely, judged on this criterion alone, info-gap decision theory cannot by any stretch of the imagination be described as a "standard decision method".

As for the characteristics attributed to info-gap decision theory.

The first point to note is that the statement in the table regarding info-gap's account of

preferencesis simply in error. Info-gap decision theorydoes most definitelyseek to establish a rigorouspreference relationamong decisions. Indeed, preferences are determined as follows: the more robust the decision (for the critical performance level under consideration) the better it is. And if decisions are evaluated on the basis of theiropportuneness,then the smaller the opportuneness the better the decision. In other words, info-gap decision theory claims to offer decision models that stipulate a preference relation whereby decisions are ranked according to their robustness (or, opportuneness) levels. Hence, theoptimaldecision is that with the highest (largest) robustness, or the lowest (smallest) opportuneness.In the same vein, the statement in the table claiming that info-gap's uncertainty sets are `close' to the `best guess' is also incorrect. The fact of the matter is that info-gap's uncertainty sets can be vast so as to encompass areas of the uncertainty space that are significantly distant from the `best guess'. Indeed, according to Ben-Haim (2001, 2006, 2010), info-gap's uncertainty sets are very often

unbounded.Third, to set aright the (wrong) impression given by the fact that this table places info-gap decision theory on an equal footing with other methods, I remind the reader that ... info-gap's robustness model and info-gap's decision model are in fact ...

simple Maximin models(more on this below).In short, the information provided by this table about info-gap decision theory misrepresents the facts about this method!.

One can of course argue that this Brief was written by analysts/scholars specializing in

Climate Change, and not in decision theory, and that this fact explains the errors in this table and in Annex A.But consider this statement on page 13 (red color added):

This research was conducted by theCentre for Climate Change Economics and Policy (CCCEP) and the Grantham Research Institute on Climate Change and the Environment at the London School of Economics and Political Science (LSE),in consultation with a number of experts on decision-making and sector-level adaptation. The Ecosystems case study was contributed by Alice Hardiman from the Royal Society for the Protection of Birds. The research was sponsored by the secretariat to the ASC.For the sake of color-blind readers, I note that I show the clause: "in consultation with a number of experts on decision-making" in red so as to highlight this fact.

In other words, I want to call attention to the fact that the authors did the right thing: they consulted experts in decision-making. The trouble is, however, that at least insofar as info-gap decision theory is concerned, they were either ill advised, or they misunderstood the advice given to them, or both. The end result is that the discussion on info-gap decision theory in this Brief, especially in Annex A, is --- to put it mildly --- highly problematic and requires attention (correction?).

With this in mind, let us now examine the discussion in Annex A on info-gap decision theory.

## Annex A's perspective on info-gap decision theory

On page 18 of Annex A we read:A related approach, known as Info-gap decision theory (Ben-Haim 2006), provides methods that yield qualitative information about the robustness of decisions to uncertainty around a best guess parameter estimate. "Robustness" and "opportuneness" curves are generated, which measure the maximum amount of uncertainty the decision maker can be exposed to and still ensure that losses do not exceed a given level (in the case of robustness), or the minimum amount of uncertainty the decision-maker must expose herself to in order to have the possibility of achieving a windfall of a certain level (in the case of opportuneness). These curves are then fed into an informal decision process (info-gap does not account for preferences rigorously), in which the decision maker is asked to specify acceptable levels, i.e. the largest loss she is willing to sustain and the smallest windfall she wishes to have the possibility of achieving, and picks options based on their trade-offs between robustness and opportuneness at these levels. A vital part of the info-gap method is the choice of uncertainty model, i.e. how uncertainty is defined and measured. This is an ad-hoc modelling choice, and much of the art in the analysis may lie in picking a model that is suited to the application at hand. It should be emphasized that Info-gap is in one respect less general than the methods employed by (Hansen & Sargent 2008), since it does not achieve robustness to changes in model structure, but only to uncertain parameters or a model.There are a number of errors/misrepresentations in this profile of info-gap decision theory. Some are more glaring than others. This therefore leads me to conclude that the authors of this Brief misunderstood the explanations given to them (by experts?) of info-gap decision theory and what it aims to do.

For now I discuss only a number of those points that must be corrected immediately. My plan is to come back to this discussion every so often and to expand on it some more.

## Comment 1:

Take note of the assertion that info-gap decision theory " ... provides methods that yield

qualitativeinformation about the robustness of decisions to uncertainty around a best guess parameter estimate. ..." My question to the authors is as follows:Is this a typo or do the authors actually meanqualitativerather than quantitative?Because, if the authors indeed mean "qualitative", then they should take note that according to info-gap decision theory, the robustness and opportuneness of decision q∈Q are yielded by these

quantitativemathematical models:

Robustness model Opportuneness model α(q,r _{c}) = max {α≥0: r(q,u) ≥ r_{c}∀u∈U(α,û)}β(q,r _{w}) = min {α≥0: r(q,u) ≥ r_{w}for at least one u∈U(α,û)}Are the authors seriously claiming that these models are

qualitative?!!!?!?!!

Technical Remark:with no loss of generality assume that r(q,û) > r_{c}and r(q,û) < r_{w}for all q∈Q.And consider this (emphasis added):

In such situations we are dealing with “true uncertainty” in the sense of Knight (1921) who was the first to distinguish between “risk” based on known probability distributions and true uncertainty when the underlying statistical distributions are unknown. Knight’s ideas have been further developed by several authors over the years and in particular by Ben-Haim (2006) who has developed aquantitative formulationknown as information-gap decision theory. This theory has recently been shown by Sniedovich (2008) to be formally equivalent to Wald’s maximin model in classical decision theory (French, 1988).

Beresford and Thompson (2009, p. 278)

Managing credit risk with info-gap uncertainty, Journal of Risk Finance, Vol. 8 No. 1, pp. 23-34.See Comment 3 for more details on the maximin connection.

## Comment 2:

Info-gap decision theory claims to provide decision-makers two decision models (Ben-Haim 2001, 2006, 2010). A robustness model that is aimed for decision-makers whose concern is robust decision. And an opportuneness model aimed for decision-makers whose concern is with opportuneness. Decision-makers concerned with robustness select a decision from the set of decisions whose robustness is high (large), whereas decision-makers concerned with opportuneness select decisions whose opportunenss is low (small). In short, info-gap decision theory instructs the former to select decisions whose robustness is the highest (largest), and the latter to select decisions whose opportuneness is the lowest (smallest). To this end, info-gap decision theory puts forth

two decision modelsformulated as follows:

Decision model for robustness Decision model for opportuneness max {α(q,r _{c}): q∈Q}min {β(q,r _{w}): q∈Q}Now, the info-gap literature (notably its primary texts (Ben-Haim 2001, 2006, 2010)) indicates in no uncertain terms that decision makers dealing with situations that are subject to severe uncertainty are concerned mostly with robustness rather than with opportuneness. Indeed, this literature overwhelmingly discusses applications dealing with robustness rather than with opportuneness.

The implication therefore is that the following narrative in the Brief:

These curves are then fed into an informal decision process (info-gap does not account for preferences rigorously), in which the decision maker is asked to specify acceptable levels, i.e. the largest loss she is willing to sustain and the smallest windfall she wishes to have the possibility of achieving, and picks options based on their trade-offs between robustness and opportuneness at these levels.misrepresents info-gap decision theory's basic claims and its intended application.

I want to particularly single out the phrase "...and picks options based on their trade-offs between robustness and opportuneness at these levels." Because, this phrase attests to an astounding misapprehension of the info-gap methodology.

According to Ben-Haim (2006, pp. 169-171),

## Comment 3:

The reference to the methods employed by Hansen and Sargent (2008) is grossly misleading. The Brief claims that

It should be emphasized that Info-gap is in one respect less general than the methods employed by (Hansen & Sargent 2008), since it does not achieve robustness to changes in model structure, but only to uncertain parameters or [sic] a model.The trouble with this comparison is that it conceals a great deal more than it reveals about info-gap decision theory.

To begin with, if the intention is to point at the differences between info-gap decision theory and other methods, to indicate which method is more general, then surely the most obvious candidate for such a comparison to top the list would be Wald's maximin model. And to appreciate why this is so consider this fact:

Theorem 1:Info-gap robustness model and info-gap decision model for robustness are simple instances of Wald's Maximin model (circa 1940).To be even more accurate:

Theorem 2:Info-gap robustness model is a very simple instance of the famous Radius of Stability model (circa 1960) itself a simple instance of Wald's Maximin model.You can find formal proofs of these theorems in two recent papers on this topic:

- Sniedovich, M. (2010) A bird's view of Info-Gap decision theory,
Journal of Risk Finance, 11(3), 268-283.

- Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty,
International Transactions in Operational Research, available on line.But if you want immediate access to such proofs, have a look at these:

- Mobile Maximin Theorem (show/hide).

- Mobile Radius of Stability Theorem (show/hide).
These two theorems

debunkinfo-gap decision theory outright (see why ....). The bottom line is that info-gap's robustness model is a re-invention of a very famous wheel (universally known as Radius of Stability), and ... a square one at that!I should also set the record straight on the information conveyed by the phrase: "... it does not achieve robustness to changes in model structure, but only to uncertain parameters or [sic] a model." Hence,

## Comment 4:

As correctly indicated by the authors (elsewhere in the above quoted paragraph), info-gap's robustness analysis is indeed conducted around a

best guessparameter estimate. Hence, as (almost) correctly indicated by this phrase, info-gap's robustness analysis seeks to "...achieve robustness ... only to uncertain parameters ... ".But ... given that this is the case, surely this should have indicated to the authors that it is their duty to explain how such a method can possibly be suitable for robust decision-making under conditions of

deep uncertainty? On what grounds can it possibly be claimed that info-gap decision theory provides the requisite methodology that is called for by this task? Are the authors suggesting that a theory oflocalrobustness (to a "best guess") is the appropriate tool for dealing with thedeep uncertaintythat characterizesclimate changephenomena of the type examined in the Brief?## Comment 5:

On page 16-17 of Annex A we read (emphasis added):

Aside from their lack of dependence on probabilities, a feature of many robust decision methods is thatthey do not attempt to maximize anything.Rather they aim to find decision strategies that satisfice, i.e. achieve an acceptable level of some, possibly conflicting, objectives.This is true of Info-gap decision theory, as well as the global robustness methods advocated by Lempert (2002) and Lempert et al. (2004), and applied to adaptation decisions by Dessai & Hulme (2007).I want to single out for special attention the hair raising statement: "... a feature of many robust decision methods is that

they do not attempt to maximize anything." I leave it to the reader to reflect on the claim made by this statement. At this stage all I do is put the following question to the authors: are the authors seriously claiming that info-gap decision theory does not maximize anything?!In that case, how do they square this claim with the following statement in Dessai et al. (2009, p. 74) which is one of their main references (emphasis added)?

Info-gap decision theory is a non-probabilistic decision theory seeking tooptimizerobustness to failure, or opportunity of windfall. This differs from classical decision theory, which typically maximizes the expected utility.And this:

We use info-gap theory for satisficing (not minimizing) the probability of detection, while at the same timemaximizing the robustness to uncertainty.

Lior Davidovitch, Richard Stoklosa, Jonathan Majer, Alex Nietrzeba, Peter Whittle, Kerrie Mengersen, Yakov Ben-Haim.

Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island

Journal of Environmental Management, Volume 90, Issue 8, Pages 2785-2793, 2009.And how do the authors square their claim with this statement (color added)?

And with this statement (emphasis added):Satisfice profit, maximize robustness.We now consider a robust-satisficing strategy which both guarantees no less than a specified minimum profit (if possible) and which maximizes the robustness or immunity to uncertainty.

Ben-Haim, Y. Info-Gap Decision Theory (2006, p. 92).This is the allocation whichmaximizes the robustnesswhile satisficing the capital reserve requirement

Ben-Haim, Y. Info-Gap Economics: an Operational Introduction (2010, p. 132).Those readers who are not familiar with info-gap decision theory are advised that Prof. Y. Ben-Haim is the Father of this theory.

The question is of course, how could such a (bewildering) misconception of the very nature of info-gap decision theory, possibly have infiltrated this Brief?

Readers seeking answers to this intriguing question should obviously direct it to the authors of the Brief. All I am going to say here, at this stage, is that this seems to be an interpretation of the incessant (misleading) babble in the info-gap literature about the difference between optimizing and satisficing

gone wrong.In particular, this seems to be a misconstruction of the incessant (misleading) babble in the info-gap literature claiming that info-gap decision theory is a robust-satificing theory rather than a utility-maximizing theory.All the same, I want to point out that I'd be delighted to discuss this matter with anyone interested in this issue, over a cup of coffee (soy latte, please).

And there is more ...

## Environment Agency, UK

It is surprising that the authors seem to be unaware of the 2009 Department for Environment Food and Rural Affairs (DEFRA) report. To be precise, what I am particularly referring to is the following paragraph on page 75 of the report:

More recently, Info-Gap approaches that purport to be non-probabilistic in nature developed by Ben-Haim (2006) have been applied to flood risk management by Hall and Harvey (2009). Sniedovich (2007) is critical of such approaches as they adopt a single description of the future and assume alternative futures become increasingly unlikely as they diverge from this initial description. The method therefore assumes that the most likely future system state is known a priori. Given that the system state is subject to severe uncertainty, an approach that relies on this assumption as its basis appears paradoxical, and this is strongly questioned by Sniedovich (2007).Mervyn Bramley, Ben Gouldby, Anthony Hurford, Jaap-Jeroen Flikweert

Marta Roca Collell, Paul Sayers, Jonathan Simm, Michael Wallis

Delivering Benefits Through Evidence

PAMS (Performance-based Asset Management System)

Phase 2 Outcome Summary Report

Project: SC040018/R1

Product code: SCHO1209BRRA-E-E

SubTitle: Report SC040018/R1

Pages: 132

ISBN: 9781849111638

Environment Agency -- December 2009

UK

Department for Environment Food and Rural Affairs

Note:The URL of the report keeps changing. Try this one (downloaded on March 6, 2011):

http://publications.environment-agency.gov.uk/epages/eapublications.storefront/4d72dd4801b700e6273fc0a80296066c/Product/View/SCHO1209BRRA-E-E### Deep uncertainty

The term

Deep Uncertaintyseems to be used in this Brief to put across the severity, or extremity, of the uncertainty that this document is concerned with. Indeed, the impression one gets is that the termDeep Uncertaintyis intended to convey the message that the uncertainty that the authors are concerned with is in some way different from the, if you will, Knightian (non-probabilisitc) uncertainty that is considered in classical decision theory. Yet, there is no discussion in this document to explain in what sense is the uncertainty designated byDeep Uncertaintydifferent, if at all, from the (non-probabilisitc) uncertainty that is postulated by classical decision theory. In brief what is it that qualifies the uncertainty that this policy brief is concerned with for the designatordeep uncertainty?I would therefore like to call the authors' and the readers' attention to the following paragraph:

... Another concern of the committee regarding the content of this chapter involves the use of the concept of "robustness." The committee finds that this term is insufficiently defined. A plausible argument can be made that there is no meaningful distinction from usual optimality analysis and that the concept discussed in this report is a matter of a poorly defined utility function. If indeed there is a real technical distinction to be made, the authors should consider expanding and supporting the discussion of this concept. Furthermore, the committee suggests that the authors address the concept of adaptive management in conjunction with discussions of robustness and in particular address how different sources of uncertainty affect different kinds of decisions. Finally, the committee would appreciate a further elucidation of what the author considers to constitute "deep uncertainty" (page 34 and other locations). The committee understands that there is overlap between this concept and the others defined in this section (e.g., "robust"), but nevertheless finds that it is not entirely clear when the author considers the situation inappropriate for use of conventional methods for characterizing uncertainty.

Review of the U.S. Climate Change Science Program's Synthesis and Assessment Product 5.2

"Best Practice Approaches for Characterizing, Communicating, and Incorporating

Scientific Uncertainty in Climate Decision Making"

pp. 17-18, 2007

http://www.nap.edu/catalog/11873.htmlMy suggestion to the authors is that they would do well to reflect on this statement, as well as on Golomb's (1970) 9-th don't of mathematical modeling, quoted below, before they next set out to write on this topic:

Don't apply terminology of subject A to the problems of subject B if it is to the enrichment of neither. Catch Phrase: 'New names for old.'

Golomb, S.W., 1970. Mathematical Models - Uses and Limitations.Simulation,4 (14), 197-198.## What's next?

Despite the crystal clear technical assessments of info-gap decision theory that are easily accessible to the public, for some odd reason, analysts/scholars in the areas of applied ecology and climate change seem to have great difficulties in comprehending the basic facts about info-gap decision theory.

Considering then that the basic facts about this theory continue to elude scholars/analysts in a number of research centers in the UK, I am tempted to extend my info-gap campaign to the UK!

Stay tuned, as there is more to come ...

- Ben-Haim (2001, 2006): Info-Gap Decision Theory: decisions under severe uncertainty.

- Regan et al (2005): Robust decision-making under severe uncertainty for conservation management.

- Moilanen et al (2006): Planning for robust reserve networks using uncertainty analysis.

- Burgman (2008): Shakespeare, Wald and decision making under severe uncertainty.

- Ben-Haim and Demertzis (2008): Confidence in monetary policy.

- Hall and Harvey (2009): Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.

- Ben-Haim (2009): Info-gap forecasting and the advantage of sub-optimal models.

- Yokomizo et al (2009): Managing the impact of invasive species: the value of knowing the density-impact curve.

- Davidovitch et al (2009): Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island.

- Ben-Haim et al (2009): Do we know how to set decision thresholds for diabetes?

- Beresford and Thompson (2009): An info-gap approach to managing portfolios of assets with uncertain returns

- Ben-Haim, Dacso, Carrasco, and Rajan (2009): Heterogeneous uncertainties in cholesterol management

- Rout, Thompson, and McCarthy (2009): Robust decisions for declaring eradication of invasive species

- Ben-Haim (2010): Info-Gap Economics: An Operational Introduction

- Hine and Hall (2010): Information gap analysis of flood model uncertainties and regional frequency analysis

- Ben-Haim (2010): Interpreting Null Results from Measurements with Uncertain Correlations: An Info-Gap Approach

- Wintle et al. (2010): Allocating monitoring effort in the face of unknown unknowns

- Moffitt et al. (2010): Securing the Border from Invasives: Robust Inspections under Severe Uncertainty

- Yemshanov et al. (2010): Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest

- Davidovitch and Ben-Haim (2010): Robust satisficing voting: why are uncertain voters biased towards sincerity?

- Schwartz et al. (2010): What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making

- Arkadeb Ghosal et al. (2010): Computing Robustness of FlexRay Schedules to Uncertainties in Design Parameters

- Hemez et al. (2002): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

- Hemez et al. (2003): Applying information-gap reasoning to the predictive accuracy assessment of transient dynamics simulations

- Hemez, F.M. and Ben-Haim, Y. (2004): Info-gap robustness for the correlation of tests and simulations of a non-linear transient

- Ben-Haim, Y. (2007): Frequently asked questions about info-gap decision theory

- Sprenger, J. (2011): The Precautionary Approach and the Role of Scientists in Environmental Decision-Making

- Sprenger, J. (2011): Precaution with the Precautionary Principle: How does it help in making decisions

- Hall et al. (2011): Robust climate policies under uncertainty: A comparison of Info--Gap and RDM methods

- Ben-Haim and Cogan (2011) : Linear bounds on an uncertain non-linear oscillator: an info-gap approach

- Van der Burg and Tyre (2011) : Integrating info-gap decision theory with robust population management: a case study using the Mountain Plover

- Hildebrandt and Knoke (2011) : Investment decisions under uncertainty --- A methodological review on forest science studies.

- Wintle et al. (2011) : Ecological-economic optimization of biodiversity conservation under climate change.

- Ranger et al. (2011) : Adaptation in the UK: a decision-making process.

Also, see my complete list of articles

Moshe's new book! - Sniedovich, M. (2012) Fooled by local robustness,
Risk Analysis, in press.

- Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty,
International Transactions in Operational Research, in press.

- Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis,
Applications of Mathematics,56(5), 499-509.

- Sniedovich, M. (2011) Dynamic programming: introductory concepts, in
Wiley Encyclopedia of Operations Research and Management Science(EORMS), Wiley.

- Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem,
OR Spectrum, 33(4), 815-929, 2011.

- Sniedovich, M. (2011)
Dynamic Programming: Foundations and Principles, Second Edition,Taylor & Francis.

- Sniedovich, M. (2010) A bird's view of Info-Gap decision theory,
Journal of Risk Finance, 11(3), 268-283.

- Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.

- Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds),
.Safety, Reliability and Risk Analysis: Theory, Methods and Applications,pp. 2071-2079, Taylor and Francis Group, London.

- Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)

- Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.),
11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation,DIPTEM, Genova, 89-94.

- Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)

- Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)

- Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08

This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )

- Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory,
Decision Point,24, 10.

- Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty,
Decision Point,22, 8-9.

- Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!,
Journal of Risk Finance,9(3), 287-291.

- Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)

In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.

- Sniedovich. M. (2008) The Mighty Maximin! (PDF File)

This paper is dedicated to the modeling aspects of Maximin and robust optimization.

- Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty,
Decision Making in Manufacturing and Services,1-2, 111-136. (PDF File) .

- Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)

In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.

- My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)
This is an exciting development!

- Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.

- Ben-Haim is not familiar with the fundamental concept
point estimate. He does not realize that a function can be a point estimate of another function.So when you read my papers make sure that you do not misinterpret the notion

point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!

- A formal proof that
Info-GapisWald's Maximin Principlein disguise. (December 31, 2006)

This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)

It shows thatInfo-Gapis not a new theory but rather a simple instance ofWald's famous Maximin Principledating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context ofGame Theory(1928).

- A proof that
Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)

This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .

It shows that becauseInfo-Gapdeploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.

- A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)

This is a very short article entitled The GAP in Info-Gap (PDF File) .

It is a math-free version of the paper above. Read it if you are allergic to math.

- A long essay entitled
What's Wrong with Info-Gap? An Operations Research Perspective(PDF File) (December 31, 2006).

This is a paper that I presented at theASOR Recent Advances in Operations Research(PDF File) mini-conference (December 1, 2006, Melbourne, Australia).

If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.

Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.

- ASOR Recent Advances, 2011, Melbourne, Australia, November 16 2011. Presentation: The Power of the (peer-reviewed) Word. (PDF file).

- Alex Rubinov Memorial Lecture The Art, Science, and Joy of (mathematical) Decision-Making, November 7, 2011, The University of Ballarat. (PDF file).

- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Invited tutorial, ALIO/INFORMS Conference, Buenos Aires, Argentina, July 6-9, 2010).

- A Critique of Info-Gap Decision theory: From Voodoo Decision-Making to Voodoo Economics(PDF File) .

(Recent Advances in OR, RMIT, Melbourne, Australia, November 25, 2009)

- Robust decision-making in the face of severe uncertainty(PDF File) .

(GRIPS, Tokyo, Japan, October 16, 2009)

- Decision-making in the face of severe uncertainty(PDF File) .

(KORDS'09 Conference, Vilnius, Lithuania, September 30 -- OCtober 3, 2009)

- Modeling robustness against severe uncertainty (PDF File) .

(SOR'09 Conference, Nova Gorica, Slovenia, September 23-25, 2009)

- How do you recognize a Voodoo decision theory?(PDF File) .

(School of Mathematical and Geospatial Sciences, RMIT, June 26, 2009).

- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, Info-Gaps, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Department of Econometrics and Business Statistics, Monash University, May 8, 2009).

- The Rise and Rise of Voodoo Decision Theory.

ASOR Recent Advances, Deakin University, November 26, 2008. This presentation was based on the pages on my website (voodoo.moshe-online.com).

- Responsible Decision-Making in the face of Severe Uncertainty (PDF File) .

(Singapore Management University, Singapore, September 29, 2008)

- A Critique of Info-Gap's Robustness Model (PDF File) .

(ESREL/SRA 2008 Conference, Valencia, Spain, September 22-25, 2008)

- Robust Decision-Making in the Face of Severe Uncertainty (PDF File) .

(Technion, Haifa, Israel, September 15, 2008)

- The Art and Science of Robust Decision-Making (PDF File) .

(AIRO 2008 Conference, Ischia, Italy, September 8-11, 2008 )

- The Fundamental Flaws in Info-Gap Decision Theory (PDF File) .

(CSIRO, Canberra, July 9, 2008 )

- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .

(OR Conference, ADFA, Canberra, July 7-8, 2008 )

- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .

(University of Sydney Seminar, May 16, 2008 )

- Decision-Making Under Severe Uncertainty: An Australian, Operational Research Perspective (PDF File) .

(ASOR National Conference, Melbourne, December 3-5, 2007 )

- A Critique of Info-Gap (PDF File) .

(SRA 2007 Conference, Hobart, August 20, 2007)

- What exactly is wrong with Info-Gap? A Decision Theoretic Perspective (PDF File) .

(MS Colloquium, University of Melbourne, August 1, 2007)

- A Formal Look at Info-Gap Theory (PDF File) .

(ORSUM Seminar , University of Melbourne, May 21, 2007)

The Art and Science of Decision-Making Under Severe Uncertainty(PDF File) .

(ACERA seminar, University of Melbourne, May 4, 2007)

- What exactly is Info-Gap? An OR perspective. (PDF File)

ASOR Recent Advances in Operations Researchmini-conference (December 1, 2006, Melbourne, Australia).

Disclaimer:This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.