Reviews of publications on Info-Gap decision theory
Review # 29 (Posted: June 22, 2011. Updated: February 3, 2012)
Hall, J., R. J. Lempert, K. Keller, A. Hackbarth, C. Mijere, and D. J. McInerney
Robust climate policies under uncertainty: A comparison of Info-‐Gap and RDM methods
in review at Risk Analysis
This study compares two approaches for robustness analysis of decision problems: the Info-‐gap method originally developed by Ben-Haim and the RDM (robust decision making) approach originally developed by Lempert, Popper, and Bankes. This study uses each approach to evaluate alternative paths for climate-‐altering greenhouse gas emissions given the potential for non-‐linear threshold responses in the climate system, significant uncertainty about such a threshold response and a variety of other key parameters, as well as the ability to learn about any threshold responses over time. Info-‐gap and RDM share many similarities. Both represent uncertainty with sets of multiple plausible representations of the future and seek to identify robust strategies whose performance is as far as possible insensitive to uncertainties. Yet they also exhibit important differences, as they arrange their analyses in different orders, treat losses and gains in different ways, and take different approaches to imprecise probabilistic information. This study finds that the two approaches reach similar but not identical policy recommendations and that their differing attributes raise important questions about their appropriate roles in decision support applications. The comparison not only improves understanding of these specific methods, it also suggests some broader insights into robustness approaches and a framework for comparing them.
This article is of particular interest to me because its first author is: Hall, J.
Let me explain.
For the benefit of readers who are not familiar with this collection of reviews, I should begin by pointing out the following. In the past two years I reviewed two articles (see Review 6 and Review 15) by Hall in which he advocated info-gap decision theory as a methodology for decision under severe uncertainty. In my reviews of these articles, I pointed out that apparently, in response to my sustained criticism of info-gap decision theory, Hall had made a brave attempt to introduce a fundamental fix in this theory, so as to correct a deep rooted flaw in it, in fact a flaw that renders info-gap decision theory a voodoo decision theory par excellence. Of course, the main objective of my reviews was to show that Hall's attempts at fixing info-gap decision theory's fundamental ills were themselves very problematic, so that rather than providing a remedy, they exacerbated the problem even more!
So, having now come across this new article, I was curious to find out whether my response to Hall's previous attempts to revamp info-gap decision theory would be reflected in the new article.
But, not only did I not find so much as a remote echo of my critique of Hall's failed attempts at "amending" info-gap decision theory. I did not find so much as an echo of his previous implied admission of the necessity to introduce some such correction in info-gap decision theory, and what is more of the necessity to state these corrections explicitly.This seems to suggest that in this article Hall has given up on his attempts to fix the theory, which means of course that this new article calls for harsher criticism than the criticism that was directed at the previous articles (see Review 6 and Review 15).
I want to assure all those readers who might find this story a bit complicated that it is in fact quite simple to follow. Simply read on!
However, before I can proceed to unravel the relevant details that will make sense of this story, it is important that I call attention to a no less important fact about this article. This has to do with what seems to be an attempt on the part of the authors to "repackage" info-gap decision theory, again I suspect in an effort to circumvent the censure that info-gap decision theory is so obviously vulnerable to! And by this I mean that, by using a clearly modified rhetoric, the authors are now seeking to dress info-gap decision theory in new clothes.So, contrary to what, to date, has been trumpeted in the entire info-gap literature (some of Hall's previous articles included) as info-gap decision theory's great forte, namely its capabilities as a method yielding robust decisions under severe uncertainty, in this article, is (deliberately?) being played down.
And so, in this article info-gap decision theory is no longer claimed to provide the decision-maker with a mechanism for ranking decisions! No, not at all!
According to the rhetoric in this article, all that info-gap decision theory furnishes the decision maker is some sort of general approach which gives him some (indeterminate?) counsel about the complicated business of decision under uncertainty. In a word, going by the rhetoric in this article, one would be hard pressed to recognize the info-gap decision theory described here as an "approach", as the methodology that in the entire Info-gap literature to date (some of Hall's previous articles included) is being hailed as a reliable methodology for identifying robust decisions, to be precise identifying those decisions that are (the most) robust to uncertainty.
The trouble is, however, that I am unable to give you a complete assessment of the so-called "Info-gap approach" outlined in this paper, because the authors did not bother to specify the details of the performance function. Consequently, it is impossible to validate the results reported on in this paper.
The missing Main Assumption
And now to the details of the story that is at the center of this review. I propose to unravel it in three stages. In the first I remind the reader of the fundamental flaws that render info-gap decision theory a voodoo decision theory. In the second, I explain how in previous articles, Hall had sought to find a way around some of these flaws. Following that I examine how the new article deals with this issue.
The fundamental flaw
For the benefit of readers who are not familiar with the "Info-gap decision theory story" I should point out that the reason that I have branded this theory: a voodoo decision theory par excellence is essentially due to its prescription for robust decision-making under severe uncertainty. In fact, you need not even be a risk analyst to immediately see that like any other "voodoo theory" this theory is of the "too good to be true" ilk. Which means of course that it is a theory whose groundless propositions can be easily exposed for what they are by means of simple examples and counter-examples.
Of particular interest to us in this discussion is info-gap decision theory's prescription for the treatment of severe uncertainty. This prescription basically instructs the following:Ignore the severity of the uncertainty.
Of course, not in so many words, still this is what this prescription comes down to.
And to appreciate my claim that you need not be a certified risk-analyst to immediately see that it is this prescription that renders this theory a voodoo decision theory par excellence, simply keep in mind that this prescription is given by a theory claiming to provide the means for tackling the severest uncertainty imaginable.
To begin with, the uncertainty is claimed to be non-probabilistic and likelihood-free, meaning that it cannot be quantified by means of "conventional" models of uncertainty or, by means of fuzziness. The quantification of the uncertainty is therefore austere in the extreme. Indeed, it comprises two elements:
- An uncertainty space, U.
- A point estimate û of the true value of the parameter of interest, call it u.
The immediate implication of the uncertainty being likelihood-free, is that there are no grounds to assume that the true value of u is more/less likely to be in the neighborhood of the estimate than in the neighborhood of any other value of u in U. And for similar reasons, the estimate must be assumed to be poor to the effect that it may well be substantially wrong indeed, no more than a wild guess of the true value of u.
And to illustrate, suppose that the uncertainty space U is this page. This means that, as the uncertainty is likelihood-free, we have not a clue where the true value of u is on this page. All we know is that it is somewhere on this page.
And even if we were to assume, for argument's sake, that the point estimate of the true value of u is exactly in the middle of this page. Given that the uncertainty is likelihood-free, this additional stipulation would not alter even by one iota the basic fact that we have no clue where the true value of u is in the page.
The implication is then that there are no grounds whatsoever, to assume that the true value of u is more/less likely to be in the neighborhood of the estimate than in the neighborhood of any other value of u in U.
And all this leads to the inevitable conclusion that to determine the robustness of a decision to uncertainty where the uncertainty is described in these terms, it is imperative to evaluate the performance of the decision over the uncertainty space U.
Yet, this is not what info-gap decision theory prescribes doing!
To the contrary, info-gap decision theory prescribes conducting a robustness analysis only in the neighborhood of this (highly questionable, doubtful etc.) estimate and nowhere else. And so, info-gap decision theory's prescription for measuring robustness against severe uncertainty is based on the following question:how much can this (poor) estimate be perturbed (in all directions) without causing a violation of the performance requirement?
Thus, if the answer is for instance 12cm, then any perturbation of size 12cm or less (in any direction) will not violate the performance requirement and will therefore be deemed "acceptable". Whereas, a slightly larger perturbation (in some direction) will be deemed unacceptable as it will cause a violation of the performance requirement.
This definition and means of measuring robustness is known universally as Radius of Stability.
For obvious reasons, the Radius of Stability is treated universally (eg. in the robust control theory literature, economics literature, etc.) as a model of local robustness. This means of course that the accepted convention is that as a measure of local robustness it cannot be counted on to provide the global robustness of a system a decision, or whatever. In other word, it cannot be counted on to indicate how well, or how poorly, a decision performs over the entire uncertainty space U. Indeed, it is elementary to devise examples demonstrating that a decision that is locally robust in the neighborhood of the estimate is very fragile globally over U, and vice versa.
In short, all this goes to show that, as a radius of stability model, info-gap's robustness model is the wrong model for the treatment of a severe uncertainty of the type that info-gap decision theory is claimed to address.
It is extremely important to note in this regard that info-gap decision theory in fact has the dubious distinction of being the only decision theory in the trade to propose that the robustness of a decision against severe uncertainty be measured by a Radius of stability model.
So, to repeat, you need not be a risk analyst to see that a theory purporting to offer a reliable methodology for robust decision to uncertainty proposing this prescription for this purpose, is fundamentally flawed.
Hall definitely saw it and this is the reason for the fixes that he proposed in his recent articles (see Review 6 and Review 15).
In response to my criticism of info-gap decision theory, Hall and Harvey (2009) have hit on an easy quick fix.
Although their paper is riddled with TUIGF, my concern here is only with the following remarkable statement made immediately after the description of info-gap's regions of uncertainty, the horizon of uncertainty α and the estimate û of the parameter of interest:An assumption remains that values of u become increasingly unlikely as they diverge from û.
Hall and Harvey (2009, p. 12 , emphasis added)
In other words, this assumption indicates that the estimate û is the most likely value of the parameter u and that the likelihood of u decreases as u deviates from û. As I pointed out in my review, it was clear that Hall and Harvey's rationale for introducing this assumption was to justify Info-Gap's fixing on the estimate as the focus of the robustness analysis.
What I particularly wanted readers to take note of was the phrasing of this assumption. I therefore, called readers attention to the word remains by raising the following (rhetorical) question:
"What exactly are we to make of remain? Does it mean that in the context of Info-Gap, which boasts of being a non-probabilistic likelihood-free theory, this assumption was all along the case and it thus remains? If so how does it square with the "official" Info-Gap decision theory? Or, is this a "new" assumption? One that the authors decided to append to the "official" theory? If it is a newly added assumption then surely this must be made clear. Whatever the case, the authors must explain how this assumption tallies with the claim that the uncertainty under consideration is severe? "
More on this in Review 6
A second attempt was made by Hine and Hall (2010), where a different assumption was introduced to get rid of, or mitigate, the above mentioned absurd in info-gap decision theory's prescription for the management of severe uncertainty. So, on page 2-3 we read the following:The main assumption is that u, albeit uncertain, will to some extent be clustered around some central estimate û in the way described by U(α,û), though the size of the cluster (the horizon of uncertainty α) is unknown. In other words, there is no known or meaningfully bounded worst case. Specification of the info gap uncertainty model U(α,û) may be based upon current observations or best future projections.
Hine and Hall (2010, pp.2-3, emphasis added)
Although this phrasing is an improvement on Hall and Harvey's (2009) quick-fix, just like its predecessor, this assumption amounts to a major indictment of info-gap decision theory. And, strictly speaking, it is misleading.
Let me explain.
- What the authors refer to here as the "main assumption", thereby creating the impression that this assumption is implicit in info-gap decision theory, is not in fact an info-gap decision theory assumption. It is not an integral part of info-gap decision theory -- as the theory is described in Ben-Haim (1996, 2001, 2006, 2010). There is no trace of it in any of these publications.
- This "main" assumption is an innovation introduced ad hoc by the authors in order to justify info-gap's absurd proposition to focus the analysis on a wild guess.
- Interestingly, there is no trace of this "main" assumption in Hall's earlier publications on info-gap decision theory.
A detailed critique of the errors, misconceptions, misinformation etc. demonstrated in the wording of this assumption is available in Review 15.
The question is then: where is this "main Assumption" in Hall et al. (2011)?
A repackaging of info-gap decision theory
But the really intriguing question that this article by Hall et al. (2011) raises is why the "sudden need" to repackage info-gap decision theory? In other words, the question that the authors must answer is this: how does their claim that info-gap decision theory does not provide a strict ranking of decisions square with the standard depictions of info-gap decision theory in all info-gap publications to date?
To enable readers who are not familiar with this literature, to see for themselves, I juxtapose Hall et al's. (2011) statement with a sample of statements --- illustrating the standard depiction of this theory --- by other info-gap scholars, including the Father of the theory (Ben-Haim) and ... Jim Hall himself!
Hall et al. 2011 Other info-gap scholars Neither Info-gap nor RDM provide a strict ranking of alternative decisions. Rather, both provide decision support, summarizing tradeoffs for decision makers to help inform their judgments about the robustness of alternative decision options.
Hall et al. (2011, p. 2)
The best decision is then chosen as the one that is most robust to uncertainty, i.e. is guaranteed to give acceptable outcomes under the greatest degree of uncertainty.
Halpern, Regan, Possingham and McCarthy (2006, p. 3)
The best decision is the one that is most robust to uncertainty, by guaranteeing an acceptable outcome under the greatest degree of uncertainty.
McCarthy and Lindenmayer (2007, p. 554)
The robustness can be evaluated even though there is no known worst case. Furthermore, the robustness function generates preferences on the decisions, q: a decision which is more robust for achieving aspiration rc is preferred over a decision which is less robust. Robust-satisficing decision making maximizes the robustness and satisfices the reward at the value rc, without specifying a limit on the level of uncertainty:
where Q is the set of available decisions.
Davidovitch and Ben-Haim (2010, p. 268)
The robustness function generates a preference ordering on the available decisions: a more robust decision is preferred over a less robust decision. Satisficing means doing well enough, or obtaining an adequate outcome. A satisficing decision strategy seeks a decision whose outcome is good enough, though perhaps sub-optimal. A robust-satisficing decision strategy maximizes the robustness to uncertainty and satisfices the outcome.
Schwartz, Ben-Haim and Dasco (2011, p. 213)
We now have a general mathematical formulation of the problem at hand including a model, which incorporates uncertainties in the preliminary data, and a method to choose the best decision.
Sisso, Shema and Ben-Haim (2010, p. 1035)
As we have noted before, this means that ``bigger is better'' for the robustness function. Consequently, a decision maker will usually prefer decision option q over an alternative decision q' if the robustness of q is greater than the robustness of q' at the same value of critical reward rc.
Let Q be the set of all available or feasible decision vectors q. A robust-satisficing decision is one which maximizes the robustness over the set Q of available q-vectors and satisfices the performance at at the critical level rc.Ben-Haim (2006, p. 45)
Info-gap analysis allows the decision maker to identify solutions that perform satisfactorily well under the widest possible range of conditions. This is a departure from the conventional approach to decision making, that seeks to maximise performance, but under conditions of severe uncertainty, a guaranteed level of performance (up to some horizon of uncertainty) may well be more attractive than optimal performance that is vulnerable to the unexpected.
Hall and Ben-Haim (2009, p. 7)
In this way the robustness function generates preferences among available decisions. When choosing between two options, the robust-satisficing decision strategy selects the more robust option.
Ben-Haim (2010, p. 8)
Surely, this blatant distortion by Hall et al. (2011) of what info-gap claims to be cannot be an accident?!
The question is: what is its object?
Because to date, info-gap decision theory has been presented as a theory that ranks decisions according to their robustness: the larger the better. Hence, according to (what is known to the public as) info-gap theory, the best decision is that whose robustness is the largest (for the desired level of performance, rc).
Are we to conclude from this paper that this is no longer the case?
I suspect that this distortion may well be another attempt at a quick fix, aimed at getting around info-gap decision theory's fundamental ills. And this, by the way, is the norm in the info-gap literature: a repackaging of the rhetoric as a means of glossing over the fundamental ills of the theory!
Ben-Haim, the Father of info-gap decision theory, is crystal clear about how his theory establishes preferences among decisions. In the 2006 edition of his book on the theory we read:For given values of critical or windfall reward, rc or rw, each immunity function induces a preference ranking on the set of available decisions. More importantly, the immunity functions enable the decision maker to explore the desirability of different options q and different requirements rc and rw , and thus to alter earlier preferences.
Ben-Haim (2006, p. 45)
And in the 2010 book we read:Goals which are satisficed (sub-optimal but good enough) can be achieved by many alternative policies. Choose the most robust from among these alternatives.
Ben-Haim (2010, p. 10)
In short, info-gap decision theory gives specific instructions on how to rank alternative decisions, policies, options, etc: choose the most robust!
Can there be any doubt about that?
So, the message is crystal clear, leaving no room for the imagination: contrary to the authors' claim, info-gap decision theory provides ranking of alternative decisions.
It should therefore be most interesting to hear from Ben-Haim himself about this attempt to revamp his theory.
So, we shall have to wait and see.
- The point raised by Hall et al. (2011) about the "tradeoff" issue has no bearing whatsoever on the fundamental fact that, as dictated by info-gap decision theory, for any given value of the critical reward rc, the best decision is a decision that maximizes the robustness function (for the specified value of rc). And it is on the basis of this determination that the "tradeoff" analysis is then conducted.
- The troubles besetting info-gap's robustness analysis have got nothing to do with this "tradeoff". The trouble is due to the fact that info-gap's robustness model is a a model of local robustness which is being misapplied for the management of severe uncertainty. More on this in the Official Mobile Debunker of info-gap decision theory.
The missing performance (reward) function
On Fri, 10 June 2011 16:42:20 +1000 (EST), I requested from one of the co-authors details on the performance function of the info-gap robustness model because this function is not specified for the numerical examples presented in the article. The request was acknowledged and was forwarded to another author.
I was informed on Wed, 13 July 2011 11:17:16 +0100, by the first author, that the reward function is not specified in the paper. The rewards are generated by a computer program (MLK DICE) that is reported on elsewhere.
I find this most surprising because the uncertain parameter under consideration (call it u) takes only 2662 values, namely the uncertainty space is discrete and contains 2662 distinct values of u. This means that for each of the four strategies under consideration, there are only 2662 rewards. The implication is then that, the reward function can be easily "specified" (online) by a relatively small spreadsheet to enable interested users to easily download and examine it.
But more than this, in view of the fact that the uncertainty space is discrete and manifestly small, determining robustness is trivial --- it can easily be done by enumeration. Indeed, given the size of this discrete uncertainty space, it is hard to comprehend why the authors conduct no more than a local robustness analysis! For their money, they could have easily performed a global analysis (over the entire uncertainty space) to come up with far more meaningful results!!!!
Regrettably, the authors do not provide such a spreadsheet so that it is impossible to check/reconstruct their results.
It is interesting to note the following reference, in the article under review, to the DICE-07 model (page 4):For this study, we use a modified version of DICE07 (McInerney, Lempert, and Keller (2009), henceforth “MLK”), that adds the possibility of a large-scale and economically costly collapse of the North Atlantic Meridional Overturning Circulation (MOC) triggered if and when atmospheric CO2 levels exceed an uncertain threshold (Keller et al. 2004).where the bibliographic details of "MLK" are as follows:McInerney, D., R. Lempert and K. Keller (2009). "What are robust strategies in the face of uncertain climate threshold responses?" Climatic Change, in revision, available at http://www.geosc.psu.edu/~kzk10/publications.html
I found a copy of this paper at http://www.geosc.psu.edu/~kzk10/McInerney_et_al_CC_09.pdf, but I could not find in it the details that are required to evaluate the reward function in question, in the article under review.
In short, I cannot see how the results reported on in the article under review, can possibly be reconstructed/checked/examined.
Summary and conclusions
The uncertainty space of the robustness model under consideration in this article consists of 2662 values of a parameter comprising 4 components. Since only four strategies are considered in this case, the robustness issue is trivial to the extent that it can easily be handled by enumeration.
This means that both methodologically and practically, it is unclear why a model of local robustness is used here to begin with.
Furthermore, as the authors do not specify the reward function, it is impossible to check the results generated by the proposed local robustness model so as to determine whether these results are consistent with the strategies’ global robustness over the uncertainty space.
Finally, there are no signs, in this article, of Hall's previous attempts to revamp info-gap decision theory (see Review 6 and Review 15).
- Ben-Haim (2001, 2006): Info-Gap Decision Theory: decisions under severe uncertainty.
- Regan et al (2005): Robust decision-making under severe uncertainty for conservation management.
- Moilanen et al (2006): Planning for robust reserve networks using uncertainty analysis.
- Burgman (2008): Shakespeare, Wald and decision making under severe uncertainty.
- Ben-Haim and Demertzis (2008): Confidence in monetary policy.
- Hall and Harvey (2009): Decision making under severe uncertainty for flood risk management: a case study of info-gap robustness analysis.
- Ben-Haim (2009): Info-gap forecasting and the advantage of sub-optimal models.
- Yokomizo et al (2009): Managing the impact of invasive species: the value of knowing the density-impact curve.
- Davidovitch et al (2009): Info-gap theory and robust design of surveillance for invasive species: The case study of Barrow Island.
- Ben-Haim et al (2009): Do we know how to set decision thresholds for diabetes?
- Beresford and Thompson (2009): An info-gap approach to managing portfolios of assets with uncertain returns
- Ben-Haim, Dacso, Carrasco, and Rajan (2009): Heterogeneous uncertainties in cholesterol management
- Rout, Thompson, and McCarthy (2009): Robust decisions for declaring eradication of invasive species
- Ben-Haim (2010): Info-Gap Economics: An Operational Introduction
- Hine and Hall (2010): Information gap analysis of flood model uncertainties and regional frequency analysis
- Ben-Haim (2010): Interpreting Null Results from Measurements with Uncertain Correlations: An Info-Gap Approach
- Wintle et al. (2010): Allocating monitoring effort in the face of unknown unknowns
- Moffitt et al. (2010): Securing the Border from Invasives: Robust Inspections under Severe Uncertainty
- Yemshanov et al. (2010): Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest
- Davidovitch and Ben-Haim (2010): Robust satisficing voting: why are uncertain voters biased towards sincerity?
- Schwartz et al. (2010): What Makes a Good Decision? Robust Satisficing as a Normative Standard of Rational Decision Making
- Arkadeb Ghosal et al. (2010): Computing Robustness of FlexRay Schedules to Uncertainties in Design Parameters
- Hemez et al. (2002): Info-gap robustness for the correlation of tests and simulations of a non-linear transient
- Hemez et al. (2003): Applying information-gap reasoning to the predictive accuracy assessment of transient dynamics simulations
- Hemez, F.M. and Ben-Haim, Y. (2004): Info-gap robustness for the correlation of tests and simulations of a non-linear transient
- Ben-Haim, Y. (2007): Frequently asked questions about info-gap decision theory
- Sprenger, J. (2011): The Precautionary Approach and the Role of Scientists in Environmental Decision-Making
- Sprenger, J. (2011): Precaution with the Precautionary Principle: How does it help in making decisions
- Hall et al. (2011): Robust climate policies under uncertainty: A comparison of Info--Gap and RDM methods
- Ben-Haim and Cogan (2011) : Linear bounds on an uncertain non-linear oscillator: an info-gap approach
- Van der Burg and Tyre (2011) : Integrating info-gap decision theory with robust population management: a case study using the Mountain Plover
- Hildebrandt and Knoke (2011) : Investment decisions under uncertainty --- A methodological review on forest science studies.
- Wintle et al. (2011) : Ecological-economic optimization of biodiversity conservation under climate change.
- Ranger et al. (2011) : Adaptation in the UK: a decision-making process.
Also, see my complete list of articles
Moshe's new book!
- Sniedovich, M. (2012) Fooled by local robustness, Risk Analysis, Early View.
- Sniedovich, M. (2012) Black swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty, International Transactions in Operational Research, 19(1-2), 253-281 (Available free of charge)
- Sniedovich, M. (2011) A classic decision theoretic perspective on worst-case analysis, Applications of Mathematics, 56(5), 499-509.
- Sniedovich, M. (2011) Dynamic programming: introductory concepts, in Wiley Encyclopedia of Operations Research and Management Science (EORMS), Wiley.
- Caserta, M., Voss, S., Sniedovich, M. (2011) Applying the corridor method to a blocks relocation problem, OR Spectrum, 33(4), 815-929, 2011.
- Sniedovich, M. (2011) Dynamic Programming: Foundations and Principles, Second Edition, Taylor & Francis.
- Sniedovich, M. (2010) A bird's view of Info-Gap decision theory, Journal of Risk Finance, 11(3), 268-283.
- Sniedovich M. (2009) Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.
- Sniedovich M. (2009) A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds), Safety, Reliability and Risk Analysis: Theory, Methods and Applications, pp. 2071-2079, Taylor and Francis Group, London..
- Sniedovich M. (2009) A Classical Decision Theoretic Perspective on Worst-Case Analysis, Working Paper No. MS-03-09, Department of Mathematics and Statistics, The University of Melbourne.(PDF File)
- Caserta, M., Voss, S., Sniedovich, M. (2008) The corridor method - A general solution concept with application to the blocks relocation problem. In: A. Bruzzone, F. Longo, Y. Merkuriev, G. Mirabelli and M.A. Piera (eds.), 11th International Workshop on Harbour, Maritime and Multimodal Logistics Modeling and Simulation, DIPTEM, Genova, 89-94.
- Sniedovich, M. (2008) FAQS about Info-Gap Decision Theory, Working Paper No. MS-12-08, Department of Mathematics and Statistics, The University of Melbourne, (PDF File)
- Sniedovich, M. (2008) A Call for the Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia (PDF File)
- Sniedovich, M. (2008) Info-Gap decision theory and the small applied world of environmental decision-making, Working Paper No. MS-11-08
This is a response to comments made by Mark Burgman on my criticism of Info-Gap (PDF file )
- Sniedovich, M. (2008) A call for the reassessment of Info-Gap decision theory, Decision Point, 24, 10.
- Sniedovich, M. (2008) From Shakespeare to Wald: modeling wors-case analysis in the face of severe uncertainty, Decision Point, 22, 8-9.
- Sniedovich, M. (2008) Wald's Maximin model: a treasure in disguise!, Journal of Risk Finance, 9(3), 287-291.
- Sniedovich, M. (2008) Anatomy of a Misguided Maximin formulation of Info-Gap's Robustness Model (PDF File)
In this paper I explain, again, the misconceptions that Info-Gap proponents seem to have regarding the relationship between Info-Gap's robustness model and Wald's Maximin model.
- Sniedovich. M. (2008) The Mighty Maximin! (PDF File)
This paper is dedicated to the modeling aspects of Maximin and robust optimization.
- Sniedovich, M. (2007) The art and science of modeling decision-making under severe uncertainty, Decision Making in Manufacturing and Services, 1-2, 111-136. (PDF File) .
- Sniedovich, M. (2007) Crystal-Clear Answers to Two FAQs about Info-Gap (PDF File)
In this paper I examine the two fundamental flaws in Info-Gap decision theory, and the flawed attempts to shrug off my criticism of Info-Gap decision theory.
- My reply (PDF File) to Ben-Haim's response to one of my papers. (April 22, 2007)
This is an exciting development!
- Ben-Haim's response confirms my assessment of Info-Gap. It is clear that Info-Gap is fundamentally flawed and therefore unsuitable for decision-making under severe uncertainty.
- Ben-Haim is not familiar with the fundamental concept point estimate. He does not realize that a function can be a point estimate of another function.
So when you read my papers make sure that you do not misinterpret the notion point estimate. The phrase "A is a point estimate of B" simply means that A is an element of the same topological space that B belongs to. Thus, if B is say a probability density function and A is a point estimate of B, then A is a probability density function belonging to the same (assumed) set (family) of probability density functions.
Ben-Haim mistakenly assumes that a point estimate is a point in a Euclidean space and therefore a point estimate cannot be say a function. This is incredible!
- A formal proof that Info-Gap is Wald's Maximin Principle in disguise. (December 31, 2006)
This is a very short article entitled Eureka! Info-Gap is Worst Case (maximin) in Disguise! (PDF File)
It shows that Info-Gap is not a new theory but rather a simple instance of Wald's famous Maximin Principle dating back to 1945, which in turn goes back to von Neumann's work on Maximin problems in the context of Game Theory (1928).
- A proof that Info-Gap's uncertainty model is fundamentally flawed. (December 31, 2006)
This is a very short article entitled The Fundamental Flaw in Info-Gap's Uncertainty Model (PDF File) .
It shows that because Info-Gap deploys a single point estimate under severe uncertainty, there is no reason to believe that the solutions it generates are likely to be robust.
- A math-free explanation of the flaw in Info-Gap. ( December 31, 2006)
This is a very short article entitled The GAP in Info-Gap (PDF File) .
It is a math-free version of the paper above. Read it if you are allergic to math.
- A long essay entitled What's Wrong with Info-Gap? An Operations Research Perspective (PDF File) (December 31, 2006).
This is a paper that I presented at the ASOR Recent Advances in Operations Research (PDF File) mini-conference (December 1, 2006, Melbourne, Australia).
If your organization is promoting Info-Gap, I suggest that you invite me for a seminar at your place. I promise to deliver a lively, informative, entertaining and convincing presentation explaining why it is not a good idea to use — let alone promote — Info-Gap as a decision-making tool.
Here is a list of relevant lectures/seminars on this topic that I gave in the last two years.
- ASOR Recent Advances, 2011, Melbourne, Australia, November 16 2011. Presentation: The Power of the (peer-reviewed) Word. (PDF file).
AIRO 2011, Brescia, Italy, September 6 -- 9, 2011. Presentation: Fooled by robustness: the role of local and global robustness in risk analysis.
- IFORS 2011, Melbourne, Australia, July 10 -- 15, 2011. Presentation: Fooled by robustness: a perspective from the Land of the Black Swan.
- Alex Rubinov Memorial Lecture The Art, Science, and Joy of (mathematical) Decision-Making, November 7, 2011, The University of Ballarat. (PDF file).
- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .
(Invited tutorial, ALIO/INFORMS Conference, Buenos Aires, Argentina, July 6-9, 2010).
- A Critique of Info-Gap Decision theory: From Voodoo Decision-Making to Voodoo Economics(PDF File) .
(Recent Advances in OR, RMIT, Melbourne, Australia, November 25, 2009)
- Robust decision-making in the face of severe uncertainty(PDF File) .
(GRIPS, Tokyo, Japan, October 16, 2009)
- Decision-making in the face of severe uncertainty(PDF File) .
(KORDS'09 Conference, Vilnius, Lithuania, September 30 -- OCtober 3, 2009)
- Modeling robustness against severe uncertainty (PDF File) .
(SOR'09 Conference, Nova Gorica, Slovenia, September 23-25, 2009)
- How do you recognize a Voodoo decision theory?(PDF File) .
(School of Mathematical and Geospatial Sciences, RMIT, June 26, 2009).
- Black Swans, Modern Nostradamuses, Voodoo Decision Theories, Info-Gaps, and the Science of Decision-Making in the Face of Severe Uncertainty (PDF File) .
(Department of Econometrics and Business Statistics, Monash University, May 8, 2009).
- The Rise and Rise of Voodoo Decision Theory.
ASOR Recent Advances, Deakin University, November 26, 2008. This presentation was based on the pages on my website (voodoo.moshe-online.com).
- Responsible Decision-Making in the face of Severe Uncertainty (PDF File) .
(Singapore Management University, Singapore, September 29, 2008)
- A Critique of Info-Gap's Robustness Model (PDF File) .
(ESREL/SRA 2008 Conference, Valencia, Spain, September 22-25, 2008)
- Robust Decision-Making in the Face of Severe Uncertainty (PDF File) .
(Technion, Haifa, Israel, September 15, 2008)
- The Art and Science of Robust Decision-Making (PDF File) .
(AIRO 2008 Conference, Ischia, Italy, September 8-11, 2008 )
- The Fundamental Flaws in Info-Gap Decision Theory (PDF File) .
(CSIRO, Canberra, July 9, 2008 )
- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .
(OR Conference, ADFA, Canberra, July 7-8, 2008 )
- Responsible Decision-Making in the Face of Severe Uncertainty (PDF File) .
(University of Sydney Seminar, May 16, 2008 )
- Decision-Making Under Severe Uncertainty: An Australian, Operational Research Perspective (PDF File) .
(ASOR National Conference, Melbourne, December 3-5, 2007 )
- A Critique of Info-Gap (PDF File) .
(SRA 2007 Conference, Hobart, August 20, 2007)
- What exactly is wrong with Info-Gap? A Decision Theoretic Perspective (PDF File) .
(MS Colloquium, University of Melbourne, August 1, 2007)
- A Formal Look at Info-Gap Theory (PDF File) .
(ORSUM Seminar , University of Melbourne, May 21, 2007)
- The Art and Science of Decision-Making Under Severe Uncertainty (PDF File) .
(ACERA seminar, University of Melbourne, May 4, 2007)
- What exactly is Info-Gap? An OR perspective. (PDF File)
ASOR Recent Advances in Operations Research mini-conference (December 1, 2006, Melbourne, Australia).
Disclaimer: This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.