The Spin Stops Here!
Decision-Making Under Severe Uncertainty  
Faqs | Help | @ | Contact | home  
voodoostan info-gap decision theory info-gap economics severe uncertainty mighty maximin robust decisions responsible decisions

Second Call for The Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia

In November 2008, I issued a Call for The Reassessment of the Use and Promotion of Info-Gap Decision Theory in Australia. This was connected to the campaign that I had launched at the end of 2006 to contain the spread of info-gap decision theory in Australia.

This First Call for Reassessment reflected my position at the time (November 2008) that it was important that info-gap adherents face up to the sharp differences between these two conflicting evaluations of info-gap decision theory:

At the time (November 2008) I argued that what was needed was not yet another workshop on Info-Gap Applications, but rather a forum that would examine the question: What Exactly is Amiss With Info-Gap Decision Theory and Why It Is Important to be Fully Informed on This.

This was very much in order in November 2008, and it remains so at present.

Since I have been active on this front for more than seven years now, I am fully aware of the difficulties involved in getting info-gap users to face up to the profound disparity between the rhetoric describing info-gap's alleged role, mode of operation, capabilities, etc. and the hard facts attesting to what this theory actually is and does.

But ... these facts are there for all to see so that they must be accepted.

This Second Call for Reassessment was prompted by the recent publication of the following CSIRO report:

Hayes KR (2011). Uncertainty and Uncertainty Analysis Methods.
Final report for the Australian Centre of Excellence for Risk Analysis (ACERA),
CSIRO Division of Mathematics, Informatics and Statistics, Hobart, Australia,
130 pp.

Read my discussion on this report.

Still, I want to call attention to one of the points made in this report about info-gap decision theory (IGT), so as to shed more light on my decision to issue this second call. Consider then the following statement (emphasis added):

Analysts who were attracted to IGT because they are very uncertain, and hence reluctant to specify a probability distribution for a model's parameters, may be disappointed to find that they need to specify the plausibility of possible parameter values in order to identify a robust management strategy.
Hayes (2011, p. 88)

The point to note here is that, Hayes (2011) reiterates a fact about info-gap decision theory that I had brought to light a long time ago and have been arguing since, which is: that info-gap decision theory is utterly unsuitable for the treatment of a severe uncertainty of the type (severity) that it claims to address (see footnote). Because, as Hayes (2011) observes, while on the face of it, info-gap decision theory claims to enable analysts to handle situations that are subject to severe uncertainty (non-probabilistic and likelihood free), when it comes to implementing it, all this comes to naught. For, contrary to these claims, unless one quantifies the plausibility/likelihood of the values of the parameter(s) in question, one would be unable to apply info-gap's prescription for dealing with severe uncertainty. In a word, info-gap's recipe for dealing with severe uncertainty is to ... ignore the severity of the uncertainty altogether.

Much as I admire Keith's use of language here, I am afraid that for the benefit of all those who might contemplate turning to info-gap, and for the benefit of info-gap scholars in particular, it is imperative to elaborate the issues involved and to make explicit the messages hidden in the above diplomatically phrased statement.

Here then is the bottom line:

The good old days of ALCHEMY are long gone!

It is generally accepted nowadays that creating something out of nothing is well-nigh impossible.

This means that if a model of uncertainty is claimed to be probability-free, likelihood-free, plausibility-free, and so on, then the results yielded by this model are by necessity ... probability-free, likelihood-free, plausibility-free, and so on. Namely, no amount of rhetoric will possibly change this fact to induce the model to yield results that are not likelihood-free and so on.

But this universally accepted fundamental does not apply in info-gap decision theory. Thus, while a big fuss is made in the info-gap literature about info-gap's model of uncertainty being probability-free, likelihood-free, plausibility-free, info-gap's prescription for the management of uncertainty flies in the face of this fundamental stipulation.

This means that info-gap's recipe for the management of severe uncertainty is in fact ALCHEMY par excellence, but this fact is obscured from view by spin and rhetoric.

In greater detail, the rhetoric in info-gap publications describing the model and the results yielded by this model is utterly incongruous with what this model actually is and does, indeed what this model is capable of doing.

The implication is then that misleading rhetoric obscures from view that info-gap decision theory is utterly unsuitable for the treatment of severe uncertainty of the type (severity) that it claims to address.

As I can afford to be less diplomatic than Keith, I shall spell out the fundamental trouble in info-gap decision theory more clearly, indeed more bluntly, as follows:

Info-gap decision theory cannot possibly deliver on what it claims to deliver.

This is so because info-gap decision theory is proclaimed to be a non-probabilistic and likelihood-free decision theory. But to justify the application of its prescription for the management of severe uncertainty, one must impose a specific plausibility/likelihood structure on the uncertainty space.

That is, one cannot even begin to implement the info-gap methodology without first of all giving the uncertainty space a specific plausibility/likelihood structure. Because, short of such a stipulation, one would be unable to justify info-gap's prescription to focus the entire robustness analysis on a single (poor) point estimate and its neighborhood. In other words, short of such a stipulation, one would be unable to justify the inherently local nature of info-gap's notion of robustness.

Surely, it does not take a risk analyst to immediately see how utterly at odds this prescription is with the proclaimed objective of the theory, which is the treatment/management of a severe uncertainty that is quantified in a non-probabilistic, likelihood/plausibility free fashion.

But, more than this, it is important to appreciate that this contradiction is inescapable, because unless one imposes this specific plausibility/likelihood structure on the uncertainty space, applying the info-gap decision methodology would make no sense at all!

And the upshot of all this is clear: given this foundational contradiction, which lies at the very heart of this theory, the info-gap methodology cannot possibly be suitable for the treatment/management of severe uncertainty of the type (severity) it claims to manage.

That said, I should point out that even ardent supporters of info-gap decision theory now "accept" my formal proofs that info-gap's robustness model is neither new nor radically different (as claimed by Ben-Haim 2001, 2006, 2010). Namely, even ardent supporters of info-gap decision theory now concede that info-gap's generic model is a simple instance of Wald's famous Maximin model (circa 1940).

So, one may well ask: why issue a Second Call for Reassessment?

My reply to this is that the publication of the CSIRO report, which discusses --- albeit not in great depth and in a highly diplomatic language --- some of the flaws afflicting info-gap decision theory, lends support to my position that it is high time to reassess the use and promotion of this theory in Australia. For one thing, it is important to show (through such a reassessment) that the large number of (Australian) publications, in peer-reviewed journals, describing info-gap applications in applied ecology and conservation biology, is no testimony to this theory's merit.

Indeed, it is important to show (through such a reassessment) that the errors and misconceptions propounded by this fundamentally flawed theory can have unsavory consequences. Witness for instance, a recent Australian publication in a peer-reviewed journal, proposing info-gap decision theory as a means for tackling Black Swans and Unknown Unknowns in sound environmental management (see Review 17).

Clearly then, such a reassessment is long overdue.

Finally, given ACERA's involvement in the promotion of info-gap decision theory in Australia over the past seven years, it is only natural to expect ACERA to play a central role is such a reassessment.

Moshe Sniedovich
Melbourne
The Land of the Black Swan
July 25, 2011

Note:

Some info-gap scholars conveniently argue that it is pointless to debate whether info-gap decision theory is suitable for the management of severe uncertainty. Because, so they claim, there is no clear cut, universally accepted definition of the term "severe uncertainty".

This is a rather odd proposition, because as these scholars are well aware, info-gap decision theory, as attested by its founder, was designed specifically for the management of a profoundly severe uncertainty. This fact is repeatedly underscored in the primary texts on info-gap decision theory, and in many articles discussing its application by statements asserting that info-gap decision theory is designed to deal with "Knightian uncertainty", "True Knightian uncertainty" and so on. Indeed, the subtitle of the first two books on the theory (Ben-Haim 2001, 2006) is "decisions under severe uncertainty"?

But what is more, if the meaning of "severe uncertainty" is indeed shrouded in such ambiguity, on what grounds do these scholars advocate the theory as a reliable tool for the management of "severe uncertainty"?

The point is then that these scholars apparently prefer to conveniently overlook the fact that the main texts on the theory, (Ben-Haim 2001, 2006, 2010), and the many articles discussing its application, do indeed make clear the meaning that "severe uncertainty" has within the framework of info-gap decision theory.

Thus, the fact is that the meaning that info-gap decision theory (Ben-Haim 2001, 2006, 2010) ascribes to the term severe uncertainty is given by three properties, expressed through these working assumptions:

It is important to note with regard to the first assumption that, according to Ben-Haim (2001, 2006), " ... most of the commonly encountered info-gap models are unbounded ...".

The third assumption entails that there are no grounds to assume/believe that the true value of the parameter is more/less likely to be in the neighborhood of a given point in the uncertainty space, say u', rather than in the neighborhood of any other given point, say u". Thus, there are no grounds to assume/believe that the true value of the parameter is more/less likely to be in the neighborhood of the given point estimate, rather than in the neighborhood of any other point in the uncertainty space.

Clearly, one need not be a risk analyst to see that this characterization of severe uncertainty implies that there are no grounds to assume/believe that the robustness of decisions to variations in the value of the parameter, in the neighborhood of the point estimate, is indicative of their robustness to variations in the value of the parameter over the entire vast uncertainty space. This means that, methodologically, a robustness analysis in the neighborhood of the point estimate does not/cannot meet the challenges encountered in the pursuit of robustness against variations in the value of the parameter over its vast (e.g. unbounded) uncertainty space.

However, since info-gap's robustness model measures the robustness of decisions against small perturbations in the value of the point estimate, the inference is that, methodologically, info-gap decision theory is utterly unsuitable for the treatment of severe uncertainty of the type (severity) that it claims to address. In fact, info-gap decision theory's prescription for "dealing" with the severe uncertainty that it stipulates is to ... ignore the severity of the uncertainty.

Remark:
I should also point out, especially for the benefit of info-gap scholars who mistakenly believe that info-gap decision theory provides a framework for tackling Unknown Unknowns (see Review 17), that in the info-gap framework, the uncertainty space and the point estimate of the parameter of interest are given. In other words, the values of these two objects are assumed to be known to the analyst at the outset.

Disclaimer: This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.


Last modified: