The Spin Stops Here!
Decision-Making Under Severe Uncertainty  
Faqs | Help | @ | Contact | home  
voodoostan info-gap decision theory info-gap economics severe uncertainty mighty maximin robust decisions responsible decisions

The Illusionary Gap

Preface

This page examines the short note: Information-gap decision theory creates a gap in ecological applications and then fills it by Professor David Fox, which was posted on the website of Environmetrics Australia on May 14, 2014, henceforth Fox (2014).

Both the title of this page, namely The Illusionary Gap, and the title of Fox's (2014) note, refer to a Letter to the Editor by Burgman and Regan (2014) entitled Information-gap decision theory fills a gap in ecological applications which was published recently in the journal Ecological Applications. Fox (2014) labels the gap referred to by Burgman and Regan (2014) illusionary, but he does not elaborate a justification for this labeling, nor does he discuss my response to Burgman and Regan (2014) on this matter.

So, before I can continue, I need to point out that in my response to Burgman and Regan (2014), entitled The elephant in the rhetoric on info-gap decision theory (Sniedovich 2014), I explain in detail why the gap referred to by Burgman and Regan (2014) is spurious, indeed non-existent, furthermore, that it was created by Burgman and Regan (2014) for the sole purpose of filling it with info-gap decision theory. The story about this non-existent gap is elaborated in the long appendix to Sniedovich (2014) entitled A perspective on a nonexistent gap in the state of the art, which is available free of charge at

http://www.esapubs.org/archive/appl/A024/013/appendix-A.pdf.

In brief.

As I argue in Sniedovich (2014), the gap in the state of the art (in applied ecology), purportedly identified by Burgman and Regan (2014), is in fact a non-existent gap:

Information-gap decision theory's (IGDT) central concept is "info-gap robustness". This concept is a reinvention of the well-established concept radius of stability (circa 1960). Furthermore, IGDT's robustness model is a very simple maximin model (circa 1940). It follows therefore that there is nothing whatsoever that IGDT does that cannot possibly be done, indeed is being done for decades, by radius of stability models, let alone maximin models, which, needless to say, do a great deal more than IGDT's robustness model! In a word, there is no gap in the state of the art that IGDT does fill, or can possibly fill. Indeed, the gap created by Burgman and Regan's (2014) was designed to serve one purpose only: to justify the use of IGDT in ecological applications.

Furthermore, in Sniedovich (2014) I also reiterate the central point that I expound elsewhere (e.g. Sniedovich 2010, 2012, 2012a, 2014), which is that IGDT is not just a reinvention of the important, intuitive, concept radius of stability, but a fundamentally flawed reinvention thereof. The reason that IGDT is fundamentally flawed is that it advocates a misapplication of radius of stability models by virtue of prescribing their use in situations that are subject to a severe uncertainty of the type stipulated by IGDT. The point is that radius of stability models, which are by definition inherently local in nature, are incompatible with the severity of the uncertainty postulated by IGDT.

This latter point is eloquently explained in Hayes et al. (2013, p. 609) as follows:

Plausibility is being evoked within IGDT in an ad hoc manner, and it is incompatible with the theory's core premise, hence any subsequent claims about the wisdom of a particular analysis have no logical foundation. It is therefore difficult to see how they could survive significant scrutiny in real-world problems. In addition, cluttering the discussion of uncertainty analysis techniques with ad hoc methods should be resisted.

Now, while Fox's (2014) note is most certainly a welcome contribution to the ongoing discussion on IGDT, it falls far short of identifying the real issues surrounding this theory. For this reason, I comment on a number of statements made in Fox's (2014) note, with the view to yet again, call attention to my formal, rigorous analysis of IGDT that followers of IGDT would be well-advised to consult, especially before they take it upon themselves to comment on it.

Remark. The maximin (minimax) models referred to in this discussion are Wald-Type models (Wald 1939, 1945, 1950), which, it ought to be pointed out, differ from the maximin (minimax) models of Game Theory that are designed to formulate 2-person, zero-sum games. In Wald-Type models the 'decision maker' plays against an antagonistic opponent ('Nature') whose sole interest is to harm the `decision maker' to the greatest possible degree. In this framework, the 'decision maker plays first, whereupon 'Nature' responds to the decision maker's decision/move.

Table of Contents

Overview

Fox's (2014) note (see also Fox 2008) joins a discussion on IGDT and its role and place in the state of the art (particularly in ecological applications), that was initiated, at the end of 2006, in my Info-Gap campaign. I took this unorthodox step when I realized that there were plans afoot to "sell" this flawed theory to the scientific and business communities in Australia.

Regrettably, Fox's (2014) note is not only a little too late, it is too little.

The most problematic point about this note is that it views IGDT as a new paradigm whose principal shortcoming is in its being untried and untested relative to more traditional methods. This view of IGDT clearly ignores the fact that this supposedly new paradigm is actually a reinvention of the well-known radius of stability paradigm which has been staple fare in various disciplines, at least since the 1960s.

And to top of it all off, Fox (2014) claims that there is nothing fundamentally wrong with IGDT. This claim clearly attests to a lack of appreciation of the difference between local and global robustness, hence of the fact that the type of uncertainty postulated by IGDT must be tackled by a model of global robustness. But, as IGDT's robustness model is a model of local robustness, it clearly lacks the capabilities to do the job.

It is also rather odd that Fox's (2014) analysis is completely oblivious to the assessment of IGDT documented in the most comprehensive peer-reviewed examination of IGDT ecological applications to-date, entitled Severe uncertainty and info-gap decision theory, by Keith Hayes, Simon Barry, Geoffrey Hosack and Gareth Peters (2013).

As for its position regarding my criticism of IGDT, Fox's (2014) note seems to adopt the convenient shoot the messenger strategy that proponents of IGDT have resorted to, from the start, in order to avoid dealing with the substance of my technical arguments that expose IGDT for what it is: a reinvention of the wheel, and a square one at that!.

And to begin, let us first examine the preamble to Fox (2014).

The Preamble

The preamble to Fox (2014) requires attention both for what it says and for what it does not say. For the reader's convenience I quote its content here, as it was posted on the website of Environmetrics Australia on 8:08AM, May 27, 2014:

Information-gap decision theory creates a gap in ecological applications and then fills it
May 14, 2014

You may not of heard of Info Gap Decision Theory (IGDT) but don't worry, not many people have.

While the theoretical foundations of IGDT have been well developed and articulated by its architect Yakov Ben-Haim at the Israel Institute of Technology, controversy continues to surround its legitimacy as a credible alternative to existing methodologies.

The issue has again resurfaced with the publication of a letter to the Editor of Ecological Applications by Professor Mark Burgman and Dr. Helen Regan arguing that IGDT is both useful and credible.

Professor David Fox, a one-time IGDT follower, weighs into the the debate. His views are expressed below.

I would agree with Professor David Fox that, relatively speaking, not many people heard about IGDT. And to illustrate, a search for "Information-gap decision theory'' on Google Scholar generates 530 links, whereas a search for "stability radius" generates 2560 links, and a search for "minimax" generates 136,000 links. To reiterate, the connection between "info-gap decision theory", "stability radius" and "minimax" is as follows: the concept "info-gap robustness" (circa 2000) is a reinvention of the concept "stability radius" (circa 1960). Also, "stability radius" models are (very simple) maximin models (circa 1940). Hence, IGDT robustness models are very simple maximin models.

But if this is so,

Good questions!

I address these questions below. At this stage, I want to point out that even if it is the case that the number of those engaged with IGDT is small, the fact remains that, in Australia, IGDT had gained considerable popularity especially among applied ecologists and conservation biologists. Indeed, it was promoted by major research centres. Furthermore, looking at this matter from a purely "academic" and "scholarly" point of view, should remind us how important it is to remain alive to the fact that things can go wrong even under the watchful eyes of well-established peer-reviewed journals.

As for IGDT's legitimacy as a credible alternative to existing methodologies, alluded to by Professor David Fox, it should be stressed that the issue is in fact far more fundamental. The real question regarding IGDT is not whether IGDT is a credible alternative to existing methodologies. Rather, the real question is whether IGDT is an alternative at all! Namely:

The real question
As a decision theory, what new ideas, concepts, models, techniques, and so on, does IGDT actually contribute to the state of the art?
And the answer is crystal clear:
Answer
None!
The concept "info-gap robustness" (circa 2000) is a reinvention of the well-established concept "stability radius" (circa 1960). What is more, robustness models of this type are incompatible with the severity of the uncertainty postulated by IGDT.

I should point out that I am not aware of any controversy surrounding the Answer.

The controversy surrounding IGDT is about a different matter altogether, namely

Controversy*
Although conceding that "IGDT's robustness model" is a radius of stability model, and a simple maximin model, some scholars, e.g. Burgman and Regan (2014), argue that for all that, IGDT nevertheless does fill a gap in ecological applications.

In contrast, Sniedovich (2014) points out that such arguments are without any merit because: given that "info-gap robustness" (circa 2000) is a reinvention of the well-established concept radius of stability (circa 1960), and given that IGDT's robustness model (circa 2000) is a very simple maximin model (circa 1940), there cannot possibly be a gap in the state of the art for IGDT to fill.

*That said I wish to point out that one can hardly speak of a controversy over this issue, because the fact of the matter is that there is no ``real'' controversy here. Proponents of IGDT do not put forward formal arguments to refute my formal, rigorous proof that IGDT's robustness model is a radius of stability model etc. All they do is bandy about statements that are aimed to contradict my position. For example, Burgman and Regan (2014) do not show that although IGDT's robustness model is a radius of stability model, hence also a very simple maximin model, IGDT does nevertheless fill a gap in the state of the art.

In short, unsubstantiated claims versus formal rigorous arguments can hardly count as a controversy. More on this below.

The Note

Fox's (2014) verdict about the gap referred to by Burgman and Regan (2014) is expressed very clearly in the opening paragraph of the note. It reads as follows (color added):

In their recent letter to Ecological Applications, Burgman and Regan (2014) provide counter arguments to some of Sniedovich's (2012) severe, and mostly harsh criticisms of Ben-Haim's info-gap decision theory (IGDT) (Ben-Haim, 2006). While I have a deep respect for Professor Burgman and Dr. Regan, I believe their unwavering faith in info-gap theory is misplaced. As the title of this note suggests, I agree with Sniedovich (2014) that 'the gap' referred to by Burgman and Regan (2014) is illusionary.

Now, although Fox's (2014) claim that this "gap" is illusionary is perfectly valid, the claim itself is not explained, much less is it substantiated in Fox's (2014) analysis. The only clue one has as to why Fox (2014) might claim that the gap is illusionary is in the following statement:

As noted by Burgman and Regan (2014) there already exists a plethora of 'conventional' tools to deal with uncertainty and, unlike IGDT these come `certified' by virtue of their long history of use and acceptance by the broad scientific community. Outside the isolated pockets of support for IGDT, the theory remains largely unknown. Certainly within statistical circles, no one I've spoken to has heard of Ben-Haim or IGDT.
Fox (2014)

But the whole point is that by claiming that there is a plethora of existing methods one does not thereby prove that no gap exists, or in other words, that the gap is "illusionary". Indeed, a plethora of 'conventional' tools to deal with uncertainty may well be available, but a gap might still exist. It is only when one shows formally and rigorously, as done in Sniedovich (2010, 2011, 2012, 2012a, 2014), that the concept "info-gap robustness" is a reinvention of the concept "radius of stability" (circa 1960), what is more that IGDT's robustness model is a simple instance of generic maximin models (circa 1940), that one shows that there is no gap that IGDT can possibly fill. Because, what this argument effectively demonstrates is that, whatever the IGDT paradigm can do, the "radius of stability" has been doing for decades. Therefore, whatever the IGDT paradigm does the maximin paradigm has been doing, and a great deal more, for decades.

However, as Fox's (2014) analysis is completely oblivious to the fact that IGDT's central concept "info-gap robustness" is a reinvention of the well-established concept "radius of stability" (circa 1960), it is unaware that IGDT's robustness model very much "comes 'certified' by virtue of [a] long history of use and acceptance by a broad scientific community." Because, IGDT's robustness model, which is known universally as "radius of stability", or "stability radius", or "robustness radius", has been in use (to perform the task for which it was envisioned: measure small perturbations in a parameter) in a wide cross-section of disciplines at least since the 1960s.

The implication therefore is that the trouble with IGDT is not, as suggested in Fox (2014), in its robustness model being new, and largely unknown, therefore untested by the broad scientific community.

Rather, the trouble with IGDT lies elsewhere. The trouble with IGDT is in its prescribing this well-known model of local robustness to tackle a severe uncertainty whose treatment requires models of global robustness. It is this incongruity between the robustness model proposed by IGDT and the severity of the uncertainty postulated by the theory that makes IGDT fundamentally flawed.

This fundamental flaw is eloquently described as follows in the most comprehensive study of ecological applications of IGDT to-date, namely the article Severe Uncertainty and Info-Gap Decision Theory by Keith Hayes, Simon Barry, Geoffrey Hosack and Gareth Peters (2013, p. 609):

Plausibility is being evoked within IGDT in an ad hoc manner, and it is incompatible with the theory's core premise, hence any subsequent claims about the wisdom of a particular analysis have no logical foundation. It is therefore difficult to see how they could survive significant scrutiny in real-world problems. In addition, cluttering the discussion of uncertainty analysis techniques with ad hoc methods should be resisted.

The inference therefore is that Fox's (2014) analysis attests to a lack of familiarity with the reasoning justifying the claim that IGDT is fundamentally flawed and its lacking a logical foundation. Indeed, it is odd that Fox's (2014) analysis does not mention, let alone discuss, Hayes et al.'s (2013) comprehensive assessment of ecological applications of IGDT.

Incidentally, the same applies to the recent article entitled Decision science for effective management of populations subject to stochasticity and imperfect knowledge by Hiroyuki Yokomizo, Shaun R. Coutts, and Hugh P. Possingham (2014), that was published as a Special Feature: Review in the journal Population Ecology. There is no reference in Yokomizo et al. (2014) to the fact that "info-gap robustness" is a reinvention of the well established concept radius of stability and that IGDT's robustness model is a very simple maximin model. As a matter of fact, there is no reference at all to the maximin paradigm in Yokomizo et al.'s (2014) survey.

It should also be pointed out that this survey, in which IGDT represents theories dealing with non-probabilistic uncertainty, does not mention its obvious, and no so obvious "limitations" and their implications. And finally, it is surprising that such a survey does not mention the most comprehensive study to-date of ecological applications of IGDT, i.e. Hayes et al. (2014)!

It is even more surprising that neither Burgman and Regan (2014), nor Yokomizo et al. (2014), nor Fox (2014) take to heart Hayes et al.'s (2013) clear warning about the incompatibility of IGDT's treatment of uncertainty/plausibility with the theory's core premise and the consequences of this incompatibility (Note: these issues are also discussed in Sniedovich 2010, 2011, 2012, 2012a, 2014 and Hayes 2011).

About the Campaign

As Fox's (2014) note seems to allude to my info-gap campaign (2006-2013), I ought to point out that details about it can be found on the project's website. Here is a short list of links to some of the sub projects.

Also, considering the general tenor of Fox's (2014) note, I think it apropos to point out the following about my "info-gap experience" over the past 11 years.

While it did not take much effort on my part to identify and describe the problematic issues afflicting IGDT, a considerable effort had to be expended (and this remains the case), to bring senior scholars/analysts around to acknowledge the inevitable conclusions deriving from my formal, rigorous assessment of IGDT. This difficulty seems to be due, among other things, to the fact that IGDT was embraced, without sufficient scrutiny, by a number of research centers in Australia to be promoted by them as a new tool for the treatment of severe uncertainty.

And to illustrate, consider the following quote from the A guide to AEDA's Info sheets (2007, p. 5; color added):

AEDA is also supporting the education of interested parties in the use of cutting edge spatial prioritisation tools (such as MARXAN) and decision support methodologies (such as Bayesian analysis and info gap analysis) and training the next generation of Australia's theoretical ecologists.

Details about the Applied Environmental Decision Analysis (AEDA) research facility can be found here.

One imagines that it is rather difficult to accept that a theory that was being advanced as offering cutting edge tools, turns out to be a reinvention of a well-established tool, and that the theory's implementation of this tool is fundamentally flawed.

The publication of the CSIRO/ACERA report Uncertainty and Uncertainty Analysis Methods (Hayes 2011), followed by the publication of the seminal peer-reviewed article Severe Uncertainty and Info-gap Decision Theory by Keith Hayes, Simon Barry, Geoffrey Hosack and Gareth Peters (2013), where my criticism of IGDT is properly acknowledged and the conclusions emanating from it are drawn and eloquently articulated, marked a turning point in my campaign.

Fox's (2014) note gives some hope that scholars/analysts who still adhere to IGDT might still reach the inevitable conclusions regarding IGDT.

Obviously, one wouldn't be able to say the same about those IGDT adherents in Australia (see above) who continue to ignore not only Sniedovich (2007, 2008, 2009, 2010, 2011, 2012, 2012a, 2014), but also Hayes (2011) and Hayes et al. (2013).

Shooting the messenger

I particularly wish to call attention to the reference in Fox's (2014) note to Burgman and Regan's (2014) claim that

To label the application of information-gap decision theory in ecology and conservation biology as "voodoo decision making" is disingenuous, unscientific and unhelpful.
Burgman and Regan (2014, p. 223)

which it interprets to imply that my criticism of IGDT amounts to playing the man not the ball.

In a word, without attempting even the slightest explanation as to what in my technically backed-up criticism of IGDT (e.g. Sneidovich 2007, 2008, 2009, 2010, 2011, 2012, 2012a, 2014) justifies this claim, Fox's (2014) note expresses approval of the view that my criticism of IGDT amounts to playing the man not the ball.

I therefore wish to make it abundantly clear that, although my criticism of IGDT is severe, and the language in which this criticism is sometimes coached, is "uncompromising", it cannot by any stretch of the imagination be characterized as playing the man not the ball.

I imagine that my criticism of IGDT appears to some as (unduly) harsh due to:

The fact of the matter is, however, that much as my criticism is harsh, and justifiably so, it does not amount to a "personal" criticism of scholars/analysts who advocate and/or promote the use of IGDT.

Don't shoot the messenger:
You might miss the message
You might miss the message
You might miss the message
You might miss the message
You might miss the message
You might miss the message
You might miss the message
You might miss the message
And to illustrate, by indicating that Ben-Haim (2010, 2012) continues to propound the same erroneous claims about IGDT and Maximin, I am not "playing the man", I am merely reporting on the state of affairs in the IGDT literature, specifically that certain articles, e.g. Ben-Haim (2010, 2012), continue to propound the same technical errors. The fact that a criticism (harsh or otherwise) is directed at a theory and its literature, does not constitutes a "personal" criticism of the Founder of the theory, even if he/she is strongly identified with it. The implication is that by showing that the IGDT ball is flawed necessarily implies that those who kick around the IGDT ball, or advocate its use, are kicking around a ... flawed ball. But this does not amount to "playing the men/women" who kick around the IGDT ball.

Similarly, by showing that Wintle et al.'s (2010) claim that IGDT provides a suitable framework for handling "Unknown Unknowns" is absurd, I do not "play the man". I merely explain that the arguments advanced in the article written by Wintle et al. (2010) are woefully misguided.

Which brings me to my use of the term voodoo decision-making to label the advocacy of IGDT's methodology as especially suitable for the treatment of a severe uncertainty of the type stipulated by IGDT. The point is this: as explained in detail in Sniedovich (2014), the claim that IGDT's methodology is particularly suitable for situations characterized by an unbounded uncertainty, a point estimate that is a wild guess, and a likelihood-free quantification of uncertainty, fully deserve the label voodoo decision making, where, as explained in Sniedovich (2014, p. 231), the term voodoo implies (according to The American Heritage Dictionary of the English Language, Fourth Edition, 2000): "Based on unrealistic or delusive assumptions: voodoo economics."

In other words, I use the term "voodoo" precisely in the same manner as it is used in phrases such as "voodoo science", "voodoo economics", "voodoo mathematics", "voodoo statistics", "voodoo accounting", and so on.

It seems rather obvious that the routine use of these phrases in the broad scientific literature attests to the fact that they are not regarded as "disingenuous", or "unscientific", or "unhelpful".

An example of my usage of the term voodoo decision-making is illustrated in my description of Wintle et al.'s (2010) proposition that IGDT is suitable for the treatment of "Unknown Unknowns". Or, in my description of the following proposition:

Information-gap (henceforth termed 'info-gap') theory was invented to assist decision-making when there are substantial knowledge gaps and when probabilistic models of uncertainty are unreliable (Ben-Haim 2006). In general terms, info-gap theory seeks decisions that are most likely to achieve a minimally acceptable (satisfactory) outcome in the face of uncertainty, termed robust satisficing.
Burgman et al. (2008, p. 8)

Because:

No wonder, therefore, that based on a comprehensive analysis of ecological applications of IGDT, Hayes et al. (2013) conclude the following:

Plausibility is being evoked within IGDT in an ad hoc manner, and it is incompatible with the theory's core premise, hence any subsequent claims about the wisdom of a particular analysis have no logical foundation. It is therefore difficult to see how they could survive significant scrutiny in real-world problems. In addition, cluttering the discussion of uncertainty analysis techniques with ad hoc methods should be resisted.
Hayes et al. (2013, p. 609)

This deservedly harsh criticism of IGDT does not amount to "playing the man not the ball".


Westminister Consulting
I submit, therefore, that rather than my criticism of IGDT being a case of "playing the man and not the ball", the attitude of IGDT proponents to my criticism is in fact a case of shooting the messenger!

I suggest, therefore, that instead they .... keep their eyes, and minds, on my criticism of the IGDT ball, for they will discover that my harsh criticism of IGDT is not only fully justified, it is instructive, constructive, and fair.

I therefore take this opportunity to challenge Professor David Fox to substantiate his claim that my criticism of IGDT amounts to "playing the man and not the ball".

Reinventing the wheel

Don't reinvent it!
Improve it!

Fox's (2014) (mistaken) claim that as a new methodology IGDT lacks the credentials of traditional methods, also affords me the opportunity to elaborate somewhat on my labeling IGDT a "reinvention of the wheel".

I wish to make it clear then that my constant harping on this phrase to characterize the IGDT methodology was, from the start, intended to make the following statements about this theory. First, to underscore that far from being new, IGDT's central concept/model, namely "info-gap robustness", is in fact a replication of a staple concept/model that is being used for decades in various disciplines. Second, to bring home that the latter comes laden with a wealth of knowledge and expertise.

To my puzzlement, many followers of IGDT seemed to be totally unperturbed by this fact, and continued to treat the IGDT methodology as a new approach for tackling problems (especially in applied ecology) that are subject to a severe uncertainty, as indeed attested by the string of articles on IGDT and its application that continued to be published, especially in applied ecology journals.

The question is then so what? Why fret about the fact that scholars/analysts refer to, or treat, a methodology as "new" when it is not new? Does it make any difference to the manner in which they use this methodology?

Keeping in mind Fox's (2014) position on this matter, my point is that we most certainly need to concern ourselves with these issues, not only because established academic traditions demand it! But, because of the ramification that may ensue. And to illustrate consider this:

  • Hypothetical Situation:
    Suppose that to tackle a new project, you consider using Methodology A, which is which is advanced and promoted as revolutionary and radically different from existing methodologies. As an alternative, you consider using Methodology B, which is well established as attested by its vast knowledge-base (literature, software, etc.).

I submit that in this case it might be vitally important to know whether Methodology A is indeed new. For, it is easy to envision a scenario where opting for Methodology A might be detrimental to your project:

Worst-case scenario of opting for Methodology A:
Unbeknownst to you, Methodology A is in fact a replica of Methodology C, which is well-established, as attested by its vast knowledge-base (literature, software, etc.).
However, it turns out that this methodology is utterly unsuitable for your project.

As indicated above, Fox's analysis (2014) does indirectly allude to this possibility through the claim that as a new methodology, IGDT has not gone through the grind of traditional methods and therefore remains unaccredited.

The point is, though, that while Fox's (2014) assertion that "... Outside the isolated pockets of support for IGDT, the theory remains largely unknown..." may well be true, it is nevertheless (unwittingly) misleading because:

A search on Google Scholar generates 177 links for "info-gap robustness", 2560 links for "stability radius", 342 links for "radius of stability", and 160 links for "robustness radius".

The implication is that a lack of awareness of the fact that, far from being a new concept, "info-gap robustness" is no more than a reinvention of what is universally known as "stability radius", "radius of stability", "robustness radius", and so on, may insulate one from the vast literature on "stability radius" and may well result in bad science.

Because to reiterate: radius of stability models are by definition inherently local in nature. They are therefore incompatible with the severity of the uncertainty postulated by IGDT.

And by the same token, Hayes et al.'s (2013) comprehensive study of ecological application of IGDT should also lead one to the conclusion that enough knowledge about IGDT's methodology is available to indicate that it is unsuitable for the treatment of the severe uncertainty it postulates:

Plausibility is being evoked within IGDT in an ad hoc manner, and it is incompatible with the theory's core premise, hence any subsequent claims about the wisdom of a particular analysis have no logical foundation. It is therefore difficult to see how they could survive significant scrutiny in real-world problems. In addition, cluttering the discussion of uncertainty analysis techniques with ad hoc methods should be resisted.
Hayes et al. (2013, p. 609)

Conclusions

The analysis in Fox (2014) addresses a number of issues about IGDT that most definitely require pointing out. However, as it remains oblivious to the substance of the formal, rigorous analysis of IGDT in Sniedovich (2007, 2010, 2011, 2012, 2012a, 2014), and to that in Hayes al. (2013), which has particularly important implications for the ecological applications of IGDT, it errs with regard to the real issues that are detrimental to this theory.

To be precise, because Fox's (2014) note is oblivious to the fact that the concept "info-gap robustness" is a reinvention of the well-established concept "radius of stability" (circa 1960), whose credentials have been proved decades ago, it mistakenly proposes that the trouble with IGDT is that it is new, hence in need of testing, verification, certification, and so on.

The fact of the matter is of course that far from being an untested methodology (relative to other established methodologies), as suggested in Fox (2014), the mathematical models which IGDT employs have been staple fare in various disciplines for more than half a century. They are therefore well-known, well-understood, easy to represent graphically, tried and tested both theoretically and practically and are being implemented routinely in various areas of science, in engineering, economics, management etc. so that the literature about them is vast.

Of course, the IGDT's literature (both its core texts and the articles deriving from them) is totally oblivious to this body of literature by dint of its being oblivious to the fact that IGDT's robustness model is a radius of stability model (circa 1960), and as such, a simple maximin model (circa 1940). As a result, readers of the IGDT literature, especially followers of the theory who are not conversant with decision theory, optimization theory, control theory, etc., are practically cut off from a huge body of knowledge and expertise that has accrued for more than half a century, about the methodology that, in IGDT circles, is viewed as "info-gap decision theory".

All this goes to show that, contrary to the claims in Fox's (2014) note, the trouble with IGDT is not that it is "new", hence untested (relative to other traditional theories). The real trouble with it is that its robustness model, which by definition is a model of local robustness, is (mis)applied to situations calling for models of global robustness. Thus IGDT's central proposition is akin to a proposition to administer a local anaesthetic in a cases requiring a global anaesthetic.

Postscript

Having issued The Third and Final Call for the Reassessment of the Use and Promotion of IGDT in Australia (May 14, 2013), I am reluctant to issue yet another, post-final, Call for the Reassessment.

Instead, I take this opportunity to issue a

First Call for the Assessment of The Lessons Learnt from the Info-Gap Experience in Australia (2003-2014)

Such an assessment should prove especially beneficial to scholars/analysts who continue to hold that IGDT's robustness model is new, hence in need of accreditation, as well as to those who do not appreciate the implications of the radius of stability concept and therefore the maximin paradigm for IGDT's robustness model.

It should also prove beneficial to those who do not appreciate the distinction between local and global robustness and its implications of a correct assessment of IGDT and its ecological applications.

And it should definitely prove beneficial to scholars/analysts who are in search for a non-existent gap in the state of the art for IGDT to fill.

Anyone interested in taking part in such an Assessment, can contact me via e-mail.

Moshe Sniedovich

Bibliography

  1. Bedow, B. 1977. The ten natural laws of operations analysis. Interfaces 7(3):122.
  2. Ben-Haim, Y. 2001. Information Gap Decision Theory. Academic Press, Amsterdam, The Netherlands.
  3. Ben-Haim, Y. 2006. Info-Gap Decision Theory. Academic Press, Amsterdam, The Netherlands.
  4. Ben-Haim, Y. 2010. Info-gap Economics: An Operational Introduction. Palgrave, New York, USA.
  5. Ben-Haim, Y. 2010a. Frequently asked questions about info-gap. URL http:// www.technion.ac.il/yakov/IGT/faqs01.pdf .
  6. Ben-Haim, Y. 2012. Doing Our Best: Optimization and the Management of Risk. Risk Analysis 32(8):1326-1332.
  7. Ben-Haim, Y. 2012a. Why Risk Analysis is Difficult, and Some Thoughts on How to Proceed. Risk Analysis 32(10):1638-1646.
  8. Ben-Haim, Y. 2012b. Modeling and design of a Hertzian contact: an info-gap approach. The Journal of Strain Analysis for Engineering Design 47(2):153-162.
  9. Ben-Haim, Y (2014). Robust satisfying and probability of survival. {\it International Journal of Systems Science 45(1):3-19.
  10. Ben-Tal, A., Golany, B., and Shtern, S. 2009. Robust multi-echelon multi-period inventory control. European Journal of Operational Research 199:922-935.
  11. Burgman, M.A., Wintle, B.A., Thompson, C.A., Moilanen, A., Runge, M.C. and Ben-Haim, Y. 2010. Reconciling uncertain costs and benefits in Bayes nets for invasive species management. Risk Analysis 30:277-284.
  12. Burgman, M.A., Wintle, B.A., Thompson, C.A., Moilanen, A., Runge, M.C. and Ben-Haim, Y. 2008. Reconciling uncertain costs and benefits in Bayes nets for invasive species management. Final Report, Project 0601-0611, Australian Centre of Excellence for Risk Analysis. URL http://www.acera.unimelb.edu.au/materials/endorsed/0601_0611.pdf .
  13. Burgman, M.A., and Regan, H.M. 2014. Information-gap decision theory fills a gap in ecological applications. Ecological Applications 24(1):227-228.
  14. Carmel, Y., and Ben-Haim, Y. 2005. Info-gap robust-satisficing model of foraging behavior: do foragers optimize or satisfice? The American Naturalist 166:633-641.
  15. Fox, D.R. 2008. To IG or not to IG?-- that is the question. Decision Point 24:10-11.
  16. Fox, D. 2014. Information-gap decision theory creates a gap in ecological applications and then fills it. URL: http://www.environmetrics.net.au/index.php?news&nid=73
  17. French, S.D. 1988. Decision Theory. Ellis Horwood, NY.
  18. Halpern, B.S., Regan, H.M., Possingham, H.P. and McCarthy, M.A. 2006. Accounting for uncertainty in marine reserve design. Ecology Letters 9:2-11.
  19. Hayes, K.R. 2011. Uncertainty and Uncertainty Analysis Methods. Final report for the Australian Centre of Excellence for Risk Analysis (ACERA), CSIRO Division of Mathematics, Informatics and Statistics, Hobart, Australia, 130 pp.
  20. Hayes, K.R., Barry, S.C., Hosack, G.R., and Peters, G.W. 2014. Severe uncertainty and info-gap decision theory. Methods in Ecology and Evolution 4:601-611.
  21. Hillson, D. 2007. `Unknown-Unknowns' and risk. Project Management Today 19(12):22-27.
  22. Hodgson, D.J., and Townley, S. 2004. Linking management changes to population dynamic responses: the transfer function of a projection matrix perturbation. Journal of Applied Ecology 41:1155-1161.
  23. Milne, W.E., and Reynolds, R.R. 1962. Fifth-order methods for the numerical solution of ordinary differential equations. Journal of the ACM 9(1):64-70.
  24. Nicholson, E., and Possingham, H.P. 2007. Making conservation decisions under uncertainty for the persistence of multiple species. Ecological Applications 17:251-265.
  25. Peterson, M. 2009. An Introduction to Decision Theory. Cambridge University Press.
  26. Rawls, J. 1971. Theory of Justice. Belknap Press, Cambridge, MA.
  27. Regan, H.H., Ben-Haim, Y., Langford, B., Wilson, W.G., Lundberg, P., Andelman, S.J. and Burgman, M.A. 2005. Robust decision-making under severe uncertainty for conservation management. Ecological Applications 15:1471-1477.
  28. Resnik, M.D. 1987. Choices: an Introduction to Decision Theory. University of Minnesota Press, Minneapolis, MN.
  29. Sniedovich, M. 2007. The art and science of modeling decision-making under severe uncertainty. Decision Making in Manufacturing and Services 1(1-2):111-136.
  30. Sniedovich, M. 2008. From Shakespeare to Wald: modeling worst-case analysis in the face of severe uncertainty. Decision Point 22:8-9.
  31. Sniedovich, M. 2008a. Wald's Maximin Model: A Treasure in Disguise! Journal of Risk Finance 9(3):278-291.
  32. Sniedovich M. 2009. A Critique of Info-Gap Robustness Model. In: Martorell et al. (eds), Safety, Reliability and Risk Analysis: Theory, Methods and Applications, pp. 2071-2079, Taylor and Francis Group, London, 2009.
  33. Sniedovich M. 2009a. Modeling of robustness against severe uncertainty, pp. 33- 42, Proceedings of the 10th International Symposium on Operational Research, SOR'09, Nova Gorica, Slovenia, September 23-25, 2009.
  34. Sniedovich, M. 2010. A bird's view of info-gap decision theory. Journal of Risk Finance 11(3):268-283.
  35. Sniedovich, M. 2011. Info-gap decision theory: a perspective from the Land of the Black Swan. URL: http://info-gap.moshe-online.com/acera.html
  36. Sniedovich, M. 2011a. A classic decision theoretic perspective on worst-case analysis, Applications of Mathematics 56(5):499-509, 2011.
  37. Sniedovich, M. 2012. Fooled by local robustness: an applied ecology perspective. Ecological Applications 22(5):1421-1427.
  38. Sniedovich, M. 2012a. Black Swans, new Nostradamuses, voodoo decision theories and the science of decision-making in the face of severe uncertainty. International Transactions in Operational Research 19(1):253-281.
  39. Sniedovich, M. 2012b. Fooled by Local Robustness. Risk Analysis 32(10):1630-1637.
  40. Sniedovich, M. 2013. From local robustness to irresponsible decision-making, Proceedings of the BALCOR 2013 Conference, September 7-9, 2013, Belgrade, Serbia, pp. 695-704.
  41. Sniedovich, M. 2013a. The many faces of worst-case analysis. Proceedings of the SYM-OP-IS 2013 Conference, September 9-11 2013, Belgrade, Serbia, pp. 887-882.
  42. Sniedovich, M. 2014. The elephant in the rhetoric on info-gap decision theory. Ecological Applications 24(1):229-233.
  43. Wald, A. 1939. Contributions to the theory of statistical estimation and testing hypotheses. Annals of Mathematical Statistics 10(4):299-326.
  44. Wald, A. 1945. Statistical decision functions which minimize the maximum risk. The Annals of Mathematics 46(2):265-280.
  45. Wald, A. 1947. An essentially complete class of admissible decision functions. The Annals of Mathematical Statistics 18:549-555.
  46. Wald, A. 1950. Statistical Decision Functions. John Wiley, NY.
  47. Wilf, H.S. 1960. Maximally stable numerical integration. Journal of the Society for Industrial and Applied Mathematics 8(3):537-540.
  48. Wintle, B.A. 2010. On the look out for black swans: monitoring for unknown unknowns. Decision Point 43:4-7.
  49. Wintle, B.A., Runge, M.C., and Bekessy, S.A. 2010. Allocating monitoring effort in the face of unknown unknowns. Ecology Letters 13(11):1325-1337.
  50. Yokomizo, H., Coutts, S.R., and Possingham, H.P. 2014. Decision science for effective management of populations subject to stochasticity and imperfect knowledge. Population Ecology 56:41-53.
  51. Zlobec, S. 1987. Survey of input optimization. Optimization 18:309-348.

Disclaimer: This page, its contents and style, are the responsibility of the author (Moshe Sniedovich) and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.


Last modified: