| Reference: | HELEN M. REGAN, YAKOV BEN-HAIM, BILL LANGFORD, WILLIAM G. WILSON, PER LUNDBERG,
SANDY J. ANDELMAN, AND MARK A. BURGMAN Robust decision-making under severe uncertainty for conservation management. Ecological Applications, 15(4), 1471-1477, 2005. |
| Abstract | In conservation biology it is necessary to make management decisions for endangered and threatened species under severe uncertainty. Failure to acknowledge and treat uncertainty can lead to poor decisions. To illustrate the importance of considering uncertainty, we reanalyze a decision problem for the Sumatran rhino, Dicerorhinus sumatrensis, using information-gap theory to propagate uncertainties and to rank management options. Rather than requiring information about the extent of parameter uncertainty at the outset, information-gap theory addresses the question of how much uncertainty can be tolerated before our decision would change. It assesses the robustness of decisions in the face of severe uncertainty. We show that different management decisions may result when uncertainty in utilities and probabilities are considered in decision-making problems. We highlight the importance of a full assessment of uncertainty in conservation management decisions to avoid, as much as possible, undesirable outcomes. |
| Scores | TUIGF:100% SNHNSNDN:200% GIGO:100% |
| Acknowledgement: | We thank Dan Berleant, Mark Colyvan, Martin Drechsler, John Harwood, Tom Hobbs, and an anonymous reviewer for useful discussions and comments that assisted in the preparation of this paper. This work was conducted as part of the Setting priorities and making decisions for conservation risk management Working Group supported by the National Center for Ecological Analysis and Synthesis, a Center funded by NSF (Grant #DEB-0072909), the University of California, and the Santa Barbara campus. |
This is a relatively old paper. Still, it is on my list because, to the best of my knowledge, it is the first publication discussing the application of Info-Gap in applied ecology. So, it may shed some light on how the uncritical use of Info-Gap decision theory evolved in this discipline.
The rather high SNHNSNDN score reflects the authors' distorted portrayal of the tools provided by classical decision theory for the treatment of severe uncertainty. Consider this (page 1471, emphasis is mine):
For decision-making under uncertainty, the usual procedure is to assign probabilities to each of the relevant states and utilities to each of the outcomes. The approach usually taken is to maximize expected utility.
Clearly, the implication is that classical decision theory has no other means to offer for the treatment of "unusual" non-probabilistic cases. Or, that using other procedures for this purpose is not common practice in classical decision theory, or something to this effect. The inference, therefore, is that given classical decision theory's manifest shortcomings, one turns to Info-Gap decision theory to fill in the gap!
But the point is that other than making this sweeping statement the authors do not bother to show this to be the case. This is unfortunate because a quick look at one of the main references cited by the authors, namely Resnik (1987), reveals that Chapter 2 is entitled Decisions Under Ignorance. It is divided into the following sections:
Chapter 2: Decisions Under Ignorance
- 2-1: Preference ordering
- 2-2: The Maximin Rule
- 2-3: The Optimism-Pessimism Rule
- 2-4: The Principle of Insufficient Reason
- 2-5: Too many Rules?
- 2-6: An Application in Social Philosophy:
Rawls vs Harsanyi- References
The probabilistic methods of classical decision theory are discussed in Chapter 3: Decisions Under Risk: Probability.
Apparently, the authors did not realize that relevant to their discussion in Resnik (1987) is not Chapter 3: Decisions Under Risk: Probability, but rather Chapter 2: Decisions Under Ignorance!?
For, had they studied this chapter, namely Chapter 2, they would have realized that Info-Gap's robustness model is simply a Maximin model.
Then, on page 1473 we read this (emphasis is mine):
The power and novelty of the info-gap approach is in the ability to explore the sensitivity of the decision to a wide range of different types of parameter, functional, and structural errors and uncertainties simultaneously, given that we do not know the extent of uncertainty in the system at the outset.
Of course, the precise opposite is the case. The purported "power and novelty" attributed to Info-Gap decision theory are in fact its two major failings.
- Firstly, the robustness model is not novel: it is a simple Maximin model (circa 1940).
- Secondly, Info-Gap's purported ability to "cope" with an unknown level of uncertainty in effect turns out to be a blatant disregard of the severity of the uncertainty under consideration. For this purported special power of Info-gap in effect amounts to a fundamentally flawed "procedure": Instead of evaluating the robustness of decisions on the basis of their performance relative to the entire stipulated region of uncertainty, Info-Gap examines the immediate neighborhood of the estimate first, to then explore away from this estimate, no further than allowed by the performance requirement.
The exploration further away from the estimate terminates once the performance requirement is violated.
Conceptually, it is instructive to think about Info-Gap's definition of robustness as a play in three acts:
- Act I: Initialization:
Position yourself at the estimate, û. Equivalently: set α=0. - Act II: Movement:
Slowly move away from the estimate. Equivalently: slowly increase the value of α. - Act III: Termination:
Stop immediately when any further movement away from the estimate will violate the performance requirement for some u in the increased region of uncertainty.
Equivalently: stop when α=α* where α* is such that for any ε>0 there is a u∈U(α*+ε,û) such that r(d,u)< r*.
Note that in this context α* represents the robustness of decision d, namely α*=α(d,û).
- Act I: Initialization:
Here is a schematic representation of the situation:
| û | ||
 | ||
| <-------- U --- Complete Region of Uncertainty --- U --------> | ||
The black area represents the complete region of uncertainty U and the white dot represents the estimate û of the parameter of interest.
Now, with this representation in front of us, I should call attention to the following: No assumption whatsoever must be made as to whether any particular value of u is more/less likely than any other value of this parameter. This is totally in line with the fact that Info-Gap decision theory not only makes no assumptions about "likelihood", it bans any talk of likelihood.
Furthermore, given that, as emphatically pointed out in this paper, the uncertainty is severe, we have not the slightest clue which value of u is the true value.
This means that to determine the robustness of a decision it is essential to establish how well the decision performs relative to the entire region of uncertainty U, or a proper approximation thereof.
To see then what Info-Gap's purported "powerful" approach accomplishes, consider the following schematic representation of the results generated by Info-Gap's robustness analysis for decision d. The red area around the estimate, denoted by U(α(d,û),û), represents the largest safe region around the estimate as determined by Info-Gap's robustness model.
| No Man's Land | û | No Man's Land |
|
| U(α(d,û),û) |
| Safe |
Take special note of Info-Gap's No Man's Land. This is the subset of the complete region of uncertainty that Info-Gap's robustness model does not explore. Roughly, this is the collection of points in U that are not in the safe region U(α(d,û),û) around the estimate û.
This schematic representation vividly illustrates why I regard Info-Gap decision theory as a classic example of voodoo decision-making: rather than tackling the severity of the uncertainty under consideration, one is instructed to ignore it altogether.
In my lectures/seminars I often describe this fundamental flaw in Info-Gap decision theory as a "treasure hunt":
Treasure Hunt
- The island represents the complete region of uncertainty under consideration (the region where the treasure is located).
- The tiny black dot represents the estimate of the parameter of interest (estimate of the location of the treasure).
- The large white circle represents the region of uncertainty pertaining to info-gap's robustness analysis.
- The small white square represents the true (unknown) value of the parameter of interest (true location of the treasure).
So, basing our search plan on Info-Gap Decision Theory, we may zero in on the neighborhood of downtown Melbourne, while for all we know, the true location of the treasure may well be in the Middle of the Simpson desert, or perhaps just north of Brisbane.
Perhaps.
Note that in this picture the estimate is placed at a considerable distance from the true value of the parameter of interest. This, of course has a point and purpose. It depicts the contention (Ben-Haim 2006, pp. 280-281 ) that under severe uncertainty the estimate is a poor indication of the true value of the parameter and is likely to be substantially wrong.
Remark:
I have been accused by Info-Gap scholars for exaggerating the size of the No Man's Land relative to the size of the "safe" area around the estimate.
So let me explain, yet again, why — from a methodological point of view — the No Man's Land must be depicted as much larger than the "safe" area, in the schematic representations of the working of Info-Gap's robustness model.
According to Ben-Haim (2006, p. 210, color is mine):
Most of the commonly encountered info-gap models are unbounded.The inference to be drawn then from this statement is that in most of the commonly encountered info-gap models, the "safe" area around the estimate is infinitely minuscule — in comparison to the complete region of uncertainty. Consequently, not only is it the case that my sketches do not exaggerate the flaw in Info-Gap decision theory. This flaw is so severe that it cannot possibly be exaggerated: the complete region of uncertainty is unbounded yet the robustness analysis is a priori confined to the neighborhood of a poor estimate that is likely to be substantially wrong.
More on this can be found in FAQ # 26 and FAQ # 71.
And last but not least, no reference whatsoever is made in the paper to the thriving area of optimization theory that deals specifically with robust decisions, namely Robust Optimization. For those who are unfamiliar with this field of optimization theory I should point out that Robust Optimization concerns itself with -- among other things -- non-probabilistic methods for robust decision-making under severe uncertainty. Not surprisingly, Wald's Maximin paradigm is used extensively in this field.
