U N E S C O – E O L S S S A M P L E C H A P T E R S DECISION MAKING UNDER UNCERTAINTY
David Easley and Mukul Majumdar
Department of Economics, Cornell University, USA
Keywords: uncertainty, decision, utility, risk, insurance, games, learning
Contents
1. Introduction
2. Expected Utility
2.1 Objective Expected Utility
2.2. Risk Aversion
2.3 Subjective Expected Utility
3. Sequential Decision Making
3.1 Discounted Dynamic Programming
3.2 Characterization of Optimal Policies
3.3 Learning
4. Games as Multi-Person Decision Theory
4.1 Nash Equilibrium
deductible4.2 Bayes Nash Equilibrium
5. Uses and Extensions
Glossary
Bibliography
Biographical Sketch
Summary
Often decision makers are uncertain about the consequences of their choices. Expected utility theory provides a model of decision making under such uncertainty. This theory deals with both objective and subjective uncertainty. It provides insights into actual decisions and it may be used as a guide for decision making. The theory has been extended to incorporate decisions made over time and the learning that results from these decisions. It also provides the basis for the analysis of interacting decision makers in a game.
1. Introduction
“The basic need for a special theory to explain behavior under conditions of uncertainty”, noted Kenneth Arrow, “arises from two considerations: (1) subjective feelings of imperfect knowledge when certain types of choices, typically involving commitments over time, are made; (2) the existence of certain observed phenomena, of which insurance is the most conspicuous example, which cannot be explained on the assumption that individuals act with subjective certainty”. The literature is too vast for a survey, and, in several directions lead to subtle issues of philosophy, economics and probability theory. At one extreme are models that focus on a single decision-maker (an investor, a central planner). At the other extreme are models - in the tradition of Walras - with a large number of agents. In between are models - in the tradition of Cournot -
U N
E S C O – E O L S S S A M P L E C H A P T E R S with a small number of interacting agents. The earliest treatments of decision making under uncertainty dealt with uncertain cash flows and assumed that only the expected value mattered. The St. Petersburg paradox (a random cash flow with infinite expected value that is clearly not worth more than a finite amount) showed that this approach was unsatisfactory. In 1738, Daniel Bernoulli proposed valuing uncertain cash flows according to the expected value of the utility of money using a logarithmic utility function. Hence, both expected value and risk matters. This approach was arbitrary, but it seemed more reasonable than assuming that decision makers care only about expected values. (It does not, however, solve the St. Petersburg paradox. Consider repeated tossing of a fair coin that pays exp(2n ) if a head appears for the first time on the n th toss.) In 1944, von Neumann and Morgenstern, in their analysis of games, provided a set of axioms for decision makers preferences over uncertain objects that lead to Bernoulli’s formulation with general utility functions over the objects. This approach had the advantage that the reasonableness of the axioms would be more easily judged than could the direct assumption of expected utility maximization. von Neumann and Morgenstern’s formulation dealt only with objective uncertainty. This is a limitation as often uncertainty is not objective, and can only be subjectively acces
sed. In 1954, Leonard Savage extended the theory to deal with this complication. His approach is elegant, but difficult. In this article we follow a simple treatment. 2. Expected Utility For models with a single agent, a basic agenda of research has been to cast the problem of optimal choice under uncertainty in terms of maximization of “expected” utility. We begin with the case in which the uncertainty the decision-maker faces is objectively known. The basic ingredients of the single agent model of choice under uncertainty are: 1. A set X = {x 1,...,x n } a finite set of prizes or consequences. 2. A set P = {(p 1,...,p n ) ∈ R p n i t n +=∑:1 = 1} of probabilities, or lotteries, on X . 3. Preferences ≥ defined on P . Formally, preferences ≥ are a binary relation on P . That is, pairs of alternatives, in P are ranked. If the decision-maker regards probability p to be “at least as good as” probability q , then we write p ≥ q . These preferences reflect the decision-maker’s valuation of prizes as well as his attitude toward risk.
2.1 Objective Expected Utility
The challenge has been to isolate axioms that enable one to impute to the decision-maker a utility function u on X , representing the decision-maker’s preferences. One shows that, under some assumptions on preferences, the decision maker prefers one probability p to another probability q if and only if the first probability yields a higher expected utility, i.e. E p (u (x )) > E q (u (x )) where the ex
pectation operation is taken with respect to the probability distribution p or q on X .
U N
E S C O – E O L S S S A M P L E C H A P T E R S The requirements for such a representation to exist are: 1. Completeness: for all p,q ∈ P either p ≥ q , q ≥ p or both. 2. Transitivity: for all p,q,r ∈ P if p ≥ q and q ≥ r, then p ≥ r . 3. Continuity: for all p,q,r ∈ P the sets {α ∈[0,1]: αp + (1-α)q ≥ r } and {α ∈[0,1]:r ≥ αp + (1-α)q } are closed. 4. Independence: for all p,q,r ∈ P and α ∈(0,1), p ≥ q if and only if αp + (1-α)r ≥ αq + (1-α)r . To interpret independence it is useful to break the probability αp + (1-α)r into two lotteries. Consider the (compound) lottery with probability α on “prize” p and probability 1-α on “prize” r . The two lotteries αp + (1-α)r and αq + (1-α)r place probability 1-α on the same prize r. With the remaining probability, α, the first gamble gives p and the second gives q where p ≥ q . So it seems intuitive that p ≥ q if and only if α p + (1-α) r ≥ αp + (1-α) r as long as the decision-maker cares only about the consequences of gambling and not the process of gambling itself. Theorem 1. A preference relation ≥ on P satisfies completeness, transitivity, continuity and independence if and only if there exists a function u : X → R 1 such that for any two probabilities p and q on X , we have p / q if and only if E p (u (x )) ≥ E q (u (x )). Clearly, the representation u(⋅) given in Theorem 1 is not unique. If u (⋅) is an expected utility function for some preferences /, then so i
s V (x ) = a + b u (x ) for any numbers a and b > 0. Expected utility theory, which is developed here for the case of finite prize sets, extends straightforwardly to continuous prizes. We focus on prizes x ∈ ;R 1+ think of amounts of money. The distribution on outcomes can be described by a cumulative distribution function
F : R +1 → [0,1]. To tie this notation back to our earlier notation for discrete prizes note that in the discrete case F (x ) = p x i x x i ()<∑ where p (x i ) = p i . For continuous prizes, P is the space of cumulative distribution functions on R +1. If a decision-maker has preferences on P that satisfy the axioms above then there is utility function u: R R +→11 such that for any F ,
G ∈ P we have F ≥ G if and only if
()()()().x dG x u x dF x u ∫∫≥
2.2. Risk Aversion
A decision-maker who dislikes uncertainty prefers the expected value of any distribution to the distribution itself. Such an individual is said to be risk averse .
Definition: A decision-maker is risk averse if for any cumulative distribution function F ,
U N E S C O – E O L S S S A M P L E C H A P T E R S
()()
()().x dF x u x xdF u ∫∫≥
This definition is equivalent to concavity of the utility function u . The curvature of the individual’s utility function provides a measure of his degree of risk aversion. This curvature cannot be measured by u”(Α) as the second derivative is not uniquely by ≥. However; u”(x )/u’(x ) is invariant to the representation chosen and it can be used as a measure of risk aversion.
Definition: The Arrow-Pratt coefficient of (absolute) risk aversion for an expected utility function u (x ) is
λ(x ,u ) = -u”(x )/u’(x ).
This measure is positive for all x , for any risk averse decision maker. The measure is increasing in the curvature of u (⋅) and thus it is a reasonable measure of risk aversion. Formally, if u (x ) = f (v (x )), for all x , for an increasing concave function f (⋅) then λ(x ,u ) ≥ λ(x ,v ) for all x.
A typical application of this theory is to the choice of insurance. Suppose that an individual begins with
wealth w > 0. With probability p 1 he will lose L 1, with probability p 2 he will lose L 2 and with probability 1 - p 1 - p 2 he will retain his initial wealth. He is offered a menu of insurance policies that pay πi in the event of loss L i with cost or premium C = α(p 1π1+p 2π2). The individual can choose any level πi ≤ L i , and he pays a premium determined by C . If α = 1 then this actuarially fair insurance. Suppose that the individual is risk averse with utility function on money given by u (⋅). Then an optimal insurance contract maximizes expected utility p 1u (w -C-L 1+π1) + p 2u (w-C-L 2+π2) + (1-p 1-p 2)u (w-C ) over feasible payoffs.
For actuarially fair insurance it is immediate from the first order conditions for this maximization problem that πi = L i for all i . That is, the individual fully insures and his wealth will be w - C . For α > 1, the solution involves a deductible D. The optimal policy is characterized by L i -πi =D > 0 for all i , where the optimal deductible depends on how risk averse the individual is and on how unfair the insurance is.
-
-
-
TO ACCESS ALL THE 12 PAGES OF THIS CHAPTER,
Visit: lss/Eolss-sampleAllChapter.aspx
Bibliography
Allais M. (1953). Le comportement de l’homme rationnel devant le risque, critique des postulats et axiomes de l’école Américaine. Econometrica 21, 503-546. [A paradox that challenges expected utility
U N
E S C O – E O L S S S A M P L E C H A P T E R S theory]
Anscombe F. and Aumann R. (1963). A definition of subjective probability. Annals of Mathematical Statistics 34, 199-205. [A modern treatment of subjective expected utility theory]
Arrow K. (1971). Essays in the Theory of Risk Bearing . Chicago: Markham. [A collection of essays on choice under uncertainty, some of which are landmarks in the progress of economic theory]
Bernoulli D. (1738). Specimen theoriae novae de mensura sortis, in Commentarii Academe Scientiaru
m Imperialis Petrogolitanae , 5, 175-192. [An early article arguing that expected utility of prizes rather than the expected value of prizes is relevant for decision making]
Berry, D.A. and Friestedt, B. (1985). Bandit Problems , Chapmen and Hall: London [A monograph dealing with bandit problems has an extended list of references]
Blackwell, D. (1965). Discounted dynamic programming. Annals of Mathematical Statistics 36 226-235 Fudenberg D. and Tirole J. (1991). Game Theory . Cambridge: MIT Press. [A standard game theory textbook]
Harsanyi J. (1967-68). Games with incomplete information played by Bayesian players. Managment Science 14, 159-182, 320-334, 486-502. [A series of articles on decision making and games with incomplete information] Majumdar M. (1998). Organizations with Incomplete Information . (ed. Mukul Majumdar) Cambridge: Cambridge University Press [A collection of essays and review articles dealing with decision making with incomplete information] Mas-Colell, A.,Whinston, M.D. and Green, J.R. (1995). Microeconomic Theory , Oxford University Press: New York [Chapter 6 provides a useful exposition of choice under uncertainty] Myerson R. (1991). Game Theory: Analysis of Conflict . Cambridge: Harvard University Press. [A standard game theory textbook] Savage L. (1954). The Foun
dations of Statistics . New York: Wiley [The pioneering work in subjective probability] Simon, H.A. (1972). Theories of bounded rationality. In Decision and Organizations (eds. McGuire, C.B. and Radner, R.), North Holland: Amsterdam 161-176 Von Neumann J. and Morgenstern O. (1944). Theory of Games and Economic Behavior . Princeton: Princeton University Press. [The original axiomatic development of expected utility theory] Biographical Sketches David Easley is H. Scarborough Professor of Economics at Cornell University. A Fellow of the Econometric Society, he is a leading contributor to the literature on decision making under uncertainty. Mukul Majumdar is H.T. and R.I. Warshow Professor of Economics at Cornell University and has made wide-ranging contributions to economic theory.
发表评论