Abstract
This paper provides a survey on probabilistic decision graphs for modeling and solving decision problems under uncertainty. We give an introduction to influence diagrams, which is a popular framework for representing and solving sequential decision problems with a single decision maker. As the methods for solving influence diagrams can scale rather badly in the length of the decision sequence, we present a couple of approaches for calculating approximate solutions. The modeling scope of the influence diagram is limited to so-called symmetric decision problems. This limitation has motivated the development of alternative representation languages, which enlarge the class of decision problems that can be modeled efficiently. We present some of these alternative frameworks and demonstrate their expressibility using several examples. Finally, we provide a list of software systems that implement the frameworks described in the paper.
Similar content being viewed by others
References
Ahlmann-Ohlsen KS, Jensen FV, Nielsen TD, Pedersen O, Vomlelova M (2009) A comparsion of two approaches for solving unconstrained influence diagrams. Int J Approx Reason 50: 153–173
Bhattacharjya D, Shachter RD (2007) Evaluating influence diagrams with decision circuits. In: Proceedings of the twentythird conference on Uncertainty in Artificial Intelligence (UAI). pp 9–16
Bhattacharjya D, Shachter RD (2008) Sensitivity analysis in decision circuits. In: Proceedings of the twentyfourth conference on Uncertainty in Artificial Intelligence (UAI). pp 34–42
Bielza C, Ríos-Insua S, Gómez M, del Pozo JAF (2000) Sensitivity analysis in ictneo. In: Lecture notes in statistics, Robust Bayesian analysis, vol 152. Springer, Berlin, pp 317–334
Cano A, Gómez M, Moral S (2006) A forward-backward Monte Carlo method for solving influence diagrams. Int J Approx Reason 42(1–2): 119–135
Charnes JM, Shenoy PP (2004) Multistage monte carlo method for solving influence diagrams using local computation. Manag Sci 50: 405–418
Cobb B (2006) Continuous decision MTE influence diagrams. In: Proceedings of the third European workshop on probabilistic graphical models. pp 67–74
Cobb BR, Shenoy PP (2004) Hybrid influence diagrams using mixtures of truncated exponentials. In: Proceedings of the Twentieth Conference on Uncertainty in Artificial Intelligence (UAI). pp 85–93
Cobb BR, Shenoy PP (2008) Decision making with hybrid influence diagrams using mixtures of truncated exponentials. Eur J Oper Res 186(1): 261–275
Cooper GF (1988) A method for using belief networks as influence diagrams. In: Proceedings of the fourth conference on uncertainty in artificial intelligence. pp 55–63
Cooper GF (1990) The computational complexity of probabilistic inference using Bayesian belief networks. Artif Intell 42(2–3): 393–405
Darwiche A (2003) A differential approach to inference in Bayesian networks. J ACM 50(3): 280–305
Darwiche A (2009) Modeling and reasoning with Bayesian networks. Cambridge University Press, Cambridge
Dechter R (2000) An anytime approximation for optimizing policies under uncertainty. In: In workshop on decision theoretic planning, at the 5th international conference on artificial intelligence planning systems (AIPS-2000)
Dittmer SL, Jensen FV (1997) Myopic value of information in influence diagrams. In: Geiger D, Shenoy PP (eds) Proceedings of the thirteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 142–149
Felli JC, Hazen GB (1999) Sensitivity analysis and the expected value of perfect information. Med Decis Mak 18: 95–109
Gal Y, Pfeffer A (2003) A language for modeling agents’ decision making processes in games. In: Proceedings of the second international joint conference on autonomous agents and multiagent systems. ACM Press, New york, pp 265 – 272
Gal Y, Pfeffer A (2008) Networks of influence diagrams: a formalism for representing agent’s beliefs and decision making processes. J Artif Intell Res 33: 109–147
Garcia-Sanchez D, Druzdzel M (2007) An efficient exhaustive anytime sampling algorithm for influence diagrams. In: Lucas P, Gmez J, Salmern A (eds) Advances in probabilistic graphical models, studies in fuzziness and soft computing, vol 214. Springer, Berlin, pp 255–273
Gmytrasiewicz PJ, Durfee EH, Wehe DK (1991) A decision-theoretic approach to coordinating multiagent interactions. In: Proceedings of the twelfth international joint conference on artificial intelligence. pp 62–68
Horsch MC, Poole D (1998) An anytime algorithm for decision making under uncertainty. In: Cooper GF, Moral S (eds) Proceedings of the fourteenth conference on Uncertainty in Artificial Intelligence (UAI). Morgan Kaufmann Publishers, Los Altos, pp 246–255
Howard RA, Matheson JE (1981) Influence diagrams. In: Howard RA, Matheson JE (eds) The principles and applications of decision analysis vol 2, chap 37. Strategic Decision Group, Menlo Park, pp 721–762
Jensen F, Jensen FV, Dittmer SL (1994) From influence diagrams to junction trees. In: Mantaras RL, Poole D (eds) Proceedings of the tenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 367–373
Jensen FV, Gatti E (2010) Information enhancement for approximate representation of optimal strategies for influence diagrams. In: Proceedings of the fifth European workshop on probabilistic graphical models. pp 102–9
Jensen FV, Nielsen TD (2007) Bayesian networks and decision graphs, 2nd edn. Springer, New York ISBN: 0-387-68281-3
Jensen FV, Vomlelova M (2002) Unconstrained influence diagrams. In: Darwiche A, Friedman N (eds) Proceedings of the eighteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 234–241
Jensen FV, Nielsen TD, Shenoy PP (2006) Sequential influence diagrams: a unified framework. Int J Approx Reason 42: 101–118
Kjaerulff UB, Madsen AL (2008) Bayesian networks and influence diagrams, 1st edn. Springer, New York ISBN: 978-0-387-74100-0
Koller D, Friedman N (2009) Probabilistic graphical models: principles and techniques. MIT Press, Cambridge
Koller D, Milch B (2001) Multi-agent influence diagrams for representing and solving games. In: Proceedings of the seventeenth international joint conference on artificial intelligence. pp 1027–1036
Koller D, Milch B (2003) Multi-agent influence diagrams for representing and solving games. Games Econ Behav 45(1): 181–221
Korb KB, Nicholson AE (2004) Bayesian artificial intelligence, 1st edn. Chapman and Hall, London ISBN:1-58488-387-1
Lauritzen SL, Nilsson D (2001) Representing and solving decision problems with limited information. Manag Sci 47(9): 1235–1251
Li Y, Shenoy PP (2010) Solving hybrid influence diagrams with deterministic variables. In: Proceedings of the twentysixth conference on Uncertainty in Artificial Intelligence (UAI). pp 322–331
Liao W, Ji Q (2008) Efficient non-myopic value-of-information computation for influence diagrams. Int J Approx Reason 49: 436–450
Luque M, Díez FJ (2010) Variable elimination for influence diagrams with super value nodes. Int J Approx Reason 51: 615–631
Madsen AL (2008) New methods for marginalization in lazy propagation. In: Jaeger M, Nielsen TD (eds) Proceedings of the fourth European workshop on probabilistic graphical models. pp 193–200
Madsen AL, Jensen F (2005) Solving linear-quadratic conditional Gaussian influence diagrams. Int J Approx Reason 38: 263–282
Madsen AL, Jensen FV (1999) Lazy evaluation of symmetric Bayesian decision problems. In: Laskey KB, Prade H (eds) Proceedings of the fifteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 382–390
Madsen AL, Nilsson D (2001) Solving influence diagrams using HUGIN, Shafer-Shenoy, and lazy propagation. In: Proceedings of the seventeenth conference on Uncertainty in Artificial Intelligence (UAI). Morgan Kaufmann Publishers, Los Altos, pp 337–345
Moral S, Rumí R, Salmerón A (2001) Mixtures of truncated exponentials in hybrid Bayesian networks. In: Proceedings of the sixth European conference on symbolic and quantitative approaches to reasoning with uncertainty, lecture notes in artificial intelligence, vol 2143. Springer, Berlin, pp 145–167
Nielsen TD (2001) Decomposition of influence diagrams. In: Benferhat S, Besnard P (eds) Proceedings of the sixth European conference on symbolic and quantitative approaches to reasoning with uncertainty. Springer, no. 2143 in Lecture Notes in Artificial Intelligence, pp 144–155
Nielsen TD, Jensen FV (1999) Well-defined decision scenarios. In: Laskey KB, Prade H (eds) Proceedings of the fifteenth Conference on Uncertainty in Artificial Intelligence (UAI). Morgan Kaufmann Publishers, Los Altos, pp 502–511
Nielsen TD, Jensen FV (2003) Sensitivity analysis in influence diagrams. IEEE Trans Syst Man Cybern Part A Syst Hum 33(2): 223–234
Nilsson D, Lauritzen SL (2000) Evaluating influence diagrams using LIMIDs. In: Boutilier C, Goldszmidt M (eds) Proceedings of the sixteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 436–445
Pearl J (1988) Probabilistic reasoning in intelligent systems. representation and reasoning. Morgan Kaufmann Publishers, San Mateo. ISBN 0-934613-73-7
Raiffa H, Schlaifer R (1961) Applied statistical decision theory. MIT press, Cambridge
Shachter RD (1986) Evaluating influence diagrams. Oper Res 34(6): 871–882
Shachter RD (1999) Efficient value of information computation. In: Laskey KB, Prade H (eds) Proceedings of the fifteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 594–601
Shachter RD, Bhattacharjya D (2010) Dynamic programming in influence diagrams with decision circuits. In: Proceedings of the twentysixth conference on Uncertainty in Artificial Intelligence (UAI). pp 509–516
Shachter RD, Kenley CR (1989) Gaussian influence diagrams. Manag Sci 35(5): 527–550
Shachter RD, Peot MA (1992) Decision making using probabilistic inference methods. In: Dubois D, Wellman MP, D’Ambrosio B, Smets P (eds) Proceedings of the eighth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 276–283
Shafer GR, Shenoy PP (1990) Probability Propagation. Ann Math Artif Intell 2: 327–352
Shenoy PP (1992) Valuation-based systems for Bayesian decision analysis. Oper Res 40(3): 463–484
Shenoy PP (2000) Valuation network representation and solution of asymmetric decision problems. Eur J Oper Res 121(3): 579–608
Shenoy PP, West JC (2009) Mixtures of polynomials in hybrid bayesian networks with deterministic variables. In: Proceedings of the eighth workshop on uncertainty processing. pp 202–212
Virto MA, Martín J, Insua DR, Moreno-Díaz A (2002) Approximate solutions of complex influence diagrams through mcmc methods. In: Proceedings of the first European workshop on probabilistic graphical models
Von Neumann J, Morgenstern O (1944) Theory of games and economic behavior, 1st edn. Wiley, New York
Yuan C, Wu X (2010) Solving influence diagrams using heuristic search. In: Proceedings of the eleventh international symposium on artificial intelligence and mathematics
Yuan C, Wu X, Hansen E (2010) Solving multistage influence diagrams using branch-and-bound search. In: Proceedings of the twentysixth conference on Uncertainty in Artificial Intelligence (UAI). Catalina Island, CA
Zhang NL (1998) Probabilistic inference in influence diagrams. In: Cooper GF, Moral S (eds) Proceedings of the fourteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 514–522
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jensen, F.V., Nielsen, T.D. Probabilistic decision graphs for optimization under uncertainty. 4OR-Q J Oper Res 9, 1–28 (2011). https://doi.org/10.1007/s10288-011-0159-7
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10288-011-0159-7