Nothing Special   »   [go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/inm/ordeca/v5y2008i1p43-52.html
   My bibliography  Save this article

Getting the Right Mix of Experts

Author

Listed:
  • Jason R. W. Merrick

    (Department of Statistical Sciences and Operations Research, Virginia Commonwealth University, Richmond, Virginia 23284)

Abstract
The Bayesian approach to combining expert opinions is well developed, providing a decision maker's posterior beliefs after receiving advice from people with deep knowledge in a given subject. A necessary part of these models is the inclusion of dependencies between the experts' judgments, often justified by an overlap in the information on which the experts base their judgments. In this paper, we propose a hierarchical structure different than those previously proposed, where the mixing distribution is treated nonparametrically with a Dirichlet process. This makes our overall model a Dirichlet process mixture and allows for experts' model parameters to be equal in the mixture. We apply this approach to published expert judgment data, demonstrating that the decision maker's posterior distributions on the quantities of interest are not restricted to specific parametric forms, even allowing for multiple modes, and are thus more intuitively appealing.

Suggested Citation

  • Jason R. W. Merrick, 2008. "Getting the Right Mix of Experts," Decision Analysis, INFORMS, vol. 5(1), pages 43-52, March.
    • Handle: RePEc:inm:ordeca:v:5:y:2008:i:1:p:43-52
    DOI: 10.1287/deca.1080.0108
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/deca.1080.0108
    Download Restriction: no
    File URL: https://libkey.io/10.1287/deca.1080.0108?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Mohamed N. Jouini & Robert T. Clemen, 1996. "Copula Models for Aggregating Expert Opinions," Operations Research, INFORMS, vol. 44(3), pages 444-457, June.
    2. French, Simon, 1981. "Consensus of opinion," European Journal of Operational Research, Elsevier, vol. 7(4), pages 332-340, August.
    3. Joseph Lipscomb & Giovanni Parmigiani & Vic Hasselblad, 1998. "Combining Expert Judgment by Hierarchical Modeling: An Application to Physician Staffing," Management Science, INFORMS, vol. 44(2), pages 149-161, February.
    4. Dennis Lindley, 1983. "Reconciliation of Probability Distributions," Operations Research, INFORMS, vol. 31(5), pages 866-880, October.
    5. Robert T. Clemen & Robert L. Winkler, 1999. "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 19(2), pages 187-203, April.
    6. Robert T. Clemen & Terence Reilly, 1999. "Correlations and Copulas for Decision and Risk Analysis," Management Science, INFORMS, vol. 45(2), pages 208-224, February.
    7. W. R. Gilks & P. Wild, 1992. "Adaptive Rejection Sampling for Gibbs Sampling," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 41(2), pages 337-348, June.
    8. Peter A. Morris, 1983. "An Axiomatic Approach to Expert Resolution," Management Science, INFORMS, vol. 29(1), pages 24-32, January.
    9. Peter A. Morris, 1974. "Decision Analysis Expert Use," Management Science, INFORMS, vol. 20(9), pages 1233-1241, May.
    10. Robert T. Clemen, 1987. "Combining Overlapping Information," Management Science, INFORMS, vol. 33(3), pages 373-380, March.
    11. Robert L. Winkler, 1981. "Combining Probability Distributions from Dependent Information Sources," Management Science, INFORMS, vol. 27(4), pages 479-488, April.
    12. Peter A. Morris, 1977. "Combining Expert Judgments: A Bayesian Approach," Management Science, INFORMS, vol. 23(7), pages 679-693, March.
    13. Roberts, G. O. & Smith, A. F. M., 1994. "Simple conditions for the convergence of the Gibbs sampler and Metropolis-Hastings algorithms," Stochastic Processes and their Applications, Elsevier, vol. 49(2), pages 207-216, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Anubhab Pattanayak & K. S. Kavi Kumar, 2014. "Weather Sensitivity Of Rice Yield: Evidence From India," Climate Change Economics (CCE), World Scientific Publishing Co. Pte. Ltd., vol. 5(04), pages 1-24.
    2. L. Robin Keller & Ali Abbas & Manel Baucells & Vicki M. Bier & David Budescu & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Ahti Salo & George Wu, 2010. "From the Editors..," Decision Analysis, INFORMS, vol. 7(4), pages 327-330, December.
      • L. Robin Keller & Manel Baucells & Kevin F. McCardle & Gregory S. Parnell & Ahti Salo, 2007. "From the Editors..," Decision Analysis, INFORMS, vol. 4(4), pages 173-175, December.
      • L. Robin Keller & Manel Baucells & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Gregory S. Parnell & Ahti Salo, 2008. "From the Editors..," Decision Analysis, INFORMS, vol. 5(4), pages 173-176, December.
      • L. Robin Keller & Manel Baucells & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Gregory S. Parnell & Ahti Salo, 2009. "From the Editors ..," Decision Analysis, INFORMS, vol. 6(4), pages 199-201, December.
    3. L. Robin Keller, 2008. "From the Editor..," Decision Analysis, INFORMS, vol. 5(1), pages 1-4, March.
    4. Jessop, Alan, 2014. "IMP: A decision aid for multiattribute evaluation using imprecise weight estimates," Omega, Elsevier, vol. 49(C), pages 18-29.
    5. Sarfraz Khan & Elaine Mauldin, 2021. "Benefit or burden? A comparison of CFO and CEO outside directorships," Journal of Business Finance & Accounting, Wiley Blackwell, vol. 48(7-8), pages 1175-1214, July.
    6. L. Robin Keller & Kelly M. Kophazi, 2012. "From the Editors ---Copulas, Group Preferences, Multilevel Defenders, Sharing Rewards, and Communicating Analytics," Decision Analysis, INFORMS, vol. 9(3), pages 213-218, September.
    7. L. Robin Keller & Kelly M. Kophazi, 2011. "From the Editors---Deterrence, Multiattribute Utility, and Probability and Bayes' Updating," Decision Analysis, INFORMS, vol. 8(2), pages 83-87, June.
    8. Bin Dan & Can Liu & Guangye Xu & Xumei Zhang, 2014. "Pareto Improvement Strategy for Service-Based Free-Riding in a Dual-Channel Supply Chain," Asia-Pacific Journal of Operational Research (APJOR), World Scientific Publishing Co. Pte. Ltd., vol. 31(06), pages 1-27.
    9. Saurabh Bansal & Yaroslav Rosokha, 2018. "Impact of Compound and Reduced Specification on Valuation of Projects with Multiple Risks," Decision Analysis, INFORMS, vol. 15(1), pages 27-46, March.
    10. Christopher W. Karvetski & Kenneth C. Olson & David R. Mandel & Charles R. Twardy, 2013. "Probabilistic Coherence Weighting for Optimizing Expert Forecasts," Decision Analysis, INFORMS, vol. 10(4), pages 305-326, December.
    11. L. Robin Keller & Kelly M. Kophazi, 2010. "From the Editors..," Decision Analysis, INFORMS, vol. 7(2), pages 151-154, June.
    12. Karner, Alex & Niemeier, Deb, 2013. "Civil rights guidance and equity analysis methods for regional transportation plans: a critical review of literature and practice," Journal of Transport Geography, Elsevier, vol. 33(C), pages 126-134.
    13. Jason R. W. Merrick & Fabrizio Ruggeri & Refik Soyer & L. Robin Keller, 2012. "From the Editors---Games and Decisions in Reliability and Risk," Decision Analysis, INFORMS, vol. 9(2), pages 81-85, June.
    14. Yuyu Fan & David V. Budescu & David Mandel & Mark Himmelstein, 2019. "Improving Accuracy by Coherence Weighting of Direct and Ratio Probability Judgments," Decision Analysis, INFORMS, vol. 16(3), pages 197-217, September.
    15. L. Robin Keller, 2011. "From the Editor ---Multiattribute and Intertemporal Preferences, Probability, and Stochastic Processes: Models and Assessment," Decision Analysis, INFORMS, vol. 8(3), pages 165-169, September.
    16. Robert F. Bordley, 2011. "Using Bayes' Rule to Update an Event's Probabilities Based on the Outcomes of Partially Similar Events," Decision Analysis, INFORMS, vol. 8(2), pages 117-127, June.
    17. Lizhong Yang & Shaobo Liu & Ping Rao & Kongjin Zhu, 2012. "Subconscious Environmental Information Perceiving Behavior And Its Attenuation In Information-Based Evacuation Experiment," International Journal of Modern Physics C (IJMPC), World Scientific Publishing Co. Pte. Ltd., vol. 23(07), pages 1-17.
    18. Lichtendahl Jr., Kenneth C., 2009. "Random quantiles of the Dirichlet process," Statistics & Probability Letters, Elsevier, vol. 79(4), pages 501-507, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jason R. W. Merrick & J. Rene van Dorp & Amita Singh, 2005. "Analysis of Correlated Expert Judgments from Extended Pairwise Comparisons," Decision Analysis, INFORMS, vol. 2(1), pages 17-29, March.
    2. Robert T. Clemen & Robert L. Winkler, 1999. "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 19(2), pages 187-203, April.
    3. Ali E. Abbas, 2009. "A Kullback-Leibler View of Linear and Log-Linear Pools," Decision Analysis, INFORMS, vol. 6(1), pages 25-37, March.
    4. Erin Baker & Olaitan Olaleye, 2013. "Combining Experts: Decomposition and Aggregation Order," Risk Analysis, John Wiley & Sons, vol. 33(6), pages 1116-1127, June.
    5. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    6. Knut Are Aastveit & James Mitchell & Francesco Ravazzolo & Herman van Dijk, 2018. "The Evolution of Forecast Density Combinations in Economics," Tinbergen Institute Discussion Papers 18-069/III, Tinbergen Institute.
    7. James E. Smith & Detlof von Winterfeldt, 2004. "Anniversary Article: Decision Analysis in Management Science," Management Science, INFORMS, vol. 50(5), pages 561-574, May.
    8. Anne Sofie Jore & James Mitchell & Shaun P. Vahey, 2010. "Combining forecast densities from VARs with uncertain instabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 621-634.
    9. Garratt, Anthony & Mise, Emi, 2014. "Forecasting exchange rates using panel model and model averaging," Economic Modelling, Elsevier, vol. 37(C), pages 32-40.
    10. Neil A. Stiber & Mitchell J. Small & Marina Pantazidou, 2004. "Site‐Specific Updating and Aggregation of Bayesian Belief Network Models for Multiple Experts," Risk Analysis, John Wiley & Sons, vol. 24(6), pages 1529-1538, December.
    11. Dr. James Mitchell, 2009. "Measuring Output Gap Uncertainty," National Institute of Economic and Social Research (NIESR) Discussion Papers 342, National Institute of Economic and Social Research.
    12. Francesco Billari & Rebecca Graziani & Eugenio Melilli, 2014. "Stochastic Population Forecasting Based on Combinations of Expert Evaluations Within the Bayesian Paradigm," Demography, Springer;Population Association of America (PAA), vol. 51(5), pages 1933-1954, October.
    13. Paola Monari & Patrizia Agati, 2001. "Fiducial inference in combining expert judgements," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 10(1), pages 81-97, January.
    14. Wolden Bache, Ida & Sofie Jore, Anne & Mitchell, James & Vahey, Shaun P., 2011. "Combining VAR and DSGE forecast densities," Journal of Economic Dynamics and Control, Elsevier, vol. 35(10), pages 1659-1670, October.
    15. Patrizio Frederic & Mario Di Bacco & Frank Lad, 2012. "Combining expert probabilities using the product of odds," Theory and Decision, Springer, vol. 73(4), pages 605-619, October.
    16. Hall, Stephen G. & Mitchell, James, 2007. "Combining density forecasts," International Journal of Forecasting, Elsevier, vol. 23(1), pages 1-13.
    17. Ali Mosleh & George Apostolakis, 1986. "The Assessment of Probability Distributions from Expert Opinions with an Application to Seismic Fragility Curves," Risk Analysis, John Wiley & Sons, vol. 6(4), pages 447-461, December.
    18. Christian Kascha & Francesco Ravazzolo, 2010. "Combining inflation density forecasts," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 29(1-2), pages 231-250.
    19. Wilson, Kevin J., 2017. "An investigation of dependence in expert judgement studies with multiple experts," International Journal of Forecasting, Elsevier, vol. 33(1), pages 325-336.
    20. Pennings, Clint L.P. & van Dalen, Jan & Rook, Laurens, 2019. "Coordinating judgmental forecasting: Coping with intentional biases," Omega, Elsevier, vol. 87(C), pages 46-56.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ordeca:v:5:y:2008:i:1:p:43-52. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.