Cited By
View all- Angelopoulos SBendoly EFransoo JHoberg KOu CTenhiälä A(2023)Digital transformation in operations managementJournal of Operations Management10.1002/joom.127169:6(876-889)Online publication date: 4-Sep-2023
Demand planning for the digital supply chain: : How to integrate human judgment and predictive analytics
Pages 965 - 982
Abstract
Our research examines how to integrate human judgment and statistical algorithms for demand planning in an increasingly data‐driven and automated environment. We use a laboratory experiment combined with a field study to compare existing integration methods with a novel approach: Human‐Guided Learning. This new method allows the algorithm to use human judgment to train a model using an iterative linear weighting of human judgment and model predictions. Human‐Guided Learning is more accurate vis‐à‐vis the established integration methods of Judgmental Adjustment, Quantitative Correction of Human Judgment, Forecast Combination, and Judgment as a Model Input. Human‐Guided Learning performs similarly to Integrative Judgment Learning, but under certain circumstances, Human‐Guided Learning can be more accurate. Our studies demonstrate that the benefit of human judgment for demand planning processes depends on the integration method.
Highlights
•
Human judgment is an essential element of demand forecasting but requires integration into the forecasting process.
•
Giving people too much influence may introduce more noise than signal. Our research examines different ways of integrating human judgment into a forecasting process and shows that an effortless way of doing so‐‐by allowing people to indicate that a special event is affecting a forecast and an algorithm to estimate the impact of that event‐‐performs remarkably well in comparison to other methods.
References
[1]
Adya, M., & Lusk, E. J. (2016). Development and validation of a rule‐based time series complexity scoring technique to support design of adaptive forecasting DSS. Decision Support Systems, 83, 70–82.
[2]
Alvarado‐Valencia, J., Barrero, L. H., Önkal, D., & Dennerlein, J. T. (2017). Expertise, credibility of system forecasts and integration methods in judgmental demand forecasting. International Journal of Forecasting, 33(1), 298–313.
[3]
Arvan, M., Fahimnia, G., Reisi, M., & Siemsen, E. (2019). Integrating human judgment into quantitative forecasting methods: A review. Omega, 86, 237–252.
[4]
Baecke, P., de Baets, S., & Vanderheyden, K. (2017). Investigating the added value of integrating human judgment into statistical demand forecasting systems. International Journal of Production Economics, 191, 85–96.
[5]
Baker, J. (2021). Maximizing forecast value added through machine learning and "nudges" (pp. 8–15). Foresight.
[6]
Blattberg, R. C., & Hoch, S. J. (1990). Database models and managerial intuition: 50% model+ 50% manager. Management Science, 36(8), 887–899.
[7]
Bunn, D., & Wright, G. (1991). Interaction of judgmental and statistical forecasting methods: Issues & analysis. Management Science, 37(5), 501–518.
[8]
Chen, D. L., Schonger, M., & Wickens, C. (2016). oTree—An open‐source platform for laboratory, online, and field experiments. Journal of Behavioral and Experimental Finance, 9, 88–97.
[9]
Collopy, F., & Armstrong, J. S. (1992). Rule‐based forecasting: Development and validation of an expert systems approach to combining time series extrapolations. Management Science, 38(10), 1394–1414.
[10]
de Baets, S., & Harvey, N. (2020). Using judgment to select and adjust forecasts from statistical models. European Journal of Operational Research, 284(3), 882–895.
[11]
Elgers, P. T., Lo, M. H., & Murray, D. (1995). Note on adjustments to analysts' earnings forecasts based upon systematic cross‐sectional components of prior‐period errors. Management Science, 41(8), 1392–1396.
[12]
Fildes, R. (1991). Efficient use of information in the formation of subjective industry forecasts. Journal of Forecasting, 10, 597–617.
[13]
Fildes, R., Goodwin, P., Lawrence, M., & Nikolopoulos, K. (2009). Effective forecasting and judgmental adjustments: An empirical evaluation and strategies for improvement in supply‐chain planning. International Journal of Forecasting, 25, 3–23.
[14]
Fildes, R., Goodwin, P., & Önkal, D. (2019). Use and misuse of information in supply chain forecasting of promotion effects. International Journal of Forecasting, 35(1), 144–156.
[15]
Fildes, R., Ma, S., & Kolassa, S. (2022). Retail forecasting: Research and practice. International Journal of Forecasting, 38(4), 1283–1318.
[16]
Fildes, R., & Petropoulos, F. (2015). Improving forecast quality in practice. Foresight: International Journal of Applied Forecasting, 36, 5–12.
[17]
Franses, P. H., & Legerstee, R. (2010). Do experts' adjustments on model‐based SKU‐level forecasts improve forecast quality? Journal of Forecasting, 29(3), 331–340.
[18]
Franses, P. H., & Legerstee, R. (2013). Do statistical forecasting models for SKU‐level data benefit from including past expert knowledge? International Journal of Forecasting, 29(1), 80–87.
[19]
Goodwin, P. (2000). Correct or combine? Mechanically integrating judgmental forecasts with statistical methods. International Journal of Forecasting, 16(2), 261–275.
[20]
Goodwin, P. (2002). Integrating management judgment and statistical methods to improve short‐term forecasts. Omega, 30(2), 127–135.
[21]
Goodwin, P., Önkal, D., & Lawrence, M. (2011). Improving the role of judgment in economic forecasting (pp. 163–192). Oxford Handbook of Economic Forecasting. Oxford University Press.
[22]
Green, K. C., & Armstrong, J. S. (2013). Demand forecasting: Evidence‐based methods. In C. Thomas & W. Shughart (Eds.), The Oxford handbook of evidence‐based management. Oxford University Press.
[23]
Griffin, D., & Tversky, A. (1992). The weighing of evidence and the determinants of confidence. Cognitive Psychology, 24(3), 411–435.
[24]
Hardin, J. W., Hardin, J. W., Hilbe, J. M., & Hilbe, J. (2007). Generalized linear models and extensions. Stata Press.
[25]
Harrison, G. W., & List, J. A. (2004). Field experiments. Journal of Economic Literature, 42(4), 1009–1055.
[26]
Hewage, H. C., Perera, H. N., & de Baets, S. (2022). Forecast adjustments during post‐promotional periods. European Journal of Operational Research, 300(2), 461–472.
[27]
Hyndman, R. J. (2014). Measuring forecast accuracy (pp. 177–183). Practical Problems and Solutions.
[28]
Hyndman, R. J., & Koehler, A. B. (2006). Another look at measures of forecast accuracy. International Journal of Forecasting, 22(4), 679–688.
[29]
Ibrahim, R., Kim, S.‐H., & Tong, J. (2021). Eliciting human judgment for prediction algorithms. Management Science, 67(4), 2314–2325.
[30]
Kahneman, D., & Smith, V. (2002). Foundations of behavioral and experimental economics. Nobel Prize in Economics Documents, 1(7), 1–25.
[31]
Kasparov, G. (2017). Deep thinking: Where machine intelligence ends and human creativity begins. Public Affairs.
[32]
Katok, E. (2011). Using laboratory experiments to build better operations management models. Foundations and Trends® in Technology, Information and Operations Management, 5(1), 1–86.
[33]
Khosrowabadi, N., Hoberg, K., & Imdahl, C. (2022). Evaluating human behaviour in response to AI recommendations for judgemental forecasting. European Journal of Operational Research, 303(3), 1151–1167.
[34]
Kremer, M., Siemsen, E., & Thomas, D. J. (2016). The sum and its parts: Judgmental hierarchical forecasting. Management Science, 62(9), 2745–2764.
[35]
Lawrence, M., Goodwin, P., O'Connor, M., & Önkal, D. (2006). Judgmental forecasting: A review of progress over the last 25 years. International Journal of Forecasting, 22(3), 493–518.
[36]
Levitt, S. D., & List, J. A. (2009). Field experiments in economics: The past, the present, and the future. European Economic Review, 53(1), 1–18.
[37]
Makridakis, S., & Hibon, M. (2000). The M3‐competition: Results, conclusions and implications. International Journal of Forecasting, 16(4), 451–476.
[38]
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2022). The M5 competition: Results, findings, and conclusions. International Journal of Forecasting, 38(4), 1346–1364.
[39]
Massey, C., & Wu, G. (2005). Detecting regime shifts: The causes of under‐ and overreaction. Management Science, 51(6), 932–947.
[40]
Moravec, H. (1988). Mind children: The future of robot and human intelligence. Harvard University Press.
[41]
Nakano, M., & Oji, N. (2010). The transition from a judgmental to an integrative method in demand forecasting: A case study of a Japanese company. International Journal of Operations and Production Management, 32(4), 386–397.
[42]
Narayanan, A., & Moritz, B. B. (2015). Decision making and cognition in multi‐echelon supply chains: An experimental study. Production and Operations Management, 24(8), 1216–1234.
[43]
Perera, H. N., Hurley, J., Fahimnia, B., & Reisi, M. (2019). The human factor in supply chain forecasting: A systematic review. European Journal of Operational Research, 274(2), 574–600.
[44]
Petropoulos, F., Fildes, R., & Goodwin, P. (2016). Do 'big losses' in judgmental adjustments to statistical forecasts affect experts' behaviour? European Journal of Operational Research, 249(3), 842–852.
[45]
Petropoulos, F., Kourentzes, N., Nikolopoulos, K., & Siemsen, E. (2018). Judgmental selection of forecasting models. Journal of Operations Management, 60, 34–46.
[46]
Petropoulos, F., & Siemsen, E. (2023). Forecast selection and representativeness. Management Science, 69(5), 2672–2690.
[47]
Rooderkerk, R. P., DeHoratius, N., & Musalem, A. (2022). Retail analytics: The quest for actionable insights from big data on consumer behavior and operational execution. Working paper.
[48]
Sanders, N. R., & Ritzman, L. P. (1992). The need for contextual and technical knowledge in judgmental forecasting. Journal of Behavioral Decision Making, 5(1), 39–52.
[49]
Sanders, N. R., & Ritzman, L. P. (2004). Integrating judgmental and quantitative forecasts: Methodologies for pooling marketing and operations information. International Journal of Operations and Production Management, 24(5), 514–529.
[50]
Sanders, N. R., & Wood, J. D. (2019). The Humachine: Humankind, machines, and the future of Enterprise. Routledge.
[51]
Schweitzer, M. E., & Cachon, G. P. (2000). Decision bias in the newsvendor problem with a known demand distribution: Experimental evidence. Management Science, 46(3), 404–420.
[52]
Seifert, M., Siemsen, E., Hadida, A. L., & Eisingerich, A. B. (2015). Effective judgmental forecasting in the context of fashion products. Journal of Operations Management, 36, 33–45.
[53]
Siemsen, E., & Aloysius, J. (2020). Supply chains analytics and the evolving work of supply chain managers. Research report for Association of Supply Chain Management.
[54]
Stephens, G. (2011). Restrictiveness and guidance in support systems. Omega, 39(3), 242–253.
[55]
Theil, H. (1971). Applied economic forecasting. North‐Holland Publishing Company.
[56]
van Donselaar, K. H., Gaur, V., van Woensel, T., Broekmeulen, R. A., & Fransoo, J. C. (2010). Ordering behavior in retail stores and implications for automated replenishment. Management Science, 56(5), 766–784.
[57]
Wakker, P. P. (2007). Message to referees who want to embark on yet another discussion of the random‐lottery incentive system for individual choice. URL: http://people.few.eur.nl/wakker/miscella/debates/randomlinc.htm, Access Date, 3 June 2020
[58]
Zhou, H., Qian, W., & Yang, Y. (2022). Tweedie gradient boosting for extremely unbalanced zero‐inflated data. Communications in Statistics‐Simulation and Computation, 51(9), 5507–5529.
Recommendations
A collaborative demand forecasting process with event-based fuzzy judgements
Highlights We develop a structured collaborative demand forecasting based on fuzzy judgements. We examine differences in adjustments between single and multiple forecasters. Errors are reduced thanks to the combination of fragmented information. Fuzzy ...
Examining the merits of feature-specific similarity functions in the news domain using human judgments
AbstractOnline news article recommendations are typically of the ‘more like this’ type, generated by similarity functions. Across three studies, we examined the representativeness of different similarity functions for news item retrieval, by comparing ...
Select-the-Best-Ones: A new way to judge relative relevance
In the traditional evaluation of information retrieval systems, assessors are asked to determine the relevance of a document on a graded scale, independent of any other documents. Such judgments are absolute judgments. Learning to rank brings some new ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© 2023 The Authors. Journal of Operations Management published by Wiley Periodicals LLC on behalf of Association for Supply Chain Management, Inc.
This is an open access article under the terms of the Creative Commons Attribution‐NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.
Publisher
John Wiley & Sons, Inc.
United States
Publication History
Published: 15 May 2023
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 30 Nov 2024
Other Metrics
Citations
Cited By
View all- Angelopoulos SBendoly EFransoo JHoberg KOu CTenhiälä A(2023)Digital transformation in operations managementJournal of Operations Management10.1002/joom.127169:6(876-889)Online publication date: 4-Sep-2023
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in