Bayesian Ensembles of Unlabeled Forecasts: A Rationale for Downweighting Extreme Forecasts

Speaker: Xiaojia Guo, University of Maryland

Abstract: In combining forecasts from several experts or models, the trimmed mean often offers improvement over the simple mean because it removes extreme forecasts that can severely bias the simple mean. The trimmed mean represents a crude form of unsupervised learning-a way to draw inferences from the current forecasts alone, without any past forecasts or realizations of the quantity of interest. In this paper, we introduce a more sophisticated unsupervised ensemble. Our ensemble follows from a Bayesian model of the forecasts and the experts’ biases, but without exact knowledge about which expert is least biased, second-least biased, etc., which is often the case in practice. This model learns only from the order statistics of the experts’ point forecasts, as if the forecasts were otherwise unlabeled. In other words, the model learns anonymously, without knowing the identity of the experts and how they performed in the past. According to the model, the more extreme a forecast is, the more likely it is to be biased. The forecasts more likely to be highly biased get lower weights in our ensemble. In an empirical study of time series forecasts from the M4 competition, we demonstrate that our Bayesian ensemble can outperform the simple and trimmed means and the best combination model from the competition. Our ensemble can also be used to produce prediction intervals or quantiles, which makes it a flexible tool for use in practice.  

If time permitted, I will also briefly talk about another paper on forecasting product diffusions using a Bayesian ensemble approach. In this paper, we introduce a unified, robust, and interpretable approach to producing pre- and post-launch distributional forecasts for a new product. Our model learns from past life cycles of comparable products, and can adapt to local changes in the marketplace. When quantile forecasts are needed, our model has the potential to provide meaningful economic benefits.