Quotation Fissler, Tobias, Frongillo, Rafael, Hlavinova, Jana, Rudloff, Birgit. 2020. Forecast Evaluation of Quantiles, Prediction Intervals, and other Set-Valued Functionals.




A functional is elicitable (identifiable) if it is the unique minimiser (zero) of an expected scoring function (identification function). Elicitability and identifiability are essential for forecast ranking and validation, M- and Z-estimation, both possibly in a regression framework. To account for the set-valued nature of many interesting functionals such as quantiles, systemic risk measures or prediction intervals we introduce a theoretical framework of elicitability and identifiability of set-valued functionals. It distinguishes between exhaustive forecasts, being set-valued and aiming at correctly specifying the entire functional, and selective forecasts, content with solely specifying a single point in the correct functional. Uncovering the structural relation between the two corresponding notions of elicitability and identifiability, we establish that a set-valued functional can be either selectively elicitable or exhaustively elicitable. Notably, selections of quantiles such as the lower quantile turn out not to be elicitable in general. Applying these structural results to Vorob'ev quantiles of random sets, we establish their selective identifiability and exhaustive elicitability. In particular, we provide a mixture representation of elementary scores, leading the way to Murphy diagrams. Our paper is complemented by a comprehensive literature review elaborating on common practice in forecast evaluation of set-valued functionals.


Press 'enter' for creating the tag

Publication's profile

Status of publication Published
Affiliation WU
Type of publication Working/discussion paper, preprint
Language English
Title Forecast Evaluation of Quantiles, Prediction Intervals, and other Set-Valued Functionals
Year 2020
URL https://arxiv.org/abs/1910.07912


Fissler, Tobias (Details)
Hlavinova, Jana (Details)
Rudloff, Birgit (Details)
Frongillo, Rafael (University of Colorado Boulder, United States/USA)
Institute for Statistics and Mathematics IN (Details)
Google Scholar: Search