Modes, Medians and Means: A Unifying Perspective

There is a much wider generalization here which is studied under the name of "property elicitation" in computer science, machine learning, and statistics. The generic question is: Given a loss function, what "property" of the distribution minimizes average loss; and given a "property", characterize all such loss functions. For example, Bregman divergences are (essentially) all losses that "elicit" the mean of a distribution. If you have any monotone continuous function g(), then |g(x) - g(s)| actually also elicits…

One can generalize this approach further to take any given loss function, not just a p-norm, and ask what statistic is achieved by minimizing loss. A field that studies this is called property elicitation, from the idea that the loss function elicits a certain kind of prediction be it mean, mode, etc. This field actually originated with human experts like weather forecasters - what is a loss function (or “scoring rule”) that incents them to predict accurately? Squared loss is…