1
$\begingroup$

I think I understand how MASE works when I have a single time series. But what if I have several, for which I want to obtain an overall accuracy measure?

It's straightforward to compute an aggregate statistic for other measures (MAPE, MAE, RMSE, etc), since there are only 2 vectors of numbers to consider: the predicted values and the actuals. But with MASE, you also need to take into consideration how to compute the scaling factor, which requires a training set.

Code that uses Rob Hyndman's Tidyverts framework would also be great to see, but not essential.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

The standard approach is to calculate the MASE separately for each series, using that series' scaling factor (classically, the in-sample MAE from the random walk forecast). Then aggregate these MASEs across series, e.g., by again taking the mean, or the median, or even weighting summary measures by the importance of your series.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Ta. Perhaps the default should be to weight by the length of each series? That would mimic the behaviour for the other measures. $\endgroup$ Commented May 27, 2020 at 7:51
  • $\begingroup$ It depends on what "the behavior for the other measures" is. For the MAPE, for instance, one would usually do the same as I outlined here, taking MAPEs separately per series and then averaging the MAPEs (possibly with some weighting, but that's uncommon). $\endgroup$ Commented May 27, 2020 at 7:56
1
$\begingroup$

A bit late to the party, but my answer could be useful for anyone else dropping to this question at a later stage.

What I'd do is plot the different MASE's in the form of a boxplot. In that way, you prevent taking another mean/median which reduces interpretability and simultaneously allow for easy graphical comparison (compare means, but also spreads and possible outliers).

Disclaimer: I'm absolutely not an expert in this field, merely sharing my personal approach.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.