Skip to main content
Cross Validated

Timeline for What are the shortcomings of the Mean Absolute Percentage Error (MAPE)?

Current License: CC BY-SA 4.0

33 events
when toggle format what by license comment
May 27 at 14:52 history edited Stephan Kolassa CC BY-SA 4.0
Updated Sluijterman reference after publication
May 21 at 9:16 history edited Stephan Kolassa CC BY-SA 4.0
added pointer to Sluijterman
Apr 18, 2024 at 19:35 history edited Stephan Kolassa CC BY-SA 4.0
changed URL for Mean APEs picture
Apr 18, 2024 at 7:55 history edited Stephan Kolassa CC BY-SA 4.0
added Mean APEs picture by Ivan Svetunkov
Dec 13, 2023 at 10:03 history edited Stephan Kolassa CC BY-SA 4.0
added link to DS.SE question
Mar 26, 2023 at 17:47 history edited kjetil b halvorsen♦ CC BY-SA 4.0
added 29 characters in body
Dec 19, 2022 at 9:07 history edited Stephan Kolassa CC BY-SA 4.0
added 85 characters in body
Sep 9, 2022 at 15:48 history edited Stephan Kolassa CC BY-SA 4.0
added link to thread discussing incrementing both the numerator and the denominator by 1
Dec 13, 2021 at 9:26 history edited Stephan Kolassa CC BY-SA 4.0
added 170 characters in body
Jul 26, 2021 at 19:23 comment added Stephan Kolassa @RDizzl3. One actor's negative outflow is another actor's positive inflow, so I would have no compunctions about swapping the signs. (And MAPE still has issues, per above.)
Jul 26, 2021 at 16:07 comment added RDizzl3 Hey @StephanKolassa thank you for the response. I was hoping that would be your suggestion as that is what I have been doing. The data I am doing with are monetary outflows - I think our clients just define them as negative but that just reflects a positive amount of money leaving a bank account.
Jul 23, 2021 at 4:53 comment added Stephan Kolassa @RDizzl3: to be honest, I have never come across that situation. For one, all percentages will be negative (what does a negative percentage error even mean?), and larger MAPEs (i.e., closer to zero) will be better. Depending on your use case, it might be easiest to just multiply everything by $-1$ to make everything positive, and then you are in the "standard" case again.
Jul 22, 2021 at 15:31 comment added RDizzl3 @StephanKolassa how does the interpretation of the MAPE change if all the values are strictly negative?
Mar 3, 2021 at 8:35 history edited Stephan Kolassa CC BY-SA 4.0
formatting
Feb 9, 2021 at 18:38 history edited Stephan Kolassa CC BY-SA 4.0
added 230 characters in body
Nov 19, 2020 at 16:26 comment added Stephan Kolassa @SrikanthRaja: that the MAPE tends to be smaller for larger volume series is a consequence of their usually having a smaller coefficient of variation. Just looking at a MAPE (or an accuracy number) is meaningless on its own - we need to take into account how easily forecastable a series is. 10% MAPE is unachievably good for some series, and ridiculously inadequate for others (where an extremely simple forecast might already achieve a MAPE of 5%).
Nov 19, 2020 at 16:14 comment added SrikanthRaja @Stephan Kolassa et al, one observation : MAPE tends to be lower for targets measured on larger scale. Say, population forecasts in a small town which is expected to be in 1000's . 1 - MAPE if considered as an accuracy metric would give a false impression that the model is accurate even though the low R-square is hinting at a poor model. Do you agree?
May 4, 2020 at 12:36 history edited Stephan Kolassa CC BY-SA 4.0
added caveat about differentiability of MAPE and possible mitigation
Nov 5, 2019 at 10:56 history edited Stephan Kolassa CC BY-SA 4.0
Updated Kolassa (2020) reference
Sep 18, 2019 at 12:21 history edited Stephan Kolassa CC BY-SA 4.0
added McKenzie reference
Jul 10, 2019 at 14:08 history edited Stephan Kolassa CC BY-SA 4.0
added reference Kolassa (2019)
Jan 26, 2019 at 21:15 history edited Stephan Kolassa CC BY-SA 4.0
added 170 characters in body
Sep 25, 2018 at 10:30 comment added Pere @Ben Percentages of absolute temperature are legitimate, but differences of temperature are easier to understand - at least, when we deal with temperatures in the everyday range; when forecasting star core temperature it may be the other way.
Sep 25, 2018 at 10:04 history edited Stephan Kolassa CC BY-SA 4.0
deleted 1 character in body
Sep 25, 2018 at 9:49 history edited Stephan Kolassa CC BY-SA 4.0
added 312 characters in body
Sep 25, 2018 at 9:24 history edited Stephan Kolassa CC BY-SA 4.0
added 141 characters in body
Jul 11, 2018 at 7:46 history edited Stephan Kolassa CC BY-SA 4.0
added 49 characters in body
Jun 6, 2018 at 6:25 comment added Stephan Kolassa @Ben: in that case, we won't divide by zero. However, the asymmetry is still a slight problem. If your forecast is 293K and the actual is 288K, you have an APE of 1.74%, and if the forecast is 288K while the actual is 293K, the APE is 1.71%, so the second forecast looks better, though both are off by 5K. (Translate into C or F as needed.) Essentially, the same absolute errors are penalized more strongly for lower actuals. Plus, interpretation of percentage errors for temperatures is not easy.
Jun 6, 2018 at 0:34 comment added Ben I agree with most of this, however, wouldn't it be legitimate to deal with ratios of temperatures so long as they are on their proper scale (i.e., the Kelvin scale)?
Feb 8, 2018 at 8:41 history edited Stephan Kolassa CC BY-SA 3.0
added 92 characters in body
Aug 25, 2017 at 12:55 comment added user78229 Excellent Q&A. I would add that all of these metrics have two big underlying assumptions -- the series is iid and stationary. If one or both of these assumptions are not met, which happens frequently in practice, then their validity is questionable.
Aug 25, 2017 at 10:32 history edited Stephan Kolassa CC BY-SA 3.0
deleted 2 characters in body
Aug 25, 2017 at 8:49 history answered Stephan Kolassa CC BY-SA 3.0
toggle format