Questions tagged [metric]
A metric is a function that outputs a distance between 2 elements of a set & meets certain strict criteria (some 'distance' functions are not metrics).
421 questions
5
votes
1
answer
124
views
Is there a "better" approach when it comes to model evaluation on multiple test datasets?
I have two models trained and validated on the same training/validation data.
Now I need to evaluate them on multiple independent test datasets (e.g., 10 different datasets of the same measure).
Which ...
- 131
0
votes
0
answers
21
views
Efficiency metric for the estimation of a binary periodic signal with errors
Consider a binary sequence coming from a binary periodic signal with random value errors ($1$ instead of $0$ and vice versa) and synchronization errors (deletions and duplicates). I would like to ...
9
votes
3
answers
392
views
What makes a metric representative and reliable when comparing different processes?
I’d like to ask a general question about metrics.
Recently, while working as an intern, I was asked to evaluate the success of different marketing journeys (in a CRM/Marketing Cloud context). The data ...
1
vote
1
answer
73
views
Average precision in integral form
Recently I came across this wonderful guide explaining the nature of average precision:
https://datascience-intro.github.io/1MS041-2022/Files/AveragePrecision.pdf
My question is about Lemma4, ...
- 11
2
votes
1
answer
115
views
Intuition on divergences
The distance between two objects can be computed using a metric function. A divergence is similar to a metric function but it does not need to satisfy the triangle inequality or be symmetric. ...
- 675
2
votes
1
answer
86
views
Metric choice for Machine Learning algorithm
I am currently building a ML model for a binary classification problem.
I am currently using a curated dataset that was provided in a research paper, that has been perfectly balanced. However, it is ...
- 21
0
votes
0
answers
99
views
Comparison of GLM & XGB for Zero-Inflated Count Data
I am recent to XGB and I am now doing a project to model counts of claims for a certain insurance context and compare GLM & XGB.
Acronyms:
GLM - Generalized Linear Models;
XGB - Extreme Gradient ...
- 1
5
votes
4
answers
2k
views
Is the term "AUROC curve" actually correct or meaningful when referring to the curve itself?
When I got into Machine Learning over 15 years ago, I learned that AUC stands for "Area Under the Curve", meaning "area under the ROC curve" and ROC being the "Receiver ...
- 681
0
votes
0
answers
46
views
Computing/calculating SSB (sum of squares between clusters) for k-modes
I am trying to calculate metrics for validation/calculation of K-modes performance. I am doing my thesis and I need to group patients from binary variables (diseases) I recently read that SSW and SSB ...
5
votes
3
answers
1k
views
Is there something like Z-score but for highly skewed distributions?
I'm analyzing my anki reviews answer times. Since I have a lot of data (over 14 years) with gaps between reviews and different study patterns I want to analyze this in cohorts.
One metric I want to ...
- 59
3
votes
0
answers
74
views
"Centroid" induced by metric
A metric $d(x,y)$ on a vector space $V$ can be used to induce the definition of a centroid of a points set $\{x_1,\ldots,x_n\}$ with respect to this metric as a (not necessarily unique) point $c$ that ...
- 6,265
1
vote
2
answers
508
views
Cross validation doesn't work [closed]
I'm currently at the local uni listening to a data science presentation. We're here from work, I'm in DS/ML and have been for the last 7 years but my degree is comp sci, and thus I'm mostly self-...
- 217
1
vote
1
answer
134
views
Potential Sign Issues in a Composite Performance Metric for Model Selection
I am analyzing the results of various machine learning models for a regression task, using four metrics: RMSE, MAE, MAPE, and $R^2$. My approach involves two types of analyses:
Individual Metric ...
- 13
3
votes
1
answer
61
views
Metric for Measuring Similarity of Z2-Symmetric binary Vectors
Apologies if this question is somewhat physics-related, but I believe it has a strong statistical component that fits within the scope of Cross Validated SE.
I have two vectors $S$ and $S_{\text{noise}...
0
votes
0
answers
153
views
Average precision vs Average recall in object detection
There are two popular metrics for object detection: Average precision and Average recall. Do you can explain with examples, what are the cases to use AP, and what are the cases to use AR?
I agree that ...
- 41