3
$\begingroup$

I am using a scalar CFA to compare latent variable means for different groups to the overall latent mean in a kind of effects coding approach (I constrain the overall latent mean of the whole sample to be 0). Aside from statistical significance (is the latent mean of this variable for one group significantly different from the latent mean for the whole sample?), I would like to know about the effect size of this difference. If I were comparing groups to one another instead of the the mean, I would use:

$$\text{Cohen's } d = \frac{\mu_a-\mu_b}{SD_{pooled}},$$

where

$$SD_{pooled} = \sqrt{\frac{n_as_a^2 + n_bs_b^2}{n_a + n_b}}$$

(or maybe add some -1s to the mix for sample standard deviation).

How can I get some standardized effect size (would it be called a Cohen's $d$?) for my comparison of one group to the overall mean?

Some options I was imagining were:

  1. Same equation, except treat the overall mean as a group ($\mu_{overall}$)
  2. Do 1 (i.e. the numerator is $\mu_a-\mu_{overall}$), but account for the different levels of variation in different groups (I have 6) in the denominator: $SD_{pooled} = \sqrt{\frac{n_1\sigma_1^2 + n_2\sigma_2^2 + \dots + n_6\sigma_6^2}{n_1 + n_2 + \dots + n_6}}$

If you have citations and/or keywords (do we call this a Cohen's $d$?) around this I'd also be very happy to see that, too.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
5
$\begingroup$

some sort of standardized effect size

The question is, what standard is meaningful to communicate? Cohen's standardized effect sizes were motivated by his work in power analysis, which motivated separating the effect of $N$ from the test statistic (e.g., $t = d \times \sqrt{N}$, $F = f^2 \times \frac{df_1}{df_2} = \frac{R^2}{1 - R^2} \times \frac{df_1}{df_2}$). Cohen's $d$ wasn't necessarily chosen because a pooled $SD$ is a meaningful standard to interpret the size of a mean difference. It can be, or perhaps a reference/control group's $SD$ might be a meaningful reference (that would be the related measure, Glass' $\Delta$), but that doesn't sound like the situation you are in:

for my comparison of one group to the overall mean?

For this comparison, you might consider each group's own $SD$ as a meaningful standard. By constraining the overall $M=0$, each group's estimated mean is the estimated difference from the grand mean.

  • Note: If you want to account for different population proportions, you can use weights for your effects codes proportional to group size; Cohen, Cohen, West, & Aiken's (2003) classic regression textbook discusses this in Section 8.4

The standardized solution (commonly available in, e.g., lavaan or Mplus) expresses that latent mean(-difference) in units of that group's latent SD, so no special calculation needed in that case. But if you have another meaningful standard to divide your unstandardized latent mean(-difference) estimate by, that calculation should be straight-forward.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.