Collapse

Describe the variety of ways that one can assess the validity of an instrument in terms of our self-esteem measure. If we were creating a new measure of self-esteem, what is the process that we would use to validate this measure?

 omplete the following readings from your textbook, Psychological Testing: A Practical Approach to Design and Evaluation:

• Chapter 9 omplete the following readings from your textbook, Psychological Testing: A Practical Approach to Design and Evaluation:
• Chapter 9

For this week's assignment, you will continue to focus on the two instruments you selected to assess constructs related to your dissertation topic. This time, your focus is on validity.

Your first paragraph should briefly remind the reader what you are studying. Then, under each construct subheading, define the construct and describe one instrument used to measure it. This week, your focus should be on how the researchers demonstrated validity of the instrument.

You should include:

• What types of validity were assessed (e.g., content, construct, convergent, discriminant, criterion),
• How they were assessed (e.g., correlations with related or unrelated constructs, factor analysis),
• And any relevant findings that support the validity of the instrument.

Again, I’m using an example no one in the class is allowed to use (Machiavellianism) so this is just to guide you! No one picked Machiavellianism as a construct so no one is allowed to use these examples ;) 

MACH-IV Example 

The MACH-IV (Christie & Geis, 1970) is a 20-item scale originally developed to assess Machiavellianism as a personality trait reflecting manipulation, cynicism, and pragmatic morality. Items are rated on a Likert-type scale (commonly 5- or 7-point), with some reverse-scored.

Recent work by Belaus et al. (2022) provides strong evidence for the structural, discriminant, and predictive validity of the MACH-IV in a Spanish-speaking Argentinian sample. After dropping problematic items 19 and 20, the authors supported a one-factor structure with adequate model fit (CFI = .92, RMSEA = .06) and reliability (α = .72; ω = .72). The scale showed discriminant validity through a negative correlation with Social Value Orientation (r = –.27), reflecting distinctiveness from prosocial tendencies.

In terms of predictive validity, higher MACH-IV scores were associated with lower cooperation in economic decision-making tasks, such as the Dictator Game (r = –.26 in DG-Take format). Participants high in Machiavellianism were more likely to expect selfish behavior from others and act selfishly themselves, even in hypothetical, non-monetary contexts.

An earlier study by Kaestner et al. (1977) explored predictive validity in a clinical sample of drug users using a behavioral "ten-dollar game." While the test-retest reliability was strong (r = .82), the game results did not clearly align with MACH-IV scores. The authors suggested this may have been due to confounds such as limited education or institutionalization. This emphasizes the importance of population-specific validation when using behavioral outcomes.

Trimmed MACH* Example

To assess Machiavellianism in my dissertation, one instrument I am considering is the Trimmed MACH* (Rauthmann, 2013), a short-form measure derived from the original MACH-IV. This five-item version was designed using item response theory (IRT) to retain only the most psychometrically sound items from the original scale. Items reflect key aspects of Machiavellianism (particularly cynicism and distrust) and are rated on a 5-point Likert scale.

Construct validity was confirmed through confirmatory factor analysis (CFA), which supported a unidimensional model (CFI = 1.00, RMSEA = .02). All items had high discrimination parameters in IRT modeling (a > 1.20), indicating that they strongly differentiate between levels of the trait across the continuum.

Convergent validity was demonstrated by strong correlations with both the original MACH-IV (r = .63; disattenuated r = .84) and another validated German Machiavellianism scale (r = .60). The Trimmed MACH* also showed discriminant validity, correlating more modestly with narcissism (r = .15) and psychopathy (r = .36), helping to distinguish it from adjacent constructs in the "Dark Triad."

No predictive validity or behavioral task outcomes were reported in this study; however, Rauthmann (2013) also examined criterion validity by correlating the Trimmed MACH* with real-world behavioral tendencies like manipulation tactics (e.g., betrayal, ingratiation). The Trimmed MACH* showed meaningful, positive correlations with tactics like betrayal (r = .40) and revenge (r = .39), indicating that it predicts behaviors theoretically linked to Machiavellian attitudes.

In summary, the Trimmed MACH* provides robust evidence for construct, convergent, discriminant, and criterion validity, making it a psychometrically sound and efficient instrument for assessing Machiavellianism, especially when a shorter survey is desired or response burden is a concern (Rauthmann, 2013).