Most popular data collection method for field research in psychology

Why are they used so much?

1. Some variables difficult to assess any other way (e.g., Internal states: attitudes, emotions)
2. Ethics: Allowing person to control what is revealed
3. Efficient: Especially with questionnaires and mass administration (in group or mailed)
4. Available: Research has provided methods to use
5. Huge literature in many areas provides nomological networks
6. Gives us good results (Lots of significant findings)

Criticisms

1. Method variance: All things correlate (.30?) just because they are assessed with the same procedure
2. MV criticism depends on domain: Strong with job stress; weak with job performance
3. Poor convergence with other sources suggests invalidity (but often other sources poor)
4. Self-reports are not objective

Method variance idea: Campbell and Fiske (1959)

Explanation: A certain amount of the variance in measurement is attributable to the particular method used.

Apparatus effects with Skinner boxes

Item formats with questionnaires/scales

This leads to the conclusions that:

• All things assessed with the same method will be related.
• Scales using the same format will correlate more than using different format.

An examination of the correlation matrix in almost any study shows the first is not true.

I showed in a 1987 study that the second was not true. Format itself does not inflate correlations.

Each measure (operationalization of a specific variable) has three components:

• True score (What we wish to assess)
• Error (Nonsystematic or random influences)
• Bias (Systematic influences not intended)

O = T + E + B

Each measure has it’s own mix of the three components, depending upon

• The theoretical construct being assessed
• The method of assessment

Questionnaires: Biased by literacy level

Face-to-face interviews: Biased by attractiveness

Phone interviews: Biased by tone of voice

Sensitive questions: Biased by social desirability

Ratings of performance: Biased by leniency tendency

When two measures share a bias, it affects their correlation. Can make it bigger or smaller

Model of Self-Report: Perception of Environmental Condition, e.g., Stressor

Copyright Paul E. Spector, All rights reserved, Last modified November 9, 1998.