4931ScaleDevelopment

SCALE CONSTRUCTION

1. Define the construct of interest very specifically

2. Design the format of the scale.

Summated rating
Bipolar vs. unipolar
Response categories (agreement, evaluation, frequency
Number of response choices (usually 4 to 7)
Special instructions

3. Write the initial item pool.

4. Administer to a sample (usually 100 or more).

5. Conduct item analysis to select items and determine internal consistency.

6. Continue to administer to collect norms and evidence for reliability and validity.

Examples of Response Choices

Agreement

Disagree very much

Disagree moderately

Disagree slightly

Agree slightly

Agree moderately

Agree very much

Evaluation

Terrible

Inferior

Passable

Good

Excellent

Frequency

Rarely

Seldom

Sometimes

Occasionally

Most of the time

Item Analysis

Analysis to determine how well each item relates to all other items.

Internal consistency reliability: The extent to which the items of a scale are related.

Coefficient alpha: Index of internal consistency
Ranges from 0 (no relation) to +1

Item-remainder coefficient: Correlation of an item with the sum of all other items in a scale.

Used to select items.
Items kept that meet a criterion, usually .30 to .40.

Coefficient alpha is rechecked after weak items are removed. It should increase.

Item analysis done in SAS with PROC CORR

Example of Item Analysis with SAS

data scale;

input item1-item4;

datalines;

1 1 1 1

1 2 2 1

1 2 1 2

1 1 2 2

4 4 3 4

3 4 3 4

4 4 4 4

2 2 2 2

3 3 3 3

2 3 2 2

;

proc corr alpha nomiss;

var item1-item4;

run;

Item Analysis SAS Output

Shows Coefficient Alpha and Item-Remainders

Simple Statistics

Variable N Mean Std Dev Sum Minimum Maximum

ITEM1 10 2.2000 1.2293 22.0000 1.0000 4.0000

ITEM2 10 2.6000 1.1738 26.0000 1.0000 4.0000

ITEM3 10 2.3000 0.9487 23.0000 1.0000 4.0000

ITEM4 10 2.5000 1.1785 25.0000 1.0000 4.0000

Cronbach Coefficient Alpha

for RAW variables : 0.964371

for STANDARDIZED variables: 0.966585

Raw Variables Std. Variables

Deleted Correlation Correlation

Variable with Total Alpha with Total Alpha

ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

ITEM1 0.957508 0.940045 0.957751 0.943961

ITEM2 0.908002 0.954255 0.904919 0.959225

ITEM3 0.882034 0.965772 0.881464 0.965883

ITEM4 0.924374 0.949408 0.922420 0.954210

Reliability and Validity

Reliability

Internal consistency: Coefficient alpha, item analysis.

Test-retest: Done on subsequent sample/s.

Time frame depends on theoretical stability.

Validity

Long term continuing process

Series of studies to assess various aspects

Convergent: Administer with similar measures
Discriminant: Administer with dissimilar measures
Content: Judgment by subject matter experts, SME
Criterion: Administer with other measures predicted to relate
Known groups: Compare groups predicted to vary
Factor structure: Does scale fit hypothesized structure?