Organizational Research Methods (ORM)

INP7097.005

Fall 2004

Instructor

Dr. Paul Spector

Office: PCD (Psychology) 4138

Voice: 974-0357

E-mail: spector@chuma.cas.usf.edu

Available on campus every Thursday, and by e-mail and phone always.

Website: http://chuma.cas.usf.edu/~spector

Time and Place

PCD 2124, Thursdays 2-4:50

Overview

Organizational research is a considerable challenge due to severe limitations in how we can study humans in naturalistic settings, especially the workplace where employers can be reluctant to allow participation in studies. To fully exploit the opportunities that become available, the organizational researcher must be prepared to apply a large variety of methodologies, often in very creative ways. The organizational researcher must be skilled in assessment, design, and statistics. This course will be an overview of methods used in organizational research, focusing largely on research design/strategies. The course assumes basic knowledge of both psychometrics/measurement and statistics, and is designed to follow the psychology PSY6217 two semester sequence or the equivalent. The approach will be mainly (although not entirely) conceptual as opposed to quantitative.

Objectives

1. To acquire knowledge of organizational research methods used in I/O Psychology,

Organizational Behavior, and related fields.

2. To enhance skills in evaluating methodology in existing literature.

3. To provide experience in presenting results of research.

4. To gain expanded appreciation of how research is conducted and disseminated.

Grading

Grades will be based on the project and presentations, as well as classroom participation.

Missed Classes

Students are expected to attend all classes. All missed classes must be made-up by writing a two page (double spaced) overview of the topic that was discussed that week. The overview is due the week immediately following the absence. It can be e-mailed to me. Missing more than the first half hour of a class will be considered an absence, unless prior arrangements are made.

Readings

Required Text: Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference, Boston: Houghton Mifflin.

Weekly Readings. Weekly readings, that form the bulk of the material for the class, are in the attached reading list. The schedule shows which readings will be discussed each week. Most are from the major journals, and can be found in the library or online. I have no objection to students splitting the copying work, with each person making copies of certain articles and everyone trading, but this has caused problems in the past when certain people did not provide the articles they were supposed to on time. Each student is still responsible for acquiring all readings and being prepared. Be sure to acquire all readings during the first two weeks of the semester (before things get too busy). If journals are not available, be sure to notify me within the first two weeks of the semester.

Research Project

There will be a research project involving the collection of data to test a hypothesis. Each student will develop a hypothesis involving 2 or 3 variables, choose measures for each variable, and collect survey data. Both a written report in journal article format and brief 5-6 minute presentation are required.

Presentations of Own Articles

Each week students will do 10-12 minute Powerpoint presentations of an article they choose that describes an organizational research study. You should role-play as if you were the author of the particular paper. The presentations should be like those in symposium sessions at a major conference. The presentation should clearly cover the purpose/rationale of the study, hypotheses, method, results and conclusions. It might be necessary to simplify the study considering the time limitations. At the conclusion of the presentation, there will be questions of the presenter by the class, again similar to a conference. Each student will do two such presentations during the semester. The time limit must be adhered to, as part of this exercise is to get used to presenting conference papers in limited time.

Article Analysis

Some weeks one of the assigned readings will be an example of an organizational study (indicated with superscript 1 in the schedule). The purpose is to analyze, critique, and discuss the methodology used. In particular, as you read each article pay attention to the following:

1. Purpose/rationale of the study. How do the authors setup the study? How do they position it? What is the contribution to the literature, i.e., what gaps does it address?

2. Hypotheses (if any). Are they causal or merely relational? Do they involve 2 variables or more than two variables? Are they directional? Are the relations direct or are there mediational or moderator relations expected. Is there much of a basis for the hypotheses?

3. Method. What is the design of the study? How are variables operationalized? Are measures reliable? Are measures established with strong evidence for construct validity? Is this a single-source or multi-source design? How were the data collected, i.e., what is the procedure? Are their ethical issues involved in the study? What population was sampled? What are the limitations to generalizability in both population and setting?

4. Results. How were the data analyzed? How were hypotheses tested? Did results support or fail to support hypotheses? Did analyses make sense? Were they overly complex? Did the authors give a good rationale for what they did? If complex statistics were used, were the simpler building blocks also shown/discussed such as descriptive statistics and correlations?

5. Discussion. What do the authors conclude? Do the conclusions flow logically from the method/results or are they stretched? Do you agree with the conclusions? Are there feasible alternative explanations for results? Were limitations discussed and if so, did you find this discussion to be informative/thoughtful or merely obvious?

6. General. Was the paper clearly written/presented and did it do a good job of communicating the results of the study?

Class Sessions

There will be 6 major activities that will take place in class.

1. Instructor mini-lecture about methodology

2. Student symposium-style presentation of own articles

3. Article analysis

4. Discussion of weekly readings

5. Work on research project

6. Student presentation of project (last week of class)

Taping policy: Notes and tapes cannot be taken for the purpose of sale.

Schedule

The schedule lists the topics and assignment due dates. The OHP book chapter assignments are here, and specific readings are in the week by week reference list.

Date	Topic	Reading/Assignment
8/26	Course Introduction
9/2	Causality and correlation	SCC-1, Meehl, Moyle¹
9/9	Validity and threats to validity	SCC-2, Daft, Frese¹
9/16	Construct and external validity	SCC-3, Podsakoff, Van Dyne¹
9/23	Quasi-experiments 1	SCC-4, Stone-Romero, Barling¹
9/30	Quasi-experiments 2	SCC-5, Spector 94, Ludwig¹
10/7	Longitudinal designs	SCC-7, Zapf, Laschinger¹
10/14	Field experiments	SCC-8, Wainer, Bedeian
10/21	Survey methods	Edwards, Spector VanKatwyk, Schaffer
10/28	Levels	Hofmann, Pirola-Merlo¹, Sutton
11/4	Meta-analysis and reviews	SCC-13, Judge¹
11/11	Veteran’s Day Holiday
11/18	Ethical/practical issues	SCC-9-10, Aguinis, Von Glinow, Papers due
11/25	Thanksgiving
12/2	Wrap-up	SCC-14, Project presentations

Note: SSC is the Shadish et al. book, and the numbers refer to chapters.

¹Empirical study to analyze/discuss.

Readings

Aguinis, H., & Henle, C. A. (2002). Ethics in Research. In S. G. Rogelberg (Ed.). Handbook of research methods in industrial and organizational psychology (pp. 34-56). Malden, MA: Blackwell.

Bediean, A. G. (1987). Of fiction and fraud: A comment on Barry and Elmes’ article. Academy of Management Review, 22, 840-842. [Can be found on Art Bediean’s website at Louisiana State University.]

Daft, R. L. (1983). Learning the craft of organizational research. Academy of Management Review, 8, 539-546.

Edwards, J. R., & Bagozzi, R. P. (2000). On the nature and direction of relationships between constructs and measures. Psychological Methods, 5, 155-174.

Frese, M. (1985). Stress at work and psychosomatic complaints: A causal interpretation. Journal of Applied Psychology, 70, 314-328.

Hofmann, D. A. (2002). Issues in multilevel research: Theory, development, measurement, and analysis. In S. G. Rogelberg (Ed.). Handbook of research methods in industrial and organizational psychology (pp. 247-274). Malden, MA: Blackwell.

Judge, T. A., Heller, D., & Mount, M. K. (2002). Five-factor model of personality and job satisfaction: A meta-analysis. Journal of Applied Psychology, 87, 530-541.

Laschinger, H. K. S., Finegan, J. E., Shamian, J., & Wilk, P. (2004). A longitudinal analysis of the impact of workplace empowerment on work satisfaction. Journal of Organizational Behavior, 25, 527-545.

Ludwig, T. D., & Geller, E. S. (1997). Assigned versus participative goal setting and response generalization: Managing injury control among professional pizza deliverers. Journal of Applied Psychology, 82, 253-261.

Meehl, P. E. (1971). High school yearbooks: A reply to Schwarz. Journal of Abnormal Psychology, 77, 143-148.

Moyle, P. (1995). The role of negative affectivity in the stress process: Tests of alternative models. Journal of Organizational Behavior, 16, 647-668.

Pirola-Merlo, A., & Mann, L. (2004). The relationship between individual creativity and team creativity: Aggregating across people and time. Journal of Organizational Behavior, 25, 235-257.

Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879-903.

Schaffer, B. S., & Riordan, C. M. (2003). A review of cross-cultural methodologies for organizational research: A best-practices approach. Organizational Research Methods, 6, 169-215.

Spector, P. E. (1994). Using self-report questionnaires in OB research: A comment on the use of a controversial method. Journal of Organizational Behavior, 15, 385-392.

Spector, P. E., Van Katwyk, P. T., Brannick, M. T., & Chen, P. Y. (1997). When two factors don’t reflect two constructs: How item characteristics can produce artifactual factors. Journal of Management, 23, 659-678.

Stone-Romero, E. F. (2002). The relative validity and usefulness of various empirical research designs. In S. G. Rogelberg (Ed.). Handbook of research methods in industrial and organizational psychology (pp. 77-98). Malden, MA: Blackwell.

Sutton, R. I., & Staw, B. M. (1995). What theory is not. Administrative Science Quarterly, 40, 371-384.

Van Dyne, L., & Pierce, J. L. (2004). Psychological ownership and feelings of possession: Three field studies predicting employee attitudes and organizational citizenship behavior. Journal of Organizational Behavior, 25, 439-459.

Von Glinow, M. A., & Novelli, L., Jr. (1982). Ethical standards within organizational behavior. Academy of Management Journal, 25, 417-436.

Wainer, H. (1999). The most dangerous profession: A note on nonsampling error. Psychological Methods, 4, 250-256.

Zapf, D., Dormann, C., & Frese, M. (1996). Longitudinal studies in organizational stress research: A review of the literature with reference to methodological issues. Journal of Occupational Health Psychology, 1, 145-169.

Recommended Readings Outside of Class

Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.

Denzin, N. K., & Lincoln, Y. S. (Eds.) (1994). Handbook of qualitative research methods. Thousand Oaks, CA: Sage.

Fields, D. L. (2002). Taking the measure of work: A guide to validated scales for organizational research and diagnosis. Thousand Oaks, CA: Sage.

James, L. R., Demaree, R. G., & Wolf, G. (1993). Rwg: An assessment of within-group agreement. Journal of Applied Psychology, 78, 306-309.

Klein, K. J., & Kozlowski, S. W. (Eds.) (2000). Multilevel theory, research, and methods in organizations: Foundations, extension, and new directions. San Francisco: Jossey Bass.

Rosenthal, R. (1991). Meta-analytic procedures for social research. Thousand Oaks, CA: Sage.

Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428.

Van de Vijver, F., & Leung, K. (1997). Methods and data analysis for cross-cultural research. Thousand Oaks, CA: Sage.