Discussion: Measurement and Data Collection
Data collection requires evaluators to consider a wide and diverse variety of data sources. These include both primary and secondary data sources. Data collection can significantly impact your evaluation project. Evaluators sometimes overlook problems where secondary data is involved and overlook the fact that data for comparison purposes can affect validity. The types of data in the Learning Resources are not meant to be exhaustive but are the most common types in program evaluation.
To prepare for this Discussion, choose one of the cases provided, review the specified Learning Resources, and complete the corresponding prompts.
Case 1
Steven Levitt, the author of Freakonomics, is exemplary in his ability to make data tell “stories.” Students of public administration and organizational behavior are familiar with the famous Hawthorne effect. Levitt and List’s (2009) reanalysis of the original Hawthorne study (which uses incomplete, archived, and secondary data) shows how management programs that were developed to increase worker productivity can be evaluated or reevaluated, even though the experiments were conducted almost a century ago.
Discuss a descriiption of at least two data collection problems that the reanalysis of Levitt and List might bring to the Hawthorne effect.
Resources:
Langbein, L. (2012). Public program evaluation: A statistical guide (2nd ed.). Armonk, NY: ME Sharpe.
• Chapter 7, “Designing Useful Surveys for Evaluation” (pp. 209–238)
McDavid, J. C., Huse, I., & Hawthorn, L. R. L. (2019). Program evaluation and performance measurement: An introduction to practice (3rd ed.). Thousand Oaks, CA: Sage.
• Chapter 4, “Measurement for Program Evaluation and Performance Monitoring” (pp. 161-199)
