Briefly restate the general nature of your team’s RFP’s program (The ABC University Sexual Harassment).
How might you use an experimental design to test a question that would be relevant to your program evaluation (e.g., compare an outcome for a group that received the program versus one that did not during the same time period, especially if the participants are randomly assigned to be in the program/no program groups)? What is a specific question you could address for your example?
Describe the specifics of your design for addressing this question. What is your independent variable? What might be a viable dependent variable to answer the question? What hypothesis would you be testing for the proposed evaluation?
What would be one method you might use to reduce threats to internal validity with this design?
Sample Answer
The general nature of the program being evaluated is a face-to-face, two-part training on sexual harassment offered by the ABC University Counseling Center to new-admission students. The training focuses specifically on attitudes, beliefs, and norms to encourage bystander intervention by peers. The university is seeking an evaluation of the training's impact on the overall student community's response to sexual harassment.
Experimental Design for Program Evaluation
To rigorously test the training's effectiveness, an experimental design could be implemented, focusing on the immediate impact of the training on students' intention to intervene.
The general nature of the program being evaluated is a face-to-face, two-part training on sexual harassment offered by the ABC University Counseling Center to new-admission students. The training focuses specifically on attitudes, beliefs, and norms to encourage bystander intervention by peers. The university is seeking an evaluation of the training's impact on the overall student community's response to sexual harassment.
Experimental Design for Program Evaluation
To rigorously test the training's effectiveness, an experimental design could be implemented, focusing on the immediate impact of the training on students' intention to intervene.
Specific Question
Does receiving the ABC University's two-part sexual harassment training significantly increase a new student's self-efficacy and stated intention to intervene as a direct or indirect bystander in a simulated sexual harassment scenario, compared to a control group?
Design Specifics
Design Type
A Pretest-Posttest Control Group Design would be the most suitable experimental design.
Random Assignment: All new-admission students who elect to participate would be randomly assigned to either the Intervention Group (receives the training immediately) or the Control Group (placed on a waitlist to receive the training at a later date).
Variables and Hypothesis
| Component | Description |
| Independent Variable (IV) | Participation in the Two-Part Sexual Harassment Training. This is the manipulation, with two levels: 1) Intervention Group (received training) and 2) Control Group (did not receive training). |
| Dependent Variable (DV) | Bystander Intervention Intentions and Self-Efficacy. This would be measured using a validated, Likert-scale questionnaire administered immediately after the training, assessing factors like: (a) likelihood of confronting a harasser (direct intervention), and (b) likelihood of offering support to a victim (indirect intervention). |
| Hypothesis | Students randomly assigned to and who complete the two-part sexual harassment training will report significantly higher scores on the Bystander Intervention Intentions and Self-Efficacy dependent variable compared to students in the waitlist Control Group. |
Export to Sheets
Reducing Threats to Internal Validity
One major threat to internal validity in this design is history (events outside the program occurring between the pretest and posttest that could affect the outcome). However, given the immediate nature of this test, a more salient threat is Testing/Instrumentation.
Threat to Validity: Testing. The act of taking the pretest itself could prime the participants in both groups to think about sexual harassment and intervention, leading to a change in their posttest scores independent of the training program.
Method to Reduce the Threat: Incorporate a Solomon Four-Group Design.
While more complex and resource-intensive, the Solomon Four-Group design adds two additional groups that only receive the posttest.