For the following situation, consider the type of inference that the test user desires to make from the test score and indicate the type of validation study that would be most appropriate (content, criterion-related predictive, criterion-related concurrent, or construct): An observational rating scale is being developed to rate the classroom performance of first-year teachers, and experienced classroom teachers are asked to examine the items on the rating scale to see if they are relevant to effective performance.
Identify and describe two (2) problems that can affect the results of a criterion-related validity study.
A psychologist has been studying delayed recall of certain words using a new memory measure. On this measure, she has found that the distribution follows a normal curve. Using the normal curve table (provided on Canvas), what percentage of students have z-scores below -1.12?
- There is a 0.66 correlation between the number of movies that Nicolas Cage appeared in and the number of people who drowned in a pool. How much of the variance in the relationship between these two variables is accounted for?
Full Answer Section
. Problems in Criterion-Related Validity Studies
- Criterion Contamination: The chosen criterion measure itself might be imperfect or contain irrelevant factors, leading to inaccurate estimates of validity.
- Sample Restriction: The study sample might not be representative of the target population, leading to generalizability issues and inaccurate validity estimates for the broader population.
3. Percentage of Students with z-scores below -1.12
Using the normal curve table, look for the z-score value of -1.12. The corresponding area to the left of this value represents the percentage of students with lower z-scores. This area is approximately 0.1230. Therefore, around 12.3% of students have z-scores below -1.12.
4. Variance Explained by Correlation
The square of the correlation coefficient (r^2) represents the proportion of variance in one variable explained by the other. In this case, r^2 = 0.66^2 = 0.4356. Therefore, approximately 43.56% of the variance in the number of people who drowned in pools is accounted for by the number of movies Nicolas Cage appeared in. However, it is crucial to remember that correlation does not imply causation, and this association might be due to other factors or chance.
Sample Answer
1. Validation Study for Classroom Performance Rating Scale
Type of Inference: Content validity
Most appropriate validation study: Content-related validation
Reasoning: The goal is to assess the relevance of the rating scale items to effective teaching, not to predict future performance or compare performance to other measures. Content experts (experienced teachers) can judge the items' ability to capture key dimensions of effective teaching.