Applying a Modern, Missing Data Approach (Multiple Imputation) to Range Restriction Involving Selection of Students: A Simulation Study (73874)

Session Information: Educational Research, Development & Publishing
Session Chair: Johnson Li

Thursday, 23 November 2023 13:40
Session: Session 3
Room: Room 608
Presentation Type: Oral Presentation

All presentation times are UTC + 9 (Asia/Tokyo)

Range restriction is a common methodological problem arising from a selection procedure that reduces the range of scores for evaluation. Taking the association between high-school SAT and university GPA scores as an example, the observed correlation is often smaller than it should be because not all high-school students have a chance to be admitted to university, and hence, their university GPA scores are missing. Conventional approaches addressing this issue involve Thorndike’s (1949) bias-correction formulae where the variance of the unrestricted scores (e.g., variance of high-school SAT scores) can plugged into those formulae. Despite their popularity, this variance is not always available in practice. Pfaffel et al. (2016) proposes the application of a modern, missing-data handling approach—Multiple Imputation (MI)—to range restriction. To further explore its application to additional selection procedures and mechanisms (e.g., multiple selection factors such as a composite score based on SAT, teacher’s evaluation, etc., inclusion of mediating paths), this study addresses a Monte Carlo simulation study that evaluates the performance of Thorndike’s and MI approaches based on various manipulations: selection ratio, sample size, data distribution, and effect size. The results showed that the MI outperforms Thorndike's approach in terms of the bias of the estimates and coverage probability of the 95% confidence intervals. This offers a useful approach to correct for the bias due to various selection procedures in educational contexts, especially it does not require the variance of the unrestricted scores. Implications of the findings to educational research involving selection of students will also be discussed.

Authors:
Johnson Li, University of Manitoba, Canada


About the Presenter(s)
Dr Johnson Li is a University Associate Professor/Senior Lecturer at University of Manitoba in Canada

See this presentation on the full scheduleThursday Schedule



Conference Comments & Feedback

Place a comment using your LinkedIn profile

Comments

Share on activity feed

Powered by WP LinkPress

Share this Presentation

Posted by Clive Staples Lewis

Last updated: 2023-02-23 23:45:00