Analysis of Individual Test Of Astronomy STandards (TOAST) Item Responses

Abstract

The development of valid and reliable strategies to efficiently determine the knowledge landscape of introductory astronomy college students is an effort of great interest to the astronomy education community. This study examines individual item response rates from a widely used conceptual understanding survey, the Test Of Astronomy Standards (TOAST). The TOAST, a 27-item, multiple-choice format, criterion-referenced test, addresses both the full range of topics commonly taught in a one- or two-semester undergraduate introductory astronomy survey courses, and concepts described in various national science education standards, frameworks, and reform documents. The present study involves an examination of responses by 1104 participants, allowing for a rigorous item-by-item and distractor-by-distractor analysis of students’ responses. The results suggest that each individual TOAST item is functioning appropriately across a broad range of students, and has sufficient sensitivity to identify notable student misconceptions. These results also provide an opportunity to identify target areas of opportunity for astronomy education researchers that remain largely unstudied.