Avoiding unintentionally difficult test items for immigrant minority students

Tamara van Schilt-Mol

1: IVA - Tilburg University, Netherlands, The 2: Babylon (Centre for Studies of the Multicultural Society) - Tilburg University

Paper

Avoiding unintentionally difficult test items for immigrant minority students

Previous research into the Dutch ‘Final Test of Primary Education’ has shown that this test contains a number of unintentionally and therefore unwanted, difficult test items for immigrant minority students whose parents speak Turkish or Arabic/Berber at home.

These extra-difficult test items are known as differential Item Functioning (DIF). DIF occurs when students of different subgroups (in the present study, native Dutch students and students whose parents speak Turkish or Arabic/Berber at home) do not have the same chances of answering a given item correctly, even though they are equally skilled in the construct to be measured, and thus have a comparable achievement level in a particular subject matter area. When the source of DIF is irrelevant to the construct to be measured, i.e., if answering the item correctly requires skills and/or knowledge other than those intended to be measured, there is a case of item bias (van Schilt-Mol,2007). Knowledge of sources of DIF and itembias are necessary for test constructors to avoid DIF and itembias in future tests as much as possible.

In this study two statistical procedures, Mantel-Haenszel (MH) and One Parameter Logistic Model (OPLM), have been used in order to identify DIF items in the ‘Final Test of Primary Education 1997’. Subsequently, five experiments have been conducted to detect possible sources of DIF, using three different research procedures: a Content-Response-Code procedure, a Think-Aloud procedure and a Rewrite procedure. On the basis of the results of these experiments, predictions were formulated concerning the possible sources of DIF to the disadvantage and advantage of Turkish and Moroccan students.

These predictions have been used in order to manipulate original DIF items into intentionally DIF-free items. These manipulated items were statistically analyzed with MH and OPLM again. Interpretation of the statistical results lead to the conclusion that five predictions about the sources of DIF could be confirmed. These predictions are: items specific characters, and the use of standard answer possibilities and difficult language can be sources of DIF to the disadvantage of Turkish/Moroccan students, questions concerning identifying spelling errors in verbs and questions concerning religion related subjects can be sources of DIF to the advantage of these students. Finally, the decision has been made whether the possible source of DIF was relevant to the construct that the test claimed to measure. Content analysis showed that nine items contained item bias (out of the 26 items for which the cause of DIF was confirmed). The remaining 17 items only had DIF. Analysis of these 17 items showed that five items contained ‘pitfalls’. Therefore, the advice is formulated to adapt or remove these five items as well as the item bias items.

In the presentation the results of the study will be presented. Also, recommendations will be given to detect and avoid DIF items.

Schilt-Mol, T. van (2007), Differential Item Functioning en itembias in de Cito-Eindtoets Basisonderwijs. Oorzaken van onbedoelde moeilijkheden in toetsopgaven voor leerlingen van Turkse en Marokkaanse afkomst. Amsterdam: Aksant Academic Publishers.

Session: Paper session
Bilingual Education 3
Thursday, April 3, 2008, 15:45-17:15
room: 08