Issue: Untersuchung des Einflusses der Vorgabe von chemischen Fachinformationen auf die Testleistung in Kompetenztests

The National Educational Standards (NES) for the subject Chemistry define students’ learning outcomes in Chemistry for the 10th grade without a description of compulsory topics for all German states. In order to be able to evaluate the achievement of the standards it is necessary to develop competence models as well as adequate test items for an assessment. While  at school the content plays an important role, the contents are not defined in the NES explicitly. Instead, they are defined by the different federal states independently which results in only a small overlap between all German states when students graduate from middle schools.

Test items focusing on content knowledge are for this reason to a high degree unfair within the frame of a national assessment. As a result in the project ESNaS (Evaluation of the National Educational Standards for Natural Sciences at the Lower Secondary Level) two types of standard-based test items were developed. The first type provides content information in the item stem while the second task type does not include additional content information.

This study had two main aims. The first aim was to develop standard and model based test items which can be used for the assessment. The second goal was to validate the model and the item design used for the evaluation.

The most important method was the comparison of the two different types of tasks. Except for the item stem, identical items were used in the validation study. The items were distributed to booklets in an incomplete block design. Data were analysed using the probabilistic test theory which enables us to compare the different items on one scale although not every item was answered by every student. Regression and correlation analyses were conducted to validate the previously defined characteristics which had caused difficulty.

These characteristics are complexity of the content and the necessary cognitive processes. Both characteristics could be proved as influencing item difficulty for the items with content information in the item stem. For the items having no additional content information only the influence of the complexity could be proved.  Additionally; correlations to other important competences  such as reading ability and intelligence were analysed.

Termtime

• 2007 - 2010

Supervisor

• Elke Sumfleth & Maik Walpuski

Dissertation

• Mathias Ropohl