Summary of dissertation project Modelling of competences in the field of chemistry – development and evaluation of a competence structure model for the sub-area of organic chemistry

Theoretical background

Natural science degree programmes have high dropout rates not only internationally, but also in Germany. Over the past few years, the dropout rates for chemistry have risen continuously and have recently reached approx. 47 % (Heublein et al., 2020). Against this background, there is an ever-greater need for evaluation and further development of science education processes in the higher education sector.

One possible approach to evaluating education processes at higher-education level can be found in the modelling and measuring of competences. The process of competence diagnostics has already been used in the school context since the formulation of specific competences in educational standards, and has already been successfully tested in international comparative studies (PISA, TIMSS). The modelling of competences also lends itself to the higher-education context, since the concept of competences has also been an integral part of university teaching since the Bologna reforms of 1999. In the course of this research project, the potential of competence modelling and measuring is being transferred to the subject of chemistry – initially using the example of organic chemistry here.

Questions

The aim of the study is to derive difficulty-generating characteristics for tasks in organic chemistry from theory and to determine their influence on the empirical task difficulty of performance test tasks. This consideration gives rise to the following research questions:

1. Which specific task characteristics can be identified for the sub-field of organic chemistry?

2. What influence do the specific task characteristics have on the task difficulty of performance test tasks in organic chemistry?
    

Method

In order to identify the task characteristics specific to organic chemistry, two approaches are initially being pursued: On the one hand, characteristics are being derived from relevant literature, and on the other, examination tasks are scrutinised for specific task characteristics. The characteristics identified are understood as partial dimensions of the competence to be depicted in order to develop a competence structure model for organic chemistry at university level. For the development, researchers are using a three-dimensional competence structure, since this has proved useful for mapping competences in natural sciences (Mayer & Wellnitz, 2013). The model developed is then operationalised by means of test items. The psychometric basis for the construction is the Rasch model, since this takes into account the characteristics of both the task and the person in parallel (Bond & Fox, 2007). As part of a pilot test, the suitability of the test instrument for the Rasch model is first reviewed and then further developed. The further developed test instrument is used again as part of the main study to determine the correlation between the specific task characteristics and level of difficulty with the help of correlation and regression analyses.

Literature:

Bond, T. G. & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the Human Sciences. London: Lawrence Erlbaum Association, Inc.

Heublein, U. & Schmelzer, R. (2018). Die Entwicklung der Studienabbruchquoten an den deutschen Hochschulen. Berechnungen auf Basis des Absolventenjahrgangs 2016 DZHW Project Report, July 2018. Hannover: DZHW, 7.

Mayer, J. & Wellnitz, N. (2014). Die Entwicklung von Kompetenzstrukturmodellen. In D. Krüger, I. Parchmann & H. Schecker (eds.), Methoden in der naturwissenschaftsdidaktischen Forschung (p. 19-29). Berlin et al: Springer.