Improvement of Experimental Learning Environments

Prof. Dr. Elke Sumfleth - Research Interests

 

Improvement of experimental learning environments

 

^ top

Improvement of experimental learning environments by teacher feedback and structuring aids

A preceding intervention study shows that students working in small groups perform significantly better in achievement tests than those taught by the teacher in front of the class. However, the video data of both kinds of instruction produce evidence for the following conclusions: While the direct instruction is carried out at its best, the group investigation shows two main deficiencies.

1.    Correct ideas tend to remain partly unconsidered and conclusions are often missing.

2.    The students do not recognize their own mistakes.

While the teacher in the control group collects the students' relevant ideas and corrects mistakes during experimentation, the students of the intervention group are often unable to do so. In group-work situations the students often forget the ideas they want to test, they describe observations of experiments and consider them to be conclusions, and they do not relate their results to the ideas or hypotheses from the beginning. As a consequence, a more intense structuring of the group work is necessary and the scientific gain of knowledge, which mainly depends on the three steps idea - experiment - conclusion, must become more evident for the students. Although experiments are a natural part of science instruction, they are often used inadequately and structured by narrow prescriptions like a "recipe". Learners are given very little opportunity to discuss or test their own hypotheses. "To many students, a 'lab' means manipulating equipment and not manipulating ideas" (Lunetta, 1998, p. 250).

Therefore an instructional approach has to consider generating hypotheses, testing them systematically in experiments, and integrating the new information thus obtained into students' knowledge structures. Klahr and Dunbar differ between a hypothesis space and an experiment space (SDDS; Klahr, Dunbar, & Fay, 1993; van Jooling & de Jong, 1997) and describe successful strategies for discovering or producing new information. It seems there is very strong evidence that students have to be explicitly taught in the requested skills.

Therefore the research should be directed on the specific problems of group work aiming at improving the task formulation, the training of the students with regard to basic competencies in planning experiments and testing interesting ideas and in drawing conclusions from experimental results.

Thus the hypotheses for this study are:

  • The success of the learning process can be increased by improving the structure of the experimental working procedures in group work situations.
  • The success of the learning process depends on the group's ability to deal with mistakes.

These hypotheses lead to two types of intervention, one type concerning the structuring of the group work, the other is feedback given to correct mistakes. In the first case, the students receive idea-cards and experiment-cards as well as a flow-chart in order to help them structure the group work. The task for the students is to write down every idea. Furthermore they have to note which experiment they conduct and which conclusion they draw. This process should help ensure that the students do not forget to review their ideas and to draw conclusions on every experiment. In addition they are supported by the flow-charts which represent an "optimal" way for conducting scientific studies. The second type of intervention is concerned with feedback to correct mistakes. In order to avoid correct ideas being wrongly disproved because of operational mistakes during the group work, students have the opportunity to ask the teacher questions at two defined points of time. On the one hand, they can ask the teacher whether the planned experiment fits with their idea before they conduct the experiment. On the other hand, the students can ask the teacher after the experiment whether the result is the expected one or not. In addition, there is a short lecture before each lesson summarizing the possible results of the last lesson to show students the mistakes they possibly made and to give them the chance to consider the results of the last lesson when conducting experiments for the present lesson, referring to the idea of cumulative learning.

The two types of intervention lead to a 2*2 design, resulting in four different treatments. The control-group (A) does not receive any additional support. Group B receives only the structuring aids, group C feedback to correct mistakes and group D both types of intervention.

The sample was taken from seven secondary schools in North Rhine-Westphalia. The groups were balanced by an intelligence test. There are three small groups in each treatment at every school resulting in a total number of 21 small groups in each treatment. Each small group consists of four students making a sample of 336 students in total. One group in each treatment at every school was video-taped for a detailed analysis of the group work. Since the intervention took five lessons, there were made 35 videos of the group work in each treatment, resulting in 140 videos in total.

The data were collected in a pre-post-test design with accompanying tests after each lesson. The pre-test consists of four different types of tests: a test on cognitive-skills, a multiple-choice-test on chemistry knowledge and a test on scientific procedures as well as a questionnaire on the social background. Short multiple-choice-tests on chemistry knowledge are used as accompanying tests. As a post-test there is again a test on scientific procedures, a multiple-choice-test on chemistry knowledge, completed by a connectivity-test on chemistry knowledge. In addition the same test on chemistry knowledge is used for a follow-up test.

As expected there are no significant differences in the pre-test data of the four treatments. The analyses of the pre-post data were done by using standardized residuals. With regard to the pre-post-test difference it is possible to say that the control group differs significantly from all other treatments except treatment B which received only the structuring intervention. Treatment D also differs highly significantly from all other treatments except treatment C. As a result one can assume that the intervention feedback has much more influence on the results than the intervention structuring. In order to verify this assumption the contrasts between the treatments were computed reporting highly significant results supporting the assumption. A test on between-subjects effects shows that the factor feedback is highly significant with p < .001 and F(1;295) = 12.75. With a partial eta-square of .041 it accounts for 4 % of the variance. Considering the multiple different factors influencing learning this may not be inessential. The same results (with lower significance) can be found analysing the accompanying tests.

The results show that students have deficiencies in organizing and reflecting their own working process. Since an external presentation of structuring aids did not appear to be helpful the succeeding study aims at optimizing students' learning outcomes by a structuring training. This training offers students in particular an introduction to the appropriate use of structuring aids in order to support self-regulated scientific inquiry.

Funded by DFG

Further information:

Maik Walpuski

 

^ top

Structuring training to improve experimental work in small groups

Since an external presentation of structuring aids did not appear to be helpful this study aims at optimising students' learning outcomes by a structuring training. This training offers students in particular an introduction to the appropriate use of structuring aids in order to support self-regulated scientific inquiry. The study is conducted as a laboratory study with a one-factorial control-group design. The efficacy of the training will be analysed using a pre-post-test design and additional video analyses. Thus the hypotheses for this study are:

  • The success of the learning process of small groups in chemical education can be increased by the implementation of a structuring training.
  • The quantity and quality of students' mistakes are changed by the implementation of the training mentioned above.

The study is conducted as a lab-study with a 2*2-1 design in three secondary schools in North Rhine-Westphalia, each with the same number of small groups. There are three different groups: A without any further support, B getting the teacher summary at the end of each group-work phase and C treated as B, but with additional structuring training. At five consecutive days students have to work independently on different problems within the subject area acids and bases.

The efficacy of the mentioned training is analysed in a pre-post-test design. While students are working in small groups three of these groups are video-taped at each school in order to collect video data showing how students deal with mistakes. Analysis of the video data with respect to the types of mistakes will show whether quantity and quality between the two treatments have changed.

The comparison of pre- and post-test data shows a highly significant increase in the learning outcome for all treatments (p < 0.001, F(1;165) = 17.06, eta² = 0.174). Furthermore, standardised residuals show that treatment C achieves a higher increase in content knowledge than expected, while treatment A and B fall short of the expected theoretically expected values. The analysis of the video data shows that the students of treatment C solve the given task more often than the other students. This may be due to better-structured work in this treatment. The analysis of the video data with regard to the sequence of idea, experiment and conclusion shows that the students in treatment A make the most omission mistakes, the students in treatment C the fewest. This is a further hint that students in treatment C work in a more structured way. The most frequent omission mistake of all students is the omission of experiment and conclusion. In these cases only an idea is formulated. This result was to be anticipated, because students formulate a lot of ideas, discuss these ideas in different dimensions, decide about the right idea and forget about the others. The second most frequent omission mistake is the omission of idea and conclusion, only the experiment is conducted. This mistake can be observed especially in treatment A, which adverts again to unplanned testing. This type of mistake can be rarely seen in treatment C. The omission of the experiment is less in all treatments, because all students like to work with chemicals. Also the omission of idea and experiment can be seen rarely, because the formulation of a conclusion is very difficult without an idea and / or an experiment. This situation can be observed when students use their previous knowledge and are convinced of the correctness. In this case students often formulate statements in terms of a conclusion without reasoning or testing.

Funded by DFG

Further information:

Isabel Wahser

For people interested in this field it might be helpful to have a look at the projects concerning context-oriented chemistry teaching.

 

^ top

Instructional quality of lab work

In research on quality in education, many different characteristics influencing the learning process of individuals have been identified (e.g. Brophy & Good, 1986; Tobin & Fraser, 2003; Helmke, 2007. Helmke (2002) and Ditton (2000) state that looking at general quality of education exclusively is outdated, so they call for specific characteristics of quality in specific subjects. With regard to this, the aim of the study is to identify those experiment-specific characteristics which enhance interest and have a positive effect on achievement in chemistry education. This will be achieved by observational research (video-analysis) based on a category system and paper-pencil-tests.

The project is divided into two phases.
In school year 2008/2009 (first phase), 18 lessons in chemistry education were videotaped. The topic was 'Alcohols' in grade 10 of secondary schools (middle track). Teachers (N=16) were instructed to demonstrate a typical chemistry lesson under the condition that at least one experiment should be conducted. These 18 classes are the first control-group.
One year later (second phase), 10 of these teachers participated in an intervention in which the characteristics of quality in phases of experiments were given by a short teacher training with a feedback based on the videos from the first phase of the project. The direct control-group is formed by parallel classes (without intervention) taught by the same teachers, which will be videotaped previous to the intervention-group.

In order to analyse the videos, especially the experimental phases, a high- and low-inferent category system were constructed. This category systems are based on the systemic model of quality in education (Reusser & Pauli, 2003), the offer and use model (Helmke, 2007) and results of the general quality of education research (e.g. Biggs, 1979; Harvey & Green, 1993; Smith, 1985). Beyond a mere description of chemistry instruction, the cognitive ability test (Heller & Perleth, 2000), an achievement test (pre-post), a test on scientific procedures (Klos et al., 2008) and a questionnaire about interest and motivation (pre-post) are used.

First results: It could be observed that the planning of the experiments averages 16 % of the teaching time, the procedure 58 % and the analysis 19 %. Thus, 92 % of the teaching time is used for experimentation. This is not surprising because of the condition that the teachers had to conduct at least one experiment. Mostly, the procedure of the experiments is organised in group work (94 %). In certain cases, the students have to work at different stations (2 %) or have to conduct a demonstration-experiment (4 %). During the group work, the students spend more time to conduct the same experiment (92 %) compared to conducting different experiments. Furthermore, it could be observed that experiments are mostly used at the beginning of the lessons (61 %). Conducting an experiment to test a hypothesis is seen in 27% of the videos. The third function of an experiment, which could be observed, is confirming a hypothesis (12 %). A comparison of the results of both studies is still under work.

Funded by a grant of the Research Training Group nwu-essen (DFG)

Further information:

Alexandra Schulz,
Collaborative Project with Maik Walpuski, Osnabrück

 

^ top

The influence of the quality of subject-specific statements on the learning outcome in small-groups

This project deals with the analysis of student-student communication in cooperative group work. The topic of the small-group work is acids and bases in the 7th grade of secondary schools (high-level). The small-group work consists of five lessons.

The aim is to clarify the influence of subject-specific statements in small-groups on the learning outcome. The following research questions are due to the objectives:

  • Which characteristics of subject-specific communication can be found analysing the statements of successful and unsuccessful small-groups?
  • Is it possible to increase the quality and amount of communication by well-directed instructions of subject-specific communication situations?
  • Is it possible to enhance the learning outcome by the improvement of the small group communication?

The few existing systems for categorising students' statements are insufficient to investigate the quality of the statements in detail. The adapted instruments are:

  • instruments to analyse the amounts of speech of each participant,
  • instruments to describe inquiry processes (process plots: e.g. Walpuski & Sumfleth, 2009),
  • instruments to analyse content-related quality (e.g. Hänze &Berger, 2007; Franke-Braun, 2008).

Developing an instrument to analyse the content wise quality of statements it is not far to adapt models of competence. The ESNaS model of competence (Walpuski et al., 2008) was taken into account. We assume that - in addition to the complexity and the cognitive processes - the correctness of the statements is also relevant for the quality of subject-specific communication. So, this category system contains among others the category content which includes the correctness, the complexity and cognitive processes of the content-related statements. To test the category system, 30 videos (15 videos of very successful groups and of 15 very unsuccessful groups) of the preceding project were reanalysed using the program videograph®.

The assumed characteristics for successful small groups were found in reanalysis. The successful groups state more right content-based statements than the other groups. Regarding only the right content-related statements the successful groups reveal a higher rate of statements in all cases. These differences are significant in the subcategory "selection of a relation/relations" and very highly significant in the subcategories "organisation of a relation/relations" and "integration of a relation/relations".

But due to the fact that this reanalysis does not prove any causality an additional intervention study is needed. Therefore, in a second step, it will be investigated in an intervention study if it is possible to increase the quality and amount of subject-specific statements by modified instructions for the small groups. Furthermore, it will be analysed if the learning outcome can be enhanced at the same time. Therefore in the school year 2009/2010, 48 different small groups (4 students each) were videotaped. The investigation of the effectiveness of the revised instructions is realised in a classic control-group-design so that no artefacts are produced. As control variables the cognitive ability test (Heller & Perleth, 2000) and a test on scientific procedures (NAW) (Klos, Henke, Kieren, Walpuski, & Sumfleth, 2008) were performed. In addition to this, a test on chemical content knowledge was used (pre-test and post-test). Additionally, in each lesson a test on content knowledge, referring to the contents of the corresponding lessons, was conducted.

Funded by DFG

Further information:

Rebecca Knobloch,
Collaborative Project with Maik Walpuski, Osnabrück

 

^ top

Advancing Experimental Methods in the Science Classroom

The study just started.

Funded by DFG

Further information:

Meike Bergs

Project Experimentation (Wirth, Walpuski, Leutner, Sumfleth)
Collaborative Project with Maik Walpuski, Osnabrück,