Welcome

Welcome to the project page of ECON 2022, where you will find general information about the study, its objectives and design.

Thank you for your interest in the project.

Your ECON 2022 team

About ECON 2022

Economic literacy is the prerequisite for achieving economic independence and helping to shape societies. The core of economic literacy is the individual’s ability to understand economic relationships on different levels, to assess them and to make informed decisions. Economic education is responsible for creating environments where effective learning can take place, so that students can acquire economic literacy. Therefore, the curriculum is of high importance, as it provides the basis for teaching and for creating learning materials. Through the ECON 2022 project, a base for further developing the curriculum and the instructional design of economic education will be established.

Four goals will be pursued:

1. The subject related curricula of the German state of North Rhine-Westphalia will be analysed.
2. The results of the analysis will aid the modelling process of the domain of economic literacy Existing subject specific instructional design approaches will be considered, as well as the current scientific discussions on assessing competences.
3. A test will be developed in alignment with the curriculum of North Rhine- Westphalia.
4. The status quo on economic literacy of students in grade 8 will be assessed.

The ECON 2022 project thus generates empirical evidence on the current state of economic education in the state of North Rhine-Westphalia. This will serve to provide recommendations for further developing the curriculum of the subject of economics. At the same time, it will facilitate monitoring the acquisition process of economic literacy.

Approach and design of the study

The term literacy is based on a functional understanding of education that focusses on the continuous learning throughout people’s lives. The acquisition of relevant competences, consisting of knowledge, skills, and attitudes, should enable learners to be an active member of society and take responsibility in economic situations.

Measuring economic literacy, as well as the development of a valid test using authentic tasks require a thorough analysis of the economic domain. The levels of competence will be developed along three roles regarding expected economic activities: Consumers, employees and economic citizens. On that basis, the competence requirements are in accordance with the aim of the curriculum to successfully manage economic tasks in life.

The preparation of the test will identify the leading ideas and contents of the curriculum in North Rhine-Westphalia (cf. Entwurf Kernlehrplan Wirtschaft, 2020) and further provide a comparative analysis of the relevant curricula as well as including the current state of research in the field. In this part of the study, the aim is to develop a model that represents the domain adequately.

Based on the domain analysis, a representative survey will be conducted. The development of the test for the authentic assessment will consider that economic literacy requires linguistic-argumentative as well as mathematical-analytical competences to portray holistic economic competences that enable learners to act in complex economic situations.

Existing tests in the field such as the Test of Economic Knowledge/ Test of Economic Literacy (cf. Soper & Walstad, 1987; Walstad, Watts & Rebeck, 2007) its German adaptation (Wirtschaftskundlicher Bildungs-Test by Beck & Krumm, 1998), or the PISA Financial Literacy study by the OECD (cf. PISA 2012; PISA 2015; PISA 2018) will provide blueprints for the development of test items.

For gathering the data, a national survey will be conducted in addition to the International Civic and Citizenship Education Study (ICCS 2022), which provides the time frame for ECON 2022. Regarding the content, the two tests will be independent. Socio-demographic data about the students or data about the characteristics of the schools will be collected and provided for ECON 2022 by the ICCS 2022 assessment. Therefore, the ECON 2022 survey can focus on economic contents. The sampling for the study is carried out jointly for ECON 2022 and ICCS 2022.

A computer-based assessment allows the development of innovative and interactive test items that reflect authentic economic real-life situations of students. This way, students’ knowledge, skills, and attitudes regarding economic participation can be detected. These are of increasing importance, considering the current national and international societal challenges.

The main survey will take place in the first half of 2022 after the successful testing of the pre-test around early 2021.  Students in grade 8 from 150 schools in North Rhine-Westphalia will be included in the main survey. A national report with the results of the survey is expected to be released in the first quarter of 2024; in addition, ECON 2022 will provide feedback for the participating schools.

Management and implementation of the study

The research project ECON 2022 is a cooperation project in the fields of vocational education & training and educational research & schooling at the University of Duisburg-Essen (UDE) and is carried out in coordination with ICCS 2022. The two PIs of ECON 2022 are:

• Prof. Dr. Esther Winther (Managing Director of the Institute for Vocational and Continuing Education, University of Duisburg-Essen) esther.winther@uni-due.de
• Prof. Dr. Hermann Josef Abs (Head of the national ICCS study center, University of Duisburg-Essen) 
  h.j.abs@uni-due.de

They coordinate a project team at the University of Duisburg-Essen working on domain and curriculum analyses, test development design and test analysis

• M. Sc. Fabio Fortunati (Junior Researcher) fabio.fortunati@uni-due.de
• M. A. Fenna Henicz (Junior Researcher) fenna.henicz@uni-due.de
• M. A. Nina Welsandt (Junior Researcher) nina.welsandt@uni-due.de

All field operations are implemented by the IEA-Hamburg.

Funding

The project is funded by the Ministry of School and Education of the State of North Rhine-Westphalia (MSB NRW).

Milestones​​

 

I. Modelling the Domain

The first fundamental step was developing a model of the economic domain. By means of an analysis of the relevant literature of the domain, a model was developed that depicts the specifics and guiding ideas of the domain - such as content-related sub-areas, domain-specific approaches or targeted knowledge acquisition processes.

The model forms the basis for the development of an assessment in alignment with the curriculum, for measuring competencies in economic education. Within the framework of analyzing the curricula, the model of the domain  was validated in order to be transferred into a psychometric model. In this phase of the project, a scientific advisory board was convened, which was consulted at regular intervals throughout the project.

II. Analyzing Curricula

Selected curricula from different states and school types in Germany were studied using a comparative content analysis. Using a code book developed on the basis of the domain model, the curricula were analyzed with regard to subject and learning process categories. In this way, it was possible to examine the extent to which state-specific and school-type-specific differences emerged. With regard to the domain model, it could be shown that it adequately represents the domain of economic education, since the elements of the model were found in all curricula, albeit in different forms.

III. Classifying Existing Measurements Instruments

The development of the instrument was preceded by intensive research of existing German and international questionnaires and testing instruments regarding Economic Literacy, as well as the classification of items for the measurement of Economic Literacy. In the process, content-related focal points, learning process-related categories such as the targeted types of knowledge and learning taxonomy levels, domain-specific approaches, and the real life relevance of eighth-grade students of the instruments were identified and compared. This resulted in a systematic overview that revealed strengths and gaps of existing instruments, which served as a guideline for the generation of questionnaire and test items for the ECON-2022 assessment. 

IV. Developing the Instruments

Based on the previous steps, the authentic ECON 2022 assessment was developed using the empirically proven difficulty-generating features of different degrees of specificity of the economic content, the modeling of the items, as well as the cognitive demand level of the 36 test items. 

In order to meet the requirement of a high level of relevance for eighth-grade students, authentic life situations of adolescents were modeled as a framework. The accompanying questionnaire included scales on economic information sources and learning opportunities, on attitudes and experiences as a consumer, on economic self-efficacy and on the perceived quality of teaching in economic school lessons.

V. Field Test

The field test took place, with a slight delay due to the pandemic, in spring 2021. In close cooperation with the IEA Hamburg, manuals and scripts were developed for those responsible for coordination and test administration at the schools that were drawn.

Using the data collected from the field test with 817 students from North Rhine-Westphalia, the empirical difficulties of the test items, the reliability, and the general fit of the test were determined using the IRT-based partial credit model. The results of the validation showed that the test reliably distinguishes between low and high ability overall. The scales of the questionnaire were subjected to factor and reliability analyses and modified in some places based on the results obtained. 

VI. Validating the Instrument

The second part of the validation of the test instrument was carried out by means of an expert survey. For this purpose, 25 people with expertise in the fields of economics, school and teaching, or test development were acquired and asked to evaluate each test item on the basis of the difficulty-generating features. In addition, the authenticity and usability of the entire instrument were rated. The ratings were then examined for discrepancies among themselves as well as discrepancies with the ratings of the test development team. Items with higher deviations from the ratings were then reexamined and revised as necessary with reference to the results of the empirical item analyses. 

VII. Main Survey

From March to June 2022, the revised assessment with 31 test items was used at 143 schools in North Rhine-Westphalia. Despite the pandemic, there were only a few dropouts, so that a satisfactory amount of data could be collected. Random quality checks by the project team and the IEA on the test days verified that the tests were conducted properly. Until the delivery of the data by the IEA is completed, the open text answers of the students were coded. For each item, 30 percent of the cases were coded by three independent persons with the aim of achieving high intercoder reliability. This ensured that the open answers could be adequately translated into numerical values.

VIII. Generating the Dataset and Analysis

Data processing and analysis with regard to answering the research questions of the project are expected to be completed at the beginning of 2024 with the publication of the project report. Subsequently, the findings of the assessment will be processed at the level of the individual schools so that school-specific characteristics of the test results can be identified and reported back. This will allow schools to identify and address their development potential. Finally, the project will result in the development of an empirically validated competency model for economic education.

Publications

Published/accepted articles

Conference contributions

Literature of the project

Beck, K., Krumm, V. (1998) Wirtschaftskundlicher Bildungstest (WBT) [German adoption of the Test of Economic Literacy from Soper and Walstad, 1987]. Hogrefe, Göttingen.

Ministerium für Schule und Bildung des Landes Nordrhein-Westfalen (2020). Kernlehrplan für die Sekundarstufe I Realschule in Nordrhein-Westfalen -Wirtschaft -Entwurf Verbändebeteiligung, 25.02.2020

OECD (2014), PISA 2012 Results: Students and Money: Financial Literacy Skills for the 21st Century (Volume VI), PISA, OECD Publishing, Paris. http://dx.doi.org/10.1787/9789264208094-en

OECD (2017), PISA 2015 Results: Students’ Financial Literacy (Volume IV), PISA, OECD Publishing, Paris. https://doi.org/10.1787/9789264270282-en 

OECD (2020), PISA 2018 Results: Are Students Smart about Money? (Volume IV), PISA, OECD Publishing, Paris, https://doi.org/10.1787/48ebd1ba-en.

Soper, J. Ch.&Walstad,W. B.(1987).Test of economic literacy, 2nd edn. Joint Council on Economics Education, New York.

Walstad, W. B., Watts, M.,& Rebeck, K. (2007). Test of understanding in college economics: Examiner'smanual(4thedn.). NewYork, NY: NationalCouncilonEconomic Education.