Hungarian Educational Longitudinal Program

The first Hungarian longitudinal research program on national representative sample was launched in Szeged, in 2003, tracking students' development throughout the years of public education.

Among others the Hungarian Educational Longitudinal Program aims at constructing the scientific foundations, infrastructure and organizational framework of follow-up studies. In addition, the program intends to examine the starting phase of primary education, to explore the reasons of falling behind, to analyze and model paths of students' individual development, to describe the stability of student development, to construct diagnostic assessment instruments creating the possibility of early diagnosis, and to discover when and how school might intervene effectively in the process of development and what preventive actions it might take in order to evade learning difficulties.

The program was launched in the 2003/2004 academic year in grades 1, 5, and 9. In 2007 the three longitudinal segments formed a continuous line of data, which made the global description and analysis of the 12 years of public education (4 years of lower primary, 4 years of upper primary and 4 years of secondary education) possible with the combination of longitudinal and cross-sectional data, collected in three cohorts, in this first period of the project. A Further cohorts entered the program in 2007, 2011, 2015 and 2019 when they entered primary school in September. For further information click on the Assessment sample.

The Research Group gradually shifted from paper-based testing to online (computer-based) assessment. Since 2015 most of the data collection was carried out online by applying the eDia system.

 

 

Program topics

 

The Hungarian Educational Longitudinal Program investigates the main characteristics of and changes in the major cognitive constituents of knowledge, the attitudes towards learning and the affective and sociocultural determinants of the learning process.


The first reference point in the analysis of individual development was the result of DIFER subtests. From 2015 a newly developed online school entry test battery has been used. The test battery was developed with the cooperation of the Szeged Centre of Research on Learning and Instruction.
In lower grades research was centered around the development of reading and mathematical skills, and in higher grades science as the third major literacy domain was also investigated, along with combinative ability, inductive reasoning and complex problem solving. Foreign language skills and information technology knowledge, two essential fields of communication in the information society, were assessed at the end of the primary and secondary levels of public education.
The program follows the development of reading skills, collects data about learning strategies, self-concept, social problem solving skills, civic competencies, cognitive factors influencing science knowledge and explores the characteristics of family and cultural background, school related attitudes and intention of further studies.


The instruments used in the assessments might be grouped into two categories according to their construction. Most of the utilized tests and questionnaires are self-developed, however some of them were adapted from international studies, making it possible to compare the achievements of Hungarian students to international results.

 

Assessment sample

 

The series of longitudinal data collection started in 2003 on three nationally representative samples, in grade 1, 5 and 9. In the first four years of the program (2003-2007) nearly 12.000 students were involved. The first phase of the research program was completed in 2007, since the oldest cohort, sample 1, left public education (completing grade 12). Sample 2 left primary school (completing grade 8) – their tracking and further involvement in the assessments is one of the objectives of the program. Sample 3 left primary school in 2010 and completed their maturation exam in 2014. This is the first sample where we collected data throughout their public education. Sample IV. left primary school in 2014. New samples were launched in 2011, 2015 and 2019 (sample V., VI. and VII.). 


The samples and data collections of the program


Academic year

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

Sample 6

Sample 7

2003/04

grade 9

grade 5

grade 1

 

 

 

 

2004/05

grade 10

grade 6

grade 2

 

 

 

 

2005/06

grade 11

grade 7

grade 3

 

 

 

 

2006/07

grade 12

grade 8

grade 4

 

 

 

 

2007/08

 

 

grade 5

grade 1

 

 

 

2008/09

 

 

grade 6

grade 2

 

 

 

2009/10

 

 

grade 7

grade 3

 

 

 

2010/11

 

 

grade 8

grade 4

 

 

 

2011/12

 

 

 

grade 5

grade 1

 

 

2012/13

 

 

 

grade 6

grade 2

 

 

2013/14

 

 

 

grade 7

grade 3

 

 

2014/15

 

 

 

grade 8

grade 4

 

 

2015/16

 

 

 

 

grade 5

grade 1

 

2016/17

 

 

 

 

grade 6

grade 2

 

2017/18

 

 

 

 

grade 7

grade 3

 

2018/19

 

 

 

 

grade 8

grade 4

 

2019/20

 

 

 

 

 

grade 5

grade 1

2020/21

 

 

 

 

 

grade 6

grade 2

2021/22

 

 

 

 

 

grade 7

grade 3


Sample sizes in the first year of the program



Quantity

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

Sample 6

Sample 7

Number of participating students

3131

3881

5286

6081

6755

6842

8637

Number of participating classes

111

179

244

302

230

318

408

Number of participating schools

57

102

127

172

172

187

241

 

 

DIFER

 

The studies conducted at the Department of Education, University of Szeged, lead by József Nagy have shown, that (1) the development of basic skills determine the success of school learning in the long term, and that (2) there are major differences in the level of intellectual and social development of students at the same age.

The assessment of the participating students was started with the subtests of the widely used DIFER (Hungarian acronym of Diagnostic System of Assessing Development), constructed by József Nagy and his colleagues. The first data collection takes place individually, at the end of the first school year.

The Hungarian Educational Longitudinal Program covers five of the seven fields of prerequisite knowledge of readiness for school:
1. writing-movement coordination
2. relational vocabulary
3. basic calculations
4. experimental deduction
5. basic social skills.

The data-collection is done by class teachers and external experts.

 

Reading skills

 

 

The assessment of reading skills has been part of national and international large-scale studies from the 1990's. However, the mostly cross-sectional design cannot provide sufficient explanations for the differences found between various age groups. Thus the longitudinal program tracks the development of reading skills from grade 2 onwards.
The tests contain texts of different length and type, corresponding to the age specialities of the investigated population. Two different text types are used in the assessment of grade 1 and 2 students, and three types are used in higher grades. The tasks connected to each text operate at the different levels, defined in the international literature. Hence, various operations are involved from easier information search to the more complex text-evaluation. The comparability of the achievement of different grades are ensured by the application of anchor texts.
 

 

Mathematics knowledge

 

The tests of every second grade consist of items similar to the subject-matter, whereas every first grade is assessed with items suitable for exploring the effect of mathematics on thinking development. One of the focal points of the tests in grade 3 and 5 is the concept of proportion together with the understanding of linear relationships. 
The test of basic calculations in grades 1 and 2 were constructed by Krisztián Józsa with the contribution of Zsuzsanna Szántó (teacher, Ottó Általános Iskola, Budapest). The test of subject matter knowledge and basic skills for grade 6 was developed by Csaba Csíkos and the test of mathematical thinking was developed by Benő Csapó.

 

Science knowledge

 

In today's society there is a universal need for knowledge based scientific thinking and science literacy. These are essential not only in the understanding of natural phenomena, but in interpreting economic and social processes as well. The program approaches the quality of school based science knowledge from the perspectives of (1) conceptual development and (2) real life applicability. 


Foundation of science literacy – Basic science concepts
Erzsébet Korom and Lászlóné Nagy investigate the enrichment and structuration of basic science concepts from grade 4 to grade 12. They collect data at the end of each characteristic phase of science education, applying two complementary instruments. The one entitled „Foundations of science literacy”is a diagnostic knowledge test that aims at exploring the acquisition and level of differentiation of basic science concepts (e.g. matter, living being, symptoms of life, attributes and changes of materials).The other instrument, entitled „Development of science concepts”, provides an extended, more differentiated view of students' understanding of the concepts involved in the knowledge test. It mainly uses open question format to investigate the application of these concepts in the interpretation of everyday phenomena. 
 
Application of science knowledge I, II
The test entitled „Science knowledge application” measures the application of school based science knowledge in a realistic, everyday context. The research aims at investigating if students who are about to leave primary (grade 7) and secondary (grade 11) education are able to understand and interpret those phenomena they encounter day by day based on the knowledge they had acquired in school.
The first assessment was conducted in Szeged, in 1995, repeated two times, in 1999 and 2006, by the Research Group on the Development of Competencies on a national sample.
These tests measure science knowledge in realistic – scientific, but not classroom based – context, where the selected curricular contents are manifest. The style of the tasks is natural, avoiding rigid, school based terminology. The tests were developed by Mária B. Németh.

 

Additional domains

 

Cognitive skills
Inductive reasoning
Inductive reasoning means generalization based on individual experiences. It is a fundamental means of acquiring new knowledge and applying the acquired knowledge in novel situations. Inductive reasoning is one of the most widely researched cognitive skills. It is compared to a number of cognitive phenomena: general intelligence, problem-solving skills or learning ability.
The tests used in grades 3-11 were constructed by Benő Csapó. The instruments applied in grades 3 and 4 consist of 4 subtests: letter sequences, number analogies, word analogies and number sequences. 
The test used to measure inductive reasoning in grades 1 and 2 consists of 37 figural items and was developed by Gyöngyvér Molnár. Considering the age of the target population special efforts were made to construct a nonverbal test, i. e. to include the most pictures and figures possible and to minimize text quantity. The structure of the test is based on Klauer's definition of inductive reasoning, thus individual thinking operations (generalization, discrimination, cross-classification, recognizing relations, discriminating relations, system formation) are measured on individual subtests.
 
 
Combinative reasoning
The assessment of the development of combinative reasoning in Hungary started in the Department of Education, József Attila University, Szeged at the end of the 1970's. The structure of the tasks was constructed based on the eight combinative operations identified in the preliminary work of theoretical analysis. The test consisted of altogether 12 – 6 formal and 6 pictorial – tasks, each with six different structures: repetitive and non-repetitive variations, non-repetitive combinations, all repetitive variations, all subsets and Descartes multiplications. In the formal tasks students had to construct a structure of numbers and letters, according to the predefined conditions. As opposed to pictorial tasks, in formal tasks no direct image can foster the work of solving the task. The test was constructed by Benő Csapó.
 
Complex problem solving
Complex problem solving has been part of OECD PISA program since 2003. The first assessments in the field in Hungary were conducted by Gyöngyvér Molnár at the turn of the millennium – prior to the PISA program. The Hungarian Educational Longitudinal Program utilizes her instruments. The tests intend to measure how students can transfer and apply their knowledge in new, realistic situations. The complex problem solving tasks span across school subjects and require students to solve problems they have never encountered in the same format before. The most problems in the tests focus on basic knowledge that is essential for all the students, and has already been acquired in the several years of mathematics and science education.
 
Foreign language
The program investigates text comprehension in English and German. Language skills are measured with tasks based on linguistic meaning that require students to understand the text. The tasks do not measure drilled structures, vocabulary or knowledge about the language, rather realistic language skills applicable in everyday situations. The task content and level of difficulty fits to the language levels, topics, task- and text-types described in the National Core Curriculum (1995, 2003) and the frame curricula (2000). In addition, they correspond to the six level requirement system of the Common European Framework (2002). In terms of this six level system, the level of task difficulty reaches level B1 in case of grade 8 students and B2 in case of grade 12 students. The development of the measures was led by Marianne Nikolov.
 

 

Influencing factors

 

Reading habits
According to earlier international findings, reading related beliefs strongly correlate with reading achievement. This relationship might be considered to be of causal nature. It is investigated which reading related beliefs can be measured in Hungarian students with the adapted IRA questionnaire. The findings might imply new suggestions in terms of teaching methods. The Index of Reading Awareness (IRA) questionnaire was adapted by Csaba Csíkos.

 
 
Attitude questionnaire
Non-cognitive aspects of learning related to the quality of measured knowledge is conducted with a questionnaire constructed by Benő Csapó. The questionnaire is intended to measure attitudes towards school subjects and towards school in general, intentions about further study and level of parents qualification.
 



Environmental conditions of learning

The questionnaire used in the program provides the possibility of comparing the collected data on students' family and cultural background to the data collected in the Hungarian National Competence Assessment. The instrument inquires about school learning and its effectiveness, e.g. the number of years spent in kindergarten, the number of failures and school changes and curricular aspects of the class (if it is specialized in any field, bilingual, etc.). The sociocultural background of the family is investigated through the received allowances, accommodation circumstances (e.g. number of rooms), the number of people living together, level of parents' qualification or internet access. The questionnaire was constructed by Mária B. Németh, based on the work of Gábor Kertesi (Institute of Economics of the Hungarian Academy of Sciences) and Gábor Kézdi (Institute of Economics of the Hungarian Academy of Sciences).

 


 
Learning habits (PISA 2000 student questionnaire)
The Learning habits questionnaire consists of the questions used in PISA 2000. It explores the individual characteristics of learning techniques (learning strategies and learning environment from the socio-cognitive elements) and personality features (motivation and self-concept) that influence the effectiveness of learning. Students rate each item on a four-step Likert-scale according to how characteristic the given statement is of him or her and how frequently.
The Hungarian version was constructed by Mária B. Németh, Csaba Csíkos, Anita Habók and Erzsébet Korom.