Open Access

Technologies based on Geographic Information System (GIS) are widely used in society and are increasingly being integrated into school curricula and practice. Many claims have been made that the use of GIS in class has positive effects on a wide range of achievement and affective variables. However, empirical evidence for that, especially in the German situation, has been scarce. Systemic thinking has been central to the guiding objective of German geography education for many years and constitutes an important contribution to prepare students for life in a complex world. Yet, so far, specific test instruments and studies elucidating factors that help students improve this competency have been far from extensive. This dissertation aims at exploring the influence of a short ‘working with GIS’ vs. ‘working with maps’ unit on students’ achievement in geography, specifically, the systemic thinking competency. Based on literature a definition of geographic systemic thinking and an associated competency model were developed. In total, three one test time and two pre-/posttest with control group studies were conducted to develop test instruments and a treatment as well as to study the question at hand. The treatment used the topic ‘tourism in Kenya’. Partly Desktop-, partly Web-based GIS versions were used. In study 5, there were two different types of materials, which contained parallel contents/tasks. While one used an overview sheet of relevant GIS functionality (‘old’), the other integrated more step-by-step instruction directly into the text (‘new’). Variables included were systemic thinking, sex, age, stream/type of geography study/pre-score, grade/semester, language and migration background, pre-experience, affective variables, pre-basic spatial thinking skills, treatment and material type. Not all variables were used in every study. The largest study (study 5) used the results of 932 seventh grade students for analyses. The sample contains both high and middle stream students from three German federal states. The study highlights issues such as e.g. test time constraints, open task coding, partly ceiling effects and item difficulties partly deviant from the model expectations. For the analyses, both raw average scores and WLE estimates obtained by a Rasch analysis are used. Additionally, based on the WLE scores, HLMs are calculated. Overall, in study 5 GIS students do not improve pre- to posttest in systemic thinking. Consequently, GIS has no positive, and partly a significantly negative impact compared to maps, e.g. in a HLM with all other variables having significant effects included. Results for material type are mixed. For instance, on the one hand, t-tests show no significant difference in pre-posttest-change between students working with ‘old’ and ‘new’ WebGIS materials. On the other hand, the overall HLMs with other variables included show a significant negative effect only for the ‘old’ but not for the ‘new’ WebGIS materials. Only 23 students could be included in the ‘having already worked with an educational GIS’-sub-group (vs. 520). The improvement of these students pre- to posttest is not significant, but has an effect size above 0.2. A calculation with the ‘no pre-experience’ sub-group being reduced to students with similar characteristics (e.g. in terms of stream, GIS type) leads to 19 vs. 84 students and similar results. In both cases, students with pre-experience perform not significantly, but with an effect size above 0.2, better than students without pre-experience. Overall, the results could hint at students needing more pre-experience so as to not have so much mental capacity tied to getting used to the software and being able to concentrate more on the system interrelationships. However, due to the sample characteristics and study design, this cannot be proven by the present data and thus needs to be explored in further studies. Other variables (age, sex, migration and language background, stream, pre-score, pre-spatial thinking score) show mixed results depending on the analysis method used. This underlines the impact of methodological choices and the need for large sample studies in order to be able to take a closer look at individual sub-groups. Furthermore, the HLM results point to not all influencing variables having been included. In general, the impact of variables such as pre-achievement/ stream and sex on pre-posttest change evident in some of the analyses points to the need for more research to develop differentiated learning materials. The conducted studies also show, e.g. through deviations from the assumed model of systemic thinking, that there is still a great need for more studies in terms of test- and model development for systemic and spatial thinking in a geographic context.