Retrieval or testing effect

Research has shown that the key to long-term knowledge is retrieval. Unless the learned material is consciously reviewed, people forget about half of the newly gained knowledge within days or weeks (Ebbinghaus, 1913; Averell & Heathcote, 2011; Murre & Dros, 2015). Retrieval practice works against forgetting. Retrieval practice - also called test-enhanced learning - is a learning strategy where we do not focus on getting information “in”, but on getting information “out” of students’ heads. It can refer to any activity (such as questions during class, quizzes, flashcards, brain dump, and examination questions) that requires retrieving information from memory without the help of any external sources. Retrieval practice is an effective strategy for improving learning outcomes and improving performance. It needs no technology, no money - it only needs a little bit of your time and effort as retrieving information requires mental effort. Through this effort, learning takes place. Several studies showed that knowledge acquired by retrieval practice is more resistant to interference effects and leads to a lower forgetting rate (Kliegl & Bauml, 2016; Racsmány & Keresztes, 2015; Szpunar, McDermott, & Roediger, 2008). In addition, it has a promoting effect on learning: it produces better organization of the acquired knowledge, enhances its transfer to new contexts, and produces faster access to learned information (Jacoby, Wahlheim, & Coane, 2010; Racsmány, Szőllősi, & Bencze, 2018; Zaromb & Roediger, 2010; Chan et al., 2018).

The testing effect is the phenomenon that retrieving information from short- or long-term memory can — under many circumstances — strengthen one’s memory of the retrieved information (Donoghue & Hattie, 2021; Rowland, 2014). Benefits of the retrieval effect (testing effect) compared to re-reading have been investigated several times in the past decade (Rowland, 2014; Adesope et al., 2017). Test-enhanced learning has been proved efficient on diverse learning materials including foreign word learning (e.g., Keresztes et al., 2014), text memorizing (Roediger & Karpicke, 2006), and skill learning (Kromann et al., 2009). Research on test-enhanced learning has been conducted in laboratory circumstances, in simulated school environments and in real school situations. However, the application of the testing effect in real mathematics educational environments needs to be examined from several aspects (Buchin & Mulligan, 2019; Dunlosky et al., 2013; Lyle et al., 2020; Peterson & Wissman, 2018), more applied research is needed in mathematics learning (Agarwal et al, 2021). Several factors can affect retention, such as context, subject and material complexity. It is not evident whether or not testing is an effective tool for learning mathematics. For this reason, our aim is to gain a better understanding of how to incorporate the testing effect in classroom settings and to lay the groundwork for further experiments. Although only a few studies investigated the testing effect on learning mathematics in a classroom environment, recent studies suggest that increasing retrieval practice may be an effective way of learning (Fazio, 2019; Lyle & Crawford, 2011; Lyle et al., 2016, 2020; Szeibert et al., 2022).

Our experiments

In a previous experiment, we showed that test-enhanced learning can be implied in secondary mathematics education (Szeibert et al., 2022). Participants of the experiment were ninth-grade students from a vocational school and an elite grammar school. Our experimental group was from the vocational school —this group was the weakest in mathematics with the lowest entrance examination scores. There were two control groups in the experiment. One control group was formed by all the other groups from the ninth grade of the vocational school, the other consisted of students from the elite school, i.e., all the members of two classes of 9th grade. In each group (control and experimental), the structure of the lessons followed the well-known model: checking the homework, learning something new, and practicing the new material through tasks and problems. In the vocational school, the allotted time for the topic learned during the experiment was 4 weeks, 3 lessons per week, altogether 11 lessons. In the grammar school, they had 6 weeks, 4 lessons per week, altogether 24 lessons for the same topic. Furthermore, both teachers were constantly consulting with us and with each other to make sure that the experiment goes fluently and that students get familiar with the same concepts, types of problems, and tasks. The only difference in the structure of the lessons was that in the experimental group, students wrote a test at the end of each lesson on the material learned on the given day. In these tests, there were two problems, mostly one theoretical and one practice problem to be solved—in some cases, problems were easy calculations. The control groups were taught in a “reported” way, which means that the solutions to the same problems were told by the teacher at the end of the lessons. At the end of the topic, students wrote the same final test. On this final test members of the experimental group outscored their schoolmates and reached statistically the same scores as the control-grammar group from an elite school. Using the method, we were able to reduce the performance gap in a given Geometry topic between students from the elite grammar school and students from the urban high-needs school.

In another experiment, our aim was to explore the efficacy of test-enhanced learning used for teaching mathematics at the university level (Szabó et al 2023). The experiment was carried out in classroom settings, concerning an obligatory course. The participants of the experiment were all first-year mathematics students who took the “Algebra and Number Theory 1” course. The course, which the students attended in six groups, consisted of one 60-minute lecture and one 90-minute problem-solving seminar per week for a total of 13 weeks. Each student completed a competence-level test in the initial class. Three of the six groups were randomly selected as the experimental group, the other three were the control group. Three groups learned Number Theory using the testing effect, and the other three learned using traditional methods. The problem-solving seminars consisted of tasks based on the theoretical subject matter of the previous week’s lecture, which were solved collectively with the help of a professor. The structure of each lesson for the control group was the following: at the beginning of the class, they wrote a short test on the previous week’s material (as is traditional in the case of this subject). This was followed by the discussion of homework and the main part, which is problem-solving with the aid of the professors. In the experimental group, the structure was almost identical, the only difference was that there was no test at the beginning of the lesson, instead, they had a test at the end of the class. The experimental and control groups learned the exact same information in the lecture and wrote the same final test. On the final test, the experimental group performed significantly better than the control group, although their performance on the initial competence exams was significantly worse. The results indicate that test-enhanced learning has a significant advantage in solving complex mathematical problems. To examine the effect of differences in individual competence, we divided the students in both experimental and control groups into low-, middle-, and high-performing groups. The efficacy of test-enhanced learning was demonstrated in all three performance levels. Regarding the three pairs of groups, members of the experimental group using test-enhanced learning performed better than those of the control group.

In our latest experiment on this topic, we compared the effectiveness of worked examples and a specific type of retrieval practice in an abstract algebra course concentrating on medium and long-term knowledge. Retrieval practice and studying worked examples can be both effective learning methods in the field of problem-solving. However, it is still a question of which one is more effective in the medium and the long term when learning to solve abstract mathematical problems. To see which one is better to use in abstract mathematics, we experimented with second-semester pre-service mathematics teachers in the frame of “Algebra and Number Theory 2”. During the semester we divided the class into two groups. One of the groups studied the material using retrieval practice while the other one learned with worked examples. They wrote two tests: one on the sixth week of the semester and another one at the end of the semester on the material they learned on the topic of polynomials. This way we measured their medium-term knowledge. The semester was closed by an oral exam. Later, we measured the long-term effects of these two methods by a “surprise” post-test four months after their algebra exam. According to the results of this research, there was no difference between the effectiveness of the two methods in the medium term. However, retrieval practice was more beneficial in the long term. Our findings suggest that testing can have a meaningful, long-lasting impact on learning and solving abstract mathematical problems.

Publications

Szabó, C., Zámbó, C., Muzsnay, A., Szeibert, J., & Bernáth, L. (2023). Investigación de la eficacia de práctica de recuperación en matemáticas universitarias. Revista De Educación, 401(1). https://doi.org/10.4438/1988-592X-RE-2023-401-584

Szeibert, J., Muzsnay, A., Szabó, C., & Bereczky-Zámbó, C. (2022). A Case Study of Using Test-Enhanced Learning as a Formative Assessment in High School Mathematics. International Journal of Science and Mathematics Education, 21(2), 623–643. https://doi.org/10.1007/s10763-022-10264-8

References:

Adesope, O. O., Trevisan, D. A., & Sundararajan, N. (2017). Rethinking the Use of Tests: A Meta-Analysis of Practice Testing. Review of Educational Research, 87(3), 659–701. https://doi.org/10.3102/0034654316689306

Averell, L., & Heathcote, A. (2011). The form of the forgetting curve and the fate of memories. Journal of Mathematical Psychology, 55(1), 25–35. https://doi.org/10.1016/j.jmp.2010.08.009

Butler, A. C. (2010). Repeated testing produces superior transfer of learning relative to repeated studying. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(5), 1118–1133. https://doi.org/10.1037/a0019902

Carpenter, S. K., Cepeda, N. J., Rohrer, D., Kang, S. H. K., & Pashler, H. (2012). Using spacing to enhance diverse forms of learning: Review of recent research and implications for instruction. Educational Psychology Review, 24(3), 369–378. https://doi.org/10.1007/s10648-012-9205-z

Chan, J. Y., Meissner, C. A., & Davis, S. L. (2018). Retrieval potentiates new learning: A theoretical and meta-analytic review. Psychological Bulletin, 144(11), 1111–1146. https://doi.org/10.1037/bul0000166

Donoghue, G. M., & Hattie, J. A. C. (2021). A meta-analysis of ten learning techniques. Frontiers in Education, 6:581216. https://doi.org/10.3389/feduc.2021.581216

Ebbinghaus, H. (1913). Memory: A contribution to experimental psychology. In Teachers College Press eBooks. https://doi.org/10.1037/10011-000

Fazio, L. K. (2019). Retrieval practice opportunities in middle school mathematics teachers’ oral questions. British Journal of Educational Psychology, 89(2), 653–669. https://doi.org/10.1111/bjep.12250

Jacoby, L. L., Wahlheim, C. N., & Coane, J. H. (2010). Test-enhanced learning of natural concepts: Effects on recognition memory, classification, and metacognition. Journal of Experimental Psychology: Learning, Memory and Cognition, 36(6), 1441–1451. https://doi.org/10.1037/a0020636

Keresztes, A., Kaiser, D., Kovács, G., & Racsmány, M. (2014). Testing promotes long-term learning via stabilizing activation patterns in a large network of brain areas. Cerebral Cortex, 24(11), 3025–3035. https://doi.org/10.1093/cercor/bht158

Kliegl, O., & Bäuml, K. T. (2016). Retrieval practice can insulate items against intralist interference: Evidence from the list-length effect, output interference, and retrieval-induced forgetting. Journal of Experimental Psychology: Learning, Memory and Cognition, 42(2), 202–214. https://doi.org/10.1037/xlm0000172

Lyle, K. B., & Crawford, N. A. (2011). Retrieving essential material at the end of lectures improves performance on statistics exams. Teaching of Psychology, 38(2), 94–97. https://doi.org/10.1177/0098628311401587

Lyle, K. B., Hopkins, R. F., Hieb, J. L., & Ralston, P. A. (2016). Spaced retrieval practice increases college students’ short- and long-term retention of Mathematics Knowledge. Educational Psychology Review, 28(4), 853–873. https://doi.org/10.1007/s10648-015-9349-8

Lyle, K. B., Bego, C. R., Hopkins, R. F., Hieb, J. L., & Raltson, P. A. (2020). How the amount and spacing of retrieval practice affect the short- and long-term retention of mathematics knowledge. Educational Psychology Review, 32, 277–295. https://doi.org/10.1007/s10648-019-09489-x

Murre, J. M. J., & Dros, J. (2015). Replication and Analysis of Ebbinghaus’ Forgetting Curve. PLoS ONE, 10(7), e0120644. https://doi.org/10.1371/journal.pone.0120644

Racsmány, M., & Keresztes, A. (2015). Initial retrieval shields against retrieval-induced forgetting. Frontiers in Psychology, 6, 657. https://doi.org/10.3389/fpsyg.2015.00657

Racsmány, M., Szőllősi, Á., & Bencze, D. (2018). Retrieval practice makes procedure from remembering: An automatization account of the testing effect. Journal of Experimental Psychology: Learning, Memory and Cognition. https://doi.org/10.1037/xlm0000423

Roediger, H. L. III, & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255. https://doi.org/10.1111/j.1467-9280.2006.01693.x

Rowland, C. (2014). The effect of testing versus restudy on retention: A meta-analytic review of the testing effect. Psychological Bulletin, 140(6), 1432–1463. https://doi.org/10.1037/a0037559

Szpunar, K. K., McDermott, K. B., & Roediger, H. L. (2008). Testing during study insulates against the buildup of proactive interference. Journal of Experimental Psychology: Learning, Memory and Cognition, 34(6), 1392–1399. https://doi.org/10.1037/a0013082

Zaromb, F. M., & Roediger, H. L. (2010). The testing effect in free recall is associated with enhanced organizational processes. Memory & Cognition, 38(8), 995–1008. https://doi.org/10.3758/mc.38.8.995