When designing and marketing an edtech app, we know that the features that sell are very often the features that tell the data story: dashboards for teachers and analytics and reports for administrators.
We agree that edtech applications should indeed improve student outcomes in a verifiable way. That is why our application provides key decision-makers with the tools and analyses they need to make data-informed decisions.
At the same time, as we have built and marketed our app, we have chosen to focus our development and talking points on the student experience and student success. More specifically, our developmental priorities have been on creating environments and experiences that students authentically connect with, where students feel safe and supported, and ultimately where students thrive as individuals.
One of the ways we support students through their educational journeys is by enabling dynamic customization of the learning environment and experience. Students can customize multiple features, including their avatars, dashboard, and even the theme and background colors of the app. Many of these customization options are connected with our rewards and badging system, which recognizes progress and rewards performance by unlocking advanced features for customization.
By empowering the personalization of their own spaces, students become increasingly invested in their learning, coming to see the learning environment and the experiences therein as something made by them and for them [1]. Within game-based environments, which have been shown to support risk-taking, persistence, and problem-solving [2], this personalization increases support of essential psychological needs, such as internal motivation, autonomy, and competence [3], resulting in improved task performance [4].
Research on gamification more broadly suggests its potential to significantly support student motivation [5] and improve outcomes in mathematics education more specifically [2, 6]. Existing research into the personalization of gamification is more limited, with most research failing to consider the importance of customization across the spectrum of gamification elements [3].
Effective gamification design in education comprises 21 elements across five thematic categories: ecological (immersion and realism), fictional (storytelling and narrative), social (interaction and collaboration), performance (measuring and evaluating performance), and personalization (individual preferences and motivations) [7].
One of the few studies that specifically explored differences in student outcomes between dynamic customization in gamified environments and one-size-fits-all (OSFA) approaches found that the multidimensional approach to customization improved both self regulation and intrinsic motivation [3].
Studies of adaptive gaming in STEM fields—where the learning experience responds to the needs and abilities of learners in context—consistently show increases in student interest, engagement, confidence, and motivation [e.g., 10, 11]. More specifically, within the field of elementary mathematics, motivation has been shown to positively impact performance [8], and motivational strategies–such as empowering personalization of the learning experience–have been shown to be particularly effective for lower-performing students and students with less interest in math [9].
While we promote the effectiveness of gamification and spotlight the strategies we employ to gamify the student learning experience, we do not utilize the gamification or customization of it as the primary means for improving student outcomes.
In fact, it is probably more accurate to understand our approach as gamifying a game-based approach to mathematics instruction [2]. Gamification involves adding game elements, such as badging and rewards, to the chosen instructional approach. On the other hand, game-based learning refers to using games as the primary learning experience. In this approach, the game itself is designed to teach specific educational objectives or skills. Our game-based learning approach includes elements such as problem-solving, critical thinking, and decision-making within the learning environment.
In this sense, our overall approach to mathematics education is tightly aligned with the effective mathematics teaching practices as defined by the National Council of Teachers of Mathematics (NCTM) [12]. Subsequently, we have developed these practices into and within a digital environment designed entirely to support the development of mathematical abilities in K-8 students [cf. 13, 14].
There is an interesting tension in the edtech industry, particularly when it comes to edtech applications designed to improve student outcomes. While students are the primary users of these apps, schools are the buyers. In this sense, it is ultimately the value edtech apps deliver to the schools that often most influence purchasing decisions.
Somewhat paradoxically, then, we believe that by making our largest investments into the development of the best student experience, we will have the greatest impact on student performance, and improved student outcomes will necessarily result in the positive benefits schools need to justify their purchasing decisions.
Empowering the customization of their learning journey is just one way we prioritize student learning and improved outcomes. We believe that improved student outcomes require little justification apart from themselves. That is why we have invested so heavily in developing the most compelling student learning experience, considering obvious supports such as appropriate curriculum pathways and formative feedback and less obvious supports such as customization to meet the underlying psychological needs common to all learners. These approaches combined result in increases in motivation, persistence, and ultimately improved performance.
[1] Tondello G. F., & Nacke L.E. (2020). Validation of user preferences and effects of personalized gamification on task performance. Frontiers in Computer Science, 2(29), 1-23. doi: 10.3389/fcomp.2020.00029
[2] Stohlmann, M. S. (2023). Categorization of game-based learning in middle school mathematics - A review. Middle School Journal, 5(4)5,16-26. doi:10.1080/00940771.2023.2254175.
[3] Rodrigues, L., Palomino, P. T., Toda, A. M., Klock, A. C. T., Oliveira, W., Avila-Santos, A. P., Gasparini, I., & Isotani, S. (2021). Personalization improves gamification: evidence from a mixed-methods study. In Proceedings of the ACM on Human-Computer Interaction, 5(287), 1-25. https://doi.org/10.1145/347471
[4] Altmeyer, P. L. M., Schmeer, L. V., Kruger, A. (2019). “Enable or disable gamification?” – Analyzing the impact of choice in a gamified image tagging task. CHI. 150. https://doi.org/10.1145/3290605.3300380
[5] Fadda, D., Pellegrini, M., Vivanet, G., Callegher, C. Z., (2021). Effects of digital games on student motivation in mathematics: A meta-analysis in K-12. Journal of Computer Assisted Learning, 38, 304–325. https://www.doi.org/10.1111/jcal.12618
[6] Bang, H. J., Li, L., & Flynn, K. (2023). Efficacy of an adaptive gamebased math learning app to support personalized learning and improve early elementary school students’ learning. Early Childhood Education Journal, 51, 717–732. https://doi.org/10.1007/s10643-022-01332-3
[7] Toda, A. M., Klock, A. C. T., & . . . Oliveira, W. (2019). Analysing gamification elements in educational environments using an existing Gamification taxonomy. Smart Learnimg Environments, 6(16). https://doi.org/10.1186/s40561-019-0106-1
[8] Karnes, J., Barwasser, A., & Grünke, M. (2021). The effects of a math racetracks intervention on the single-digit multiplication facts fluency of four struggling elementary school students. Insights into Learning Disabilities 18(1), 53-77. https://www.researchgate.net/publication/348488364_The_Effects_of_a_Math_Racetracks_Intervention_on_the_Single-Digit_Multiplication_Facts_Fluency_of_Four_Struggling_Elementary_School_Students
[9] Begeny, J. C., Codding, R. S., Wang, J., Hida, R. M., Patterson, S. L., Kessler, S., Fields‐Turner, F., & Ramos, K. A. (2020). An analysis of motivation strategies used within the small‐group Accelerating Mathematics Performance through Practice Strategies (AMPPS‐SG) program. Psychology in the Schools, 57(4), 540-555. https://www.doi.org/10.1002/pits.22334
[10] Gonzales, C. S. G., Munoz, V., & Toledo, P. (2016). Enhancing the engagement of intelligent tutorial systems through personalization of gamification. International Journal of Engineering Education, 32(1), 534-541. https://www.researchgate.net/publication/275970643
[11] Zourmpakis, A-I, Kalogiannakis, M.,& Papadakis, S. (2023). Adaptive gamification in science education: An analysis of the impact of implementation and adapted game elements on students’ motivation. Computers, 12, 1-20. https://doi.org/10.3390/computers12070143
[12] Index - National Council of Teachers of Mathematics (nctm.org)
[13] Hughes, J. E., Thomas, R., & Scharber, C. (2006). Assessing Technology Integration: The RAT – Replacement, Amplification, and Transformation - Framework. SITE 2006 Proceedings (techedges.org)
[14] Thomas, A., & Edson, A. J. (2019). A framework for teachers’ evaluation of digital instructional materials: Integrating mathematics teaching practices with technology use in K-8 classrooms.
Contemporary Issues in Technology and Teacher Education, 19(3), 351-372. A Framework for Teachers’ Evaluation of Digital Instructional Materials: Integrating Mathematics Teaching Practices with Technology Use in K-8 Classrooms – CITE Journal
The modern-day counterpart to stars and smiley face stickers on homework assignments, digital rewards, and badges have gained significant interest and adoption in the edtech space as methods for recognizing progress and rewarding performance.
While we are not unique in including a badging and rewards system in our app, our approach recognizes the potential pitfalls of such systems and aims to ensure that our badging and rewards strategies genuinely enhance the learning experience rather than just follow a trend. In turn, we have designed our system to ensure that it aligns with students’ intrinsic motivations while also providing tangible benefits to students to support their unique learning journeys.
Gamified systems, particularly reward-based systems, have been criticized for their potential to undermine motivation, satisfaction, and empowerment [1], or to simply have no positive effect on learning at all [2].
However, there is growing evidence suggesting that when rewards are aligned with the goals of the task, they can serve as powerful motivators without detracting from the primary objectives [3].
A 2023 study [4] involving 778 7th and 8th grade mathematics students explored the impact of gamification on student motivation. The study involved students interacting with a learning platform that incorporated badges, rewards, and other gamification elements over a semester. The results showed that the gamification elements increased senses of playfulness and accomplishment, which increased immersion in the learning process, which ultimately supported both intrinsic and extrinsic motivation. Ultimately, students viewed the badges and rewards not just as external tokens, but as genuine markers of their progress and achievement.
Similarly, a comprehensive meta-analysis, which included 30 independent studies with a total of 3083 participants, explored the effects of gamification on educational outcomes [5]. Overall, gamification was found to have a small to medium positive effect size on student outcomes. The specific gamification features of badging and reward systems had similar effect sizes, demonstrating that systems that provide recognition of progress and/or rewards for performance significantly improve student outcomes.
A more recent study found that students who learn in digital environments enriched with badging and reward-type systems outperform students in non-gamified environments across all academic measures, including a 65% increase in persistence and a 10% increase in grades [6].
Additionally, badging and reward systems have been shown to help students develop metacognitive skills as they intuitively come to understand their individual learning styles [7]. These systems also provide educators with clear indicators of student progress [6]. Educators can then use these insights into individual student trajectories to provide more tailored instruction and individualized support [8].
Of course, none of this is to say that gamification generally or badging and rewards systems more specifically provide any positive benefits in themselves. Effective gamification design is about designing for motivational experiences rather than designing the experiences themselves [9].
In other words, to support students' intrinsic motivation, autonomy, and confidence, gamified systems must do several things really well and all at the same time: offer meaningful choices, set achievable challenges, provide positive feedback, and be adaptable to individual preferences within a supportive environment [9].
This is why, within our larger user experience framework, we have designed our overall gamification strategy to leverage the benefits of badging and rewards systems while avoiding common pitfalls. Specifically, our badging and rewards system combines both badging for recognition of progress and badging for rewards for performance to maximize the potential to support student success. As a form of recognition, our badges provide students with consistent markers of achievement, providing a form of immediate feedback to encourage progress. As a form of reward, students can use their badges to personalize their own learning experiences. For example, students can unlock app customization features to individualize their learning environment, or they can access additional support tokens to help them solve more complex challenges as the curriculum progresses*.*
Addressing the Risk of Misplaced Priorities in Gamification
Despite its potential benefits, gamification can inadvertently prioritize non-essential outcomes, such as earning otherwise meaningless badges, over primary learning goals, specifically demonstrating subject matter competence [1]. At the same time, it is essential to acknowledge that students are entire persons [7]. Thus, while academic outcomes do indeed guide our work, we also acknowledge that student success is embedded within each students’ social and emotional complex. As a result, our system intentionally taps into the fundamental human desire for recognition and achievement, fostering deep senses of pride, responsibility, and agency among students.
We believe in empowering our students as whole persons. This is why our badging and rewards system is designed to both motivate students by recognizing their progress and to empower student agency by rewarding performance with tangible benefits that provide individualized benefits throughout each students’ unique learning journey. With each step of incremental progress, we want students to know–at a deeply personal level–that they can learn, that they can achieve and that they can succeed. Our badging and rewards system is just one way we communicate to students our belief in their potential.
[1] Hanus, M. D. & Fox, J. (2015). Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and performance. Computers & Education, 80, 152-161. https://doi.org/10.1016/j.compedu.2014.08.019
[2] Attila, Y., & Arieli-Attali, M. (2015). Gamification in assessment: Do points affect performance. Computers & Education, 83, 57-63. https://doi.org/10.1016/j.compedu.2014.12.012
[3] Malek, S. L., Sarin, S., & Haon, C. (2020). Extrinsic rewards, intrinsic motivation, and new product development performance. Journal of Product Innovation and Management, 37(6), 528-551. https://doi.org/10.1111/jpm.12554
[4] Alt, D. (2023). Assessing the benefits of gamification in mathematics for student gameful experience and gaming motivation. Computers & Education, 200. https://doi.org/10.1016/j.compedu.2023.104806.
[5] Huang, R., Ritzhaupt, A. D., Sommer, M., Zhu, J., Stephen, A., Valle, N., Hampton, J., & Li, J. (2020). The impact of gamification in educational settings on student learning outcomes: A meta-analysis. Educational Technology Research and Development, 68, 1875-1901. https://doi.org/10.1007/s11423-020-09807-z
[6] Chapman, J. R., Kohler, T. B., Rick, P. J., & Trego, A. (2023). Maybe we’ve got it wrong. An experimental evaluation of self-determination and Flow Theory in gamification. Journal of Research on Technology in Education. https://doi.org/10.1080/15391523.2023.2242981
[7] Costley, N. N. (2022). Teacher perceptions of gamification in K-8 classrooms. Available from ProQuest Dissertations & Theses Global. (2649294937). Retrieved from http://proxy-ln.researchport.umd.edu/login?url=https://www.proquest.com/dissertations-theses/teacher-perceptions-gamification-k-8-classrooms/docview/2649294937/se-2
[8] Thomas, E. A. (2023). The adoption of a learning management system by K-8 school teachers (Order No. 30526749). Available from ProQuest Dissertations & Theses Global; Publicly Available Content Database. (2833416528). Retrieved from http://proxy-ln.researchport.umd.edu/login?url=https://www.proquest.com/dissertations-theses/adoption-learning-management-system-k-8-school/docview/2833416528/se-2
[9] Van Roy, R., & Zaman, B. (2017). Why gamification fails in education and how to make it successful: Introducing nine gamification heuristics based on self-determination theory. In M. Ma & A. Oikonomou (Eds.), Serious games and edutainment applications (Vol. II, pp. 485–509). Springer.
From Fortnite to Candy Crush to Peloton, leaderboards are becoming standard components for gamification. By publicly displaying users' rankings based on various metrics, leaderboards incentivize participants to climb the ranks by outperforming others, driving engagement, motivation, and performance.
Despite their popularity, our app does not include a leaderboard.
This may seem like a bad design and marketing decision given the widespread adoption of leaderboards in industries as diverse as health and fitness, customer service, and, of course, education [1]. Indeed, research into leaderboards in education generally and mathematics education more specifically reports positive outcomes such as increases in student attention [2] and improved student performance [3].
However, a significant body of evidence suggests leaderboards can adversely affect students in a variety of ways. Leaderboards have been shown to increase negative emotions and decrease learner motivation [4]. However, the strongest argument against leaderboards is their tendency to make performance competitive, which can adversely affect certain student populations [5].
A study about leaderboards and female students is instructive here [6]. In the study, female participants were exposed to either a male-dominated leaderboard, a female-dominated leaderboard, or no leaderboard. Not only did exposure to the leaderboard not improve their math performance, the study found that participants in the female-dominated leaderboard condition performed comparatively worse on a math quiz. As a result, the research concluded that leaderboards tend to inspire social comparison, leading to negative effects on academic performance, particularly for female students.
Similarly, in a study of the effects of leaderboards in gamified mathematics education, researchers found that leaderboards favor male students significantly more than female students on measures of interest and enjoyment [7].
A meta-analysis of competition in gamified education found that competition typically results in dominance by high achievers, which in turn decreases the participation and performance of low achievers. In other words, competition—intrinsic to the purpose of leaderboards—widens the achievement gap, improving the performance of students already successful in the discipline while simultaneously harming less-able students [8].
Because perceived failure can lead to math anxiety, and math anxiety is negatively correlated with math achievement [9], our approach retrains student’s focus on development. Rather than spotlighting relative performance through comparison on a leaderboard, we choose to focus on individual progress and mastery. Our approach emphasizes personalized adaptation and positive feedback, catering to individual learning trajectories without the pressures of peer comparison.
Our decision not to include a leaderboard is not to say that we believe all gamification elements are detrimental. Gamification, when implemented thoughtfully, can serve as a powerful motivator [2] [3] [10]. That is why we include elements such as badges and awards, and customizable avatars to connect with and motivate our students.
Of course, many ed tech apps will continue to use leaderboards and champion their potential to engage and motivate. And there are probably many contexts where that approach makes sense.
However, the essence of any educational technology lies not just in the tool itself but in the pedagogical strategies embedded within it [11]. While gamification can serve as a motivator, its implementation must be thoughtful and aligned with overarching educational goals, ensuring genuine benefits for all students [12].
Our goals are simple—to help every student become proficient in math. By focusing on strategies that prioritize individual growth over comparative performance, we aim to create an environment that aligns with the intrinsic needs and motivations of students, ensuring better outcomes in mathematics and beyond.
[1] L. C. Amo, R. Liao, H. R. Rao, and G. Walker. (2018). Effects of leaderboards in games on consumer engagement. In SIGMIS-CPR '18, June 18–20, 2018, Buffalo-Niagara Falls, NY, USA, 2 pages. https://doi.org/10.1145/3209626.3209708
[2] Jaaska, E., Lehtinen, J., Kujala, J., & Kauppila, O. (2022). Game-based learning and students’ motivation in project management education. Project Leadership & Society, 3. https://doi.org/10.1016/j.plas.2022.100055
[3] Recabarren, M., Corvalán, B., & Villegas, M. (2023). Exploring the differences between gamer and non-gamer students in the effects of gamification on their motivation and learning, Interactive Learning Environments, 31(6), 3529-3542. https://doi.org/10.1080/10494820.2021.1933543
[4] Cao, Y., Gong, S-Y, Wang, Z., Cheng, Y., & Wang, Y-Q. (2022). More challenging or more achievable? The impacts of difficulty and dominant goal orientation in leaderboards within educational gamification. Journal of Computer Assisted Learning, 38, 845-860. https://www.doi.org.10.1111/jcal.12652
[5] Dichev, C., & Dicheva, D. (2017) Gamifying education: what is known, what is believed and what remains uncertain: a critical review. International Journal of Educational Technology in Higher Education, 14(9). https://doi.org/10.1186/s41239-017-0042-5
[6] Christy, K. R., & Fox, J. (2014). Leaderboards in a virtual classroom: A test of stereotype
threat and social comparison explanations for women's math performance. Computers & Education, 78, 66-77. http://dx.doi.org/10.1016/j.compedu.2014.05.005
[7] Gurjanow, I., & Ludwig, M. (2017). Gamifying math trails with the MathCityMap App: Impact of points and leaderboard on intrinsic motivation. In Proceedings of the 13th International Conference on Technology in Mathematics Teaching (ICTMT 13), Lyon, France, July 3-6, 2017 (pp. 108-115). ictmt13_gurjanow_ludwig.pdf (sciencesconf.org)
[8] Chen, C., Shih, C., & Law, V. (2020). The effects of competition in digital game-based learning(DGBL): A meta-analysis. Educational Technology Research and Development, 68, 1855–1873. https://doi.org/10.1007/s11423-020-09794-1
[9] Barroso, C, Ganley, C. M., McGraw, A, L., Hart, G, S., & Daucourt, M. C. (2021). A meta-analysis of the relation between math anxiety and math achievement. Psychological Bulletin, 147(2), 134-168. http://dx.doi.org/10.1037/bul0000307
[10] Karnes, J., Barwasser, A., & Grünke, M. (2021). The effects of a math racetracks intervention on the single-digit multiplication facts fluency of four struggling elementary school students. Insights into Learning Disabilities 18(1), 53-77. https://www.researchgate.net/publication/348488364_The_Effects_of_a_Math_Racetracks_Intervention_on_the_Single-Digit_Multiplication_Facts_Fluency_of_Four_Struggling_Elementary_School_Students
[11] Fadda, D., Pellegrini, M., Vivanet, G., Callegher, C. Z., (2021). Effects of digital games on student motivation in mathematics: A meta-analysis in K-12. Journal of Computer Assisted Learning, 38, 304–325. https://www.doi.org/10.1111/jcal.12618
[12] Zhu, Y., Xu, S., Wang, W., Zhang, L., Liu, D., Liu, Z., & Xu, Y. (2022). The impact of Online and Offline Learning motivation on learning performance: the mediating role of positive academic emotion. Education and Information Technologies, 27(7), 8921+.
https://link-gale-com.proxy-ln.researchport.umd.edu/apps/doc/A715923535/AONE?u=loyoland_main&sid=ebsco&xid=ea85e03e
