Nigar Tuğbagül Altan, Oğuzkaan Sarıkaya (2022). Interactive Learning: Numbers Application Based on Augmented Reality. In: Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems. AHFE Open Access, vol. 22. https://doi.org/10.54941/ahfe1001086
Children with mental disorders struggle when they are taught with conventional teaching methods and materials. This is especially a major problem in early stages of learning. Since identification of those with special needs is also difficult/challenging in early years.In this study, an interactive numbers-learning application for children with mental disorders and/or preschool children is developed. The tool uses augmented reality activated through a marker to pop-up interactive 3D numbers. With the application children in their early childhood and/or children with mental or learning disabilities are able to direct the camera of their device to a 2-D number on a page (i.e. marker) and revive a 3-D augmented image. The application also speaks out that number. The ultimate goal of the application is to speed up the learning process in an enjoyable manner.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
INTRODUCTION
Nowadays, many applications are implemented to educate children. The new technology provides the incorporation of multimodal learning and enables the reconceptualization of play (Yelland and Gilbert, 2018). Multimodal learning is presented by the use of multiple sensory modalities like visuals, audio, and text. Computer-based learning environments which utilize multimodal learning are considered as strong tools to facilitate understanding, particularly for underachieving children, and also, improve attention (Sankey et al. 2010). Playing is an important piece of early childhood development and learning. According to researchers, a mixed reality play environment, a combination of physical experiences, new interactive characteristics of the virtual environment, and user’s imagination, is good for learning. Besides, they noted that engagement and interaction are crucial properties of learning virtual environments (Hinske, 2008). Learner interaction allows the students to interact and manipulate the course materials giving a flexible, personal experience. When participants actively engage and participate in such “rich and intensive” interactive activities, they learn effectively and learners’ performance increases with satisfaction and enhanced spatial skills (Yilmaz et al. 2015). Gaming and learning based on technology should be pieces of optimal quality education. Experts supply benefits from technology encapsulated audio and visual modalities for Kindergarten and elementary schools (Bujak et al. 2013). Children with different learning abilities and capacities are taught with the same teaching methods and material causing them to pay effort in the early stages of learning. Identifying special needs in learning is difficult with traditional methods while AR can be a solution to identify some special needs and differences between children. This paper aims to purpose and develop a marker-based tracking interactive learning numbers system based on augmented reality system that will aid preschool children, in their early childhood and/or children who will have mental or learning disabilities, in learning. The paper is organized in that section II presents related works, section III describes model and system, and section IV includes experimental results. Finally, the discussion and main conclusions are presented in sections V and VI, respectively.
RELATED WORKS It is emphasized in many academic studies that augmented reality technology contributes to the learning of preschool children. On the other hand, there is an opinion in the literature that when using augmented and virtual reality technology, children may be disconnected from the real world and have difficulty distinguishing between the real and the virtual world (non- real) (Wu et al. 2013). According to Azuma, AR is a form of virtual reality where 3D virtual objects are augmented within the real world, rather than replacing it completely (Azuma, 1997). In virtual reality, the degree of the disconnection between the virtual and real worlds is higher than in the augmented reality world. There have also been studies that have collected and evaluated the studies on whether augmented reality technology contributes
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7 positively or negatively to children's learning (Oranc and Kuntay, 2019). Some of the studies that argue that augmented reality technology (ART) contributes positively to children's learning are listed below.
In the studies on ART in the field of education, the contribution of augmented reality technology to learning, its effect on students' motivation levels, students' attention and interest in lessons, and parents' perspectives on augmented reality technology were examined. Some tests were conducted by the Valencia Polytechnic University with 32 students aged 5-6 years, 14 boys and 18 girls. These tests, it is aimed to teach students letters and words. Two learning methods were used for this purpose. One of them is a traditional teaching method with 2D objects, based on pictures and books. Another is an application where ART is used and there are 3D objects in it. As a result of the tests, 81% of the students stated that they preferred the education application with ART to the traditional education application (Juan et al., 2010). In a study, it is stated that the reason why ART increases the motivation level of students is that it offers face-to-face communication and easy use (Bujak et al. 2013). In another study, it is stated that ART provides interaction between the virtual and the real worlds in the field of education, and students who are motivated by ART, are more successful (Chen and Yen-Nung, 2012). In another study, it is related to the theory of mind test based on ART. Children with mental disorders who cannot pay attention in educational activities, are easily distracted, have difficulty participating in activities and have focus problems, are attracted to the activities in which ART is used, their attention is drawn and they focus on the content of the activity (Altan and Gokturk, 2012). In another study, an application is designed using ART. This application is aimed to teach preschool children animals, colors, numbers, objects, vegetables, and fruits. The application is tested with 33 students whose ages are 5-6. According to the test results, the students state that they are satisfied with the application designed with ART and that they like it (Yilmaz, 2016).In a study with parents, it is decided that ART make contributes to the education of their children (Cascales-Martínez et al. 2013). ART-based applications are not only designed to teach children certain concepts. For example, to increase children's interest in books or activities, the different computer interfaces, ART was designed with different aims (Rohaya et al. (2012), Fazli (2019), Altan and Gokturk (2017)). These applications have been highly appreciated and demanded by the end-users. As mentioned above, there are many studies similar to these studies. As a result of the literature review, in general, it was concluded that ART contributed positively to the learning of especially preschool children. Although ART's contribution to learning is so high, there are hardly any educational applications using ART. This paper, which aims to learn ART-based numbers for preschool children, has been designed. In the next section, the model and system are described in this work.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
MODEL AND INTERACTIVE LEARNING: NUMBERS MOBILE APPLICATION The user interface designs of this application, which was designed to teach numbers to preschool children, were made with the Unity Game Engine. The functional features of the application are coded with C# in this project. Vuforia SDK and Database, which can work integrated with Unity, are used to bring ART to the application. A database was needed to store the answers given by the user in the test section of the application. Firebase Database, which serves within the Google Company, was used in the design of the application as a database. In this developed application, it is aimed to teach numbers to preschool children. While doing this, the application is designed as ART-based to attract children's attention and have fun while learning at the same time. The flow chart of the application is shown in Figure 1.
Figure 1. Flow chart of the application.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
In Figure. 1, the flow chart of the application shows the background of the program. User interface design – main screen represents “learn the numbers”, “test yourself” and “exit” from program options. When the application is run, a menu appears on the screen (Figure 2a). The system directs the user to show the markers to the device's camera where the application is installed (Figure 2b).
(a) (b)
Figure 2. (a) Main screen when application runs and (b) System directs the user to show the markers.
Each marker is used to be shown to the camera, and then tapping the play button creates a 3D view of the 2D number marker on the screen. Option “Learn the numbers” of the application is designed to teach the numbers from zero to nine. Each number is augmented as 3-D objects with sounds. The sound of the 3D number is heard when the user touches the 3D-number appearance on the screen.
Figure 3a represents the example of the appearance of 3-D “number-9”. Option “Test yourself” of the application is designed to test what the user learns. Randomly, a number is selected and asked by the program. When “Choice the number-X and show it to the camera” is written on the screen. The user tries to find X-number from all markers. Figure 3b shows the example (number-2) screen of the “Test yourself” option of the application and Figure 3c shows an augmented reality marker from zero to nine.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
(a) (b) (c)
Figure 3. (a) The example of appearance of 3-D “number” (9), (b) The example (number-2) screen of “Test yourself” option of the application and (c) Augmented reality marker "from zero to nine", respectively.
When the user choices the number which is asked by the application to the user, the screen appears as Figure 4a. If the user does not find the number, the screen appears as Figure 4b. Five questions are asked by the application randomly. Each question is answered by the users (the children). After answering five questions, Figure 4c shows an example for “Test yourself”. The last screen represents the score of true- false answers.
(a) (b) (c)
Figure 4. (a) When the user choice the number which is asked by the application to the user, (b) When the user does not choice the number which is asked by the application to the user and (c) The score of true- false answers, respectively.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
The user can do a test again when touching the return button and return the home page with the touching home page button.
CONCLUSION
In this study, we developed an application teaching aid called interactive numbers through augmented reality. During the development stages of this application, we received invaluable suggestions and help from special education and kindergarten teachers. The application was tested to a limited extent on children from special education and early childhood schools. However, tests are not completed yet because of the covid-19 pandemic.
In the next future, we will test the application and add new features according to the needs and preferences of our target audience.
ACKNOWLEDGMENTS
Thanks to Psikojenez Company for their financial and educational support (www.psikojenez.com).
REFERENCES
Altan, N. T. and Gokturk, M. (2017). Providing Individual Knowledge from Students with Autism and Mild Mental Disability Using Computer Interface. Springer AHFE’2017, Los Angeles- California. Altan, N. T. and Gokturk, M. (2019). Comparison of the Theory of Mind (ToM) Tests on the
Paper, 2D Touch Screen and Augmented Reality Environments on the Students with Neurodevelopmental Disorders (SWND). IEEE Access (SCI-E), 2019. DOI: 10, 1109/ACCESS.2019, 2902836. Azuma, R. (1997). A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments, 6, 355-385. https://doi.org/10.1162/pres.1997.6.4.355. Bujak, K.R., Radu, I., Catrambone, R., MacIntyre, B., Zheng, R., & Golubski, G. (2013). A
psychological perspective on augmented reality in the mathematics classroom. Comput. Educ., 68, 536-544. Cascales-Martínez, A. & López, D. & Contero, M. (2013). Study on Parent's Acceptance of the Augmented Reality Use for Preschool Education. Procedia Computer Science. 25. 420-427. 10.1016/j.procs.2013.11.053. Chen, Chih-Ming, and Yen-Nung Tsai. (2012). Interactive augmented reality system for
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
Fazli, M. S. (2019), Resimli Çocuk Kitaplarında Artırılmıs Gerceklik Teknolojisinin Kullanimi ve Bir Resimli Hikaye Kitabı Uygulamasi, Master Thesis, Hacettepe University. Hinske, S., Langheinrich, M., and Lampe, M. (2008). Towards guidelines for designing
augmented toy environments. DIS '08. Juan, M. Carmen, Llop, Edith, Abad, Francisco, Lluch, Javier (2010): Learning Words Using Augmented Reality. In: ICALT 2010 - 10th IEEEE International Conference on Advanced Learning Technologies 5-7 July, 2010, Sousse, Tuneisa. pp. 422-426. https://dx.doi.org/10.1109/ICALT.2010.123. Oranc, C., & Kuntay, A.C. (2019). Learning from the real and the virtual worlds: Educational
use of augmented reality in early childhood. Int. J. Child Comput. Interact, 21, 104- 111. Rohaya, Dayang ed. (2012). Design and Development of an Interactive Augmented Reality Edutainment Storybook for Preschool. IERI Procedia. 2. 802-807. 10.1016/j.ieri.2012.06.174. Sankey, Michael, Birchi D. and Garnier, M. (2010). Engaging students through multimodal
learning environments: The journey continues. ASCILITE 2010 - The Australasian Society for Computers in Learning in Tertiary Education. Wu, H.K., Lee, S.W.Y., Chang, H.Y. & Liang, J.C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62(1), 41-49. Elsevier Ltd. Retrieved January 21, 2022 from https://www.learntechlib.org/p/132254/. Yelland, N. and Gilbert, C. (2018). Transformative technologies and play in the early years:
Using tablets for new learning. Global Studies of Childhood, 8(2), 152–161. https://doi.org/10.1177/2043610617734985. Yilmaz, R. M., Baydas, O., Karakus, T., & Goktas, Y. (2015). An Examination of Interactions in a Three-Dimensional Virtual World. Computers & Education, 88, 256– 267. https://doi.org/10.1016/J.COMPEDU.2015.06.002. Yilmaz, R. (2016). Educational magic toys developed with augmented reality technology for
early childhood education. Computers in Human Behavior. 54. 240-248. 10.1016/j.chb.2015.07.040. Vuforia Engine Developer Portal (Agust.23, 2021), Website: https://developer.vuforia.com. Firebase, (September 17, 2021).), Website: https://firebase.google.com.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-7923-8988-7
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional