Alex Ackermann, Volker Paelke, Benjamin Tannert, Oliver Kück (2025). From Manual to Automated: Enhancing Inclusivity in Foreign Language Education with Technology. In: Human Factors in Design, Engineering, and Computing. AHFE Open Access, vol. 199, pp. 20–28. https://doi.org/10.54941/ahfe1006814
In the 2020/21 school year, the city state of Bremen (Germany) became the first German state to equip all students with iPads, thereby creating a key prerequisite for the digital transformation of teaching. Digital platforms open up new opportunities for interaction in the classroom and new forms of learning. This paper reports on the exploratory “MediaMeetsDiversity@School” project that aimed to investigate the viability and effectiveness of using digital learning platforms to support more inclusive foreign language education. The focus was on creating learning experiences that adapt to the needs of students with special requirements, including reading and spelling difficulties (dyslexia), multilingual students with limited exposure to German at home, and neurodivergent learners. To address the challenge of creating inclusive and adaptive digital content for diverse learners, we reviewed potential uses of AI in content creation. Since no established framework existed for automation in this context, we adapted and extended the SAE levels of automation to structure our analysis. The resulting framework distinguishes six levels of automation, ranging from Level 0 (manual content creation) to Level 5 (fully automated, unsupervised content generation at run-time). For the scope of this study, we focused on design-time automation (Levels 0–3) to ensure consistent and comparable learner experiences. We used web technologies to implement the app prototype, ensuring broad platform compatibility, while also optimizing it for the iPads used in Bremen schools. The app prototype consists of adapted content for two of the aforementioned target groups. The tasks for the dyslexia target group consist of multisensory training in phoneme-grapheme correspondence, while the tasks for the multilingual target group focus on promoting logic and metalinguistics. Both types of tasks progress from a tutorial to simple tasks (awareness), then to medium tasks (application), and finally to complex tasks (deepening). The content was evaluated in eight school classes (grades 5 to 7) from seven schools in Bremen and Lower Saxony. After a brief introduction, students worked with the prototype during a 90-minute school session. They then responded to a 22-question feedback survey with both scaled and open-text answers.The results indicate a high level of acceptance and engagement among students, who found the tasks enjoyable and motivating. However, technical issues and usability challenges were also observed during task execution. Despite the limitations identified, the study demonstrated the potential of targeted, technology-supported learning environments to meet diverse learner needs in inclusive classrooms. Further research is required to assess long-term learning outcomes and to explore higher levels of automation in adaptive content generation.
INTRODUCTION & RELATED WORK This paper reports on the exploratory “MediaMeetsDiversity@School: Promoting diverse potential in the inclusive education system with student iPads” project that aimed to investigate the viability and effectiveness of using digital learning platforms to support more inclusive foreign language education. With the overarching goal of counteracting educational inequalities through new possibilities for human-computer interaction in mind, we wanted to unlock this potential. To unlock it, it is necessary to make sustainable usage of individualized teaching methods in classrooms, harnessing the intersection of changing subject-specific teaching conditions and new technological developments for the benefit of both students and teachers. The core subject of English, as a foreign language, plays an important role in this constellation: in class, teachers can only simulate authentic moments of interaction to a limited extent, which are central to practicing effective foreign-language communication. However, targeted communication learning opportunities without human counterparts can be
From Manual to Automated: Enhancing Inclusivity in Foreign Language Education 21
designed in a multisensory way on students’ iPads, using speech recognition and artificial intelligence. In the MediaMeetsDiversity@School project, we achieved this through design, programming, implementation, and the evaluation of digital teaching and learning opportunities for three subgroups of learners: students with autistic spectrum disorders (ASD), as well as students with reading and spelling difficulties (dyslexia), and multilingual students with limited exposure to German at home.
Recent research on foreign language education has demonstrated the effectiveness of current technologies, including mobile apps, multimedia materials, and game-based learning, in enhancing motivation and engagement among pupils (Dia, 2024; Panagiotidis et al., 2023; Zhang and Zou, 2022). Specifically, pupils with dyslexia benefit from visually engaging, gamified environments and assistive technologies, which have shown promise in addressing phonetic and reading challenges (Chauhan et al., 2024). Research by Vasalou et al. indicates that digital games, when designed with social and educational principles, can promote engagement and learning among children with dyslexia (Vasalou et al., 2017). As for pupils with autism spectrum disorder (ASD), research shows that they tend to benefit from visual aids and individualized strategies (Ni Putu Intan Bidari and Yanti, 2021; Zohoorian et al., 2024). Further research by Mitchell et al. demonstrates that virtual environments (VEs) can enhance social understanding in students with ASD, supplementing traditional social skills interventions (Mitchell et al., 2007). Kahanurak, Dibyamandala, and Mangkhang show that multilingual learners gain from digital storytelling and intercultural communicative approaches (Kahanurak et al., 2023). Supporting this, Parsons et al. investigated how digital stories can be used as both a research and professional development tool in technology-enhanced learning for children with autism. Their findings indicate benefits for pupils in terms of motivation and enjoyment, as well as empowerment for both teachers and pupils (Parsons et al., 2015). Additionally, Dagenais et al. examined how ScribJab, a multilingual iPad application and website, supports students and teachers in composing, illustrating, and narrating stories in two languages within school settings. They demonstrate positive student engagement with the app and the benefits of technology for bilingual teaching (Dagenais et al., 2017).
STUDY The following chapter describes the project’s background, the approach using artificial intelligence and automation, and the implementation of the prototype, including content design.
Background
The project started in January 2022, so the Corona pandemic was still ongoing and influenced the collaboration with pupils of the target group and their teachers due to contact restrictions imposed by governmental institutions and schools to protect everyone’s health. The ability to talk via
online meetings allowed for exchanging ideas, but testing the prototype with the target group was not easy to implement.
The target group of pupils with special needs also brought in special requirements that had to be considered. Partially, the individual’s needs or arising barriers could not be named, so during the process, the teachers had to assist through classifying and clarifying the feedback.
One positive aspect of the pandemic was that in the city-state of Bremen (Germany), where we carried out our project, every pupil received their own iPad, funded by the local government. This made it easier for us to select a target device for implementing our system, ensuring that it was available and usable for everyone.
User-Centered Design and the Role of Adaptation The development of the application followed an iterative, user-centered design process, in line with established standards such as ISO 9241. A user- centered approach is essential for educational technologies, as it ensures that system requirements are derived from the needs of the intended audiences rather than from technical convenience. In this project, the importance of this perspective was heightened by the diversity of the target groups: learners with different backgrounds, varying levels of prior knowledge, and distinct accessibility requirements. For these groups, inclusive design cannot rely on generic solutions; rather, it requires the ability to tailor both content and interaction modalities to specific learner needs.
Conventional content creation and development techniques reach their limits when confronted with such diversity. Manually producing and curating multiple versions of learning materials for each target group (or even for each individual learner) can be prohibitively time-consuming and resource- intensive. One of the central challenges of the project was therefore to explore how adaptive mechanisms, potentially supported by artificial intelligence, could be integrated into the user-centered design process.
Several possible strategies for AI-supported design and adaptation were considered. To provide a conceptual structure, the project drew inspiration from the SAE levels of automation in autonomous driving (ISO/SAE PAS 22736:2021)1, translating them into a staged model of AI involvement in content creation. Our model is structured as follows: At the lowest level (Level 0, no automation), all content creation remains manual, without AI assistance. At higher levels, automation gradually increases: Level 1 (Assisted Content Creation) involves supportive tools such as intelligent autocomplete; Level 2 (Partially Automated Content Creation) extends to AI- assisted generation or adaptation of text and images through prompting; and Level 3 (Conditionally Automated Content Creation) uses automatic creation or adaptation based on a supplied syllabus, with manual intervention for edge-cases and supervised by human review. Level 4 (Highly Automated Content Creation) would cover the automated creation and deployment within predefined domains or topics and could thus work user specific at run 1ISO/SAE PAS 22736:2021, International Organization for Standardization; https://www.iso.org/ standard/73766.html.
From Manual to Automated: Enhancing Inclusivity in Foreign Language Education 23
time, while Level 5 (Fully Automated Content Creation) would correspond to independently automated, situation-aware generation that adapts content dynamically to both users and contexts.
For the purposes of this research, the focus was placed on design-time AI support (mostly at Level 2) rather than run-time adaptation (Levels 4 and 5). This restriction ensured that learners would experience comparable conditions during evaluation.
Implementation of the Prototype
Figure 1 shows the entire system in a diagram. We built the system as a web application with only the front end and local storage. The advantage here is that pupils can access and use the tool via a web interface, without requiring a login or any other limitations. We implemented the prototype using Angular 14.1.2 as the web framework and Tailwind CSS for the design. The prototype consists of two main pages: the first for tasks related to the target group with dyslexia, and the second for tasks for multilingual pupils. These two pages each include three additional components for the task types described in the next chapter, content design.
Figure 1: System and components implemented in the prototype.
Additionally, the system features an administrator page accessible via a link to reset progress. We included the required data in a CSV file and stored it locally in the browser. The CSV file stores tasks, answers, hints, links to audio aids and image materials, and markings indicating when a task has been completed. The tasks for the dyslexia target group include audio aids that provide spoken solutions or phonemes for practice. We used the Audio AI Voicemaker2 to create these spoken solutions. A project team member with a background in teaching English as a foreign language recorded the 2Voicemaker Technologies Pvt. https://voicemaker.in/.
individual phonemes. We used sound effects from the Pixabay3 online audio database, licensed under a royalty-free license.
As shown in Figure 2, we kept the design deliberately simple to avoid major distractions. The focus was on the prototype’s usability. Further, we decided on a mascot that appears throughout the entire application. As the mascot, we chose a polar bear to keep the design neutral. Additionally, we used a learning path consisting of ice floes with signs in three different colors to represent individual assignments and levels of difficulty. A star on the individual sign indicates task completion.
Figure 2: Example exercise from the tasks for pupils with dyslexia (left), the system in use on an iPad (right).
Content Design The tasks, albeit available to the broad student body, were designed to especially address three marginalised groups of pupils. Three different task categories were established to each particularly benefit one of these groups while still being approachable to pupils outside of the respective group.
The first set of tasks targeted pupils with dyslexia, an estimated 5-8% of the general student body (Gerlach, 2019). While its symptoms may vary depending on the individual and their language(s), dyslexia is most prominently associated with “reduced working memory, poor phonological processing, [and] incomplete automatization or slow central processing” (Nijakowska, 2010). Affected students require different learning strategies and tend to profit less from regular lessons than their peers (Gerlach, 2019; Nijakowska, 2010). To address this group, we created three sets of notably short tasks and focused on exercising phonemic awareness, a core issue of many dyslexic people (Gerlach, 2019). Five different phonemes were presented via sound, IPA spelling, and common words. Next, pupils were presented with words where the corresponding grapheme had been removed and had to be chosen from a list of graphemes associated with the phoneme. Finally, words including one of the phonemes in question were shown with the corresponding grapheme switched out with another grapheme commonly
From Manual to Automated: Enhancing Inclusivity in Foreign Language Education 25
associated with the phoneme. Students had to identify the incorrect grapheme and choose the correct replacement from a list of graphemes.
The second set of tasks targeted pupils on the autism spectrum. While probably less than 1% of the general student body are affected, the effects range from mild to very severe impairments regarding social interaction, communication, and interests (Lawson, 2002). Affected pupils often struggle in conversations and suffer social stigmatisation (ibid). In order to exercise social interaction without possible stigmatisation, we prepared interactions using generative AI. In those interactions, pupils were presented with a situation and a specific goal they had to achieve via communicating with a fictitious person. Task difficulty ranged from simple and short interactions such as ordering food to more sensitive topics such as cancelling a party invitation on short notice.
The third set of tasks targeted pupils with a multilingual background, probably the largest yet least consistent group. Although existing multilingualism offers many advantages for further language learning, in practice it is hardly capitalised on (Roche, 2013). The effects of knowledge transfer between languages, while not fully understood, generally improve a subconscious understanding of other languages (Roche, 2013). Two different sets of tasks with increasing difficulty were designed to take advantage of this. First, pupils were given two common words in English and three translations each into other languages in random order. They were tasked with matching the translations with the correct English word. Next, they received a number of short English utterances and their direct translation into another, likely unknown, language. After rudimentarily understanding the syntactic and grammatical properties of that language, they had to translate one last utterance themselves. Since proficiency in this task was assumed to be very diverse, pupils could freely choose between different difficulty levels.
EVALUATION & RESULTS Below, we provide an overview of both the evaluation process and the results achieved.
Study Design and Procedure Eight teachers from seven different schools in Bremen, Bremerhaven, and Lower Saxony participated in the study. Each teacher provided between one and three different classes ranging from grade 5 to 7, with no more than two consecutive lessons (around 90 minutes) per class. In order to evaluate the tasks, we created a questionnaire for all participating pupils. We were especially interested in the tasks’ motivational value and how accepting the pupils were of their unique design and their technical implementation. The items were designed along the Intrinsic Motivation Inventory (IMI) (Ryan et al., 1983) and the Technology Acceptance Model 3 (Venkatesh and Bala, 2008). The questionnaire contained 22 closed questions and space for individual feedback. All participating teachers took part in a workshop about the topics and were taught about the Web-App’s basic functions. They further received a written handout to address potential future questions. The teachers
were briefed to first explain the Web-App’s functions to their pupils, then leave them to explore the tasks on their own. Towards the end, the pupils had to anonymously complete the questionnaire. The study was conducted between March and May 2023 and occasionally shadowed by a member of our team.
Results
The questionnaires showed clear tendencies across the surveyed student body and included a surprising abundance of individual feedback. Pupils enjoyed the tasks’ uniqueness and reported relatively high intrinsic motivation. Most were able to work fully autonomously. Some, however, felt insecure because of how much the tasks differed from regular English lessons. Expectations regarding the technical implementation were high as pupils tended to compare the Web-App with long-established Apps such as Anton4 (Solocode GmbH). The ability to freely choose any task order was described as liberating to some pupils but confusing to others. All in all, pupils enjoyed puzzling over the translation tasks and responded positively towards the proposed difficulty. They however criticized the relatively little task variety along with a small number of technical issues; expectable feedback for a first trial run.
DISCUSSION
This discussion provides an analysis of the results obtained with regard to various relevant aspects.
Implementation Technologies and Platforms The technological foundation for distribution, presentation, and interaction across existing devices, such as tablets, is solid, enabling adaptive and customized content delivery as needed. In our experiments, we did not identify any hard limitations regarding the technological platforms used.
During the test phase, isolated technical problems occurred, but this was to be expected during the initial of a prototype. Most of these problems were resolved during the evaluation, so they had no significant impact.
AI for Adaptive Content Design AI tools proved valuable in accelerating content production and supporting user-centered design, particularly through assistive and prompt-based generation. However, notable limitations remain: in experiments with speech- based input, we observed that advanced AI systems often “guessed” correct results from clearly incorrect input, which undermines the purpose of language learning. This led the prototype to focus on text-based input. While AI holds strong potential for real-time, user-dependent content adaptation, careful evaluation is essential to ensure pedagogical suitability. Given the ongoing advances in AI, exploration of level 4 and 5 automations in content
4Solocode GmbH: Anton. Available online at https://www.anton.app.
From Manual to Automated: Enhancing Inclusivity in Foreign Language Education 27
generation becomes a promising area for future research, while a more in- depth examination of level 3 automation across broader content domains and additional pedagogical approaches can lay a stronger foundation for validating novel teaching concepts, especially for students with special requirements.
Didactic Perspective From an educational perspective, our approach has proven to be well- suited to supporting pupils with special needs in language learning. The feedback that the pupils enjoyed using the system and found it motivating was particularly positive. The opportunity to work on the tasks independently is also an important aspect for these pupils in terms of foster their confidence in educational situations.
Due to the limited scope of tasks during the evaluation, no long-term motivation was achieved. However, this is only a minor issue, as the content can easily be expanded in a second iteration to offer a wider selection and more diverse and motivating types of tasks.
CONCLUSION
Our approach showed that there is a big opportunity to support pupils with special needs with digital media in their everyday school life. The findings showed that an adaption to the users is an important part, and also the integration of AI can be a good way to make learning easier for our target group.
During the research, it was also recognisable that the development for this target group can only be achieved in cooperation with them to get really good and usable solutions. It was also a finding, that there has to be done more research especially for the target group of pupils with special needs and their use of digital media in school context. Here, AI can be a good support in this setting but it will be just a part of the puzzle.
ACKNOWLEDGMENT
We want to thank Prof. Dr. phil. Sabine Doff and Dr. phil. Joanna Pfingsthorn for their collaboration on this exploratory project. Further, we want to thank the city-state of Bremen for funding this project. We would also like to thank Jana Phung for her design contribution. Finally, we would like to thank all the teachers, pupils, and schools in Bremen and Lower Saxony who participated in our study.
REFERENCES
Chauhan, M. S., Raghav, I., Jha, V., Singh, K., Kumari, A., 2024. Exploring Technological Interventions for Dyslexia Across Multilingual Contexts, in: 2024 International Conference on Electrical Electronics and Computing Technologies (ICEECT). Presented at the 2024 International Conference on Electrical Electronics and Computing Technologies (ICEECT), IEEE, Greater Noida, India, pp. 1–6. https://doi.org/10.1109/ICEECT61758.2024.10739023.
Dagenais, D., Toohey, K., Bennett Fox, A., Singh, A., 2017. Multilingual and multimodal composition at school: ScribJab in action. Language and Education 31, 263–282. https://doi.org/10.1080/09500782.2016.1261893. Dia, A. A., 2024. Teaching and Learning English in the Digital Era: Opportunities
and Challenges. RTECONF 1, 13–25. https://doi.org/10.33422/rteconf.v1i1.183. Gerlach, D., 2019. Lese-Rechtschreib-Schwierigkeiten (LRS) im Fremdsprachenunterricht, Narr Starter. Narr Francke Attempto, Tübingen. Kahanurak, S., Dibyamandala, J., Mangkhang, C., 2023. Digital Storytelling and
Intercultural Communicative Competence through English as a Foreign Language for Multilingual Learners. JCT 12, 14. https://doi.org/10.5430/jct.v12n1p14. Lawson, W., 2002. Understanding and working with the spectrum of autism: an insider’s view, 2. impr. ed. Kingsley, Philadelphia, PA. Mitchell, P., Parsons, S., Leonard, A., 2007. Using Virtual Environments for Teaching
Social Understanding to 6 Adolescents with Autistic Spectrum Disorders. J Autism Dev Disord 37, 589–600. https://doi.org/10.1007/s10803–006-0189–8. Ni Putu Intan Bidari, Made Vera Yanti, 2021. Visual aids for teaching English to students with Autism Spectrum Disorder. jeta 2, 24–37. https://doi.org/10.52217/ jeta.v2i1.641. Nijakowska, J., 2010. Dyslexia in the Foreign Language Classroom, Second
Language Acquisition Ser. Multilingual Matters, Bristol. Panagiotidis, P., Krystalli, P., Arvanitis, P., 2023. Technology as a Motivational Factor in Foreign Language Learning. European Journal of Education 6, 69–84. https:// doi.org/10.2478/ejed-2023–0007. Parsons, S., Guldberg, K., Porayska-Pomsta, K., Lee, R., 2015. Digital stories as a method for evidence-based practice and knowledge co-creation in technology-enhanced learning for children with autism. International Journal of Research & Method in Education 38, 247–271. https://doi.org/10.1080/ 1743727X.2015.1019852. Roche, J., 2013. Mehrsprachigkeitstheorie: Erwerb - Kognition - Transkulturation -
Ökologie, Narr Studienbücher. Narr, Tübingen. Ryan, R. M., Mims, V., Koestner, R., 1983. Relation of reward contingency and interpersonal context to intrinsic motivation: A review and test using cognitive evaluation theory. Journal of Personality and Social Psychology 45, 736–750. https://doi.org/10.1037/0022–3514.45.4.736. Vasalou, A., Khaled, R., Holmes, W., Gooch, D., 2017. Digital games-based learning
for children with dyslexia: A social constructivist perspective on engagement and learning during group game-play. Computers & Education 114, 175–192. https:// doi.org/10.1016/j.compedu.2017.06.009. Venkatesh, V., Bala, H., 2008. Technology Acceptance Model 3 and a Research Agenda on Interventions. Decision Sciences 39, 273–315. https://doi.org/10.1111/ j.1540–5915.2008.00192.x. Zhang, R., Zou, D., 2022. Types, purposes, and effectiveness of state-of-
the-art technologies for second and foreign language learning. Computer Assisted Language Learning 35, 696–742. https://doi.org/10.1080/ 09588221.2020.1744666. Zohoorian, Z., Matin Sadr, N., Mohammadi-Nezhad, T., Sadr, F. M., 2024.
Investigating the Methods of Teaching English as a Foreign Language to Students with Autism Spectrum Disorder. JMR. https://doi.org/10.18502/jmr.v18i3.16414.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Design, Engineering, and Computing
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-75-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional