Integrating Artificial Intelligence Mediated Communications to Enhance EFL Learners' Transactional Speaking Skills at SMP Tahfidz Mutiara Al-Akbar

Introduction

English has long occupied a central position in Indonesian educational policy, being mandated as a compulsory subject across primary and secondary education (1, 2). In line with the Regulation of the Indonesian Ministry of Education, Culture, Research, and Technology No. 8 of 2024, junior high school students are expected to demonstrate communicative competence, including the ability to exchange ideas, opinions, and information in spoken interaction. Within this curricular framework, transactional speaking defined as spoken interaction primarily aimed at conveying and negotiating information is a core learning outcome (3, 4). However, empirical evidence consistently indicates that Indonesian EFL learners encounter substantial difficulties in speaking performance. Studies report persistent problems related to limited vocabulary, weak grammatical control, pronunciation inaccuracies, and low confidence (5–7). A preliminary speaking assessment conducted at SMP Tahfidz Mutiara Al-Akbar using an oral interview model and evaluated through SpeechAce revealed below-average performance (overall mean score 5/9 or 56%), with particularly low scores in vocabulary (50%) and grammar (44%) (8). Although learners demonstrated basic competence aligned with CEFR A2 level, substantial gaps remained in fluency, accuracy, and coherence. These findings highlight an urgent need for pedagogical innovation capable of systematically addressing persistent deficiencies in transactional speaking among junior secondary EFL learners.

Despite the widespread implementation of communicative approaches, including mastery learning (Bloom, 1968 in Block, 1971) and interactive assessment techniques (Clark, 1979 in Ginther, 2012), traditional classroom practices often fail to provide sufficient individualized feedback, sustained speaking opportunities, and data-driven assessment (9, 10). Artificial Intelligence (AI), particularly Automatic Speech Recognition (ASR), has recently emerged as a transformative tool in language education. AI-mediated interaction systems have demonstrated significant potential to enhance learners’ speaking proficiency, engagement, and self-regulation (11–13). Such systems offer real-time feedback on pronunciation, fluency, vocabulary, and grammar features that are difficult to consistently provide in conventional classroom settings. However, existing studies have primarily focused on higher education contexts, specific commercial applications, or limited speaking components, with insufficient exploration of AI integration among young EFL learners in junior secondary schools.

Moreover, few investigations holistically examine not only learning outcomes but also contextual factors influencing AI integration and the pedagogical challenges faced by teachers. This gap is particularly significant given that effective technology integration requires alignment with pedagogical frameworks such as Technological Pedagogical Content Knowledge (TPACK) and structured educational planning (3). While current studies highlight the technical affordances of AI tools, they often overlook how EFL teachers dynamically integrate their technological knowledge with pedagogical strategies and specific transactional speaking content. In junior secondary settings, where learners require high scaffolding, exploring AI integration through the lens of TPACK is critical to understanding not only 'if' the technology works, but 'how' it aligns with structured curricular objectives.

To address these limitations, the present study proposes the integration of Artificial Intelligence-mediated communication through SpeechAce, an ASR-based platform designed to assess and enhance speaking performance with immediate, individualized feedback. Recent research demonstrates that the use of AI-powered tools, including speech recognition and automated feedback systems, contributes to greater speaking fluency, increased learner motivation, and reduced anxiety in EFL contexts (14). Unlike prior studies, this research targets junior high school EFL learners and embeds AI within systematically developed AI-based teaching materials aligned with transactional speaking objectives. Empirical findings indicate that AI-assisted applications can significantly improve speaking performance and alleviate psychological barriers to oral production (15). 

The study adopts a mixed-method pre-experimental design to measure the effect of AI-mediated communication on learners’ transactional speaking skills, identify factors influencing AI integration from both teacher and student perspectives, and examine pedagogical challenges encountered during implementation. Additionally, evidence shows that AI-driven mobile learning can enhance learners’ speaking confidence and engagement through adaptive feedback mechanisms (16). By bridging the gap between AI innovation and classroom practice, this study contributes to the state-of-the-art in AI-assisted language education, extends theoretical discourse on AI integration and transactional speaking pedagogy, and offers practical, adaptable teaching materials for secondary-level English instruction. Ultimately, the research positions AI not merely as a technological tool, but as a pedagogically grounded innovation with the potential to transform EFL speaking instruction in resource-constrained educational contexts.

Methodology

Study Design and Rationale

This study employed a mixed-method pre-experimental design integrating quantitative and qualitative approaches. The quantitative component used a one-group pretest–posttest design to measure students’ improvement in transactional speaking skills before and after the implementation of Artificial Intelligence (AI)-mediated communication. The qualitative component was conducted to explore factors influencing AI integration and identify challenges encountered during implementation. The selection of a pre-experimental design was based on practical classroom constraints that did not allow randomization or the establishment of a control group. As noted by Clark and Creswell (2014), hypothesis testing in educational research requires systematic comparison of dependent variables before and after treatment, while Sugiyono (2013) states that pre-experimental designs are appropriate when experimental control is limited (17, 18). The independent variable in this study was AI-mediated communication using SpeechAce, and the dependent variable was students’ transactional speaking performance.

Participants, Population, and Sampling

The study was conducted at SMP Tahfidz Mutiara Al-Akbar during the 2024/2025 academic year. The population consisted of Grade IX students. A total sampling technique was employed due to the limited number of students in the cohort. Although the population consisted of 33 students, only eight students consistently attended the instructional sessions and met the inclusion criteria; therefore, these eight students participated in the study. Inclusion criteria included active enrollment as Grade IX students, regular attendance during the intervention period, and the ability to operate digital learning platforms. The study also involved one English teacher who contributed to the qualitative evaluation of implementation challenges. The focus on young learners aligns with characteristics of early adolescent language learners as discussed by Ellis (2014) (19).

Consequently, the final sample size was confined to eight students (n = 8). Due to this exceptionally small cohort, generalizability is strictly limited, and this study is explicitly framed as a preliminary pilot investigation rather than a definitive causal trial.

Instruments and Measurement

Students’ speaking performance was assessed using SpeechAce, an Automatic Speech Recognition (ASR)-based platform designed to evaluate pronunciation and fluency (20). The system automatically generated scores across four speaking components: pronunciation, fluency, vocabulary, and grammar, using a 0–9 scale aligned with CEFR proficiency levels. Pretest and posttest speaking assessments consisted of transactional speaking prompts requiring students to exchange information, express opinions, and respond to situational contexts.

In addition to the speaking test, a student evaluation questionnaire using a Likert-scale format was administered to examine perceptions of AI integration. Teacher feedback was also collected to identify contextual and technical challenges. The instructional materials were developed based on the Technological Pedagogical Content Knowledge (TPACK) framework to ensure alignment between technological tools, pedagogical strategies, and speaking content (3).

SpeechAce evaluates features on a 0–9 rubric aligned with CEFR levels. To allow granular statistical analysis via paired-samples t-test, these scores were aggregated and mathematically converted into a standard 100-point scale.

Data Collection Procedure

Data collection was conducted over three weeks comprising six instructional sessions (90 minutes per session). In the first session, students completed a pretest using SpeechAce to measure baseline transactional speaking ability. During the intervention phase, students engaged in structured learning activities that included pronunciation drills, guided role-play tasks, and independent speaking practice using SpeechAce. Students were required to perform multiple speaking attempts to utilize the automated feedback provided by the system. Teacher scaffolding was provided to clarify recurring errors in grammar, vocabulary, and pronunciation.

At the end of the intervention period, students completed a posttest using parallel speaking prompts under identical conditions. Following the posttest, students completed the evaluation questionnaire, and the teacher provided reflective feedback regarding implementation challenges.

Data Analysis Strategy

Quantitative data were analyzed using SPSS. Descriptive statistics, including means and standard deviations, were calculated to determine overall speaking performance before and after the intervention. A normality test using the Kolmogorov–Smirnov test was conducted to ensure that the data met parametric assumptions. To test the research hypothesis regarding the effect of AI-mediated communication on transactional speaking performance, a paired-samples t-test was performed at a significance level of α = 0.05. The magnitude of improvement was also examined to determine practical significance.

Qualitative data from questionnaires and teacher reflections were analyzed through data reduction, data display, and conclusion drawing to identify recurring themes related to factors influencing AI integration and implementation challenges.

Results and Discussion

The findings demonstrate that the implementation of Artificial Intelligence (AI)-mediated communication through SpeechAce led to a statistically and pedagogically significant improvement in students’ transactional speaking performance. The integration of AI-based automatic speech recognition (ASR) technology provided learners with structured speaking tasks combined with immediate, individualized feedback, thereby creating a practice environment that extended beyond conventional teacher-centered instruction. Descriptive statistics of students’ pretest and posttest scores are presented in Table 1. Importantly, the observed improvement indicates that AI-mediated learning environments can function as an effective supplementary scaffold to classroom interaction, particularly in contexts where speaking practice opportunities are limited. These results further suggest that technology-enhanced feedback mechanisms may accelerate learners’ progression toward communicative competence by promoting continuous performance monitoring and iterative linguistic refinement.

As shown in Table 1, the mean pre-test score was 47.36 (SD = 4.11), with scores ranging from 40.00 to 52.20. These values indicate that prior to the intervention, students’ transactional speaking ability was relatively moderate but clustered within a limited performance range. The relatively small standard deviation suggests that participants demonstrated comparable baseline proficiency levels, reflecting homogeneity in initial speaking competence. This iterative refinement process underscores the cognitive benefits of automated scaffolding in language acquisition. By receiving instant, low-stakes corrective feedback from SpeechAce, learners could engage in the 'Noticing Hypothesis' (Schmidt, 1990), where they actively recognize the gap between their interlanguage production and the target language model, leading to structured linguistic modification without classroom anxiety (21).

Following the implementation of AI-mediated speaking practice, the mean post-test score increased to 52.24 (SD = 4.31), with scores ranging from 44.40 to 56.70. The mean gain of 4.88 points represents an improvement of approximately 10.3% relative to the pre-test mean. Importantly, the minimum score increased from 40.00 to 44.40, indicating that even the lowest-performing students showed measurable progress. Similarly, the maximum score increased from 52.20 to 56.70, demonstrating upward achievement across the performance spectrum.

Although the standard deviation slightly increased from 4.11 to 4.31, the change was minimal, suggesting that the intervention did not disproportionately benefit only high-performing students. Instead, the improvement appears to have occurred consistently across participants. This pattern supports the pedagogical value of AI-assisted practice as an inclusive learning support mechanism.

The observed improvement may be attributed to the structured repetition and immediate corrective feedback provided by the AI system. According to Richards (2008), transactional speaking requires real-time processing of linguistic forms and meaning negotiation. The AI platform facilitated repeated speaking attempts with automated pronunciation and fluency feedback, enabling learners to refine their output iteratively. This aligns with mastery learning principles (22), which emphasize incremental performance enhancement through feedback and repetition.

The upward shift in both central tendency (mean) and score range suggests that AI-mediated communication not only enhanced overall performance but also elevated the lower performance threshold within the group. Such findings are consistent with previous studies indicating that AI-based speaking tools improve learners’ pronunciation accuracy, fluency development, and engagement in EFL contexts (11–13, 20).

Despite these positive trends, it is important to note that the relatively small sample size (n = 8) limits generalizability. As highlighted by Sugiyono (2013) and Clark and Creswell (2014), pre-experimental designs provide preliminary evidence but require cautious interpretation. Nevertheless, the consistent upward trend across minimum, mean, and maximum values provides convergent evidence of instructional effectiveness (17, 18). Overall, the descriptive findings indicate that AI-mediated communication contributed to measurable and pedagogically meaningful improvements in students’ transactional speaking performance.

After six sessions of AI-mediated instruction, the mean post-test score reached 52.24 out of 100, representing a steady upward trend across all speaking components. The overall improvement is visually presented in Figure 1.

The figure illustrates a clear upward trend in students’ speaking performance after the intervention. This improvement suggests that iterative speaking attempts supported by automated, real-time feedback enabled learners to progressively refine pronunciation accuracy, fluency control, lexical selection, and grammatical structure. As noted by Alnafisah (2022), ASR-based systems such as SpeechAce provide immediate corrective feedback that supports pronunciation development and self-monitoring processes (20). The opportunity to repeat speaking tasks multiple times appears to have strengthened learners’ metacognitive awareness, allowing them to identify recurring errors and adjust their linguistic output accordingly. Prior to hypothesis testing, normality assumptions were examined using the Kolmogorov–Smirnov test, as presented in Table 2.

As presented in the table above, it definitely shows that the significant value of the pre-speaking test = 0.200 > 0.05. It means that the sample is normally distributed. Furthermore, the significant value of post-speaking test = 0.60 > 0.05. It means that the sample is also normally distributed. Following confirmation of the normality assumption, a paired-samples t-test was conducted to determine whether the observed improvement between pre-test and post-test scores was statistically significant.

The paired-samples t-test was selected because the study involved repeated measurements from the same participants, making it appropriate for comparing mean differences within a single group over time. This statistical procedure evaluates whether the mean difference between two related conditions significantly differs from zero.

As presented in Table 3, the paired-samples t-test analysis revealed a statistically significant difference between the pre-speaking test and post-speaking test scores. The mean difference between the two measurements was −4.36250 (SD = 1.95005), indicating that post-test scores were substantially higher than pre-test scores. The negative value reflects the calculation order (pre-test minus post-test), confirming that students’ speaking performance improved following the intervention.

The standard error of the mean difference was 0.68945, and the 95% confidence interval ranged from −5.99278 to −2.73222. Because the confidence interval does not cross zero, this further confirms the presence of a statistically meaningful difference between the two conditions. The t-test result showed t (7) = −6.328, p =.000 (p <.001), indicating that the observed improvement was highly significant and unlikely to have occurred by chance.

Given the small sample size (n = 8), the magnitude of the t-value suggests a strong treatment effect relative to within-group variability. These findings provide robust statistical evidence that the implementation of AI-mediated communication indicated positive trends on students’ transactional speaking performance.

Teacher and Student Perceptions

The qualitative insights gleaned from student questionnaires and teacher reflective journals illuminated the nuanced experiential dimensions of AI integration, converging into two primary themes: heightened digital engagement and infrastructure-induced pedagogical anxiety.

Regarding the first theme, students uniformly exhibited a positive disposition toward SpeechAce. The platform's real-time automated scoring mechanism functioned as a form of gamified feedback, which suggested improvement in learners' intrinsic motivation and self-efficacy. In traditional junior secondary classrooms, students often experience a high affective filter—characterized by anxiety and fear of making mistakes in front of peers. SpeechAce provided a low-stakes, autonomous practice environment where students could iterate their spoken output without fear of negative social evaluation. This instant validation loop directly contributed to the upward shift in post-test scores, as students felt more empowered to self-correct their pronunciation and fluency. However, the second theme brought critical contextual challenges to the fore, particularly from the teacher's perspective. While acknowledging the utility of AI in outsourcing repetitive speaking drills, the teacher highlighted that technology integration is highly dependent on infrastructural stability. At SMP Tahfidz Mutiara Al-Akbar, intermittent internet connectivity occasionally disrupted the synchronization of the automated speech recognition system, leading to brief episodes of student frustration. Furthermore, the teacher emphasized a crucial pedagogical gap: early adolescent learners often lack the advanced metacognitive capacity to interpret complex AI-generated feedback independently, especially concerning structural grammar errors. This finding underscores that AI-driven tools cannot act as a standalone remedy; rather, they demand a robust TPACK framework where the human teacher remains indispensable as a mediator who translates digital metrics into actionable linguistic development.

Limitations of the Study

Several limitations restrict the scope of these findings. First, the lack of a control group makes it difficult to completely isolate the AI tool's impact from natural student maturation and concurrent classroom instruction. Second, the sample size (n = 8) is very small, representing a specific localized context which inherently limits the generalizability of the outcomes. Lastly, the short three-week intervention period only captures immediate gains; hence, the long-term sustainability and pedagogical efficacy of AI-assisted speaking development requires further longitudinal exploration with larger cohorts.

Conclusion

The findings of this mixed-method study demonstrate that the integration of AI-mediated communication through SpeechAce significantly improved junior secondary students’ transactional speaking performance, as evidenced by statistically significant gains in quantitative metrics. Beyond these statistical advancements, the qualitative data enrich our understanding of the classroom reality, revealing that while AI-driven instant feedback dramatically heightens student motivation and reduces speaking anxiety, its overall efficacy is constrained by technical infrastructure and the learners' digital literacy. The teacher’s insights highlight that AI does not replace human instruction; instead, it redefines the teacher's role as an essential scaffolding mediator who helps young learners interpret automated corrective feedback.

In conclusion, AI-assisted platforms function effectively not as autonomous teaching replacements, but as powerful pedagogical augmentation tools within a structured curriculum. Despite constraints related to the small sample size, this study offers preliminary empirical ground for embedding AI in secondary EFL contexts. Future research should utilize larger, experimental cohorts to explore optimal blended frameworks that seamlessly balance AI automated feedback with teacher-guided speaking pedagogy across diverse socio-technological school settings.