ABSTRACT
This technical report presents analyses and results from the first large-scale randomized controlled trial (RCT) of the effect of the game-based, artificial intelligence Dysolve® program on the reading achievement of the lowest-performing students in grades 3-8. Such RCT studies are widely recognized by experts as the “gold standard” for rigorous evaluation of program impacts. Additionally, the artificial intelligence (AI) system behind Dysolve generates single-use games in real time through a patent-protected method during its interaction with each user. Unlike adaptive programs of the past, Dysolve does not select from a premade pool of items or activities. This high responsivity at the individual-specific level is designed to locate and correct language processing deficits underlying each student’s reading difficulty.
In Dysolve, games are built in real time based on accumulating data in a user’s program. For example, a Dysolve game may assess Phoneme Detection, i.e., auditorily picking out a target sound in single, spoken words. This is important for learning and retaining new words, spelling and reading. In Dysolve, Phoneme Detection is delivered as a Fishing Game. The utterance of a test word is synchronized with a fish swimming across the screen at a set speed. Users ‘catch’ the fish representing a test word with the target sound. The student is told to listen for a target sound (e.g., /b/). Then they hear audio files of 10 common, single words (e.g., bed, both) in succession at a set speed. They tap on the keyboard whenever they detect the target sound in these words. Scoring is done automatically. Depending on the student’s game responses, Dysolve AI may decide to deliver another Fishing Game with new test words or a new test sound to verify or explore further or move on to a different activity.
During the 2022-23, through 2024-25 school years, a randomized controlled trial (RCT) was conducted to evaluate the effect of the Dysolve® program on reading achievement scores of students (n=848) enrolled in grades 3-8 in 32 schools from 9 states in the US. Participants included students in grades 3-8 who scored near the 10th percentile on average in reading/ELA for their grade on the previous year state or local reading/ELA test (excluding students with visual impairment, physical hearing problems, or cognitive impairment). Most participants were minority students from low-income districts. Students were randomly assigned to treatment and control conditions within grade and school. Baseline balance was confirmed through a statistical test of pre-intervention reading scores. Recommended minimum dosage was 9 hours of Dysolve in total (e.g., 15 min per day, 4 days per week, for 9 weeks). The trial commenced during the COVID-19 pandemic, and Dysolve was used as a supplemental program to regular reading instruction.
Most students in the treatment group logged a cumulative total of less than 3 hours (less than one-third of the intended minimum dosage). Impacts of the Dysolve intervention were assessed through both intent-to-treat (ITT) and treatment-on-the-treated (TOT) analyses. Results from the ITT analyses revealed a positive and marginally significant (p=.057) intent to treat effect (i.e., +.095 standard deviations). This result suggests that students randomly assigned to Dysolve had posttest reading scores that were slightly higher, on average, than students that did not receive access to Dysolve. Results from the TOT analyses suggested that greater dose is associated with greater gains—the effect of Dysolve increases by .20 standard deviations for each 10-fold increase in dosage. This suggests that the effect associated with a full dose of 27 hours (i.e., 15-20 minutes per day, 4 days per week, for 6 months) can be projected to produce a +.30 standard deviation (0.095+0.201=0.296) increase in reading scores. However, this conclusion is based on a projection from a dosage-adjusted statistical model. The fact that very few students assigned to the treatment group in this study used Dysolve for more than the minimum dose of 9 hours precludes strong inference about the impacts of full dosage.
Implications from the ITT effects in this RCT suggest that Dysolve, even at low dosage, may have positive impacts on students’ performance on standardized reading tests. This is important because Dysolve does not provide direct reading instruction. Dysolve was designed under the assumption that improvement in basic language processing enables reading development. In other words, Dysolve is designed to address language processing deficits shown to be associated with constrained reading development. Thus, to register positive impacts in this RCT, a transfer effect must occur from language processing to the broader reading skills measured by the assessments used in this study.
Unfortunately, the potential impacts of Dysolve at full dose are not well reflected in this study as the results are tempered by the relatively low usage of Dysolve by treatment students. However, as a field trial, this RCT reflects real-world implementation, unlike experimental studies in lab-like settings where researchers have strict control over dosage. This RCT was not conducted in a lab setting and without strict control over treatment dosage. The real-world implementation allowed dosage to vary naturally, albeit with few treated students receiving the recommended dosage. Therefore, this report does not reflect the potential impacts of Dysolve at a full dose. However, as a field trial, the results suggest that future studies of Dysolve are warranted.
This study represents the first external, independent evaluation of the first AI program to deliver individually adaptive intervention to address reading difficulty, without requiring teacher training or instruction as part of the program. Adults without special training (e.g., parents, teachers, tutors, paraeducators) can supervise students using Dysolve. This feature may facilitate adoption and reduce costs in comparison to other programs that target similar outcomes. We plan to conduct additional studies of Dysolve to better understand its impacts under higher dosages, as well as the relative cost-effectiveness of Dysolve versus other interventions for students experiencing reading difficulties.
