Skip to main content

Research & Evidence

Evidence-based design

Acrobatiq’s adaptive, data-driven courseware is the first in a new generation of interactive and comprehensive learning solutions that builds on the success of more than a decade of research from Carnegie Mellon University’s Open Learning Initiative.

Since 2001, OLI has been at the forefront of scientifically-based research on the impact of online learning environments on student success. Results at multiple institutions point to accelerated learning, reduced student attrition and significant correlations between learning activities and learning gains.

Highlighted research

The following studies highlight a few of the research findings:

Faster learning in hybrid courses

Researchers sought to determine if students using the Statistics course would learn at a different pace than students in a traditional face-to-face course format. Results exceeded expectations. Students completed the OLI Statistics course in 8 weeks, with 2 class meetings per week, while students in the face-to-face format completed the course in 15 weeks, with 4 class meetings per week. Although OLI students spent no more time studying statistics outside of class than their traditional peers, they demonstrated learning outcomes that were as good or better than those of their peers. They also retained the information in tests 1+ semester later.
The Open Learning Initiative: Measuring the effectiveness of the OLI statistics course in accelerating student learning. Journal of Interactive Media in Education. Marsha Lovett, Oded Meyer, & Candace Thille, 2008.

An Artificial Intelligence Tutor: A Supplementary Tool for Teaching and Practicing Braille

This study evaluated the usability and effectiveness of an artificial intelligence Braille Tutor designed to supplement the instruction of students with visual impairments as they learned to write braille contractions. Methods: A mixed-methods design was used, which incorporated a single-subject, adapted alternating treatments design as well as qualitative teacher interviews and surveys.
McCarthy, Tessa; Rosenblum, L. Penny; Johnson, Benny G.; Dittel, Jeffrey; Kearns, Devin M.; Journal of Visual Impairment & Blindness, v110 n5 p309-322 Sep-Oct 2016

Reduced time to completion and cost savings at multiple institutions

Non-profit research organization Ithaka S+R compared a hybrid version of the Statistics course with a traditional face-to face Statistics course using randomly assigned students at six public universities. Students in the hybrid format had comparable or better learning gains and took 25% less time to achieve the same outcomes. Managing Director of Ithaka S+R, Deanna Marcum wrote, “The results of this study are remarkable; they show comparable learning outcomes for this basic course, with a promise of cost savings and productivity gains over time.”
Interactive Learning Online at Public Universities: Evidence from Randomized Trials, William G. Bowen, Matthew M. Chingos, Kelly A. Lack, & Thomas I. Nygren, 2012.

Multiple courses, faster completion, and improved learning outcomes

In a major study involving several OLI courses used in U.S. community colleges, students using the courseware covered 33% more content in the same time as their peers in traditional courses and achieved a 13% learning gain, compared to 2% by peers in traditional face-to-face courses.
Open Learning Initiative Courses in Community Colleges: Evidence on Use and Effectiveness. Julia Kaufman, Sarah Ryan, Candace Thille & Norman Bier, 2013.

Independent trial of the OLI statistics course

In 2007, Carnegie Mellon conducted a series of “do no harm” studies with the OLI statistics course. The studies show that students using the OLI course, as an online course with minimal instructor contact, performed as well or better than students in traditional instructor-lead classes. Then, a study conducted by ITHAKA, a nonprofit research organization, demonstrates the same results using the OLI statistics course outside of Carnegie Mellon—in several large public institutions.
Bowen, W.G., Chingos, M.M., Lack, K.L., & Nygren, T.I. (2012). Interactive Learning Online at Public Universities: Evidence from Randomized Trials. ITHAKA.

OLI study on accelerating student learning with OLI statistics

This study, conducted at Carnegie Mellon University, shows that students using the OLI statistics course at Carnegie Mellon achieved the same or better learning outcomes as students in the traditional course in half the time.
Lovett, M., Meyer, O., & Thille, C. (2008). The Open Learning Initiative: Measuring the effectiveness of the OLI statistics course in accelerating student learning. Journal of Interactive Media in Education.

Accessible Artificial Intelligence-Based Chemistry Tutoring for Blind and Visually Impaired Students

We report the development of an artificial intelligence (AI) tutoring software system that is accessible to students who are blind or visually impaired, delivered through the Internet. In prior research, we have successfully developed, tested and brought to the classroom AI-based tutoring systems in chemistry education. This work achieved several important firsts in chemistry tutorial software: enabling students to get interactive assistance on their own problems, which they enter into the tutor, get feedback on their own work, which they also enter in detail, and ask the tutor specific questions about how the concepts being studied apply to their current situation. In controlled testing, these intelligent tutors have significantly improved student performance and understanding. The aim of the present research is to bring the full benefits of this new educational technology to students who are blind or visually impaired. The problem of quality educational software materials for the blind is known to be particularly acute. Certain unique attributes of the AI tutoring systems make them well suited for full accessibility using Internet-capable screen reader technology.
Benny G. Johnson†, Jeffrey S. Dittel, and Dale A. Holder; The Chemical Educator; Published: 17 March 2010

Additional publications