EVALUATION & IMPACT
Impact evaluation is an important part of every project we fund. All partners provide us with interim and final reports outlining successes and challenges in meeting the agreed targets. For several projects, we have received in-depth feedback looking at learning outcomes and the effectiveness of the Mobile Learning Lab (MLL) in delivering the agreed objectives.
Key takeaways from the Mobile Learning Lab experience
The Project Evaluation Report we compiled following our two-year pilot project with CAUSE Canada in 2016–18 showed:
When the evaluation project was completed in July 2018, our partner officially handed over the equipment – tablets, solar charging system and the RACHEL – to the five communities where the MLLs were set up. Since then, each community has taken it upon themselves, without funding, to keep the MLLs open for students. Teachers in one community set up their own group to use the MLL for content learning, as well as pedagogical support. In 2021 we funded another project in the area which you can read about here.
The 30-page impact study on the results of the MLL pilot project in Sierra Leone, including tests and surveys administered to students before and after the project implementation, details the results against a control group. To read the report, click here.
In 2019–20, our partner Change for Children, conducted an evaluation of the MLL during and after a project in Comitancillo, Guatemala, following 1,638 students and 87 teachers with including intervention and control groups. The report, Technology for Improved Education, concludes:
“Despite the shortened length of the Project due to school closures as a result of the COVID-19 pandemic, the findings of the research study indicated that the comparative advantages identified in existing research had potential to improve student learning in this context as well. The technology showed promise in:
- Scaling up standardized instruction;
- Facilitating differentiated instruction;
- Expanding opportunities for practice;
- Increasing learning engagement and identifying how the technology could facilitate improved learning outcomes.
In addition, the research identified further ways that the technology could potentially facilitate improved student learning outcomes:
- Free access to high-quality resources;
- Teacher capacity-building;
- Student learning to use technology; and
- Creating and sharing family, community and cultural knowledge and engagement.
[…] Free access to resources and development of digital literacy may seem obvious or immaterial in many global contexts, but for rural communities they are potentially ground-breaking and transformative.”
While learning outcomes did not improve for the intervention group, our partner notes that the project was cut short due to COVID-19 school closures and that one year does not provide sufficient time to see improvements in learning. It also noted that students in Comitancillo are starting from a very low base, with test scores on national exams averaging 28% and 34% in language and mathematics, respectively, lower than the already low national average in Guatemala.