EVALUATION & IMPACT

Impact evaluation is an important part of every project we fund. All partners provide us with interim and final reports outlining successes and challenges in meeting the agreed targets. For several projects, we have received in-depth feedback looking at learning outcomes and the effectiveness of the Mobile Learning Lab (MLL) in delivering the agreed objectives.

Key takeaways from the Mobile Learning Lab experience

• The children love it! If learning is to be meaningful for children and something they want to do, rather than an obligation, the learning experience should be fun and engaging. The wide range of content on the MLL, and the fact that the content is often interactive, and increasingly gamified, means that children are likely to find exactly the information they need in a format they enjoy using. When education is fun, children will easily spend more time learning.
• School attendance increases dramatically on days when the children know they will be able to use the MLL.
• Literacy and numeracy scores improve, as evidenced from our own evaluation reports, and from partner organizations.
• Partners can download curated educational materials based on local needs, local ministry of education requirements, text books, or custom content in local languages.
• Parents appreciate that their children have access to technology, understanding that its importance for their futures. When they see their children using ICT and learning, they become more participative and enthusiastic partners in their children’s education.
• Girls like to be able to access ‘’private’’ information on their own on sexual health and reproduction topics.

Sierra Leone

The Project Evaluation Report we compiled following our two-year pilot project with CAUSE Canada in 2016–18 showed:

• Attendance was 90% over the 8-month period of access to the MLL. Students would often stand in line when it was not their assigned day in the hope that other students might not show up and they could take their place.
• Improvement in math and literacy scores was almost twice as high as that of those students in the control group who did not have access to the MLL.
• Students using the MLL were more enthusiastic, participated more in the classroom, and asked more questions.
• Students worked together and learned from each other.
• Students appreciated the child-friendly environment of the MLL, where they could learn in a respectful and non-violent space.
• Students were engaged and motivated. Their self-confidence increased. They developed strong peer learning bonds, and their knowledge and learning extended back to their own families.

When the evaluation project was completed in July 2018, our partner officially handed over the equipment – tablets, solar charging system and the RACHEL – to the five communities where the MLLs were set up. Since then, each community has taken it upon themselves, without funding, to keep the MLLs open for students. Teachers in one community set up their own group to use the MLL for content learning, as well as pedagogical support. In 2021 we funded another project in the area which you can read about here.

The 30-page impact study on the results of the MLL pilot project in Sierra Leone, including tests and surveys administered to students before and after the project implementation, details the results against a control group. To read the report, click here.

Guatemala

In 2019–20, our partner Change for Children, conducted an evaluation of the MLL during and after a project in Comitancillo, Guatemala, following 1,638 students and 87 teachers with including intervention and control groups. The report, Technology for Improved Education, concludes:

“Despite the shortened length of the Project due to school closures as a result of the COVID-19 pandemic, the findings of the research study indicated that the comparative advantages identified in existing research had potential to improve student learning in this context as well. The technology showed promise in:

1. 1. Scaling up standardized instruction;
   2. Facilitating differentiated instruction;
   3. Expanding opportunities for practice;
   4. Increasing learning engagement and identifying how the technology could facilitate improved learning outcomes.

In addition, the research identified further ways that the technology could potentially facilitate improved student learning outcomes:

1. 1. Free access to high-quality resources;
   2. Teacher capacity-building;
   3. Student learning to use technology; and
   4. Creating and sharing family, community and cultural knowledge and engagement.

[…] Free access to resources and development of digital literacy may seem obvious or immaterial in many global contexts, but for rural communities they are potentially ground-breaking and transformative.”

While learning outcomes did not improve for the intervention group, our partner notes that the project was cut short due to COVID-19 school closures and that one year does not provide sufficient time to see improvements in learning. It also noted that students in Comitancillo are starting from a very low base, with test scores on national exams averaging 28% and 34% in language and mathematics, respectively, lower than the already low national average in Guatemala.