User-centered design of Kinyarwanda MT applications for education
We wrote recently about the experience of working with Kinyarwanda speakers to generate open-source language data for digital solutions in their language. But language data is only as useful as its application to real-world needs. To deploy the Kinyarwanda data and deliver effective applications of language AI we had to apply human-centered design principles for the purpose of understanding user needs for a specific use case. And we also wanted to do that in a way that builds broader language tech capacity to meet the needs of marginalized language speakers. Here we look at how we went about that, in partnership with Rwandan tech firm Digital Umuganda, and with what results.
We delivered far more than a machine translation integration
We do things differently at CLEAR Global. Our language technology work aims to build capacity beyond delivering specific project outputs, and to showcase the potential of a collaborative and open-source approach to helping people to get information and be heard, whatever their language.
So we didn’t take the conventional approach to product development. Rather than developing a generic machine translation (MT) model, we built a bespoke model for an educational use case capable of delivering better performance for users. Tests found that the model, built with language data sourced from the Atingi platform and other educational content, performed markedly better than generic models like Google Translate.
That improved performance is largely due to our collaboration with Rwandan partners, particularly Digital Umuganda. With their teams of Kinyarwanda speakers and in-depth knowledge of the context, they were able to identify relevant data sources to train and evaluate the model, and to assess MT performance against a strong understanding of user needs. Together with the Rwandan Cultural Heritage Academy, they were also able to validate the end product as being in line with national standards.
We wanted to demonstrate the value of developing open-source applications of language technology for improving public service delivery – in this case education. We built on user research and human-centered design principles to develop and test three ways in which MT could improve the user experience. Not only did this yield a better experience for learners, but it provides a basis for course developers to continue expanding and improving their online training offering for multilingual learners. The model itself was trained for two-way translation between English and Kinyarwanda, but can now be trained for other combinations of languages according to need.
In fact everything about this project was designed to leave something behind. We shared the model as a learning management system on the open-access platform Moodle, so training providers beyond Atingi can use and build on it. We shared the education-specific language datasets for Kinyarwanda and English on the open-source collaboration hub Hugging Face so they can be used in building other MT applications. Our knowledge exchange with Rwandan partners like Digital Umuganda leaves them with a better understanding of the steps involved in human-centered MT development, which they can add to their already impressive tech expertise in other areas. The net result is that there is increased capacity for building language technology to improve public services for speakers of less well-served languages like Kinyarwanda.
How we did it: Listening and Partnership
The use case was to improve the user experience of Rwandan learners on the Atingi training platform
The project aim was to improve service delivery by applying an open-source machine translation model in an education sector use case. With support from GIZ, CLEAR Global and Digital Umuganda partnered with digital learning provider Atingi to improve the digital learning experience for young people. The Atingi platform offers accessible, high-quality education and training worldwide. It is designed to help learners in low- and middle-income countries especially to improve their skills and knowledge. Over 300 courses and other learning materials are available online, on topics such as digital skills, entrepreneurship, and vocational training.
Atingi is part of a broader effort to leverage digital technology to enhance education and training opportunities in the interests of economic and social development. Governments, NGOs, and private-sector partners regularly use the platform to reach a diverse audience and address specific educational needs. Yet uptake in Rwanda was low, and language barriers were assumed to be one of the main obstacles. Atingi was interested to work with us to find a solution.
A user-centered design approach
Our overall aim was to assess the impact of Kinyarwanda language support on the Atingi platform’s user experience and adoption. That required a user-centered methodology comprising the following steps:
Goals | Methodology |
| Define the problem through user research | User personas, empathy mapping, localized surveys |
| Analyze the impact of the problem on end-users and stakeholders | Report building, trend, and pattern analysis |
| Develop a clear problem statement | Define who, what, and why through focus group discussions |
| Validate/invalidate hypothesis | Prototype and A/B test proofs of concept |
Users wanted the ease and depth of understanding of learning in their own language
Our user research explored the specific challenges faced by users of the platform. It found that the linguistic nuances and cultural context embedded in Kinyarwanda significantly impact comprehension and engagement. While translated content provides a basic understanding, it often fails to capture the subtleties and relatable examples that native language materials can offer.
Users also explained that learning in their own language, Kinyarwanda, feels more intuitive and less cognitively taxing than navigating translated English content. This suggested that incorporating Kinyarwanda support could improve not just the platform’s accessibility but the whole learning experience, leading to higher user engagement and satisfaction.
User personas and journeys provided a core reference for designing a solution
The partners took the conclusions from our research as a basis to identify user personas for both students and course creators around the use of MT. These were intended to ensure the design and tech team had a continuous reference to users’ needs, goals, and aspirations
The main focus was on the student persona, capturing their journey from a point of frustration to a point of satisfaction through the implementation of the proofs of concept around the use of MT to improve the service.
We developed three possible proofs of concept
The initial findings suggested three possible proofs of concept for language technology. These solutions were designed and tested for problem-solution fit:
1. Navigation in Kinyarwanda – Localize navigation text and controls in Kinyarwanda.
Reasoning – User research indicated that users are more likely to engage with a platform when its interface is in their first language. This approach ensures that users can easily understand and navigate the platform, reducing cognitive load and frustration.
2. Complete subject translation – Translate the entire course content into Kinyarwanda.
Reasoning – Full translation of course content addresses the core issue of language barriers, allowing users to fully grasp complex concepts and instructions. This method ensures that all educational material is comprehensible and culturally relevant, leading to a more effective learning experience.
3. In-line translation – Enable users to highlight and translate phrases into Kinyarwanda.
Reasoning – In-line translation provides flexibility for users, who may have varying levels of proficiency in English. It allows them to translate specific phrases or sections as needed, fostering a more personalized and supportive learning environment. This feature helps bridge the gap for users who are transitioning to learning in English but still need occasional support in their first language.
Testing validated all proofs of concept and showed highest satisfaction with the mobile experience
Our hypothesis was that the lack of Kinyarwanda navigation limits adoption of the platform, and that full content translation and in-line translation would improve user experience and adoption. A/B testing of the mobile compared with the web experience with a group of selected users gave us feedback on using the MT integration for each of the three uses: navigation, course content translation, and in-line translation.
We looked for testers who were:
- Native Kinyarwanda speakers
- Existing users of the Atingi platform
- New users of the Atingi platform
- Users with both Kinyarwanda and English fluency
- Users with specific industry expertise (AI, entrepreneurship)
Tests found that the introduction of localized navigation in Kinyarwanda, full course content translation, and the in-line translation resulted in increased satisfaction. Full course content translation yielded a test satisfaction score of 71%, while in-line translation with the widget got an 86% satisfaction score. Users tested both the mobile and web versions of the three proofs of concept and displayed higher satisfaction with the mobile experience, as mobile was their preferred channel for consuming course content.
We integrated an open-source translation model that works for learners now and in the future
In collaboration with Digital Umuganda and Atingi, we developed clear specifications for each proof of concept and built low-fidelity prototypes for the initial research. We later developed these into high-fidelity prototypes in order to validate the user experience and problem-solution fit.
We also had to build the language model to deploy machine translation for the use cases identified. To ensure that MT provided relevant context for users, our approach was to develop an education-specific model using language from the education sector generated with Kinyarwanda speakers. We fine-tuned and trained a bidirectional model for English-Kinyarwanda translation, enabling learners to translate either entire courses or highlighted sections of text between English and Kinyarwanda. The solution also allowed course creators to deploy machine translation for an entire course content and then make potential adjustments for standardized and contextual translations.
The partners then integrated the translation model into Atingi’s learning management system (LMS) to provide course content translation via a unique widget. Because the integration is open-source, the Kinyarwandan-speaking community can use it to continue improving and expanding language capabilities in the education sector.
A future where the internet speaks every language
At CLEAR Global, we believe everyone has the right to get information and be heard, whatever their language. People who want to access digital services should be able to do so in the language they feel most comfortable using. In order to have a fairer digital future, language technology beyond commercially viable and globally dominant languages is the first step.
Are you working on language technology solutions in humanitarian or development programming?
With thanks to our technology and funding partners:
Digital Umuganda
Digital Umuganda is an AI and open data company with a mission to enable access to information in local African languages. They create open-source datasets, models and tools that make it possible for natural language processing (NLP) including large language models (LLMs) to work for marginalized communities that speak low-resource languages. Learn more at digitalumuganda.comDigital Transformation Center Rwanda
The Digital Transformation Center is a Rwandan-German initiative aimed at developing impact-driven digital solutions in Africa. It provides not only advisory services and training for government institutions and local tech companies but also a modern space to boost creativity and collaboration. Learn more at digicenter.rwGIZ Fair Forward
On behalf of the German Federal Ministry for Economic Cooperation and Development (BMZ), the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) implements the project “FAIR Forward – Artificial Intelligence for All”, which works to create a more open, inclusive, and sustainable approach to AI on the international level, and more specifically, to develop artificial intelligence ecosystems locally across its seven partner countries (Rwanda, Uganda, Kenya, South Africa, Ghana, India, and Indonesia). For more information, visit FAIR Forward – Open data for AI.
