Mercedes Huertas-Migueláñez
A department created a user interface (UI) to facilitate user’s contribution to construct a linguistic resource by translating English terms into Italian. The department wanted to increase the number of contributions as it was very low.
We set out to evaluate and improve the usability of the existing UI by conducting lab testing sessions as well as unmoderated remote usability testing involving 13 Linguists.
The design process was divided in four steps inspired by Design Thinking methodology. In this process, we involved primary school teachers and we selected the methods and adapted them considering participants’ constraints.
In this step, we contacted 10 teachers by assisting to public events and following a snowball sampling. Teachers worked in 2 different educational institutions.
Through interviews and observations we got to know teachers’ practices to create, share and store their instructional materials.
Problem definition: After the analyses of the data collected, we concluded that one of the two institutions was facing issues to organise teaching materials. We decided to move forward the collaboration with them.
UI used to translate terms from English into Italian
After analysing the quantitative and the qualitative data collected, we observed that students were very enthusiastic about the task as it allowed them to learn nuances in their mother tongue as well as in English. Moreover, they could contribute from anywhere.
However, the system had issues that hindered the correct interaction resulting in low numbers of contribution through the UI: the issues mainly referred to the layout of the UI and the organisation of the terms they had to translate or evaluate.
After the analyses of the data collected, we concluded that one of the two institutions was facing issues to organise teaching materials. We decided to move forward the collaboration with them.
Considering user’s opinions and interaction data we proposed a new design for the UI.
In this step, we involved a new group of 7 participants from 2 different regions in Italy.
We conducted a paper prototype testing where participants were asked to complete a series of tasks using the paper prototypes. This time they translated terms from English into Italian and peer evaluate each other.
The aim was to evaluate the new design as well as to observe whether involving participants in earlier stages of the design process could have an effect on engagement.
After the prototype test session, we adjusted the designs according to users’ feedback and transferred the prototypes to the final UI using javascript, HTML5 and CSS.
In this phase, we conducted an unmoderated remote usability testing. We assigned each participant a number of terms to translate and evaluate using the UI. We gave them two months and a diary to annotate their impressions during this period.
During this phase, email played an important role to keep communication with participants, showing that we were “present” and notifying them whenever issues emerged.
The number of terms evaluated and translated per participant is significantly higher, compared to the exploratory study. Some participants had personal problems and could not contribute as they wished during the two months.
The total number of terms translated (250 terms out of 600 terms) and evaluated (180 out of 600 terms) is higher during the test of the new design. The Exploratory Study was conducted over three months, whereas the second was conducted over two. Although both groups of participants were committed, only the second group was engaged, which means that creating and maintaining a trust relationship with users has a direct influence in their engagement.
In this project I collaborated with developers and project managers to integrate the user perspective in the design of the new UI.
This project validated the idea of the importance of building trust relationships with participants to engage them in the design process as well as in the use of the tool.
With this study, I learnt how to manage a group of participants remotely.