Evaluation of the Mediavault tool with SINEA creatives, December 1st 2020, Barcelona, Spain.
Image courtesy of Irene Calvís Rovira.
A main goal of the TRACTION project is to use technology to support co-creation and consumption of opera content by diverse communities across the European Union. The project comprises three trials, INO (Irish National Opera), SAMP (Sociedade Artística Musical dos Pousos), and LICEU (Gran Teatre Del Liceu). Based in Dublin, the goal of the INO trial is to use new technology and novel audio-visual formats to create a new, digital community opera in Ireland. LICEU, which is based in Barcelona, concerts a creative collaboration between migrants of different cultures from the Raval neighbourhood and opera professionals at the LICEU opera house, in order to train employable creative skills. SAMP, a community project for young inmates in Leiria, Portugal, likewise aims to engage prisoners in an opera co-creation process in order to lower the rate of crime.
User-centered design is at the heart of the Traction toolset development. This approach to interactive systems development focuses on understanding and evaluating user needs and requirements by applying human factors/ergonomics, usability knowledge, and techniques through an iterative process of design and feedback. User-centered design not only enhances technology design effectiveness and efficiency, but also improves human well-being, user satisfaction, and accessibility. The design and evaluation activities of technology and its use in community opera co-creation are repeated during the project with the purpose of continuous improvement.
In this article we first overview our user-centered timeline for the three trials, then describe our user-centered methods.
First Year Timeline
Below is a timeline of our user-centered design process.
Timeline of our user-centered design process for the three trials, INO, LICEU, and SAMP.
Our first step was a series of conversations and discussions with the user groups to get an initial idea of the type of trials they wanted to run. After this, we gathered and derived an initial set of requirements through three focus groups with TRACTION partners from INO, LICEU, and SAMP. Based on these initial requirements, we developed personas and use cases with INO partners, then ran a second set of focus groups to derive a second set of requirements.
In August, we began development on the Mediavault tool. From August through October we worked with the Liceu partners to map the user journey process for the tool, first generating personas and use cases, then developing storyboards, and creating wireframes that our team used to develop the Mediavault tool. In December, we evaluated the usability and usefulness of the Mediavault with SINEA creatives and students from Escola Massana in Barcelona.
In parallel to the Mediavault, we worked with SAMP partners to develop the Performance Engine tool. In September we created personas and use cases. In October we created storyboards for the tool, and finalized a set of wireframes for the tool in November. In December, we evaluated the usability and usefulness of the Performance Engine with inmates and partners from the SAMP trial.
Development of the Mediavault and Performance Engine happened throughout the year. From January to April, developers conducted initial tests and evaluations of system functionality. In May, we began development on the underlying framework, and in August we began working on the backend of the two tools, refining those based on the requirements gathered in the focus groups with the three trials. In October, we began working on the frontend of the tools. As we went through our user-centered design process and iteratively refined the designs of the systems, we iterated through a series of designs to complete the frontend interfaces.
Our user-centered process encompassed several design and research methods. First, we gathered requirements for the TRACTION tools through a series of focus groups. Then, we mapped user journeys by developing personas, use cases, storyboards, and designed Mediavault and Performance Engine interfaces by creating a set of wireframes. After developers employed the wireframes to develop the tools, we evaluated them through a usability test paired with an in-depth interview about their experience.
A sample user persona representing a student from Escola Massana.
Focus Groups: Gathering Requirements
The goal of gathering user requirements was to inform the design of the Traction technology for the three trials, in order to build a common understanding of the project between Traction team members, by identifying potential users, and identifying and refining a set of user requirements for toolset design.
We gathered initial requirements through a set of focus groups. A focus group is a group discussion on a particular topic, used for generating information on collective views, and the meanings that lie behind those views. They are also useful in generating a rich understanding of participants’ experiences and beliefs. This discussion is loosely guided, monitored and recorded by a researcher or facilitator.
From the focus groups, we obtained a set of requirements for the Mediavault and Performance Engine designs based on the thematic analysis from the FG1 trials. These requirements specified what the two tools should do and how they should perform, and were separated into functional requirements that specify what the system should do, and non-functional requirements, that specify what constraints exist on development of the systems; such as data requirements, environmental requirements, user requirements, and usability requirements. To concretize our design goals, we separated the requirements by trial; and ranked requirements by relative interest to the trials, feasibility, and project priority.
Mapping User Journeys
A “user’s journey” is a relationship with an interaction experience over time and across different channels. For each user, it is vital to understand the following:
- Motivation – Why are they trying to do it?
- Channels – Where interaction takes place
- Actions – The actual behaviors and steps taken by users.
- Pain points – What are the challenges users are facing?
Our user journey consisted of the following steps:
- Developing User Personas: Defining archetypical users whose goals and characteristics represent the needs of a larger group of users.
- Defining Scenarios and Use Cases: Brainstorming a set of possible user scenarios, and describing how users will perform tasks for those scenarios. This helps explain how a system should behave, and help establish a list of goals that may be used to establish the cost and complexity of a system.
- Storyboarding Interaction: Sketching a set of use cases as a series of step-by-step interactions. Each step demonstrates an experience that the persona has with a tool or another person.
A sample storyboard representing interaction with the Mediavault Tool.
- Developing Wireframes: Wireframes are low-fidelity, simplified outlines of a user interface that shows a layout of different screens of an application. Wireframes often use placeholders for text and images, focusing instead on the layout of a design.
A sample wireframe of the Mediavault tool login page.
After a lengthy design and development process, we evaluated the Mediavault and Performance engine through a series of usability tests paired with in-depth interviews. The usability tests evaluated how easy the two systems were to use, while the in-depth interviews asked users to envision the usefulness of each tool in practice; for codesign, learning, and outreach.
Usability Study: A research technique to evaluate the design of a system, product, or experience by testing it directly on users. The goal of usability testing is to measure how well a design meets its intended purpose, and how intuitive it is for users with no prior experience to learn to use the tool.
In-Depth-Interview: A technique to discover the respondents’ perceptions or to probe into a subject to explore nuances and details. In-depth interviews vary from informal conversations to more formal interviews, which may be unstructured, semi-structured or structured.
Overall, the user-centered design process is vital to creating an interface that adequate supports a range of user needs. Through our work, we aim to support the goals of TRACTION, by developing technology that can be easily used by all of our TRACTION partners and participants.