Co-creating a Virtual Reality Opera

A participant playing a VR experience with a VR headset and controllers.

In-person co-creation workshops began on 19^th August 2021. It was remarkable after 18 months of not being able to be face-to-face with people, to feel the energy of a collective again. A key part of these workshops with the community was to bring the participants through the possibilities of virtual reality and the unique storytelling attributes inherent with the medium.

On that first morning, I ran a session detailing some of the unique attributes of VR. There are also many user experience design principles that need to be followed in order for users to have a comfortable experience. Some of these mechanics that are specific to the medium of VR include:

• Scale – in VR we can experience scale in profound new ways. As we are in a 360-degree spatial medium, a character’s perspective could be huge, like they are a giant, or indeed tiny, as though they were a small mouse. Experiencing scale like this is not possible in any other medium.
• Motion – in order to mitigate against motion sickness, smooth constant velocity is recommended.
• Convergence – we need to be careful when positioning objects or text in VR to keep them at a comfortable distance and to not make our users cross-eyed which can lead to headaches.

After the storytelling workshop, as this is an experiential medium, we wanted people to try it. The first demo that I asked everyone to try was First Steps from Oculus, aptly named as it is a really great ‘first step’ to getting to grips with your controllers. All of the possible buttons and mechanics are explained in a fun and experiential way which includes dancing with a robot! One of the main barriers I find with many new users of virtual reality is the limiting belief that they won’t know how to do it or will break something. I often hear “I’m not a techie” or “I’m not a gamer – I don’t know how to use controllers. What are these things in my hands?” First Steps blows all of that out of the water.

The next demo that I love to give people is heavily curated to them and their interests. I spoke with both the professional artists who were trying virtual reality and the community participants. I discovered that one of the participants has a strong interest in audio so I curated ‘Sanctuaries of Silence’ for him, a really beautiful 360 film about an audiologist looking for silence all over the planet and going deep into the Amazon to try to find it. Another participant has a big interest in Star Wars and really enjoyed Vader Immortal, where you come face-to-face with Darth Vadar. We had a community member who loves to travel and really missed traveling during the pandemic. She tried Wander, a VR experience that incorporates Google Earth data which enables you to virtually travel all over the globe. I particularly enjoy watching people having an ‘AHA!’ moment when they see the potential for the medium after experiencing a piece of work that resonated with them emotionally.

After the initial workshops, we returned with an exploration into mythological tales, where Sorcha Hegarty from Candlelit Tales recounted four Irish myths relating to some of the themes that came up during the online workshops with all of the communities; such as travel, journey, return, nature and birds in particular to name but a few.

After we went through those stories, I wanted people to see the mythology VR story, Baba Yaga from Boabob studios which is a really great example. Apart from the unique picture-book aesthetic of the piece, I love the branching narrative where you can choose to be the good witch or the bad witch. We’ve taken inspiration from that and are looking at having a branching narrative within the virtual reality opera that will be the culmination of all of this co-creation process with the community.

Parallel to the myth and creative writing sessions led by Jody O’Neill, the musicians – both professional and members of the communities – were recorded playing. We were lucky to work with Guillaume Auvray who recorded several of the compositions led by Finola Merivale with an ambisonic microphone. Ambisonics and binaural audio are types of spatialised audio used in VR production. Spatial audio provides the ability to hear a soundscape from a variety of directions as you move through environments in a VR headset. Through these workshops, with community participants from the Midlands of Ireland (the partnership with Music Generation), and the community group in Tallaght (through the partnership with the Civic Theatre), we collated a large number of assets in the form of imagery, stories (both personal and fictional), poetry and music.

Guillaume Auvray recording a community composition with an ambisonic microphone.

Press play below to hear professional musicians and community participants improvising ‘travelling across water’:

It was then time to travel to Inis Meáin bringing all of these assets (many in physical forms of post its, drawings, paper based life maps, written stories etc) and work with the communities there to build upon, adapt and extend the ideas.

What resulted was two wonderful workshop days with the community of Inis Meáin. The first being with a variety of school children who, unsurprisingly, couldn’t get enough of VR. I think several of the children on the island now want a VR headset for Christmas, which is an interesting juxtaposition in itself as there is very little technology on the island bar mobile phones. It is uncommon for people to have computers or game consoles and the WiFi on the island is very limited. We had prepared for that with our VR demos and had downloaded the experiences in advance. The second workshop day on Inis Meáin featured a broad spectrum of participants. Some had previously engaged with the online workshops while others were new to the TRACTION project. We had a really productive day of creative writing and sharing, culminating in revising the myth ‘mixes’ we had brought from the other communities. From these sessions, we had a jumping off point for our overarching narrative for the VR Opera, an exciting milestone!

Now as we embark on the pre production stage of the VR Opera, we are defining how to continue the co-creation process with community participants. We are planning to hold several interactive sessions where we collectively discuss elements of the story, visuals, style, interaction design, music etc. In addition, technology is providing new methods of co-creation possibilities. One idea we hope to implement are motion capture days with communities where they can put on a motion suit, glove and use face tracking to provide the movements for various natural elements in the world such as seaweed, jellyfish, birds and fish.

A participant testing the motion capture technology.

If you are interested in knowing more about motion capture, you may want to read this previous blog in the topic.