Live Performance and Virtual Reality

By Terry Madigan, Camille Donegan and Aisling Phelan of Virtual Reality Ireland

The arena of live performance and VR is nascent but extremely fast moving with technology evolving at a rapid pace which is only increasing in speed to keep up with new Covid-specific demands. At a time when most theatre, opera houses and other cultural institutions are closed or capacities are extremely curtailed, the cultural sector is suffering through an unprecedented time of crisis. Prior to Covid, explorations of remote performance were seen as Research and Development (R&D) based rather than having real commercial prospects. This new world we live in has changed all of that.

In a matter of months, we have seen an art and games collective in New York -Tender Claws, partner with the largest headset manufacturer in the world – Oculus, owned by Facebook, to bring a performance of the Tempest by a live performer in VR to a headset near you, all for a ticket price of $14.99. Not only that but you visit a virtual box office in a desert to purchase your ticket. This exploration was innovative, exciting, progressive and offered a light of hope to artists all around the world. Actors could get paid to perform in their bedrooms (providing they had an Oculus!) and producers could create new work on VR platforms and explore new revenue models for audiences.

Another example is Miranda, a steampunk VR Opera offered on a multitude of platforms presented by LUMA festival (see . Miranda is ‘a fully immersive musical theatre VR experience featuring live motion-capture performance during every showtime’ and is definitely breaking new ground.

What has all of this got to do with the TRACTION EU project? Well, the purpose of TRACTION is to
promote Opera to new audiences, especially disenfranchised communities via novel formats. It has a strong focus on utilising the power of technology to extend reach and offer new possibilities for the dissemination of content.

In an already jaded world of online consumption where countless theatre companies and opera
houses provide unlimited (and often free) pre-recorded content online for audiences, the public are screaming for something more, something different, something more akin to the live, physical and visceral theatre experience which occupies all your senses. This is far from easy. From the smell of the auditorium to the buzz of energy as audience members look for their seats and excited chatter of groups. A live performance is deeply sensory and to be honest, will never and could never be fully replicated via immersive technology solutions.

We have discussed this conundrum as part of the TRACTION project and carried out focus groups exploring many difficult questions such as the following:

‘Should we replicate the current Opera experience, or create entirely new possibilities for the artform?’

‘What aspects of these novel formats offer new ways for audiences to engage?’

‘How much should we focus on the audience experience versus the co-creation artistic process?’

Deep questions, worthy of a multi-year European Commission supported research and innovation project.

We believe TRACTION is more relevant now than it was pre-Covid 19 and that there is a fresh impetus within this consortium to find new avenues to satisfy audiences and to take them somewhere entirely novel are of paramount importance. To this end, we are closely monitoring the performing arts sector as it transitions from analogue to digital. We believe that this is an opportunity to explore what immersive technologies can achieve that traditional methods cannot, such as experiencing immersion, simulation, interaction and embodiment.

It’s not just opera and theatre that are looking for new ways to create and connect with audiences.
The music world, particularly live music concerts have also been exploring opportunities provided by
immersive technologies. Jean-Michel Jarre held a highly successful VR Chat concert which was
organised for Music Day on June 21st 2020 and attracted 600,000 people who experienced the event (in VR headsets) from all over the world. Additionally, major physical arts/cultural festivals like Burning Man, typically held in the Black Rock Desert in Nevada, have taken place digitally using immersive technologies.

Jean-Michel Jarre concert in VR Chat – Image courtesy of JMJ

Interestingly, both professional and amateur artists from around the world are embracing the possibilities presented by the VR Chat platform. This is illustrated by the line-up of projects chosen for curation in the Raindance Immersive programme 2020, several of which have been built and have captured live performance via VR Chat.

Virtual Reality Ireland Media have been working on numerous potential directions and areas of research relevant to this new digitally live environment including virtual production and live body, hand and facial motion capture. Working with the Rokoko Smart Suit and Smart Gloves, HTC Vive Trackers, Adobe Fuse models and facial capture technology such as the Rokoko Face Capture Studio, the team at Virtual Reality Ireland Media have set up an infrastructure and workflow where a performer’s physical movements can be captured and that motion data can then be mapped to any avatar in platforms such as Unreal Engine and VR Chat.

Immersive technology experiences offer audiences a new way to experience art, from within a world where the characters, environments and audio have all been created for the performance. They may be interactive or passive, multi user or single user, using motion capture data for movements or not, but all of these explorations are bringing us further along this journey – looking to connect audiences with live performance in new, innovative and exciting ways.