The Royal Opera House in London’s Covent Garden is pushing technological boundaries in every direction.
Joe McFadden (below) is Chief Technology Officer of the Royal Opera House. He talked to Blooloop about how technology at the ROH is affecting all aspects of their work – from everyday commerce to artistic innovation.
McFadden was IT Director at the National Theatre before joining the Royal Opera House. He has over 20 years’ experience in the fields of digital and technology. He leads the digital visitor experience initiative and is responsible for setting the strategic direction to maintain and strengthen ROH’s sector leadership in technology and digital activity.
Him and his team are also working on projects in a wide range of other areas. These include augmented reality, 360 VR, open data, app-based services and the Internet of Things (IoT – the inter-networking of smart devices, buildings and other items enabling the collection and exchange of data).
Three lanes for driving business change
Technology at the Royal Opera House is split into three main lanes, in terms of driving business change. Firstly, there’s running the operational side. “This tends to be more established IT systems from the desktop side, through to our e-commerce and booking system,” says McFadden.
Then there is new technology. “We use it to grow parts of the organisation,” says McFadden. “However, fundamentally, we are still operating on the same business models. This might be around better business intelligence, driving the uptake of membership benefits, or maximising ticketing yield revenue for our organisation.
“Finally, there are newer types of technology which have the potential to be transformational for the Royal Opera House,” he continues. “These are immersive technology, VR, and AR. Some are from an audience facing perspective, and some from the perspective of their potential in terms of changing the operating model.
“We organise our approach to technology, and the teams, with those three lanes in mind.”
Audience Labs – building new audiences
“There is a team looking at audience innovation,” says McFadden. “So we have Audience Labs, a relatively new addition for us. This brings together individuals who had already been thinking about and working on the use of technologies. We’re talking about conversational interfaces, Alexa [from Amazon], for example, as well as VR and AR.“
“We then have teams working very closely with the organisation around growth and optimisation,” says McFadden. “They tend to be organised as products based teams, so digital thinking around Lean and Agile.
“And then we have an operations team. They look after the bread-and-butter technology that various parts of the business rely on day-to-day. That includes things that are obvious to the audience, like our ticketing website. It then stretches through to things which are behind the scenes, but obviously very important. These could include scheduling systems, which are how we plan sometimes four, five, six years ahead.
“So this is quite a broad range of activity.”
Bringing VR and AR into a classic ballet
The VR and AR at the cutting edge of performance are particularly fascinating. “We’ve been looking at this for around two and a half years now,” says McFadden. “Our first experiment with VR was with 360 VR for The Nutcracker, our regular and very much loved Christmas ballet.
“That’s a piece of work which was very much trying to identify the language of virtual reality from a storytelling perspective. We set ourselves the challenge of trying to create a third person narrative.”
This narrative centred around the story of a new dancer, dancing in the Nutcracker’s corps de ballet for the very first time. “It was interesting both from a storytelling perspective, and also in terms of the technology,” says McFadden. “What worked; what didn’t.”
Capturing the sound of the Royal Opera House
The team then moved onto a set of projects working with the Royal Opera’s orchestra and chorus.
“The most recent of those was based around the opera Nabucco,” says McFadden. “Once again we were shifting to a first person narrative. We were trying to put the viewer very much in the shoes, as it were, of a member of the Royal Opera House chorus, as part of that amazing, hundred strong artistic group. First of all while they are rehearsing, then transitioning to being on stage at the Royal Opera House. Once again we were playing with different forms of narrative. However from the tech perspective, it was really about trying to capture, as authentically as possible, the unique and amazing sound quality of the chorus and the Royal Opera House itself, the acoustics within our main stage.”
This required the undertaking of a specialist piece of sound design, comprising 80+ mics around the auditorium. “We brought in specialist partners to help get the sound designers’ sound mix right,” says McFadden. “So that we could then present an ambisonic mix with the VR piece that really captures something of the unique sound quality in the Opera House.”
Now McFadden’s team is starting to think about something new. “Both of those projects were, in different ways, behind-the-scenes stories,” says McFadden. “We are now starting to move into thinking about work that is made specifically for VR, both in terms of opera and ballet.
“The first one of those is a project with Wayne McGregor, our amazing resident choreographer. He’s a ceaselessly curious and collaborative artistic professional: we are incredibly lucky to have him as part of the Royal Ballet family.
“He is very interested in working on the possibilities of virtual choreography. So, rather than being a cinematic capture placed in a VR setting, it takes the idea of mapping the dancers’ bodies. It brings them into a virtual world where you don’t need to follow all of the rules of physics and reality. You can do some interesting and innovative things with choreography in a virtual world, using the technology of a games engine rather than a cinema.”
HoloLens Technology Empowers Production Teams
In terms of AR, McFadden and his team have worked on a couple of projects. “One is actually backstage,” he says. “We are doing an R&D (research and development) project at the moment with Microsoft HoloLens. The aim is to empower the creative and design teams for productions at the Royal Opera House. We are probably one of the busiest theatres in the world, certainly in the UK. Therefore it is really important that we can be as flexible and as efficient as possible; with our rehearsal process, with our production process, and, crucially, with time on the stage.”
Inevitably every theatre has issues with sets and sightlines. Then there are technical issues around automation of pieces of set, or how that integrates into the lighting design. If these are only caught at the end of the process, they can be much harder and more expensive to fix.
“Using the HoloLens technology, we want to be able to empower production teams to play out some of those risks and issues earlier in the process,” explains McFadden. “We take set designs that are already in a 3-D pipeline, into the HoloLens environment. We are exploring gesture-based interfaces with HoloLens, and we’re also looking at the possibility of it being driven through and feeding traditional theatrical systems. So that could have lighting desks, for example, moved into a networked world in convergence with the technology it’s coming from, the world of information technology.”
Allowing Teams to Collaborate from a Distance
“I have increasingly moved in the direction of open source networking protocols,” says McFadden. “The project is the exploration of the possibility of a set and environment being collaborated on by design teams. An operator in real time would be able to drive changes to that set. Eventually to the lighting state through standard, well-known tools like professional lighting desks.”
McFadden is excited about the possibility of allowing teams to collaborate at distance.
“New forms of VR headsets are emerging,” he says. “There are going to be a whole raft of them based around Microsoft’s technology later this year. It will be possible for a designer who happens to be working in Paris to collaborate with a director in the States, and with some of the members of the team here in Covent Garden. Everyone would be working around the same model, some in augmented or mixed reality, and others joining remotely through VR. So that’s the vision.”
McFadden is currently working on this. “We’re going to be doing some demonstrations to partners in the sector towards the end of the summer,” he says. “We are then going to be looking for partners to help us take and grow that further. It wouldn’t just be for our own use, but for wider use in the creative industries.”
Reaching out to New Audiences with Augmented Reality
McFadden turns to the subject of the use of augmented reality for audience facing initiatives. “We are very excited by the potential here,” he says. “Particularly as we have seen it become much more of the focus of the large tech companies. Witness Facebook and Apple’s announcements in recent months. In addition, there was Apple’s recent AR kit announcement. What we have been doing is some small-scale R&D around location specific augmented reality. These are experiences that would deepen and re-enrich the experience of audiences. They would allow us to reach new audiences who perhaps don’t perceive opera and ballet as being for them.
“We have done a couple of small-scale R&D projects aimed at student audiences. These work with our student ambassadors to provide narratives that are, if you like, back stories to existing works. We are pretty excited for the potential of that.”
Other experiments with immersive technology within the audience labs setup include the next stages of development of using audio to create truly immersive experiences. “We’re even toying with some ideas of how we might be able to bring other senses into that kind of experience,” says McFadden. “It’s something people are starting to experiment with outside the Opera House. We are looking at that with some interest, and seeing how that might apply for us.”
It isn’t all plain sailing. Unforeseen issues inevitably arise with any kind of innovation and experimentation. “The original Nutcracker experience finishes on a quite complex shot,” says McFadden. “It still holds up today, although the technology is moving on at such a rapid clip. The shot takes you from within the corps de ballet almost as a zoom out. It’s a kind of tracking shot up into the fly tower. That’s a very impressive experience, but in terms of vertigo or nausea and things that can trigger it, that kind of shot was a problem.”
With the ambisonic mix, McFadden is very aware that, despite the painstaking work that has gone into the authentic reproduction of that sound, the playback capabilities of the equipment through which it is experienced are crucial, and could compromise its impact. “So we are also thinking about how we might ensure they will be experienced in the right setting,” says McFadden. “Whether that should be, for the moment, through front of house installations, or on the festival circuit.
Expanding the Boundaries of VR
“From a virtual reality perspective, it is still very early days,” says McFadden. “I think there are a number of questions around the ways in which these experiences can be consumed in a social setting, and the business model that will support them.
“We are also thinking about the length of experiences. The received wisdom is that VR installations should only be a certain length, around seven minutes. However, in common with some other performing arts organisations that are doing fantastic work in this area, we think that’s maybe an artificial barrier. If the material and the content is sufficiently engaging we can do more on that front.
“I think we’re going to get there in stages,” McFadden continues. “For example, with some of the made-for-VR we’re aiming at shorter pieces – around the half-hour length – before we think about a full-scale opera or a ballet. That’s two to three years away.”
In the long term, McFadden envisages that augmented reality will have a more profound impact than virtual reality. “The ways in which you might be able to include virtual elements within the performance itself are very interesting for the performing arts. The RSC’s production of The Tempest, which is just about to come to London, really sets the bar. It challenges performing arts organisations to think about what might be possible in terms of mixing virtual and physical performance, and also about how to blend those effectively.
“Certainly I think that, in the longer term, augmented reality is probably going to have the most profound impact for our industry. It will also impact the creative industries more generally. It’s going to be very exciting as more and more artists become aware of, and are given the opportunity to use these tools.
“I think that is one of the purposes of the audience labs. We will be able to provide a home for that within the family of artists that work with the Royal Opera House.”
He points out that there are many organisations now doing this kind of work. “There’s the National Theatre, with their Immersive Storytelling Studio,” he says. “Then there’s also the Lab at the Google Cultural Institute as well. I think it’s a really exciting time to explore the potential for immersive technology within the arts.”
Images credits: all ROH, except Joe McFadden kind courtesy CIO UK.