In a fast-moving, post-pandemic world, brands need to come up with new ways to reach and engage with their audience. As traditional retail flounders and COVID changes the way we interact, Sensape is coming up with new high-tech ways to solve the problem.
Sensape, a creator of unique immersive brand experiences, spoke to blooloop about the rising trend for augmented reality (AR) and how, when combined with AI technology, it can bring brands to life and create new audience connections.
“We are one of the leading AR and AI tech companies in Europe,” says Michael Lehnert, Commercial Director of Sensape, speaking from the company’s headquarters in Germany.
“Our mission is that we try to teach a computer to see and to understand its environment, like a human being. With our self-developed AI, the computer takes clues on who is in front of it. It recognises different factors to then decide, ‘How can I interact with this person?’”
Using AI to identify the audience
On interacting with a user, the AI technology can gather information to build a picture of who they are:
“For this, the system uses a camera stream and can recognise, in real-time, the different external features of a person, or many persons. For example, height, emotion, attention, 3D positioning, gender, age. Also objects that, for instance, they might be holding. And it’s not only for one person. We can do this for everyone who is in the angle of the camera, as well as 10 to 15 metres away from the camera.”
“What the AI does with this information is it then creates conclusions or ideas as to who is in front of the camera. For example, if there are three people in front of the machine or passing by; one person is female, one metre and 70 centimetres high, in her mid-30s, then there’s a male person around one metre and 85 centimetres high, mid-30s, and a third person one metre and 20 high, under 18, and they all are walking within one metre of each other.
“We know that there is a privacy cycle of around one metre. And if you are in this imaginary privacy circle then there is likely to be a deep relationship. So, with this group of three, we can conclude that they might be a family. Because they are two adults and one young person in this close proximity. And then we can say, ‘Hi family, how are you, what would you like to do?’
“For example, with our mall guiding, we can highlight on the map all of the attractions or shops which are offering products or promotions for a family. The idea is that we use this information, and then try to interact like a human being.”
Making connections in a post-COVID world
As we adjust to the new, post-COVID, normal, people are keen to get out and make connections again. However, many will not be as comfortable in busy situations as before. So they could be reluctant to take part in experiences where they will be in close contact with others. Here, AR technology can help brands to provide a memorable experience while also keeping crowds to a minimum and ensuring customers are safe.
“What marketing managers or brand managers all over the world are trying to do is to emotionalise their brand,” says Lehnert.
“Previously, this wasn’t a problem. They could put on events, for instance like the Red Bull sports events or the Coca Cola truck. Or for sports brands, they had fans in the stadium or at the training court and so on. There was not this distance between the fan or the visitor, and the brand. But now, there is a fear around close contact.
“However, with technology, we can create that deep relationship between a brand and a fan or customer base through virtual reality or augmented reality. You can have your own special moment with our solution. Then you can also capture a video or a photo so that you have a memory and you can take it home.
#PoseWithThePros
“For example, we did the #PoseWithThePros AR fan engagement experience with Bayern Munich, where people can say, “I have met Thomas Müller, I’ve met Lewandowski!”
“We can also bring this to concerts or theme parks. If you have Madonna on the stage in front of thousands of people, and everyone wants to have a picture with her, that’s not possible. But what we can create is a moment where you can take a photo with Madonna or dance with Madonna. With our AR solutions, now everyone can do this.
“That’s a huge jump into the next generation of the fan experience and also the AR experience.”
The changing face of retail
While traditional retail has been facing challenges for some time, the pandemic has had a huge impact over the past 18 months. For many retailers and brands, there is a real need to adapt and evolve in order to overcome these new challenges.
At Sensape, we help brands to create moments. We use AR and AI to create something special, an experience that you can’t get by shopping on the internet
“A trend that we have seen accelerate over the last 18 months is the switch to online shopping,” says Lehnert. “Because, as well as being a safer option at the height of the pandemic, it is easier, it’s faster, you have more transparency.
“In response, retailers need to create experiences and to offer something fun. At Sensape, we help brands to create moments. We use AR and AI to create something special, an experience that you can’t get by shopping on the internet.”
Gamified AR experiences
“We recently did a huge promotion for Rivella. The idea was that you could discover the different features of the new lemonade product by Rivella with a gamification process. You could play a game and get a free sample.
“The system detects how many players are in front of the machine. Then it explains the game, and you start to play. And the benefit is that you get as many lemonades as players are in the game. That means there is a connection between this time you had fun playing here with your friends, and the lemonade. After this, we did some research and said, “Okay, what do you think, is it positive or negative?” There was a huge impact on their positive emotions.”
“Another example for retailers is a project we did with Ritter Sport. And the idea here is that you create your own AR environment. There were three different parameters: age, gender, and favourite type of chocolate. So in this case, there was a shelf with 50 different kinds of Ritter Sport chocolate. Once the customer chose their favourite, then the machine automatically created their AR world to experience, with different aspects depending on those three parameters.
“We reached more than two and a half million users with this promotion. They had a lot of fun and, within this two and a half million recorded sessions, there were also more than 10 million social media interactions.”
Creating fun
As well as adding some fun to a brand interaction, this gamified, interactive sales model is a good way to increase upselling and extend a brand’s reach.
“For everything we do, there are two possible goals: marketing reach and sales revenue. So the question for our customer is, what are your ideas and what do you want to achieve?
“In this case, Ritter Sport wanted to increase their reach. The idea was that you can get your own photo after the experience. And then of course you also share that photo on social media. So people can see that the brand’s customers have a lot of fun.
“This is a real core value, that we can create fun. It’s not boring, it’s not complicated, and there’s a lot of fun. And that’s what people are searching for.”
New AR worlds
One other example that Lehnert highlights is a project that Sensape did with Pernod Ricard:
“The Pernod Ricard marketing team informed us of their five different customer personas for their products (Absolut Vodka, Jameson Whiskey, etc). Our recognition technology then detects the persona in front of the machine and takes them into the AR world best suited to them. For example, if they fit the persona for Absolut Vodka, they are shown more information and different recipes for that product.”
“We also change the sound and light conditions for each AR world. So, you can be in the Caribbean, or a Russian techno club, or a French Bistro.”
AR experiences for museums
Sensape can also produce immersive experiences for museums, creating an interactive world that helps institutions keep up with the trend for edutainment.
For instance, on a large screen, with AR, the visitor can see themselves placed in a different environment, such as the scene of a famous painting, a different time or even an underwater landscape. They can then control the scene or interact with it with either gestures or touch. This leads them to feel they are truly immersed in that world.
By using AR technology, museums can engage better with their audience. Plus, adding interaction also leads to a higher recall rate. One possible use is that a museum could use AI to assess a visitor and then show them the fates of people with similar characteristics. This in turn creates a stronger connection between the visitor and the topic being explored.
Projects like this which use Sensape’s technology in a museum setting can help institutions to gamify the experience, make it more interactive, and ultimately result in a more memorable visit.
Virtual queuing
Another service that Sensape provides is virtual queuing. In the wake of the pandemic, this technology is more in demand than ever. Operators are looking for ways to keep their guests safe while also improving their experience.
“We started with access management systems, for instance, solutions that recognise if the visitor is wearing a face mask, and can count the number of people coming in. Because of the pandemic many shops and attractions, especially in Central Europe, have strict limitations on the number of customers and visitors.”
Digital ordering system including table reservation, ordering, contact tracing, and an ordering overview for the staff
“But then the problem was if the system says you’re not able to go in because the capacity is full, that is a negative encounter with the visitor. And our customers needed a solution that meant they wouldn’t then lose that guest,” says Lehnert.
This was the birth of Sensape Q – a virtual queuing solution:
“We said okay, we want to create a browser-based app, where if you’re not allowed to go in because the capacity is full, you can make an appointment or directly access the virtual queue, and then you go away and do something else. When the capacity goes down, then our system informs you that you’re able to come back, that you are the next one in the queue.”
Sensape Q
Sensape Q can also connect different smaller queues to one big queue – ideal for shopping malls or theme parks.
“What if people want, for example, to go to roller coaster A and then to a roller coaster B then maybe to a show and then to roller coaster K?” asks Lehnert. “Sensape Q knows the different lengths of these virtual queues. It can route visitors perfectly to those different stations so that their waiting time is optimised.
“The system also has space to give more information. Because we then know that you are interested in shop A or shop B. So we can give you extra information about those shops or maybe special offers. Or maybe if we know that the wait time is too long, we can give you a voucher for a coffee, for instance. This allows us to level up the customer experience and customer service.”
“For example, in Ecuador, we worked with a huge pharmacy chain that was having an issue with large queues forming outside. They use our solution to optimise the traffic and also to give more information. Shoppers can even use the system to ask questions and get product advice while they are in the virtual queue.
“Also, here in Germany, there is a huge chain of bicycle stores. With our system, the customer can register for the repair service, or advice on e-bikes, for example. They get put in different queues and go straight through to the right service.”
The future of AR trends and experiences
The COVID-19 pandemic has undoubtedly accelerated the adoption of more digital tech trends. Speaking about whether we will increasingly see these types of AR solutions in retail and entertainment spaces going forward, Lehnert says:
“At the moment there is a limitation because normally for AR and also VR experiences you need devices. You need a special screen or your mobile phone. But there is a huge development in smart glasses.”
“I think, in one or two years, there will be a smarter way to wear devices directly on your face. And then there will be a lot of development – before we have been limited with the hardware devices. Yes, there are lots of possibilities now. But nobody wants to go through the world having to hold up your smartphone all the time. Because then the real world is escaping. So, I think that will be a huge jump.
“And the next thing will be that we can use our technique to bring printed media to the next level. So we can put out AR and media content that connects analogue products with digital content in a smart way. You get a direct connection between the analogue world and the digital world.”
Hands-free experiences
On what is next for the company, Lehnert says that the pandemic has meant it is also renewing its focus on touchless and hands-free solutions:
“Our system can also use gesture control because we are detecting all the time the skeleton of the user. So that means we can offer gesture control without a 3D camera, we can use a normal camera. A visitor can raise their hand and bring it to a call to action. Then the system knows that they want to click there.
“That is a very simple solution. But it means we can create gesture control for all the UI we are developing. It means that you don’t need a touchscreen to interact with the machine, and with the brand.”
Marketing for millennials
Using a new AR-based, technological approach to audience engagement is a good way of reaching today’s digitally-savvy consumer, says Lehnert:
“One of the strengths of our solution is that people, especially young people, are used to creating selfies. I read an article recently that said over 1 trillion photos are taken every year. So, people are used to this.”
“With the kind of digital content we enable, brands have direct user-generated content, because the user is in the picture. It’s an individual picture and then you can also put some brand awareness or some brand information on the picture. For instance, as we did with our Ritter Sport experience.
“Mostly, our customers begin by saying it’s hard to get in contact with their audience or with their customers. But with the help of AR, it’s very easy. Because if the user is in the middle of the whole ongoing scene, can interact with the digital media and maybe can also get control of what it is doing, then it’s very probable that they want to share this experience with friends. And then you’re in the most valuable channel you can be.”
All images kind courtesy of Sensape
