MeYouandUs: Fantasia Express
Challenge
Clients MeYouandUs, a creative technology arts company, commissioned us to develop Fantasia Express as a prototype demonstrator which they could go to potential partners with. This Innovation Fund backed R&D project, using AR and mixed reality, had two specific, objectives which on the face of things looked straightforward, they were: Create an augmented reality experience inside a moving train carriage Create an augmented reality experience outside a moving train to be viewed from inside the carriage The problem Standard Spatial Localisation and Mapping (SLAM) using a device’s gyroscope and camera to describe its location, was no good in this case simply because of the fact that the app was to be used on a moving train. Try taking a corner at speed using AR and your image will just slide off the screen. Let there be no mistake, this was groundbreaking stuff, literally nobody had done this before. With a straightforward augmented reality job we might start with an out-of-the-box AR framework and build from there but no such luck in this case, we started from scratch and built our own.
Solution
Our SLAM system had to work differently for AR inside the carriage than outside, and took inspiration from the Mars Exploration Rovers and aerospace technology. Inside the carriage we used visual odometry, whereby the device’s camera feed is analysed frame-by-frame, identifying specific feature points, then working backwards to understand where things are compared to where they were, and how they move. For AR outside the carriage the device’s gyroscope was employed differently to make assumptions about the movement of the train itself. Sensors then brought data together to estimate the motion and turn of the device and a fusion technique, similar to those used in aeroplanes, brought all this data together. Finally Kalman Filters were applied to cancel out data drift & noise and ensure steady, accurate AR images. This avante-garde, state of the art tech, whilst outstanding on its own, was, of course, nothing without the stories to go with it. Anyone that knows anything about the creative process will tell you that the journey from concept to product is multi-layered, well researched, informed and sometimes lengthy, but also that it’s well worth the effort for, as with most things in life, you get out of it what you put into it. The initial ideas that MeYouanUs came to us with were storyboarded, mood boards were drawn up, and creative workshops were run to work out how best to visualise the stories in an AR world. We worked with a BAFTA award winning writer to translate draft scripts into our AR format before taking them to the studio to be professionally recorded by a team of voice actors which we then matched up with our animation. Our client had strong, clear ideas about the characters to be used, and their personalities, but when it came to their visual design the brief was looser. From a baby dragon to stroppy Raven our animators had pretty much creative control with this aspect and ran with it creating 3D models and artwork which practically leapt out of the screen. The final part of the journey (last pun, honest) was field testing. This involved building contraptions in the studio which would upset standard AR and latterly sending the team out on random train journeys to test it on an actual, moving vehicle.
Results
The results, as you can see, are pretty special. We’re proud to have developed such an innovative and cutting edge technology and it’s something we hope to be able to take into other arenas. As a prototype demonstrator Fantasia Express gave our clients, MeYouandUs, an exceptional product to approach train companies, in the UK and worldwide, with. The app offers their passengers a unique experience, through this advanced use of AR, as well as creative assets to showcase it, and in turn opens up the possibilities of commercialisation to the companies themselves. With full support from East Coast franchisee LNER, we delivered an app where the user is entertained, they learn something new, and can even take pictures of what they see. What started as a regular, dull train journey has suddenly become an adventure. Of course the potential opened up by the project is huge with Fantasia Express becoming a showcase for what is possible on the platform – if you’ll excuse the pun. Possibilities obviously include new experiences, such as the ones we’ve seen here, but they also extend to user targeted options and commercial opportunities.
Want fresh marketing stories in your inbox?
Get the weekly email that makes you a marketing genius.
Stay informed and inspired, for free.