How SPYSCAPE Added Magic Into Their AR Game Using Nyckel

Using Nyckel ML FaaS, SPYSCAPE trained and deployed an image classification function, used as a key feature of the Augmented Reality in-game experience.
mugshot Nyckel
Aug 2022

SPYSCAPE is a destination spy museum and experience in New York City, with interactive exhibits about fictional and real spies. The museum’s permanent exhibits include interactive features and immersive games through which visitors can figure out what kind of spy they would be. Are you a handler or a technician? Special ops or cryptographer?

"Normally you have a problem and then you go for a solution... What I’ve now got is a really nice solution, and we’re actually looking for other problems we can apply it to!"
Gwyn Morfey
SPYSCAPE Tech Lead
SPYSCAPE uses Nyckel image classification to add “magic” to their AR games
10 minutes to train first classifier
Function was “spooky accurate” right away
Nyckel-powered magic camera a game highlight

The HQ also features a rolling exhibit. 007 x SPYSCAPE has recently moved out, and a physical component of a new Batman x SPYSCAPE mobile augmented reality game has moved in. Batman x Skyscape can be played anywhere in the world, but, if you happen to be in NYC, the game will also take you through the exhibition. It gives you the immersive experience of actually working with Batman and the characters from that universe.

"But we really wanted to have that magic: making it seem like, how could they have done that? It must be a person!"
Gwyn Morfey
SPYSCAPE Tech Lead

What you do in the game moves the story forward. Although you’re in fact communicating with software, SPYSCAPE wants it to feel like you’re communicating with real people. But the question was how to do this at scale? The game already used a large branching script about the length of a typical novel. This allows the characters to react to the choices that players make in the game, which begins to establish the feeling of reality.

Spyscape UX

The physical exhibits in New York use RF wristbands and NFC to maintain a personalized sense of reality throughout the game, producing an immersive magic that resonates with customers. But another powerful element of the HQ’s immersiveness comes from its use of picture recognition.

"What we were looking to do was basically Things That Are Magic, and people love the magic camera!"
Gwyn Morfey
SPYSCAPE Tech Lead

At one point in the exhibit, visitors have to go into a laboratory and take pictures with the Batcam. SPYSCAPE needed an AI to classify the photographs that visitors were taking within the game in order for the game to react to them in a realistic way.

SPYSCAPE already had a lot of infrastructure in AWS, so that was their first thought. But there were lots of competing priorities within the company at the time, and it didn’t make sense to take on a whole raft of machine learning researchers just to solve this problem. Outsourcing made sense, and that’s when Tech Lead Gwyn saw an advertisement for Nyckel on Hacker News.

"Literally over the space of ten minutes, I created an account, threw some random images at it… and it blew my mind! It was spooky-accurate!"
Gwyn Morfey
SPYSCAPE Tech Lead

There was very little to do in terms of API integration: a dozen or so example images per class of training data, uploaded to the Nyckel web interface; and a server written in Node, deployed in AWS, to do a little bit of munging of the output.

Gwyn and his team are so pleased with the magic of the ML-powered Batcam that they are now looking for other use cases within Skyscape.

"Thanks for building this thing – it’s super-cool!"
Gwyn Morfey
SPYSCAPE Tech Lead

Want to build your own classifier in just minutes?

You get 1,000 classifications a month for free. Pay only when you scale. No credit card required.

Start for free Get a demo