AR Projects

Kamui Teleportation Effect (2022)

This effect is inspired by the anime Naruto. 

Kamui is an ultimate ability in the story, and the ability is popular among audiences. This effect aims to allow fans of Naruto to enjoy casting the ability in the filter. The Gif on the right is an example of Kamui being cast in the anime. The ability can teleport other objects, people, or the casting person. When casting, it has a spiral absorption visual effect to show materials being absorbed into his eye. 

As a big fan of anime, I think it’s really cool to be able to make my own video of casting it like in the anime with an AR filter, so I made this filter with SparkAR. I believe many of Naruto’s fans will feel excited about this filter effect. 

How-To:

Tap the screen to trigger Kamui’s absorption effect. The absorption is based on the visual effects in the anime. Also, the ability can teleport you to different locations in the anime.

After the animation ends, about 5 seconds, you can tap again

The Kamui AR effect is achieved by a GLSL shader with two parts that I created. 

The first part’s shader rotates and moves coordinate and saves to a temporary texture as red and green, and transfer to the second part. In each frame, the calculation is based on the temporary texture from the previous frame. This allows me to put eye position as the pivot to rotate for each frame and make the entire effect smoother. 

Video on the left: the overlay pattern indicates the twisted coordinates for the second part of my shader to sample on my camera image at the calculated coordinates. Part 2 also adds the fade effect response to the animation progress from SparkAR Patch.

The entire interaction is achieved by patch editors. 

It’s my first time using patches and SparkAR, so I didn’t know what’s the limit of the Patches. After finishing the project, I feel that for some complex interactions or mechanisms, it would be much more straightforward to just hook up JavaScript code in there. Still, the Patches is pretty convenient for basic interactions.

Can You Bend the Curve? (2021)

I made an interactive Snapchat AR project with the Lens Studio. It’s my first time learning to do things with face tracking and Lens Studio. My thought with this project is based on the current Covid situation. I thought about letting people choose to wear a mask when they go out to bend the covid positive case curve, just like Gov. Cuomo has said all the time since April.

I tried to do it with Unity because I was more familiar with it but the file size was pretty big and I didn’t figure out how to preview the AR game in Unity editor. I need to export to my phone every time I want to test something, so I started to try Lens Studio. 

Here is the QR code for the project:

Experiment with Image tracking:

With Vuforia in Unity. I attached subways to the metro card and has interaction when two subway sections are connected

Experiment on object tracking:

I tried to achieve a force field shield that’s protecting the astronaut from fire. I attached a surface shader to the 3D-printed astronaut, and a force field attach to the 3D-printed astronaut. Then the fire is attacked to the subway card.