Apply HN: Radcam - MSQRD for Cartoon Animations

4 points by cpcat 10y ago ↗ HN

IDEA:

Radcam is a camera app that will detect as many details about your appearance along with the scene around you and instantly turns them into an exhilarating cartoon animation that you can share as a looping gif. Radcam is inspired by this video on Youtube [https://www.youtube.com/watch?v=vrPgHJ-LAsU]. I am not affiliated with the author of that video, yet ;). The cartoon animations will be hand-crafted and pre-loaded by creative artists. The choice of animation will be triggered by your facial expression.

TECH:

* iOS, computer vision, OpenGL

* Realtime mobile facial feature detection

* On-Device Image Classifier for contextual object detection

* Image filters (custom cartoon effect)

* Image Morphing

* Video Encoding

I will share videos of progress in this thread. Currently i just completed realtime facial features on iOS so i will make it presentable and post a link to a video/demo of that as a first step.

Challenges:

Memory Footprint: The most challenging part will be keeping the app's memory footprint reasonable with all the object files in there for detection.

Abstracting the scene: There is also the challenge of mapping scenes to pre-designed animations. I'll start by keeping it simple (perhaps by just using the colors in the scene) but eventually will have to implement some sort of abstraction. i think i'll figure it out along the way.

Morhping into character: What i'm most interested in is morphing; it has to be specific implementation for the app since i want to keep as much detail from the original face after morphing into the cartoon caharacter. The cartoon filter should simplify things.

My goal will be to ship an MVP and make sure this doesn't turn into a research/scientific-breakthrough kind of project. It is very easy to go that way with computer vision.

3 comments

[ 4.1 ms ] story [ 17.7 ms ] thread

buss 10y ago ↗

This looks like a serious technical challenge for a not-very-useful result. How do you know people want this? Can you use this technology for something with a higher value than cartoon scenes?

cpcat 10y ago ↗

Technically it's not as serious as it sounds thanks to the wealth of open-source computer vision libraries and their liberal licenses. For the MVP at least.

It is a challenge nevertheless, which is why i'm doing it for fun. it's unique, has a wow factor, and would personally use it for expressing myself. I have no clue if people will want it so i'll try my best to make using the app fun for them as well.

Hopefully you'll see how fascinating it is when i post a video of the real time facial feature detection i have already completed.

UPDATE:

As promised here is a video of the status update. I was waiting for a good hair day :) https://goo.gl/s5I8Cx

This is running at 30 FPS with the highest iSight resolution on a 6s. HN orange for the tracked features ofcourse.

I think i'm shifting towards a MSQRD clone. The app is in no way affiliated with MSQRD i used the name just for motivational purposes.

The next step would be to align a 3D object using Metal.