Hi, there.
On last weekend we entered a Hackathon, and come up with this new technology.
Using gyroscope data, we can pretty much use the mobile phones as a laser pointer and draw. We created a React-Native(Expo) app in which you try to imitate spell drawings by moving/rotating your phone.
The project is open-source, we haven't put it on app/play stores but you can build and install on your phones to try it out following the readme.
You can check out more details and how it works in the project readme.
I love this. I originally got into programming because I figured it was the closest thing to the magic I loved in Harry Potter: a mixture of academia, history, and pragmatism. Seeing a project like this just embodies a lot of the sense of wonder I had about getting computers to do cool new things for me.
Thanks for making something purely for fun and sharing it with us!
Nintendo sends cease and desists to a lot of free (as in beer or speech) projects. Some companies bully hard, I think Warner Bros is likely heavy-handed on Harry Potter.
Rowling is OK with fan-based HP writing if it isn't commercial, isn't obscene, and is not credited to whoever produced it and not to her [1]. There's probably a decent chance she would be OK with this if it is free.
Years ago I had an idea. A game of magic battles. The witches and wizards would search the vicinity (very much like Pokemon go) to find other witches and wizards, challenge them to a duel and fight it out with their wands (or smartphones). The spells would be triggered by using movements i.e. gyroscopes and trickier spells would require lots of wand waving and verbal cues.
I suppose when AR/VR glasses becomes the norm, we could also integrate the magical beams and cooler effects! Anyways, it brings me so much joy to see a portion of it materialize. I'll just patiently wait until someone comes up with the entire game
Image similarity is a terrible way to do this; you lose all temporal data, which would make different speeds of motion impossible to discern. You're much better off feeding your XYZ + timeseries data into a hidden markov model, then classifying gestures with SVM prediction. There's a really great C++ gesture recognition library that I've used to do just that for a VR game that featured custom spellcasting gestures [0]
Hi, I totally agree, but I had like 1-2 hours left to implement the similarity check part, so I took a simple image similarity approach. (Also, I only have images for target spells for now, need to create target data with temporal data as well)
Your idea for gesture recognition is interesting, I will look into that in more detail.
PS: I already convert 3D to 2D with filtering so I can apply comparison of 2 2D+time data:
What, I originally had in my mind was to modify and use a similarity metric such as EMD (Earth Mover’s Distance) which I believe would be a great fit for our case.
22 comments
[ 3.0 ms ] story [ 59.7 ms ] threadUsing gyroscope data, we can pretty much use the mobile phones as a laser pointer and draw. We created a React-Native(Expo) app in which you try to imitate spell drawings by moving/rotating your phone.
The project is open-source, we haven't put it on app/play stores but you can build and install on your phones to try it out following the readme.
You can check out more details and how it works in the project readme.
Also, there's a voting period of the Hackathon until wednesday. If you like the project, please vote us on https://ucbuyucuturnuvasi.com/grimmauld-meydani
You can also, check out other awesome projects there.
Thanks for making something purely for fun and sharing it with us!
In Harry Potter: You say magic words, that make things happen.
In programming: You type words in the computer, that make things happen.
https://en.wikipedia.org/wiki/Magic_number_(programming)
I wonder if it could also detect and judge pronounciation of a spell - always remember: "It's leviOsa, not levioSA!"
https://mobile.twitter.com/gothspiderbitch/status/1467693856...
[1] https://fanlore.org/wiki/Rowling_backs_Potter_fan_fiction
What I initially thought was whether this can be considered fan art, and since it’s free (and open-source) I think it might be okay.
But, I’ll check in detail and maybe replace all titles before publishing it to the stores.
[0] https://github.com/nickgillian/grt
Your idea for gesture recognition is interesting, I will look into that in more detail.
PS: I already convert 3D to 2D with filtering so I can apply comparison of 2 2D+time data:
What, I originally had in my mind was to modify and use a similarity metric such as EMD (Earth Mover’s Distance) which I believe would be a great fit for our case.