Ever since I was a kid, the LightSaber has been something that I've always wanted to build. With all the Virtual Reality and Augmented Reality trend this days, I wanted to learn how to develop software for it. I bought Google Cardboard when it first released last year and started exploring different apps. I'm amazed on how simple and easy it is to setup and run. I came across Unity3d game engine to create games for mobile including Google Cardboard. I started learning Unity3d which has vast amount of add-ons for Kinect, Oculus Rift, HTC Vive, Samsung Gear VR, Google Cardboard. I'm currently using Mattel ViewMaster VR which is based from Google Cardboard.
Virtual Reality gives immersive experience, but not having a way to interact with it, it gets boring after awhile. Came the idea to explore creating controllers in Virtual Reality. Just simple ones, like on and off. I started using Arduino via Serial port, then Bluetooth control, also tried Littlebits Cloudbit.
How about building a LightSaber controller?
Well, I found out that there's Sparkfun sells an IMU module to Photon. I can use that to get the rotation for my LightSaber.
https://www.sparkfun.com/products/13629
The tutorial is easy to follow. Thanks Sparkfun.
https://learn.sparkfun.com/tutorials/photon-imu-shield-hookup-guide
After hooking it up, I wanted to have better data coming out of the IMU. So I researched into Kalman Filters. It can be to reduce noise data from the IMU.
https://en.wikipedia.org/wiki/Kalman_filter
Research some more!
I ended up finding this module in Arduino
https://github.com/TKJElectronics/KalmanFilter
I can use that but Particle.io has particular way of adding arduino libraries. Guess what? More Research.
So I ended up, creating a fork on the KalmanFilter project to incorporate to Particle. Here are some instructions how to create a library for Particle.io
https://github.com/spark/uber-library-example/#getting-started
Here's the link to the Kalman Filter Photon Library
https://github.com/rlyle78/KalmanFilter
I modified the boilerplate code that Sparkfun provides with IMU and added the calculation needed for the Kalman Filter.
But how about the handle? How are you going to make it look like a LightSaber? Well, I know I still need to provide battery for the Photon. Then I saw this Lipstick-Size External Battery Power Bank. That's my holder!
if you want to see more projects like these, please add respect and follow me.
Now I got the "Controller". I was able to connect it to Serial Port to my macbook. I'm getting data. This is getting interesting. Now what?
VR/AR requires it to be mobile. That means, deploy to mobile phone if I'm going to use it with Google Cardboard. Since I'm getting Serial Data on my laptop maybe I can just broadcast that or something. But I don't want it tethered. There's a reason why Photon is popular, Wireless IOT. Built in WIFI.
Time to research again. Thinking about it, it's a WIFI and it's Arduino based, there's gotta be some libraries that it's using to connect to the internet.
But what protocol? Should I use HTTP? Web Sockets? TCP? UDP? I need to get my data fast. So I decided TCP or UDP. Time to dig some more.
I found it in the reference and forums
https://docs.particle.io/reference/firmware/photon/#tcpclient
https://docs.particle.io/reference/firmware/photon/#udp
https://community.particle.io/t/receiving-udp-packets-on-the-photon/12825
TCP and UDP. Cool. I can actually send data.
How do I test it out. What's the easiest way to test data sent from Photon? Well, I can do it in C#, create a network connection. I started also studying NodeJS this year. How about NodeJS? I found the documentation and this article.
https://nodejs.org/api/dgram.html
http://www.hacksparrow.com/node-js-udp-server-and-client-example.html
var dgram = require("dgram");var server = dgram.createSocket("udp4");server.on("error", function (err) { console.log("server error:\n" + err.stack); server.close();});server.on("message", function (msg, rinfo) { console.log("server got: " + msg + " from " + rinfo.address + ":" + rinfo.port);});server.on("listening", function () { var address = server.address(); console.log("server listening " + address.address + ":" + address.port);});server.bind(41234);// server listening 0.0.0.0:41234
I started getting data from Photon via UDP to NodeJS.
But how to connect to my phone? I have Android, and I can't use NodeJS on it.
How about this? I said to myself. I can make my phone as Access Point ... then connect Photon to my phone. Then I can now connect a Unity3d app to my Photon. Tada! It completes the loop.
But how do you connect a Photon to my Access Point Phone? Typically, you would use a phone to connect Photon to a wifi connection. But I'm already using my phone... solution? use another phone or a tablet, connect them both to the Access Point Phone. It actually worked.
Here's the code that I used. Make sure you know the ip address of your phone.
https://gist.github.com/rondagdag/7d139d95e5b6ea2ef8e0
I started learning Unity3d this year. This is a good platform for creating games and visualization. Especially in VR and AR.
My background is C# and .NET that's why I chose Unity3d instead of Unreal. I'm hoping I can reuse some .NET libraries for communication. It's now free to use networking stuff in Unity3d with the new version.
I'm new to Unity3d. There are good courses that I took in Pluralsight that helped me grasp the idea and the platform.
What's good about Unity3d is that it has an asset store. Looking at the asset store, there are quite a few libraries that I can use but I need to buy it.
I found this article:
http://forum.unity3d.com/threads/simple-udp-implementation-send-read-via-mono-c.15900/
so I tried the UDP receive since I only want to receive data from the photon.
And It actually worked!
I connected the Photon to Unity3d via UDP sockets.
The important part of this project is the VR part. Game programming is new to me. So is Virtual Reality and Google Cardboard.
https://developers.google.com/cardboard/unity
I downloaded the Unity Package for Google Cardboard and open the sample app.
https://github.com/googlesamples/cardboard-unity
I started using the Google Cardboard Sample App. I incorporated the UDP code in C# that I worked on. Added logs just to get the data connection going.
Here's the code that I used to receive UDP in Unity3d.
https://gist.github.com/rondagdag/84aa25133d1aaf0102cb
I deployed it in my android phone, ran the app.... It worked.
It's still work in progress... Will update soon. Let me know if you want to learn more about this project. Follow me and add respect to this project.
Comments