HoloLens + Cognitive Services = Object Recognition

I attended a HoloHack in Atlanta last Dec 2-3, 2016. It's an AR/MR smart cities themed Hackathon sponsored by thyssenkrupp and Microsoft. It's an awesome event, got to meet thyssenkrupp hololens developers.

Here's the project I presented: https://devpost.com/software/hololens-and-cognitive-services

Inspiration

I'm a fan of Terminator movies. The hack is about doing Terminator Vision in Hololens.

What it does

Do Object Detection and analysis using Microsoft Cognitive Services Vision API. The user would say 'Scan Image', hololens will snapshot the scene, A Box Assistant will respond to the request and speak out to describe, label and identify content.

How I built it

Using the Hololens camera, capture an image and send it to Microsoft Cognitive Services Vision API. Learn more here: https://www.microsoft.com/cognitive-services/en-us/computer-vision-api

Challenges I ran into

Getting the Box To talk properly. Sometimes it takes longer to send data to Cognitive Services and the user will have to wait for the results.

Accomplishments that I'm proud of

Connecting to MS Cognitive Services Vision API. Uploading photos and reading response back to the user.

What I learned

Capturing images from Hololens camera.

What's next for Hololens Cognitive Services

Trying other MS Cognitive Services API. Do OCR and Recognize Celebrities.