Hackster is hosting Hackster Holidays, Ep. 6: Livestream & Giveaway Drawing. Watch previous episodes or stream live on Monday!Stream Hackster Holidays, Ep. 6 on Monday!
shadowninjazxGrace Liu
Created May 15, 2019 © CC BY-NC-ND

trAnSLate: American Sign Language Translator

Natural language processing for gestures (sign language) with computer vision.

AdvancedShowcase (no instructions)20 hours185
Most Creative - US Based Project
2019 China-US Young Maker Competition
trAnSLate: American Sign Language Translator

Things used in this project

Hardware components

Android device
Android device
×1

Software apps and online services

Google Cloud Functions
Google AutoML
Expo: React Native

Story

Read more

Code

Custom Machine Learning Prediction API on Cloud Funciton

JavaScript
Node.js code for Google Cloud Function
This function interfaces with the machine learning model used for prediction.
// Using AutoML
const automl = require('@google-cloud/automl');
const predictionClient = new automl.PredictionServiceClient();


// Constants
const project = 'kaggle-160323';
const region = 'us-central1';
const automl_model = 'ICN6055334473235779041';
const automl_model_bad = 'ICN3108178127894216047';
const automl_model_2 = 'ICN3822943271648285140';

var output = "";


// Function to interface with AutoML model
function callAutoMLAPI(requestBody){
  return predictionClient.predict(requestBody)
    .then(responses => {
    console.log('Got a prediction from AutoML API!', JSON.stringify(responses));
    output = JSON.stringify(responses);
    return output;
  });
}


// Main driver function to respond to http request
exports.helloWorld = (req, res)=>{
  
  // Construct httpd request to send to AutoML API
  var requestBody = {
    name: predictionClient.modelPath(project, region, automl_model_2),
    payload: {
      "image": {
		"imageBytes": req.body.data
      }
    }
  }
  
  // Return response to app
  return callAutoMLAPI(requestBody).then((output)=>{
  	res.status(200).send(output);
  }).catch((err)=> {
      res.status(400).send(err);
  });
  

};

EXPO React Native Application

This repository has code for the camera app and connects with the deployed machine learning model. Download expo and connect your phone to run the app (Android). https://expo.io/

Credits

shadowninjazx

shadowninjazx

2 projects • 2 followers
Grace Liu

Grace Liu

0 projects • 1 follower
Thanks to Alan Tran and William Chen.

Comments

Awards
Most Creative - US Based Project
2019 China-US Young Maker Competition
Related channels and tags