ZeroHero: Reducing Food Waste with Object Recognition

Demo Video

Inspiration

We were looking for projects that warms our hearts. As university students, we have first hand experiences on what it feels like to throw away leftover ingredients that we didn't use. Therefore, we've decided, for the sake of reducing food waste, to create an app that provides creative dishes for people to put their leftover ingredients to good use.

What it does

ZeroHero is a fully functional cross-platform mobile app that will take a picture of your leftover ingredients and suggest healthy, delicious, and invigorating recipes from top chefs all over the world.

How we built it

ZeroHero is built using React-Native. The expo camera is used to take the picture, which is passed through a cloud function to inference on a Google AutoML Vision Model.

This model was trained on the produces we bought from Kroger during the hackathon and includes: Peppers, Tomatoes, Apples, Avocadoes, Onions, Lettuce, Bananas, and Carrots. We collected and annotated over 2,000 images by hand.

The model identifies the multiple items in the image and returns it to the app, upon which we call the Spoonacular API, which identifies the recipes that use the most of these ingredients. We employ multiple calls to the API to gather the recipes' image, nutritional information, and the link to the actual recipe.

Challenges we ran into

1) We had to create our own data set of hand-labeled images of fruits and vegetables.

2) We had to recursively query Spoonacular API for recipes and their nutritional information.

3) We had to familiarize ourselves with React Native to convert images to a base64 string to query Google AutoML.

4) Connecting our app to Spoonacular API and our deployed object classification model on Google Cloud.

Accomplishments that we're proud of

1) Our hand-labeled data set of over 2,000+ images achieved incredible results with AutoML training!

2) Successfully linking Sponnacular and Google Cloud APIs to our React Native application.

3) Familiarizing ourselves with JavaScript.

4) The potential of this app.

What we learned

1) JavaScript

2) Google Cloud Platform

3) Teamwork

4) How much two avocados cost at Kroger

What's next for ZeroHero: Reducing Food Waste with Object Recognition

In the future, ZeroHero will be able to identify a wider variety of ingredients such as meats, nuts, and even canned foods to suggest even wilder recipes that will blow your mind. Furthermore, we will also be able to take into consideration the quantity of individual ingredients to improve the recipes that we suggest. We will also be able to tell you the estimated lifespan of your ingredients before it goes bad.