Creating a Digital Menu Using FPGAs

As restaurants have been opening back up here in NYC, I've noticed they all have been switching to contactless, digital menus. Since I've recently been spending time working on wireless applications involving custom web servers running on the ARM processor of the Zynq SoC FPGA using the MiniZed and the Ultra96v2, I realized how easily FPGAs can be utilized as a solution to help restaurants adapt during COVID. I turned to my trusty MiniZed again to see how straightforward it might be to create a standalone solution for a digital restaurant menu accessible via a QR code.

The backbone of this project is the web server to running on the ARM core of the Zynq chip of the MiniZed. I've gone over how to create a custom web server for the MiniZed in the past, but I wanted to outline it again here since there is a little bit different focus for customizing it to match the needs of a restaurant. My hometown's local coffee shop posted on their social media that they were looking for a digital menu solution, so I decided to use their menu and logos in the home page of the web server.

I'm starting with the ext4 root filesystem image I created for the MiniZed in my Ouija board project. Since the web server is written in Python, the necessary Python packages (Flask and Requests) coupled with the necessary image files for the static folder of the web server require more space than the default initramfs image for the MiniZed can support.

Back End Python

For the back end of the web server written in Python, it runs a Flask application that uses HTML requests to service GET and POST requests from the front end HTML. Each webpage of the server is defined as a function that outlines how to return the rendered template with the desired data.

First, install the Python packages Flask and Requests using pip:

root@minized-emmc-enhanced-2019-2:~# pip3 install flask --trusted-host pypi.org --trusted-host files.pythonhosted.org

root@minized-emmc-enhanced-2019-2:~# pip3 install requests --trusted-host pypi.org --trusted-host files.pythonhosted.org

Then create the new file for the web server. I chose to enable the debug option (app.debug = True) which prints out the HTTP requests and resulting status codes to the serial console.

Notice the home page is defined as the home() function, with both GET and POST requests simply returning the rendered HTML template for the home page.

# MiniZed webserver Python script (webserver.py)
import os
import re
import sys
import glob
import time
import uuid
import crypt
import signal
import random
import os.path
import subprocess
import multiprocessing  
from flask import Flask, render_template, request
from multiprocessing import Pipe

CUR_DIRECTORY = os.path.split(os.path.abspath(__file__))[0]
ALERT = -1

app = Flask(__name__)
app.debug = True

@app.route('/')
@app.route('/home.html', methods=['GET','POST'])
def home():
    if request.method == "POST":
        return render_template("Home/home.html")
    else:
        return render_template("Home/home.html")

if __name__=='__main__':
    app.run(host='0.0.0.0', port=80, threaded=True)

Web Server File Structure

As I've mentioned before, there is a minimum file structure requirement to get a barebones web server running on a device like an FPGA or microcontroller.

When the render_template() method is called it looks for the HTML templates in the templates directory. The directory where templates lives in is also where the webserver.py scripts lives. The name of the HTML template file is the only required argument to be passed to the render_template() method, and optional arguments include any variables to be passed to the template engine as keyword arguments.

The other required directory is the static directory. The static directory is where the Flask application pulls the CSS for formatting, JavaScript, and image files for the web page templates. The images directory is where the files for the coffee shop's menu and logo will be placed.

Front End Templates

There is a default template that serves as the common base for all webpages. The default template handles things such as the base CSS theme, header and footer that will appear on each webpage, and the navigation toolbar listing each available page.

This template is where the coffee shop's logo is called to appear in the header of every webpage. For now, there is only the home page for the menu, so home.html is the only option in the navbar section.

Default HTML template:

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="utf-8">
        <meta http-equiv="X-UA-Compatible" content="IE=edge">
        <meta name="viewport" content="width=device-width, initial-scale=1">

        <title>{% block title %}{% endblock %}</title>

        <link href="{{ url_for('static', filename='css/bootstrap.min.css')}}" rel="stylesheet">
        <link href="{{ url_for('static', filename='css/bootstrap-theme.min.css')}}" rel="stylesheet">
        <link href="{{ url_for('static', filename='css/theme.css')}}" rel="stylesheet">
        <script src="{{ url_for('static', filename='js/jquery-3.2.1.js')}}"></script>
        <script src="{{ url_for('static', filename='js/bootstrap.js')}}"></script>
        <link rel="shortcut icon" href="{{ url_for('static', filename='images/coffee_shop_logo1.png')}}">
    </head>

    <body>
    <nav class="navbar navbar-inverse navbar-fixed-top">
        <div class="container">
            <div class="navbar-header">
                <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
                    <span class="sr-only">Toggle navigation</span>
                    <span class="icon-bar"></span>
                    <span class="icon-bar"></span>
                    <span class="icon-bar"></span>
                </button>
                <a href="/" class="pull-left"><img src="static/images/coffee_shop_logo1.png" class="img-responsive" width="250" height="50"/></a>
            </div>
            <div id="navbar" class="navbar-collapse collapse">
                <ul class="nav navbar-nav">
                    <li><a href="/home.html">Home</a></li>
                </ul>
            </div>
        </div>
    </nav>

    <div class="container">
        <div class="row row-offcanvas row-offcanvas-right">

            {% block content %}
            {% endblock %}

        <hr>
        <footer>
            <img src="static/images/knitronics_logo.png" class="img-responsive" width="75" height="15"/>
        </footer>

        </div>
    </div>
    <div id="myModal" class="img-modal">
        <span class="close">&times;</span>
        <img class="img-modal-content" id="modalContainer">
    </div>
    <script>
        var modal = document.getElementById('myModal');
        var modalImg = document.getElementById("modalContainer");
        var span = document.getElementsByClassName("close")[0];
        function showImageModal(params){
            modal.style.display = "block";
            modalImg.src = params.src;
        }

        span.onclick = function(){
            modal.style.display = "none";
        }

        function submitFormAsync(request, data){
            $.ajax({
                type: request.method,
                url: request.action,
                data: data,
                success: function(response) {}
            });
            return false;
        }
    </script>
    </body>
</html>

The home page template is where the menu image file for the coffee shop is called to be displayed. A simple CSS style can be called to have the image automatically size itself to the current window so it looks clean and neat on whatever device is accessing it.

Home page HTML template with the menu:

{% extends "Default/default.html" %}
{% block content %}

<div class="page-header">
    <h1 class="display-4"><b>{% block title %}Filling Station Menu{% endblock %}</b></h1>
</div>

<style>
img.one{
    height:100%;
    width:100%;
}
</style>

<img class="one" src="{{ url_for('static', filename='images/coffee_shop_menu-1.png') }}" width="1000" height="1000"/>

{% endblock %}

Once the HTML templates are completed, launch the web server on the MiniZed:

root@minized-emmc-enhanced-2019-2:~# python3 ./minized-webserver/webserver.py

Then use the MiniZed local IP in a browser to view the webpage. I pulled it up on my phone since the majority of the customer traffic would be from their smart phones.

Port Forwarding

In order to access the menu when from outside of the coffee shop's internal network, port forwarding can be used to route incoming requests from the Internet to the proper local IP address on the coffee shop's internal network.

Port forwarding designates a particular port (port 80 in this case) on the gateway of the local network to allow for external communication with the flask webserver that has also been attached to the same port. Port 80 is the port that a server is always listening to for web clients in the HTTP protocol structure.

Port forwarding will take the local IP address of the MiniZed FPGA on the coffee shops' local network and map it to the router's external IP address on port 80 using NAT (network address translation). This means the link to access the webpage menu will be "<the router's external IP address>:80"

Generate QR Code to for Link

Instead of making customers type out the link created using port forwarding, a QR code can be generated with the link attached to it. There are many sites that can be used to generate a simple QR code for free like this one where you simply enter the link and then can download the image file of the new custom QR code.

Summary

Overall, I really like that this solution is something I could potentially package up and send to a restaurant that may need some help adapting to the new way business during the pandemic, especially if they don't have the resources to update their website at the moment. A few tweaks I would like to add are things like a script that runs at initial boot up of the MiniZed that will connect it to the restaurant's wi-fi and launch the web server. This way, the MiniZed board doesn't need any further attention past plugging it into a power source and pressuring the power button.