Floating displays (aerial displays) exists for many years and are used, for example, to operate elevators without touching them. They are also used in reception areas, as digital signage but also as cash machines and many more applications. The narrow field of view is the ideal solution for safe data entry, such as entering a PIN number. And since the pandemic, such operating concepts have gained a lot of interest because they ensure maximum hygiene. The underlying principle is not too difficult, so I decided to build my own DIY version.
HOW IT WORKSA floating display refers to a display that shows information in mid-air, where there is no hardware. It uses an incoherent light source (e.g. a bright display) and passive optical components to converge the diverging light from the display to a specific focus plane to form a real image in the air. And just to be clear, this is different from Pepper's ghost displays, where objects can be seen behind a display glass (the glass surface creates a ghostly image by reflecting an illuminated object from the hidden area).
The device has three major optical components:
1. Bright display (e.g. a field monitor or a modern tablet computer)2. Beam Splitter/Semi-transparent mirror (e.g. plexiglass or tele-prompter glass)3. Retroreflective foil (see below)
The most important component is a reflective element called "retro-reflector". If you shine a light on a mirror with an angle of incidence - it will be reflected at an angle equal to the angle of incidence. This is the law of reflection which states that a ray of light will reflect off a surface at the same angle at which it strikes it when it comes in contact with it. A retro-reflector, however, will reflect the light back to the source, no matter which way it came from.
Retro-reflectors are used on traffic signs, bicycles and safety equipment to make them more visible.
The retro-reflection is achieved through either glass beads or microprisms embedded in the sheeting. The size of these structures is sub-millimeter, each structure is not noticeable for viewers. For our purposes, the microprisms should be as small as possible, evenly distributed, and of good quality.
Here is how can we use a retro-reflector to create a floating display: If we use a display and let the image be reflected by a reflective sheet... nothing happens, because the reflected image is formed directly on the light source itself. However, if we introduce a beam splitter, the reflected rays can be separated. The optical components interact as follows :
a) Light is emitted from a bright display in the direction of the beam splitter.b) Then it's (partly) reflected by the beam splitter towards the retro-reflector. The light returned from the retro-reflector is sent back to the beam splitter (opposite, but at exactly the same angle).c) Finally, the light passing through the beam splitter converges outside to form an aerial image.
Orientation : There are several ways to arrange the 3 optical components. The basic version is shown in the picture below ("cardboard experiment"). I changed the configuration so that the floating image appears above the machine (because I think it looks cooler...).
(Note : It is important to mention at this point that, of course, I did not invent the concept. That honor belongs to a group of scientists from Japan who have been reporting on it for over 10 years. See the links below to the available papers.)
COMPONENTS USEDA link to my source is here : https://www.reflecto.shop/(a German online shop for various reflective materials) The material I finally chose is called Oralite3010.Note: I also found a company that sells retroreflective material for professional use. However, as you may have guessed by now, it's way too expensive for hobbyist purposes. But it would certainly be nice to try it out. If you are interested, search for "Nippon Carbide RF-A". In the end, I decided to go with the affordable Oralite3010, even though the picture is not ultra-sharp and the brightness could be a little bit higher. At the end of the day, this is a DIY project with a limited budget, so I have to live with some limitations.
STRUCTUREI decided to go for a stable design of the case and to put it all together using aluminum profiles (20x20).
All the necessary parts for attaching the functional parts are 3D printed and can be found in the repository linked below:- Beamsplitter glas holders- Field monitor frame- Sensor unit elements- Reflector holder- Lattepanda frame- Case design elements (not shown above)
SENSOR UNITFrom the beginning, I wanted to be able to interact with my floating display, so I had to find a way to detect the position of a finger "touching" the screen in the air. For my version, all I need is a simple interaction, so just a couple of defined areas where I can tap on a button or something like that. My goal was to create an Arduino subsystem that would be able to process some simple sensor data.
I tested a number of sensors for this purpose.
The first sensor I found was the ZX Gesture Sensor from Sparkfun. It uses an infrared light emitter and 2 receivers and claims to be able to detect movements and position of an object above the sensor. Setup and connection is easy - all we need is 5V and a connection to the I2C bus. Gesture recognition works well with the examples provided. It can detect left, right, and up swipes almost without a problem. However, the output of the position itself was not very precise - especially when I used only a finger instead of my hand. The calculation of the position is based on the amount of reflection, which is more of an approximation than an accurate measurement. But the main problem I found was that the sensor immediately spit out garbage data when I turned on my room light. This is because many light sources not only emit light in the visible spectrum, but also contain a certain amount of infrared light. So I simply cannot rely on the readings in normal lighting conditions, which ruled this sensor out for me.
TOF Sensors
I then started looking for sensors that were not so sensitive to ambient light and turned to time-of-flight sensors. TOF sensors are now common for robotics, drones and other applications, and have become truly affordable. They are also widely used in the maker community. I started testing the well-known VL53L0X from ST Microelectronics. It has a single laser transmitter and receiver in the same package, so it's very compact.
Setting up and connecting was as easy as the previous sensor. All we need is a 5V power supply and an I2C bus connection. The sensor emits small pulses of invisible laser light which can be seen slightly by the camera sensor of my cell phone. Time of Flight sensors measure the time it takes for a laser pulse to emit, reflect off an object, and return to its sensor. This allows for accurate distance measurement. And it also works very under normal conditions of ambient light.
It can provide accurate distance values in millimeters and in a range where the floating display would be. But one sensor only provides a one-dimensional measurement. To determine a position on our floating display, we need to expand to a 2D plane. For which we obviously need multiple sensors.
For my system, I would end up needing 3 sensors to have a sufficient system field of view. I am using an Arduino Nano to read the sensor data and programmed it to send all distance measurements via serial communication. With my little sensor array I should be able to detect 3 sections within the display area, and in each section I get a vertical position for some "touch" fields.
A total of 9 touch fields must be reliably detected, which should be sufficient for this DIY version. To visualize the sensor data, I wrote a test program on the PC. It receives the data via UART (COM port) and displays the distance of each sensor as a colored dot. Now I was able to verify that I could accurately measure a finger position in the desired area.
This seemed to work well. However, the finger position will always be slightly different, so some tolerance is needed for evaluation. Next, I had to find the sensor ranges that corresponded to the real positions on my floating display "touch fields".
So I modified the test program to display a simple 3x3 grid that highlights a field when one of the sensor positions is in a defined range for each field. It took some fine tuning, but in the end I managed to find the values for all 9 touch areas.
The basic framework is now in place, so let's add some content that is more appealing than just a 3x3 test grid. For this, I used a Lattepanda single-board computer (SBC) as the central brain of this machine. It allows to have an easy method to show standard images and animations, since it's essentially a small Windows computer. The field monitor is connected via HDMI as a second display, while the integrated front screen is used for direct control of the SBC.
By it's front screen, the SBC provides diagnostics data access and allows to fine-tune the touch-field ranges during operation easily. And the touch screen is a fantastic addition to the machine, giving it a special design element that makes it look like a communication module from the future. But another reason was that the Lattepanda can be directly hooked up to the Arduino Nano sensor module. Using one of the 3 provided USB interfaces, the SBC provides power to the Nano and at the same time it reads the data via an emulated UART.The remaining 2 USB ports are connected to a double USB passthrough connector mounted on the side panel. This allows me to easily connect a keyboard, mouse, or USB thumb drive to access the unit for uploading new images, etc.
A final small modification was that I pachted the "on" switch of the field monitor and connected it to an optocoupler, controlled by the Nano. With this simple hack, the Nano would automatically turn on the display every time it boots up. This ensures that the external display is ready when the Lattepanda SBC starts to display content on the second display output.The SBC itself will be switched on by an external switch on the left panel connected to the "SW" pin (connect to GND). The switch itself is illuminated by one of the PWM ports of the build-in Arduino Leonardo. I first tried to use the Leonardo for sensor processing, but it somehow did not work, so I decided to use the external Arduino Nano, which worked great.The Arduino Nano source code is included in the project below, the Lattepanda executable can be provided on request.
APPLICATIONSSo what could it be used for besides being a science fiction gadget? Firstly, the narrow field of view is ideal for safe data entry (the floating image is visible only from a direct line of sight). This makes it perfect for anyone who wants to enter data safely and securely. This machine is perfect for entering PIN numbers in crowded places. Another example could be a voting device.And since the pandemic, such operating concepts have gained a lot of interest because they ensure maximum hygiene. So possible application include "touch" displays in areas where hygiene conditions are important - e.g. hospitals, doctor's offices or homes for the elderly. Also in restaurants where orders are typically given via touchscreens, this is definitely a good option. And I am sure there are many more use cases...
SCI-FIDESIGN
But after all the effort spend, I wanted this machine to have a cool sci-fi look. Unfortunately, this time I could not find a similar device in my favorite sci-fi worlds (e.g. Star Wars). So I was basically free to make my own version. At last, I could finally try some techniques that I had been wanting to try for a while :
Since I cannot print metal parts (yet), but I wanted to have some metallic parts, I've been using this simple trick :a) First, wrap the printed design parts in aluminum foil. The parts should have some texture, so you can press them into the foil or let them stand out from it.b) Spray paint the element with black paint. It doesn't have to be fancy. Let it dry before moving on to the next step.c) Finally, scratch away some parts of the paint again using some steel wool. This makes the element look as if it's been used for years.
- create my own stickers
Inspired by an article in the German Make Magazine :
You can make your own stickers with a laser printer, some clear tape, and a bowl of water. Attach the transparent tape to the printout with the stickers, then cut out the stickers and place them in the water. After some minutes, remove the stickers from the water and gently rub the softened paper off the back of the tape with your finger. Then stick them on. The trick is that the toner color is not dissolved by the water, so the toner color sticks to the tape.
Finally it is evident that this DIY floating display design is not without its shortcomings. The primary issue is the degradation of image quality due to diffraction by the microstructure of retroreflectors. As the floating distance increases, the image becomes increasingly blurred. Another significant challenge is the degradation of image brightness, which is caused by substantial light loss at reflection and transmission at a beam splitter and at retroreflection.Other issues are :- multiple reflections (ghost images) caused by undesired reflection of the optical elements- viewing angle is constrained (but this may be perceived as a desirable feature !)- occlusion by optical elements- bulky design- very basic touch interactions due to my simple sensor array
Here are a few potential solutions :
- use high quality prism-type retro-reflector (expensive)- reduce the floating image distance to avoid blurring- use a bright digital signage LED display for outdoor use as display source- use polarizer films and quarter wave films laminated on optical elements to enhance light effiency (perhaps not suitable for DIY versions...)- use a 3D camera to improve the "touch" interactions (expensive)
ACKNOWLEDGEMENTIt is crucial to recognize the contributions of the team of scientists who have been engaged in this field of research for over a decade. References to the published works of Prof. Yamamoto and numerous other academic professionals can be found below :
IEC TR 62629-51-1 Edition 1.0 2020-05 : "3D display devices – Part 51-1: Generic introduction of aerial display" Aerial Imaging by Retro-Reflection (AIRR) by [Yamamoto et al. 2014a].
Floating aerial LED signage based on aerial imaging by retro-reflection (AIRR) Hirotsugu Yamamoto, Yuka Tomiyama, and Shiro Suyama, 2014https://opg.optica.org/oe/fulltext.cfm?uri=oe-22-22-26919&id=303352#
CONCLUSIONThat's it for this project. I hope you find this interesting and that it might inspire you to recreate it. If you're building your own version, I'd love to hear about your ideas. Please leave a comment below. Or perhaps you have some other great ideas about what you can do with a floating display.
All the best
Mac
Comments