Wearable object detector with tactile distance feedback

Finding/avoiding things is hard when you can’t see. This wrist-worn device helps you scan the space and provides clear distance feedback.

IntermediateFull instructions provided4 hours231

Honorable Mention Visual Impairments

Build2gether 2.0 — Inclusive Innovation Challenge

Wearable object detector with tactile distance feedback

Things used in this project

Hardware components

SparkFun Distance Sensor Breakout - 4 Meter, VL53L1X

Arduino pro micro

Software apps and online services

Arduino IDE

Hand tools and fabrication machines

Soldering iron (generic)

Solder Flux, Soldering

Jumper Wire, Bundle

Story

It’s not very often that, in the bustle of life, I (and maybe most like me) think about the senses we take for granted. Now and then I happen to notice, or actually think about the fact that some people live without some of them.

I’m really hoping I won’t say anything stupid here, because I don’t really know what I’m talking about, or how people might think about it, so I ask for forgiveness if I do.

I think it can be hard to deal without seeing, probably the most important sense to understand the environment around us.

I suppose at home one can have a very controlled environment; know where (almost) everything is, but I think it’s much harder to deal with unknown places.

There are already some tools to help, like the cane, the clicker and maybe others I don’t even know.

Having watched the “Sue Thomas: F.B.Eye” TV series, and being an electronics engineer, I had all sort of ideas of devices that could be helpful to deaf people, but most were cumbersome, unpractical, or beyond my abilities. Anyway, it got me thinking about ways to improve any sense disability, including blindness.

When I first learned about the VL53L1x distance sensor, I immediately thought that this is something that could be useful: it’s a distance sensor, with a relatively narrow beam, not affected by sounds (and not using ultrasounds that might affect the guide dogs). Unfortunately, it doesn’t work well outdoors because it uses light so a bright day would make it useless, but I think there are enough indoors situations where one might find it useful: to find a doorway, to avoid obstacles that are not on the ground (the cane does that), to find something on a table, etc.

So, I thought it’s worth a try.

I designed a device that can be worn on the wrist, so it can be easily pointed to the direction of interest. In fact, the device is fast enough that, by moving/rotating the wrist, it can easily “scan” the environment, such that the user can immediately know which way objects are, and how far, within the scanned area.

The feedback is given through a tactile transducer worn on the outside of a finger, such that the palm/hand can still be used, even as it may reduce scanning ability, as one wouldn’t want to rotate the wrist while holding a cup of coffee.

After proposing the idea, the Contest Masters gave me some invaluable feedback. Distance feedback is more important than just knowing something is “there”. I went beyond the proposed change in intensity or volume of pulse or sound, which can be hard to gauge in noisy environments, to use Pulse Width/Frequency Modulation of a finger mounted haptic/tactile transducer that can be felt and easily interpreted regardless of ambient noise, and without overburdening the hearing, which I think is very important when one can’t see.

Another proposal was to have the device in the palm, to better guide the hand to pick up whatever object might be the “subject”, but I feel the palm should be free, if it’s to do the actual picking up. However, the idea opens up the possibility to wear in on the “inside” side of the wrist, with the same benefit, so, depending on the situation, one could rotate it inside/outside (like a watch) as needed. Thank you Masters!

The boldest Contest Master’s proposal was to have the device identify and describe the object using AI or such technology. That would be the bee’s knees, but is way beyond the scope of this project which is meant to be fast, small, instantly-on, independent of any cloud or other serious processing power. It seems there are services that can transmit video from a phone worn in front, that can describe the surroundings, but while very useful, they are something else and may not be ideal to easily locate a cup on a table, for example.

The prototype I made is rather ugly, using sections of a glove to hold the device and transducer on the hand, but is fully functional.

Device on wrist, transducer on finger, wire that connects them (can be tucked)

It was built inside an AA battery holder so it could be made much smaller and rechargeable. Also, I used a laptop speaker for the tactile transducer which works very well, but smaller devices could be found/designed to further reduce the size of the system.

Complete (ugly) device with transducer.

The transducer could in fact be mounted on the surface of the device that touches the skin, if worn in a sensitive location. This way there would be no wire, just the device. The prototype was made so I could experiment in multiple places.

The videos below show the device used to detect a small object (mug) on a surface, and to detect distance to a curtain. Since I can’t “show” tactile feeling, I put the tactile transducer near the camera microphone and the clicks you hear are “actuations” or movements of the membrane that would be felt on the skin when mounted. Alternate clicks show push/release, so they are easy to feel and interpret. You’ll have to turn the volume up to hear them, since they are just movements, really, not Beeps.

Detecting a small object:

Notice the change in frequency when pointing towards the mug

Detecting Distance:

Notice the change in frequency when distance changes

When nothing is within range, there are still pulses, to reassure the user that the device is on. When an object is detected nearer than the (configurable) maximum distance, the pulses become more “urgent” (faster, closer together) so that after a period of trials the user can easily tell how far the object is. The current range is just under 2 meters, I think it could be extended to 4m.

One easy way to “calibrate” one’s sense is to touch a wall, feel the fast pulses, then move back and forth; in time the distance gauging should become second nature.

On to the build.

I used an Arduino pro micro (it was the smallest I had; any, really, should work) to run the code, read the distance from a VL53L1x breakout board, run some algorithm to transform the distance into practical pulse timing, and send distance-modulated pulses to the transducer.

The sensor peeks out through a small window in the enclosure. Power switch is to the right.

In the picture above, the Arduino was lifted for clarity.

The schematic is in the schematics section and the code is in the code section; select the type of Arduino that you have, before building/uploading.

If the project is of interest, future (but hard) improvements might include some detection of object size, and feedback by changing modulations between Pulse Position, Pulse Width, and Frequency modulations to convey the extra information to the user.

Code

Wearable object detector with tactile distance feedback

/*
	Written by AnaCoda and CataCluj based on public domain/beerware code from Nathan Seidle from SparkFun Electronics
*/

#include <Wire.h>
#include "SparkFun_VL53L1X_Arduino_Library.h"

#define debug 1
#define prescaler 64

#define FREQ_MIN 2
#define FREQ_MAX 20
#if prescaler == 1
	#define OCR_MIN 8000000/FREQ_MAX	// When prescaler is 1
	#define OCR_MAX 8000000/FREQ_MIN
#elif prescaler == 64
	#define OCR_MIN 8000000/(64*FREQ_MAX)	// 12.5(10000Hz) When prescaler is 64. For some reason they become negative
	#define OCR_MAX 8000000/(64*FREQ_MIN)	// 125
#else
	#error Must be one of above
#endif

#define TONE_DELAY 1
#define DIST_MIN 100	//Not exactly mm but ballpark
#define DIST_MAX 3000

VL53L1X distanceSensor;

const int numReadings = 10;

int readings[numReadings];      // the readings from the analog input
int readIndex = 0;              // the index of the current reading
int total = 0;                  // the running total
int DistAvg = 5000;				// Start average with something to get there faster
float DistAvgWeight = 0.9;      // IIR a,Weight of new Measurement. Y[n] = aX[n] + (1-a)Y[n-1]

uint16_t Tim1Top = 0;			//This will determine Tone Frequency (Inversely-proportional)

int NewDist = 0;

//Logarithmic: more change near the higher frequencies

void setup(void)
{
	Wire.begin();
	Wire.setClock(400000); //Increase I2C bus speed to 400kHz
#if debug == 1
	Serial.begin(115200);
	Serial.println("VL53L1X");
#endif
//Initialize all readings to 0
	for (int thisReading = 0; thisReading < numReadings; thisReading++)
	{	readings[thisReading] = 0;	}
	if (distanceSensor.begin() == false)
	{	Serial.println("Sensor offline!");	}

// set up 8 MHz timer on CLOCKOUT (OC1A), Pin 9, Port B1
	pinMode (9, OUTPUT); //Speaker, Tactile Transducer
	TCCR1A = bit (COM1A0) | bit (WGM11) | bit (WGM10);	// toggle OC1A on Compare Match. WGM for Fast PWM  (WGM13:0 = 15 = 0b1111)
#if prescaler == 1	// OCR1A val of 8000 => 1000Hz; 800 => 10000Hz. Min Freq = 8000000/65535 = 122Hz
	TCCR1B = bit (WGM13) | bit (WGM12) | bit (CS10);	// Fast PWM, no prescaling
#elif prescaler == 64
//	TCCR1B = bit (WGM12) | bit (CS10) | bit (CS11);	// CTC mode, 64 prescaling. OCR1A 512000 => 1000Hz; 51200 => 10000Hz. Min Freq = 2Hz. Max = 
//	TCCR1A	= 0b01000011;	//Making any of COMnX1:0 non-zero overrides GPIO Pin Function. Clear OC1A on Compare Match
//				||||||||
//				||||||--WGM11, WGM10 (11) Fast PWM. Also set WGM13:2 in TCCRnB to 11
//				||||--	Reserved, must be 0
//				||--	COM1B1, COM1B0:
//				--		COM1A1, COM1A0: 01 = Toggle OC1A on Compare Match

	TCCR1B	= 0b00011011;//This is how we start/stop the timer, by changing clock source
//				||||||||
//				|||||---CS12:0 (001). Clock Select clkI/O / 1 (No Prescaling)
//				|||--	WGM13, WGM12 (11) Fast PWM Mode. Also set WGM11:0 in TCCRnB to 00
//				||-		Reserved, must be (0)
//				|-		^ICES1: Input Capture Edge Select (0)		
//				-		^ICNC1: Input Capture Noise Canceler (0)		
#else
	#error Must be one of above
#endif

#if debug == 2	//We toggle GPIO every iteration to see how fast it is
	TCCR1A = 0;	//Clear COM1A0 to return pin to GPIO mode. Also Fast PWM mode is irrelevant
#endif
}

void loop(void)
{
	distanceSensor.startMeasurement(); //Write configuration bytes to initiate measurement
	while (distanceSensor.newDataReady() == false)	//Poll for completion of measurement. Takes 40-50ms.
	{	delay(1);	}
	
	unsigned int uiSignalRate = distanceSensor.getSignalRate();
	if (uiSignalRate > 10)
	{
		NewDist = distanceSensor.getDistance(); 							//Get the result of the measurement from the sensor
		DistAvg = DistAvgWeight * NewDist + (1 - DistAvgWeight) * DistAvg;
	}
	uint16_t ui16_New_OCR = fscale(DIST_MIN, DIST_MAX, OCR_MIN, OCR_MAX, DistAvg, 4);

	OCR1A = ui16_New_OCR;
//	delay(TONE_DELAY);
//	delay(100);

#if debug == 1
	Serial.print("D ");		Serial.print(DistAvg);			//DistAvg. Could be OCR_MIN
	Serial.print(" O ");	Serial.print(ui16_New_OCR);		//ui16_New_OCR. Could be OCR_MAX
	Serial.print(" S ");	Serial.println(uiSignalRate);
#endif

#if debug == 2	//We toggle GPIO every iteration to see how fast it is
//	PINB = 0b00100000;	//Toggle Pin B1. This works but I don't know why "0b00100000" means Bit 1 as in Port B1
//	asm ("sbi %0, %1 \n": : "I" (_SFR_IO_ADDR(PINB)), "I" (PINB1));	//This doesn't work
	digitalWrite(9, !digitalRead(9));	//This works to toggle
#endif
}

float fscale( float originalMin, float originalMax, float newBegin, float
              newEnd, float inputValue, float curve)
{
  float OriginalRange = 0;
  float NewRange = 0;
  float zeroRefCurVal = 0;
  float normalizedCurVal = 0;
  float rangedValue = 0;
  boolean invFlag = 0;


  // condition curve parameter
  // limit range

  if (curve > 10) curve = 10;
  if (curve < -10) curve = -10;

  curve = (curve * -.1) ; // - invert and scale - this seems more intuitive - postive numbers give more weight to high end on output
  curve = pow(10, curve); // convert linear scale into lograthimic exponent for other pow function

  // Check for out of range inputValues
	if (inputValue < originalMin) 
	{	inputValue = originalMin;	}
	if (inputValue > originalMax)
	{	inputValue = originalMax;	}

	
	OriginalRange = originalMax - originalMin;	// Zero Refference the values

	if (newEnd > newBegin)
	{	NewRange = newEnd - newBegin;	}
	else
	{
		NewRange = newBegin - newEnd;
		invFlag = 1;
	}

	zeroRefCurVal = inputValue - originalMin;
	normalizedCurVal  =  zeroRefCurVal / OriginalRange;   // normalize to 0 - 1 float

  // Check for originalMin > originalMax  - the math for all other cases i.e. negative numbers seems to work out fine
	if (originalMin > originalMax )
	{	return 0;	}

	if (invFlag == 0)
	{	rangedValue =  (pow(normalizedCurVal, curve) * NewRange) + newBegin;	}
	else     // invert the ranges
	{	rangedValue =  newBegin - (pow(normalizedCurVal, curve) * NewRange);	}

	return rangedValue;
}