What Color is it?

In this project we will use the Adafruit Starter Pack for Windows 10 IoT Core on Raspberry Pi 2 kit components to create a project that uses an RGB color sensor to measure the color of an object placed on the sensor. A pushbutton is used to trigger the sensor. The color data is then written to the debug console and the approximate color detected is output on a speaker or headphones.

In this lesson you will learn how to connect a pushbutton to the Raspberry Pi using the GPIO pins, how to talk to a sensor via the I2C bus and how to convert a string to an audio output.

Hardware

Connect the Raspberry Pi2 to the breadboard and the other components as per the Fritzing diagram in the "Schematics" section below.

You will also need either a speaker or a pair of headphones for this lesson. Connect it to the AUX socket on the Pi.

Software

You can download the code starting project from https://github.com/ms-iot/adafruitsample and we will lead you through the addition of the code needed to talk to the web service and get your pin on the map. What map?

Open up "Lesson_205\StartSolution\Lesson_205.sln" and open the MainPage.xaml.cs file.

We have filled in a few methods as a starting point for you in this solution. If you want to jump ahead you can find a solution with all the code completed at: "Lesson_205\FullSolution\Lesson_205.sln"

MainPage.xaml.cs

Open the MainPage.xaml.cs file.

Add the following lines at the top of the MainPage class. These are references to the classes we will use later. Here, you will also specify the GPIO pin you will be using for the pushbutton. We have used GPIO 4 (pin 7) in this example.

 public sealed partial class MainPage : Page
    {
        //A class which wraps the color sensor
        TCS34725 colorSensor;
        //A SpeechSynthesizer class for text to speech operations
        SpeechSynthesizer synthesizer;
        //A MediaElement class for playing the audio
        MediaElement audio;
        //A GPIO pin for the pushbutton
        GpioPin buttonPin;
        //The GPIO pin number we want to use to control the pushbutton
        int gpioPin = 4;
        public MainPage()
        {
            this.InitializeComponent();
        }

Now we add code in the OnNavigatedTo method which will:

1. Create a new TCS34725 object for the color sensor and initialize the object.
2. Create a new speech synthesizer for text to speech operations.
3. Create a new media element to play the audio and initialize it.
4. Call the InitializeGpio function.

If you do not want to add a pin onto the map, remove MakePinWebAPICall();  

 protected override async void OnNavigatedTo(NavigationEventArgs navArgs)
        {
            MakePinWebAPICall();
            try
            {
                //Create a new object for the color sensor class
                colorSensor = new TCS34725();
                //Initialize the sensor
                await colorSensor.Initialize();

                //Create a new SpeechSynthesizer
                synthesizer = new SpeechSynthesizer();

                //Create a new MediaElement
                audio = new MediaElement();

                //Initialize the GPIO pin for the pushbutton
                InitializeGpio();
            }
            catch (Exception e)
            {
                Debug.WriteLine(e.Message);
            }
        }

The InitializeGpio function should do the following:

1. Create a default GPIO controller and use it to open the required GPIO pin.
   
2. Debounce the pin.
   
3. Set the pin mode to input.
   
4. Set a callback function for the GPIO pin ValueChanged event.
   

 //This method is used to initialize a GPIO pin
        private void InitializeGpio()
        {
            //Create a default GPIO controller
            GpioController gpioController = GpioController.GetDefault();
            //Use the controller to open the gpio pin of given number
            buttonPin = gpioController.OpenPin(gpioPin);
            //Debounce the pin to prevent unwanted button pressed events
            buttonPin.DebounceTimeout = new TimeSpan(1000);
            //Set the pin for input
            buttonPin.SetDriveMode(GpioPinDriveMode.Input);
            //Set a function callback in the event of a value change
            buttonPin.ValueChanged += buttonPin_ValueChanged;
        }

The buttonPin_ValueChanged function is called every time the pushbutton is pressed. The function should:

1. Check for the rising edge i.e. check if the pushbutton is released.
   
2. If true, get the approximate color value and call the SpeakColor function.
   

 //This method will be called everytime there is a change in the GPIO pin value
        private async void buttonPin_ValueChanged(object sender, GpioPinValueChangedEventArgs e)
        {
            //Only read the sensor value when the button is released
            if (e.Edge == GpioPinEdge.RisingEdge)
            {
                //Read the approximate color from the sensor
                string colorRead = await colorSensor.getClosestColor();
                //Output the colr name to the speaker
                await SpeakColor(colorRead);
            }
        }

The SpeakColor function should:

1. Create a SpeechSynthesisStream using the string with the approximate color.
   
2. Using a dispatcher, set the source of a media element object to the SpeechSynthesisStream and play it.
   

 //This method is used to output a string to the speaker
        private async Task SpeakColor(string colorRead)
        {
            //Create a SpeechSynthesisStream using a string
            var stream = await synthesizer.SynthesizeTextToStreamAsync("The color appears to be " + colorRead);
            //Use a dispatcher to play the audio
            var ignored = Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
            {
                //Set the souce of the MediaElement to the SpeechSynthesisStream
                audio.SetSource(stream, stream.ContentType);
                //Play the stream
                audio.Play();
            });
        }

TCS34725.cs 

Open the TCS34725.cs file.

First, create 2 classes to store the data read from the sensor.

 //Create a class for the raw color data (Red, Green, Blue, Clear)
    public class ColorData
    {
        public UInt16 Red { get; set; }
        public UInt16 Green { get; set; }
        public UInt16 Blue { get; set; }
        public UInt16 Clear { get; set; }
    }

    //Create a class for the RGB data (Red, Green, Blue)
    public class RgbData
    {
        public int Red { get; set; }
        public int Green { get; set; }
        public int Blue { get; set; }
    }

The first part of the TCS34725 class  is making a list of the addresses of the different registers in the TCS34725. These values can be found in the TCS34725 datasheet. The variables required for I2C and GPIO are declared following the enum.

A list of known colors is given to you. You may add or remove colors from this list based on the colors included in Windows.UI.Colors

Add a structure to store the name and value pairs of colors.

 //Create a structure to store the name and value of the known colors.
        public struct KnownColor
        {
            public Color colorValue;
            public string colorName;

            public KnownColor(Color value, string name)
            {
                colorValue = value;
                colorName = name;
            }
        };

Next comes the constructor. Here we set the GPIO pin to control the LED on the color sensor. We are using GPIO 12 (pin 32).

In the Initialize function, do the following:

1. Instantiate the I2C connection settings using the device address of the  TCS34725 sensor.
   
2. Set the I2C bus speed of connection to fast mode.
   
3. Use the I2CBus device selector to create an advanced query syntax string and create a collection using this string.
   
4. Instantiate the the  TCS34725  I2C device using the device id of the I2C bus and the connection settings.
5. Create a default GPIO controller and use it to open the required GPIO pin.
6. Set the pin mode to output.
7. Initialize the color list by calling the initColorList function.

 //Method to initialize the TCS34725 sensor
        public async Task Initialize()
        {
            Debug.WriteLine("TCS34725::Initialize");

            try
            {
                //Instantiate the I2CConnectionSettings using the device address of the TCS34725
                I2cConnectionSettings settings = new I2cConnectionSettings(TCS34725_Address);
                
                //Set the I2C bus speed of connection to fast mode
                settings.BusSpeed = I2cBusSpeed.FastMode;
                
                //Use the I2CBus device selector to create an advanced query syntax string
                string aqs = I2cDevice.GetDeviceSelector(I2CControllerName);
                
                //Use the Windows.Devices.Enumeration.DeviceInformation class to create a 
                //collection using the advanced query syntax string
                DeviceInformationCollection dis = await DeviceInformation.FindAllAsync(aqs);
                
                //Instantiate the the TCS34725 I2C device using the device id of the I2CBus 
                //and the I2CConnectionSettings
                colorSensor = await I2cDevice.FromIdAsync(dis[0].Id, settings);

                //Create a default GPIO controller
                gpio = GpioController.GetDefault();
                //Open the LED control pin using the GPIO controller
                LedControlGPIOPin = gpio.OpenPin(LedControlPin);
                //Set the pin to output
                LedControlGPIOPin.SetDriveMode(GpioPinDriveMode.Output);

                //Initialize the known color list
                initColorList();
            }
            catch (Exception e)
            {
                Debug.WriteLine("Exception: " + e.Message + "\n" + e.StackTrace);
                throw;
            }
        }

In the initColorList function, read all colors in the Windows.UI.Colors library and add the known colors to a list.

 //Method to get the known color list
        private void initColorList()
        {
            colorList = new List<KnownColor>();

            //Read the all the known colors from Windows.UI.Colors
            foreach (PropertyInfo property in typeof(Colors).GetProperties())
            {
                //Select the colors in the limited colors list
                if (limitColorList.Contains(property.Name))
                {
                    KnownColor temp = new KnownColor((Color)property.GetValue(null), property.Name);
                    colorList.Add(temp);
                }
            }
        }

The next part of the code is done for you. 

1. An enum is created for the sensor LED state and it is set to ON by default.
   
2. A set function is written to set the state of the LED.
   
3. Two enums are created using the datasheet to set the values for the different integration time and gain configurations available for the color sensor. 
4. The default value for the integration time is set to 700ms and the gain is set to 0. These values affect the resolution and sensitivity of the color sensor.
   

Next, add the following lines of code to the begin function to:

1. Read and verify the sensor signature.
   
2. Set the Init variable to true.
   
3. Set the integration time and gain.
   
4. Enable the sensor.
   

 private async Task begin()
        {
            Debug.WriteLine("TCS34725::Begin");
            byte[] WriteBuffer = new byte[] { TCS34725_ID | TCS34725_COMMAND_BIT };
            byte[] ReadBuffer = new byte[] { 0xFF };

            //Read and check the device signature
            colorSensor.WriteRead(WriteBuffer, ReadBuffer);
            Debug.WriteLine("TCS34725 Signature: " + ReadBuffer[0].ToString());

            if (ReadBuffer[0] != 0x44)
            {
                Debug.WriteLine("TCS34725::Begin Signature Mismatch.");
                return;
            }

            //Set the initalize variable to true
            Init = true;

            //Set the default integration time
            setIntegrationTime(_tcs34725IntegrationTime);

            //Set default gain
            setGain(_tcs34725Gain);

            //Note: By default the device is in power down mode on bootup so need to enable it.
            await Enable();
        }

Next write three functions. One to set the integration time, one to set the gain and one to enable the sensor. In each of these functions, different values are written to the sensor to perform the given operation. The values are determined using the datasheet.

A similar disable function is written for you.

 //Method to write the gain value to the control register
        private async void setGain(eTCS34725Gain gain)
        {
            if (!Init) await begin();
            _tcs34725Gain = gain;
            byte[] WriteBuffer = new byte[] { TCS34725_CONTROL | TCS34725_COMMAND_BIT,
                                              (byte)_tcs34725Gain };
            colorSensor.Write(WriteBuffer);
        }

        //Method to write the integration time value to the ATIME register
        private async void setIntegrationTime(eTCS34725IntegrationTime integrationTime)
        {
            if (!Init) await begin();
            _tcs34725IntegrationTime = integrationTime;
            byte[] WriteBuffer = new byte[] { TCS34725_ATIME | TCS34725_COMMAND_BIT,
                                              (byte)_tcs34725IntegrationTime };
            colorSensor.Write(WriteBuffer);
        }

        //Method to enable the sensor
        public async Task Enable()
        {
            Debug.WriteLine("TCS34725::enable");
            if (!Init) await begin();

            byte[] WriteBuffer = new byte[] { 0x00, 0x00 };

            //Enable register 
            WriteBuffer[0] = TCS34725_ENABLE | TCS34725_COMMAND_BIT;

            //Send power on
            WriteBuffer[1] = TCS34725_ENABLE_PON;
            colorSensor.Write(WriteBuffer);

            //Pause between commands
            await Task.Delay(3);

            //Send ADC Enable
            WriteBuffer[1] = (TCS34725_ENABLE_PON | TCS34725_ENABLE_AEN);
            colorSensor.Write(WriteBuffer);
        }

Now write a simple function to combine 2 bytes to create a 16-bit buffer.

 //Method to get the 16-bit color from 2 8-bit buffers
        UInt16 ColorFromBuffer(byte[] buffer)
        {
            UInt16 color = 0x00;

            color = buffer[1];
            color <<= 8;
            color |= buffer[0];

            return color;
        }

In the getRawData function, do the following:

1. Create a new ColorData object.
   
2. Check that the I2C device has been initialized.
   
3. Read the Clear data.
   
4. Repeat the same to read the Red, Green and Blue data.
   
5. Write the raw values to the debug console and return the ColorData object.
   

 //Method to read the raw color data
        public async Task<ColorData> getRawData()
        {
            //Create an object to store the raw color data
            ColorData colorData = new ColorData();

            //Make sure the I2C device is initialized
            if (!Init) await begin();

            byte[] WriteBuffer = new byte[] { 0x00 };
            byte[] ReadBuffer = new byte[] { 0x00, 0x00 };

            //Read and store the clear data
            WriteBuffer[0] = TCS34725_CDATAL | TCS34725_COMMAND_BIT;
            colorSensor.WriteRead(WriteBuffer, ReadBuffer);
            colorData.Clear = ColorFromBuffer(ReadBuffer);

            //Read and store the red data
            WriteBuffer[0] = TCS34725_RDATAL | TCS34725_COMMAND_BIT;
            colorSensor.WriteRead(WriteBuffer, ReadBuffer);
            colorData.Red = ColorFromBuffer(ReadBuffer);

            //Read and store the green data
            WriteBuffer[0] = TCS34725_GDATAL | TCS34725_COMMAND_BIT;
            colorSensor.WriteRead(WriteBuffer, ReadBuffer);
            colorData.Green = ColorFromBuffer(ReadBuffer);

            //Read and store the blue data
            WriteBuffer[0] = TCS34725_BDATAL | TCS34725_COMMAND_BIT;
            colorSensor.WriteRead(WriteBuffer, ReadBuffer);
            colorData.Blue = ColorFromBuffer(ReadBuffer);

            //Output the raw data to the debug console
            Debug.WriteLine("Raw Data - red: {0}, green: {1}, blue: {2}, clear: {3}", 
                            colorData.Red, colorData.Green, colorData.Blue, colorData.Clear);

            //Return the data
            return colorData;
        }

In the getRgbData function:

1. Create a new RgbData object.
   
2. Read the raw data using the previous function.
   
3. Calculate the RGB values using: [raw color data] * 255 / [raw clear data]
   
4. Write the RGB values to the debug console and return the RgbData object.
   

 //Method to read the RGB data
        public async Task<RgbData> getRgbData()
        {
            //Create an object to store the raw color data
            RgbData rgbData = new RgbData();

            //First get the raw color data
            ColorData colorData = await getRawData();
            //Check if clear data is received
            if (colorData.Clear > 0)
            {
                //Find the RGB values from the raw data using the clear data as reference
                rgbData.Red = (colorData.Red * 255 / colorData.Clear);
                rgbData.Blue = (colorData.Blue * 255 / colorData.Clear);
                rgbData.Green = (colorData.Green * 255 / colorData.Clear);
            }
            //Write the RGB values to the debug console
            Debug.WriteLine("RGB Data - red: {0}, green: {1}, blue: {2}", rgbData.Red, rgbData.Green, rgbData.Blue);

            //Return the data
            return rgbData;
        }

Lastly, in the getClosestColor function add the following code to:

1. Read the RGB color data from the previous function.
   
2. Calculate the Euclidean distance between the RGB data and the known colors in our list. 
   
3. Return the color with the shortest distance as the closest color.
   

 //Method to find the approximate color
        public async Task<string> getClosestColor()
        {
            //Create an object to store the raw color data
            RgbData rgbData = await getRgbData();
            //Create a variable to store the closest color. Black by default.
            KnownColor closestColor = colorList[7];
            //Create a variable to store the minimum Euclidean distance between 2 colors
            double minDiff = double.MaxValue;

            //For every known color, check the Euclidean distance and store the minimum distance
            foreach (var color in colorList)
            {
                Color colorValue = color.colorValue;
                double diff = Math.Pow((colorValue.R - rgbData.Red), 2) + 
                              Math.Pow((colorValue.G - rgbData.Green), 2) + 
                              Math.Pow((colorValue.B - rgbData.Blue), 2);
                diff = (int)Math.Sqrt(diff);
                if (diff < minDiff)
                {
                    minDiff = diff;
                    closestColor = color;
                }
            }
            //Write the approximate color to the debug console
            Debug.WriteLine("Approximate color: " + closestColor.colorName + " - " 
                                                  + closestColor.colorValue.ToString());

            //Return the approximate color
            return closestColor.colorName;
        }

You code is now ready to deploy!

Try it out!

Hold an object on top of the color sensor and push the pushbutton. The RGB data from the object will be read and written to the Output window on Visual Studio. Next, the approximate color will be spoken out on the speaker/headphones.