Safety Assistant for Industrial Control System

Control system is very essential part in any process control industry such as power generation, chemical, water and wastewater etc. In this type of industry, main part of the whole control system especially distributed control system (DCS) is installed in a separate room, from where cables are pulled out to connect to the field equipment.

First things first, issue regarding the performance of the building blocks of control system that are responsible for controlling and maintaining the process. Control system used in industries, are composed of electronics such as server(s), communication module(s), process controller(s) as well as different IOs (analog and/or digital). Proper functioning of these elements is a must in order to run the process with a predictable behavior. To ensure the proper function of such electronics (mentioned by manufacturer), we need to ensure an optimal environment under which all these electronics are performing its desired operation. Any change with the environment can severely degrade the performance of these modules. There are many factors upon which performance of such electronics depend on very much. Temperature, humidity, dust are more crucial ones among those factors. Reliability, speed, noise, power consumption are the performance metrics—we care about during system design and research shows that temperature degrades the performance of all these metrics. In a power generation station (real story from my country), they damaged the stator core and coil of their 30MW permanent-magnet synchronous generator due to lost communication between the operator workstation and the control modules. After issuing a command to start lube oil pump, which failed to execute because of communication broke down. Later they suspect that temperature of some cabinets goes incredibly high and after then communication failed due to no response from the controller inside these modules within the time span of response. Nowadays, manufacturer started applying conformal coating to combat against humidity, dust, chemical as well as high temperature in order to avoid performance degradation and corrosive attack. Though budget might go higher, but module with conformal coating significantly boost the performance of system. It might be a good idea to keep track of such parameters.

Now, let’s take a look on our valuable energy. Industries maintain the temperature of the control room by installing several air conditioners inside that particular room. During the winter session, they might not need to run the air conditioner all day long or might want to schedule all of the available air conditioners at a different time-stamp. By scheduling air conditioners at a different time-stamp, we can conserve our valuable power to make useful for others as well as we can achieve the best utilization of all available resources.

And finally let’s consider the security. All right, this is not the same security, we are used to it in the realm of hardware or software. This is basically access control. Control rooms are accessed only by few authorized personnel, nominated by authority. So, control room usually locked with conventional locker (at least in my region) and keys are supplied to the authorized personnel. If the list of authorized personnel become longer, then they might need to share the limited number of keys among themselves. This might cause some undesirable trouble in case of emergency such as a person went outside for other works, who got the key.

Proposed system

This project can be categorized as a safety monitoring system of process control industry. A safety monitoring system is basically a system, added to monitor system operation in real-time. This application is tailored to the issues illustrated in previous section.

We know that system performance metrics such as reliability, speed of execution, power consumption, noise etc. are very likely affected by the temperature. An increase in temperature can degrade the performance of all mentioned metrics here. In order to get a clear idea about the temperature of electronics inside cabinets of control room, a temperature sensor will be supplied to each of the cabinet. A humidity sensor as well as dust sensor are also provided to capture relative humidity and dust concentration (respectively) inside the control room. This arrangement may help us to consider a little bit more regarding the corrosive attack, addressed in previous section. This project also address the issue of smoke detection, caused by an unintentional fire explosion. An alarm based on smoke detection, can notify user about a catastrophic event just begin to take root. However, data from sensor modules will be plotted in real-time. Data will be stored in a database for further analysis.

A very simple algorithm is implemented in order to schedule air conditioner based on cabinet temperature inside the control room. An average temperature is calculated from the temperatures of the individual cabinets and determined whether to turn on (or off) the air conditioner with a defined threshold limit, set by user. Since, IoT2020 has an on-board real-time clock module, we can easily set a scheduler to turn on (or off) of particular air conditioner where multiple air conditioners are installed. With such scheduling, we can achieve the best utilization of available resources.

Just like many other entry reserved places such as data center, this application used an electro-magnetic locker instead of conventional locker to prevent unauthorized access by personnel. This application also tried to avoid to carry an overhead (such as card used in RFID based access control system) with web based architecture. An user identification name and password will be provided to each of the authorized personnel. Moreover, a log file of accessed personnel will be generated in database for future investigation.

The Hardware

Hardware of this project is pretty simple. It is basically composed of three sections—power supply, sensing or input section and actuation or output section. A brief on each section is given below—

• Power supply: The system is designed to power up from AC mains. A switching mode power supply, abbreviated as SMPS (12V, 5A) is used to provide power the IoT2020 development board. The output voltage of SMPS is chosen such a value that is appropriate for the IoT2020 (between 9 ~ 36 VDC) Along with the SMPS, another buck converter module (5V), based on LM2596 is used to provide power to the logic circuits such as sensor modules, switching control using bipolar junction transistors (BJTs) etc. SMPS is also responsible for powering the actuating devices such as magnetic door lock, lamp, which also run at 12V level in this pseudo application. Please do not connect the 5V output of Arduino header of IoT2020 with the output of buck converter module.

• Sensing or input section: As described earlier, we are going to use temperature sensors inside each cabinet, humidity sensor and dust sensor (also used for smoke detection). The chosen temperature sensor is DS18B20, which is low cost and having a considerable stabilized output. DS18B20 is a bi-direction one-wire sensor module. Since there is no dedicated one-wire hardware in IoT2020, I’ve to choose one-wire over UART in order to talk to the DS18B20. One-wire over UART is more efficient that conventional bit-banging approach with software, which creates unnecessary overhead to the CPU. An optical dust sensor—GP2Y1010AU from Sharp, provides analog output based on the concentration of dust in air. GP2Y1010AU detects the concentration of dust in air by means of Infrared (IR) light reflection through dust in air. An important feature of this dust sensor is that it can distinguish smoke from dust. Perfect! This is useful to notify user in advance with an auditory cue (an alarm) based on presence of smoke. To get reliable output from the sensor module, we have to control the sample timing illustrated in datasheet. The IR LED is connected to Arduino digital IO (D4) and analog output is directly feed to the analog input (A0). Humidity sensor, used in this application is a primitive one—HSM-20G. HSM-20G is an analog sensor which output resistance varies with respect to relative humidity. With a proper biasing circuitry, the output of analog humidity sensor goes to the analog input (A1) of Arduino header of IoT2020.

• Actuation or output section: This is basically the output section of this project. A magnetic door lock (12V. 1A) is used to secure the control room from unauthorized access. A LED uses as pseudo DC lamp in this application to light up the control room. Both the door lock and the lamp is used to switch using a two-channel relay module. The relay module is a negative logic device (logic voltage level of ‘0’ is greater than that of ‘1’). So, I’ve to use a simple switching circuit based on 2N3904 NPN BJT, to switch the relay module with positive logic signal. Door lock and lamp are connected to the digital IO at D8 and D9 respective of Arduino header. A piezo electric buzzer is used to indicate smoke alarm, connected to the digital IO (D3)—configured as Pulse Width Modulation (PWM) output. This final element of this section, is the remote trigger circuitry—used to turn on/off air conditioner. And we are using a signal switching relay to trigger the remote controller on/off button electronically (Below shown how to trigger remote controller electronically). The operation of signal switching relay is controlled by a BJT, connected to the digital IO (D10) of Arduino header of IoT2020.

The Software

Software is divided into two sub-divisions—low-level and high-level. Low-level software, which is written in popular C language and responsible for collection as well as storing sensor data. High-level which is Node-RED flow diagram, a flow-based programming tool used to plot sensor data in two-dimensional X-Y plane as well as provide an intuitive user interface (UI) for the users.

• Low-lever or data acquisition section: This section basically interact with low-speed skeleton peripherals such as GPIO, ADC, UART etc. of IoT2020 and to the lite-weight database called SQLite. After initialing the required hardware module, the main calls each section—temperature data read, dust sensor read and humidity sensor read in a sequential order by means of a state machine implementation. A call graph of low-level software module is shown in figure below. Please be noted that the oval represents software module whereas rectangle represents hardware module.

• High-level or UI section: This section is mainly responsible to interact with the users. This is done with Node-RED flow along with simple JavaScript scripts. This section also actuates the output devices such as light, door lock, air conditioner based on actions taken by user. As promised earlier, in order to save energy, consumed by air conditioner, a simple algorithm based on temperature of cabinet is implemented to turn on or off the air conditioner automatically. Two sliders are provided to update threshold level, which used as margin to turn on or off air conditioner. A button also provided in order to turn on or off the air conditioner in any emergencies. Smoke detection uses the data from dust sensor, which generate an alarm based on predefined threshold value (magic value determined by empirical method). To get rid of tedious continuous alarm sound, a button is supplied to acknowledge the generated alarm. Authorized user can unlock the door of control room with their corresponding user identification and password for maintenance purposes. A log will be generated based on accessed personnel in order to later analysis. Please be noted that a new user can be inserted by means of SQLite command. Finally, another button is supplied in order to turn on or off the light inside the control room. A call graph of high-level software module is shown in figure below (rectangles with black background represent a button).

Build

Few hardware hacks: Let’s do some simple hacks—reasons why we are here!

• Power Supply: I’ve chosen a buck converter to power up the logic circuitry. It’s a variable output version of LM2596 which I found at my local market. I wanted to get a fixed output voltage of 5 volts. To do so, I have gone through the datasheet of LM2596 and found the following equation (below image in slide show) —which deals with the output voltage. Now, let’s come to the module. I found the resistors (R1 and R2) and verified with beloved multimeter. Calculation is done to replace the R2 resistor to get a fixed output of 5 volts. I removed the variable R2 with a fixed one of 1K Ohm based on my calculation. You may use a fixed output version (5V) of LM2596 or any suitable one.

• Air conditioner remote controller: One of the requirement of this project is to turn on/off air conditioner automatically. I planned to use a regular remote controller than designing a new one from scratch. We know, keys of remote controller are connected as a matrix in order to avoid large number of IO requirement. One terminal of key is connected to the so-called column of imaginary matrix and another to the row. When, we press a button, two terminals of this particular key get shorted by means of carbon element under beneath of that key. This short circuit between the two terminals of a key, indicates a button pressed by intelligent controller (aka MCU) inside the remote controller. In order to do the same electronically, we have to locate the terminals of button in PCB of remote controller. Securing the location of terminals of the on/off button (only on/off button used in this application) of remote controller, we’ve to solder both terminals with an extension cable. If we short these two terminals by any means, then we will get the same effect as button pressed. I have identified both the row and column (marked conn – 1 and conn – 2 in image below) and connected to jumper cables to make a flexible connection of remote controller to the system.

Environment setup: This project uses the latest release of image version (v2.1.3) as of April 19, 2017. You can download the latest image from here. Image version v2.1.3 comes up with all the required software modules (such as Intel mraa libraries for low-speed peripherals, Node-RED, SQLite3) for this application except one, Node-RED SQLite database module.

• Image installation: If this is your very first time to install image file in a SD card, then check out document titled “installing_new_image.pdf” under documents section of my GitHub repository here.

• Inserting SD card and RTC backup battery: Insert the newly burned SD card inside the SD slot of IoT2020. Now, time to install the RTC backup battery. Though, Siemens has a recommendation regarding the battery selection, I’ve chosen a CR2032 battery (I found this one at my local market)

• Changing IP address: In this step, I would like to connect my IoT2020 with the home network. To do so, power up the board and login with SSH to change the IP address. You can choose your favorite method to SSH the IoT2020 by means of Ethernet or UART debug console. I’m using Ethernet and PuTTy as a SSH client. Open PuTTy and connect to the IoT2020 (192.168.200.1) and make sure IP address of host computer is also the same subnet i.e. something like 192.168.200.10 and follow the instructions in below images... (NB: for details please follow Siemens guide from here)

• Login with new IP: Now, shutdown your IoT2020 and connect to your home router. Login again the with the newly set IP address that is reserved address (for me192.168.0.105) in router.

• Expand unallocated space in SD card: With the latest image version (v2.1.3), expansion of unallocated space is done during the first boot. If you are using previous version of image, please follow this guide to expand unallocated space. To make sure there is no unallocated space left in SD card follow the below instruction.

• Time setting: Set the time of your Linux machine. The idea here is, first set the system time. Then system will then be copied to hardware time i.e. to RTC.

• Installing Node-RED SQLite database: This is the most daunting part of this application. Of course, in terms of time. Node-RED SQLite module consumes a fair amount of time to build. Anyway, follow the below instruction in images.

• Auto-start Node-RED: (1) Type “iot2000setup” command on terminal window to enter iot2000 setup menu. (2) Select “Software” from the setup menu using arrow keys from keyboard, then hit “Enter” button. Please be noted that selected module will be highlighted RED and first charter of word should be GREEN. (3) Just like (2), select “Manage Autostart Options” and press “Enter” button. (4) Select “Auto Start node-red” by hitting “Enter” button once again. You will see an asterisk mark “ * “ inside [ ] braces. Hit “Tab” to go to the “Done” option. After pressing “Enter” while selecting “Done” option, you will set the node-red module to automatic start mode. (5) Go back to “IoT2000” setting menu and finally “Quit” from setup window.

Database creation: Though this section titled “Database creation”, we will also create the necessary directories for the application. Please download the required files from here.

• Creation of directories: This application using a mother directory titled “ctrl_room_monitor” under “/home/root/” directory. “ctrl_room_monitor” also contains two other daughter directories titled “database” and “application” in order to store required database files and application files respectively.

• Copying of required files: Copy the required files (create_table.sql and init_insert.sql) into the directory “/home/root/ctrl_room_monitor/database/”. I’m using WinSCP as FTP client. A tutorial on how to copy files using WinSCP can be found here.

• Create database with initial values: First, verify if all required files are copied properly or not using “ls” command on terminal window. Now, time to create database titled “ctrl_db.db” with required tables. After creation of database and required tables, we will store some initials values into tables. Initials values include default data of each sensor, default status of loads, and default user accounts. NOTE: Don’t change the name of database “ctrl_db.db”, otherwise you have to make a lot of change in Node-RED flow diagram as well as in source file of low-level application.

• Schema of tables: This is the last step of this section to see the database fields (aka schema) of tables. This is particularly useful to insert data such as adding new user in later time.

Building low-level application: We are going to build our low-level application for sensing or input section, illustrated earlier. Please download the required files from here.

• Copying of application files: Just like before, I’m using WinSCP to copy the required files that is “app.c” and “Makefile” to the directory “/home/root/ctrl_room_monitor/application/”

• Build: First, ensure all required files are copied properly using "ls" command. Then build with "make" command. Upon successfully build, we will generate an executable file, titled “app”. Note, you may encounter a “clock skew” warning during building. This is because of a mismatch of time (if system time lag) between IoT2020 module and time of creation of Makefile. You have to set time during environment setup section in order to get rid of this warning.

Importing Node-RED flow diagram: In this step, we will import high-level software module (aka Node-RED flow diagram) to provide an intuitive UI for users. Since, we already installed a new Node-RED module (node-red-node-sqlite), we need to reboot our IoT2020 to make installed module effective. So, please reboot and login again with root. You can download Node-RED flow diagram script for this application from here.

• First, check whether SQLite node module installed or not. All we need is to go to Node-RED dashboard using browser. Please be noted that Node-RED services run at port 1880. Since, I tied my IoT2020 in my home network with 192.168.0.105, I'm typing 192.168.0.105:1880 to the address bar of my browser. If you are using different IP address, change the IP address field but keet the port field same.

• Chick on "menu" icon (three horizontal lines on top-right corner), then select import and finally, clicked on clipboard.

• A new window titled "import nodes" will appear. Copy the script from the downloaded file and paste into the gray-colored box in the middle of the window. Now, click on "import" button.

• Click on "Deploy" button, which background color changed (an indication of change in flow diagram).

• Wait until a new message (Successfully deployed ) come out. You will encounter a moving vertical var (same as light bar of dark knight) at the position of "Deploy" button.

• Finally, browse UI version of your Node-RED flow diagram by typing 192.168.0.105:1800/ui (you may need to change IP if you use different settings).

Auto-start low-level application: Just like Node-RED, we would like to start the sensing or input (aka low-level) application to start automatically during boot. In order to do so, please follow the below steps.

• Enter the following command to create a symbolic link of executable app file to "/usr/bin/" directory.

ln -s /home/root/ctrl_room_monitor/application/app /usr/bin/ 

• Under “/etc/init.d/” directory create a new file using text editor such as “nano”. Give a suitable name such as “launch_safety_assistant.sh”.

cd /etc/init.d/
nano launch_safety_assistant.sh

• Type or copy the following code snippet to the newly created "launch_safety_assistant.sh" file.

#! /bin/sh
# location of application executable file
./usr/bin/app &
exit 0

• Now, give permission to the file to run as program using following command.

chmod +x launch_safety_assistant.sh

• Finally, add your startup script file into the boot sequence using following command.

update-rc.d launch_safety_assistant.sh defaults

Reboot: Okay, we are all done. Now, it's time to make the final reboot to start our low-level application and high-level application will automatically start after boot. Please remember that starting of Node-RED will take time (around 2-3 minutes). You can verify whether both applications are started or not from terminal window by issuing flowing command.

ps -ef   

Video Demonstration

A demonstration video of this application is attached below.

 Few more attachments based on system runtime of two days (specified time of days). Please be noted that I like to keep the width of "Load Status" diagram fixed due to avoid overlapping of label (x-axis) in the chart.

Information

• Unix "epoch" is chosen as the format of "time" field in SQLite database.

• Two user accounts created during the initial insertion of data into database. User name and password of both users account are chosen same. User account 1 (user name: admin, password: admin), User account 2 (user name: dummy, password: dummy).

• I faced a software crash problem (actually mraa failed to perform) after deployment (not always) of new Node-RED flow. Please let me know your insight in the comment section.