Published July 11, 2023 © GPL3+

Capture data from Arduino to CSV using pySerial

Use pySerial to capture data from an Arduino system and write the result to a CSV file

BeginnerFull instructions provided1 hour2,266

Capture data from Arduino to CSV using pySerial

Things used in this project

Hardware components

Arduino Nano 33 BLE Sense

USB-A to Micro-USB Cable

Software apps and online services

Arduino IDE

python3

Story

There is often a need to capture data from an Arduino device and write this data to a file which can be used in other applications. An example would be data gathered by the Arduino as an edge device which is required to train a AI model. Completed model can then be deployed back on the Arduino device. Alternatively, the CSV file can be imported into a spreadsheet program for further analysis.

In this project, a Nano 33 BLE device is connected to a laptop and data from the Nano is captured via the Serial port by the laptop and written to a CSV file.

Here are the major components for this project

Arduino Nano 33 BLE
Laptop running Python
USB cable

These programs will work with any board in the Arduino family; all it needs is a Serial port that supports the Arduino Serial protocol.

Although not tested, It should also work with Linux, Windows or Mac operating systems.

pySerial

pySerial is a python lbrary that allows communication over any serial port on a computer. Because all Arduino devices use the Serial port for programming and communication, pySerial can be used to send and recive data from these devices.

To run pySerial, Pyhton must be installed on your computer. Many resources exist on the internet explaining how to install Python. (Most Linux distributions come with Python already installed)

www.python.org

To read the pySerial documentation:

https://pyserial.readthedocs.io/en/latest/index.html

pySerial also need to be installed before it can be used. This is easiest done in a terminal window with the pip command

You can check if pySerial is installed with

The pySerial commands used in this project are:

import serial – imports the library
serial.Serial(‘Port description’) – defines a serial port object and opens up communication
.write(data) – writes a single byte of data to the defined serial port
.readline(data) – reads a complete line of data from serial port; stops when \n encountered
.flushInput() - clears the buffer after a write operation

Python CSV Library

Python standard distribution includes the CSVmodule which implements classes to read and write tabular data in CSV format from and to files. Complete official documentation

https://docs.python.org/3/library/csv.html

This library will be used to write data received from an Arduino to a CSV file.

Schematic

Really simple

Connect Nano port to computer port using a USB cable

Flow Chart

Here is a flow chart of how the two systems interact in this project

The actual programs used are included in this project with comments.

The program exchanges integer values which are generated in the program. It is possible to send other types of variables (float, strings, characters, unsigned integers). The decode routine in the python program needs to be changed to suit.

Making it work

Here are the steps needed

Open the sketch “LoggingSketch.ino” in the Arduino IDE
Upload to the sketch to the Nano
Once this is done, close the IDE. If the python program is run while, the IDE is open, an error will occur. Two programs cannot access the same Serial port.
In a terminal window, start the python program

The program will run and print out the values received from the Nano in the terminal window. After 9 iterations, the ‘stop’ received from the Nano will stop the python program.
In the same directory as ReadSerial.py, there will now be a file ‘logging.csv’. Open the file in Text Editor or a spreadsheet application to see the results.

Easy

Code

/*
*Simple sketch to demonstrate sending data over Serial to a computer
*Computer recieves data and writes to a CSV file
*Computer uses pySerial to receive and transmit data
*This example uses interger variable, other data types are possible
*/
int counter = 0;
int incomingChar;

void setup(){
  Serial.begin(115200); // Start up Serial Port
  delay(700);// Wait for Serial to settle
  Serial.println("Simple Logging Sketch - send serial data to CSV");
}

void loop(){
  // Wait for computer to send character
  // Once character received, start sending data to computer
  if (Serial.available() > 0){
    incomingChar = Serial.read();
    
    // Send data to computer via Serial
    for (int i = 0; i < 10; i++){
      Serial.print(counter);
      Serial.print(",");
      Serial.print(counter + 1);
      Serial.print(",");
      Serial.println(counter + 2); // \n will end read by computer
      counter++;
      delay(3000);
    } 
  
  // Once data send is finished, send 'stop' to complete
  Serial.println("stop");
  Serial.flush();
  }
  
}

import serial
from datetime import datetime
import csv

#Open a csv file and set it up to receive comma delimited input
logging = open('logging.csv',mode='a')
writer = csv.writer(logging, delimiter=",", escapechar=' ', quoting=csv.QUOTE_NONE)

#Open a serial port that is connected to an Arduino (below is Linux, Windows and Mac would be "COM4" or similar)
#No timeout specified; program will wait until all serial data is received from Arduino
#Port description will vary according to operating system. Linux will be in the form /dev/ttyXXXX
#Windows and MAC will be COMX
ser = serial.Serial('/dev/ttyACM0')
ser.flushInput()

#Write out a single character encoded in utf-8; this is defalt encoding for Arduino serial comms
#This character tells the Arduino to start sending data
ser.write(bytes('x', 'utf-8'))


while True:
    #Read in data from Serial until \n (new line) received
    ser_bytes = ser.readline()
    print(ser_bytes)
    
    #Convert received bytes to text format
    decoded_bytes = (ser_bytes[0:len(ser_bytes)-2].decode("utf-8"))
    print(decoded_bytes)
    
    #Retreive current time
    c = datetime.now()
    current_time = c.strftime('%H:%M:%S')
    print(current_time)
    
    #If Arduino has sent a string "stop", exit loop
    if (decoded_bytes == "stop"):
         break
    
    #Write received data to CSV file
    writer.writerow([current_time,decoded_bytes])
            
# Close port and CSV file to exit
ser.close()
logging.close()
print("logging finished")

Credits

kpower

19 projects • 6 followers

Qualified Electrical Engineer with experience in software and hardware development

Contact

Comments

Please log in or sign up to comment.

Capture data from Arduino to CSV using pySerial

Things used in this project

Hardware components

Software apps and online services

Story

pySerial

Python CSV Library

Schematic

Flow Chart

Making it work

Schematics

Connection Diagram

Code

Logging Sketch

Python program with pySerial to read from Arduino Nano

Logging CSV file

Credits

kpower

Comments

Embed the widget on your own site

Capture data from Arduino to CSV using pySerial

Capture data from Arduino to CSV using pySerial

Things used in this project

Hardware components

Software apps and online services

Story

pySerial

Python CSV Library

Schematic

Flow Chart

Making it work

Schematics

Connection Diagram

Code

Logging Sketch

Python program with pySerial to read from Arduino Nano

Logging CSV file

Credits

kpower

Comments

Related channels and tags