Hardware components | ||||||
_ztBMuBhMHo.jpg?auto=compress%2Cformat&w=48&h=48&fit=fill&bg=ffffff) |
| × | 1 | |||
 |
| × | 1 | |||
 |
| × | 1 | |||
| × | 1 | ||||
| × | 1 | ||||
This is a brief tutorial on data storage on EEPROMs for Arduino data logger projects.
SD cards are a standard way of logging data on an Arduino. There are a lot of shield on the market for SD cards and a large number of tutorials how to use them. The disadvantage of SD cards is the large memory footprint of the SD card filesystems. Typically buffers sizes of 512 bytes have to be allocated. Card quality varies and some don't work well on different microcontrollers. SD cards need the SPI bus and block pins for other peripherals.
A low cost alternative are I2C EEPROM modules. They come in various sizes. 4kB and 32kB modules are the most common ones. 64 kB are available from many sources. They work on the I2C bus and only need two pins that they can share with other I2C devices.
There are many libraries for serial EEPROM on the market. Most of them address the EEPROM as a big storage array. Bytes, numbers or strings can be stored at specific storage location.
The solution here is different. It creates a very primitive file system on the EEPROM which a C-style API. EEPROM data is buffered in a small page buffer for read and write. Buffering makes the EEPROM reasonably fast and protects it against too many write cycles.
In addition to this, there is a raw API for buffered direct EEPROM access.
EEPROM write and read timing is exactly to the data sheet specification without any unnecessary delay() function in the code.
The library is part of my Arduino BASIC interpreter (https://github.com/slviajero/tinybasic) but described here as a standalone solution for use with C/C++ from the Arduino IDE. This standalone library can be downloaded from my repo https://github.com/slviajero/EepromFS.
EEPROM TypesThe library and test programs work any Arduino device no matter if it is a 3.3V or 5V system. It is tested on many platforms like Arduino, ESP, Raspberry Pi Pico.
A small breadboard and some wire are needed to connect the EEPROM.
EEPROM come as modules, chips, or built in in real time clock. The most common modules are either the ones on the right of the picture with a configurable I2C address and write protection or the module in the middle which have a fixed address. You can use any of these devices with this library.
You will need the Arduino IDE. I use version 1.8.15. In addition to this you need to download the library from https://github.com/slviajero/EepromFS and integrate it to your Arduino library.
Connect the EEPROMMost modules have the pinout printed on the board. Connect the SDA pin of the Arduino (on an UNO this is A4) with the SDA pin of the module and the SCL pin (A5 on an UNO) as well. Also connect power. Almost all EEPROMs can run at 5V. There are models with minimum voltages down to 1.7V. These are the 24FCxxx type which xxx denoting the EEPROM size in bits. The considerably cheaper 24Cxxx need between 2.7V and 5V.
The I2C addresses of the modules are between 0x50 and 0x57. On the configurable modules like the ones above, all jumpers including the write protection WP should be open. This way the address is set to 0x50.
EEPROM chips can be connected directly as well, leaving the address and WP pins open. No I2C pullup resistors are needed. Pulling up the address pins to 5V will change the I2C address. Pulling up WP will activate write protection. SDA is connected to pin 5 and SCL to pin 6 of the module. Power on pin 8 (VCC) and pin 4 (GND).
Clock chips have different addresses depending on the clock model.
The very common HW-084 models have 0x57 as default address of their 4kB EEPROM. These modules are rectangular. The slightly smaller quadratic "Tiny RTC" modules use 0x50. Both modules can be configured by soldering jumper connections.
Downloading the SoftwareDownload and install the EEPROM filesystem library https://github.com/slviajero/EepromFS. Open the test program examples/efstest of the library in the Arduino IDE. This program will run through a series of test of the EEPROM file system. It will be used to explain the features of the library in this tutorial.
At the beginning of the test program
#include <EepromFS.h
#include <Wire.h>
EepromFS EFS(0x50, 32768);
an EepromFS object is created. The first argument is the I2C address and the second argument is the size of the EEPROM. Set these values to your hardware configuration.
Upload and run the test program. In the output, all "Error status" lines should show result 0, i.e. no error if the EEPROM is wired correctly. The first few lines of output should look like this:
-----------------------------------------
Filesystem test
-----------------------------------------
Formating 32 slots
Format time = 171
518 bytes updated
Update baudrate = 24000
Error status: 0
-----------------------------------------
Filessystem mounted
32 slots in fs
Error status: 0
-----------------------------------------
EepromFS divides the entire EEPROM space into equal slots of a fixed length. In the example above it creates 32 slots of approximately 1kB size. A file can be as big as the slot size but not bigger. Even if there is space in other slots the file cannot grow beyond one slot. This makes the filesystem very simple as there is no file allocation table or any other complicated directory mechanism.
Filenames can be 12 characters long. Typical MSDOS 8.3 filenames are possible. The file name length is a parameter of the EepromFS library and can be changed to any other value less than 28.
Only two file can be open at the same time. One can be open for read and another for write and append.
There is one read/write buffer of 16 bytes. The buffer length can be changed to any other value less than 30.
The limitations on filename and buffer length do not come from the EepromFS library but from the underlying Arduino Wire library. One I2C transaction can only transport 32 bytes including the addressing. This limits the payload size to 30 bytes. EEPROM usually can handle bigger block sizes of 64 or 128 bytes. To make use of this one would need a Wire library with a bigger buffer size. This would speed up write access considerably. Changing the buffer parameters in the standard Arduino library would do the job. As speed is not important for my use cases, I haven't done this so far.
Write speed to the EEPROM is approximately 25000 baud or 3000 bytes per second. Read speed is 60000 bytes per second. The latter is determined by the Wire transaction speed while write speed is determined by the EEPROM write delay parameter. This parameter can be found in the EEPROM data sheets. The library uses 10 ms which is a standard parameter for many EEPROMs. Some can be faster, but not much.
The baudrate test program can be downloaded from https://github.com/slviajero/EepromFS/blob/main/examples/baudrate/baudrate.ino.
Excursion: How it works?You can skip this paragraph if you are only interested in the library itself and its APIs. This is additional information on EEPROM timing and I2C.
EEPROMs are organised in blocks. Each block can be written separately. The blocks are between 32 and 128 bytes. Writing a block is a time consuming activity and needs a suitable delay.
EEPROM can accept one block at a time for read and then need time to digest the information. For this reason it is good to buffer the data in RAM and transfer one block at a time. Unfortunately the Arduino I2C library limits the transfer block size to 32 bytes. This is set by the parameter
#define BUFFER_LENGTH 32
in Wire.h. The parameter can be changed by editing the library. This would allow larger I2C transfer sizes. Please note that also twi.h needs to be changed as the parameter appears also there and is not adapted automatically.
The EepromFS library accepts the limitation of standard Wire and buffers 16 bytes.
The read code in the library looks like this
Wire.beginTransmission(eepromaddr);
unsigned int pa=p*EFS_PAGESIZE;
Wire.write((int)pa/256);
Wire.write((int)pa%256);
Wire.endTransmission();
Wire.requestFrom((int)eepromaddr, (int)EFS_PAGESIZE);
int dc=0;
while( !Wire.available() && dc++ < 1000) delay(0);
if (Wire.available() == EFS_PAGESIZE) {
for(uint8_t i=0; i<EFS_PAGESIZE; i++)
pagebuffer[i]=(uint8_t) Wire.read();
pagenumber=p;
pagechanged=false;
} else {
ferror|=1;
return 0;
}
EEPROMs expect the address to be send as two bytes at the beginning of the transaction. This happens in the first code block. In the second code block, one buffer page (16 bytes) of data are requested. Once the library has available data, it is retrieved. There is no delay in the library except the delay(0) which is needed on ESP8266 systems as it calls yield(). Timing on read is uncritical and Wire.available() is used to synchronise EEPROM and Arduino.
The write code handles the I2C limitations and the timing
unsigned int pa=pagenumber*EFS_PAGESIZE;
Wire.beginTransmission(eepromaddr);
Wire.write((int)pa/256);
Wire.write((int)pa%256);
if (Wire.write(pagebuffer, EFS_PAGESIZE) != EFS_PAGESIZE) ferror|=1;
ferror+=2*Wire.endTransmission();
delay(EFSWRITEDELAY); // the write delay according to the AT24x datasheet
pagechanged=false;
First two bytes are transferred for the address, then Write.write() sends the buffer. EFS_PAGESIZE is 16 by default. Any size up to 30 is possible here but not more. Address and payload have to fit in a 32 byte frame.
After the transmission a delay of EFSWRITEDELAY waits for the EEPROM to digest the data. This is 10ms by default. Some tuning can be done here. 10 ms is a standard value. Some EEPROM can do 4 ms. We block the library here and the calling program to avoid that reads conflict with writes.
No other time delay is needed for EEPROMs than this. All other function of the EEPROM run at I2C bus speed.
If you only need a stable EEPROM library and are not interested in the filesystem API, you can directly go to the RAW API section. With this API you get timing and buffering and can access the EEPROM as a byte array.
Use the POSIX Style APIThe POSIX style API supports many of the standard C file IO function. Implemented API functions are
bool format(uint8_t s);
uint8_t fopen(char* fn, char* m);
uint8_t fclose(uint8_t f);
uint8_t fclose(char* m);
bool eof(uint8_t f);
uint8_t fgetc(uint8_t f);
bool fputc(uint8_t ch, uint8_t f);
bool fflush(uint8_t f);
void rewind(uint8_t f);
uint8_t readdir();
char* filename(uint8_t f);
unsigned int filesize(uint8_t f);
uint8_t remove(char* fn);
uint8_t rename(char* ofn, char* nfn);
int available(uint8_t f);
File handles are uint8_t i.e. one byte integers.
A typical program to write a file to the filesystem would look like this
#include <EepromFS.h>
#include <Wire.h>
EepromFS EFS(0x50, 32768);
char* m="hello world";
void setup()
{
Wire.begin();
Serial.begin(9600);
if (EFS.begin()) {
uint8_t f=EFS.fopen("test", "w");
for(int i=0; i<11; i++) EFS.fputc(m[i], f);
EFS.fclose(f);
Serial.println("Wrote 11 bytes");
} else
Serial.println("Mount failed");
}
void loop(){}
There is a number of examples including directory and error handling in the test program https://github.com/slviajero/EepromFS/blob/main/examples/efstest/efstest.ino.
Most function behave just like POSIX style file access function would work.
Use the Raw APIIn addition to the POSIX API, the library also offers a raw access API. This just makes use of the read/write buffering and the timing but not of the filesystem. Accessing the EEPROM with the raw API can and will destroy the file system structure.
The following library function are implemented
uint8_t rawread(unsigned int a);
void rawwrite(unsigned int a, uint8_t d);
void rawflush();
The EEPROM can be accessed as an array of bytes. These function correspond to the EEPROM.write() and EEPROM.read() function of the Arduino IDE. All access is buffered and unnecessary read and write to the I2C bus are avoided. The library can be useful for systems with no builtin EEPROM. It offers a fast and lean API to an external EEPROM.
The function rawflush() is needed to close a transaction. This should happen if no more data needs to be transferred. It assures that the current buffer is written to the EEPROM.
The example program for this API is https://github.com/slviajero/EepromFS/blob/main/examples/efstest-eeprom/efstest-eeprom.ino.
Use the Byte APIThe third API of EepromFS is similar to the raw API but protects the file system structure.
uint8_t getdata(uint8_t s, unsigned int i);
void putdata(uint8_t s, unsigned int i, uint8_t d);
unsigned int EepromFS::size();
The functions getdata() and putdata() access one particular slot of the file system. This slot is given by an integer s. No test is done if the slot is already used by a file.
A code segement for this API would look like this
// allocate a file slot and call it dummy, close it right away
uint8_t s=EFS.fopen("dummy", "w");
EFS.fclose(s);
Serial.print("Slot number is ");
Serial.println(s);
// how big is the slot
Serial.print("Data slot size is :");
Serial.println(EFS.size());
// access it through the byte API
for(int i=0; i<10; i++) EFS.putdata(s, i, i*i);
EFS.rawflush();
for(int i=0; i<10; i++) Serial.println(EFS.getdata(s, i));
The fopen() statement is only used to create a file and remember the slot s it is in. The calls putdata() and getdata() can access this file like the standard Arduino functions EEPROM.read() and EEPROM.write(). Flushing is mandatory to protect the data. Without the flush, the read part of the code would still be successful as the data is stored in the buffer but it would not be written permanently to the EEPROM.
If data is read or written beyond the size of the slot, this transaction is return. fgetc() returns a 0 outside the bounds of the slot and fputc() ignores the data. EFS.size() returns the number of bytes in a slot.
This test program can be downloaded from https://github.com/slviajero/EepromFS/blob/main/examples/efstest-byte/efstest-byte.ino.
Using EepromFS From BASICEEPROMs written with EFS can be integrated and used in the IoT BASIC interpreter https://github.com/slviajero/tinybasic as EepromFS is one of the supported file systems of the system. You can download the interpreter from https://github.com/slviajero/tinybasic/tree/main/TinybasicArduino.
With an Arduino UNO an Integer BASIC can be compiled, floating point would be too big. Open TinybasicArduino.ino and set the compiler directives
#undef BASICFULL
#undef BASICINTEGER
#define BASICSIMPLE
#undef BASICMINIMAL
#undef BASICTINYWITHFLOAT
BASICSIMPLE is defined, all others are #undef. Bigger Arduinos can use BASICFULL.
Open hardware-arduino.h and set the compiler directive
#undef USESPICOSERIAL
#undef ARDUINOPS2
#undef ARDUINOUSBKBD
#undef ARDUINOPRT
#undef DISPLAYCANSCROLL
#undef ARDUINOLCDI2C
#undef ARDUINONOKIA51
#undef ARDUINOILI9488
#undef ARDUINOSSD1306
#undef LCDSHIELD
#undef ARDUINOTFT
#undef ARDUINOVGA
#undef ARDUINOEEPROM
#define ARDUINOEFS
#undef ARDUINOSD
#undef ESPSPIFFS
#undef RP2040LITTLEFS
#undef ARDUINORTC
#undef ARDUINOWIRE
#undef ARDUINOWIRESLAVE
#undef ARDUINORF24
#undef ARDUINOETH
#undef ARDUINOMQTT
#undef ARDUINOSENSORS
#undef ARDUINOSPIRAM
#undef STANDALONE
All compiler directives are undef expect ARDUINOEFS.
Also, set the EEPROM parameters to the correct values.
#define EEPROMI2CADDR 0x050
#define EFSEEPROMSIZE 32768
Compile and upload the sketch to the Arduino.
The files created by the test programs can be listed with CATALOG
> catalog
file1 17
file2 52
test
dummy
Files can be deleted and renamed with DELETE and RENAME.
BASIC file access commands like OPEN, CLOSE, INPUT and PRINT can be used to read the data.
EEPROMs are exchangeable between systems running with C++ code and the larger BASIC based computers with graphics and IoT functions like the ones in these tutorials
Data can be shuffled this way without the overhead of SD card filesystems.
Comments