A Better SD Library with RT-Thread

Background

The SD library shipped with Arduino IDE is handy, however lacks features like exFAT, LFN (long filename) and non-English characters support. This article is introducing an alternate SD card driver (based on RT-Thread) to address those issues.

RT-Thread

RT-Thread is a free and open source (Apache License 2.0) RTOS and available in the form of Arduino library. There is another article (Multitasking on Arduino) available to get the basic idea of RT-Thread.

Let's start by installing the library through Arduino IDE's library manager. (This article is based on RT-Thread library version 0.4.4.)

SD Card Driver (TL;DR)

(If you are not interested in the implementationand just want to know how to use it, please skip this section.)

SD card support in RT-Thread library is in the form of DFS (device file system), which is part of RT-Thread architecture. FAT is one of the file system supported by RT-Thread. (In version 0.4.4 of RT-Thread library, FAT is the only supported file system.)

The FAT DFS itself is based on the excellent work of ChaN's FatFs project.

A standard RT-Thread DFS provides the following file system and file interfaces:

/* File system operations */
struct dfs_filesystem_ops
{
   const char *name;
   uint32_t flags;      /* flags for file system operations */
   /* operations for file */
   const struct dfs_file_ops *fops;
   /* mount and unmount file system */
   int (*mount)    (struct dfs_filesystem *fs, unsigned long rwflag, const void *data);
   int (*unmount)  (struct dfs_filesystem *fs);
   /* make a file system */
   int (*mkfs)     (rt_device_t devid);
   int (*statfs)   (struct dfs_filesystem *fs, struct statfs *buf);
   int (*unlink)   (struct dfs_filesystem *fs, const char *pathname);
   int (*stat)     (struct dfs_filesystem *fs, const char *filename, struct stat *buf);
   int (*rename)   (struct dfs_filesystem *fs, const char *oldpath, const char *newpath);
};
/* File operations */
struct dfs_file_ops
{
   int (*open)     (struct dfs_fd *fd);
   int (*close)    (struct dfs_fd *fd);
   int (*ioctl)    (struct dfs_fd *fd, int cmd, void *args);
   int (*read)     (struct dfs_fd *fd, void *buf, size_t count);
   int (*write)    (struct dfs_fd *fd, const void *buf, size_t count);
   int (*flush)    (struct dfs_fd *fd);
   int (*lseek)    (struct dfs_fd *fd, off_t offset);
   int (*getdents) (struct dfs_fd *fd, struct dirent *dirp, uint32_t count);
   int (*poll)     (struct dfs_fd *fd, struct rt_pollreq *req);
}

A specific DFS may implement all or some of them. When the time of mounting DFS, e.g. dfs_mount("SD", "/", "elm", 0, 0), the specific DFS will be bound to a device. In this case the DFS "elm" (FatFs) bound to device "SD".

A standard RT-Thread device provides the following interface:

/* operations set for device object */
struct rt_device_ops
{
   /* common device interface */
   rt_err_t  (*init)   (rt_device_t dev);
   rt_err_t  (*open)   (rt_device_t dev, rt_uint16_t oflag);
   rt_err_t  (*close)  (rt_device_t dev);
   rt_size_t (*read)   (rt_device_t dev, rt_off_t pos, void *buffer, rt_size_t size);
   rt_size_t (*write)  (rt_device_t dev, rt_off_t pos, const void *buffer, rt_size_t size);
   rt_err_t  (*control)(rt_device_t dev, int cmd, void *args);
};

A specific device may implement all or none (set function pointer to NULL) of them.

In the library, the device named "SD" implements SD card access related functions, which is involved a lower level device named "SPI1" in case of MKRZERO board. And the SPI device finally involved Arduino SPI library.

Data Logger Example

The Arduino SD library provides an example named "Datalogger". The same example is available also in RT-Thread library as listed in Code section below.

The difference is that, in the example code below there is one second interval between sampling and only do it for 10 times.

You may be already noticed the comment above open() function. To open an existing file and remove all its content can be done simply by replace O_APPEND with O_TRUNC flag.

When running the example below with MKRZERO board, you may observe the following output from Serial Monitor:

\ | /
- RT -     Thread Operating System
/ | \     4.0.1 build Apr 17 2019
2006 - 2019 Copyright by rt-thread team
+ Mount SD to "/"
416,347,312
finsh />436,369,335
442,375,340
449,376,338
449,375,346
429,374,341
447,369,342
449,363,338
426,363,334
419,353,327

Manipulation with Shell

The real advantage of using RT-Thread library is that it enables you to manipulate files with (FinSH) shell commands.

In Serial Monitororother serial terminal tools, enter ls() command will show the list of files in current directory (in this case "/") as below.

ls()
Directory /:
DATALOG.TXT         240
HI_UTF8.TXT         35
A_REAL~1.TXT        22
       0, 0x00000000 

The number behind the file name is the file size in bytes. In the screenshot above, the size of "datalog.txt" is 240-byte because I run the example twice.

Enter cat("datalog.txt") command will show the content of "datalog.txt" to confirm that there are 20 records.

finsh />cat("datalog.txt")
464,358,333
464,368,336
480,381,354
447,364,346
443,363,340
441,365,343
463,371,345
467,374,313
447,364,345
465,369,346
416,347,312
436,369,335
442,375,340
449,376,338
449,375,346
429,374,341
447,369,342
449,363,338
426,363,334
419,353,327
       0, 0x00000000

There are also copy() and rm() commands.

finsh />copy("datalog.txt", "copy.txt")
       0, 0x00000000
finsh />ls()
Directory /:
COPY.TXT            240
DATALOG.TXT         240
HI_UTF8.TXT         35
A_REAL~1.TXT        22
       0, 0x00000000
finsh />rm("copy.txt")
       0, 0x00000000
finsh />ls()
Directory /:
DATALOG.TXT         240
HI_UTF8.TXT         35
A_REAL~1.TXT        22
       0, 0x00000000

To list all the available commands, enter list().

finsh />list()
--Function List:
hello            -- say hello world
version          -- show RT-Thread version information
list             -- list available commands
list_mem         -- list memory usage information
list_thread      -- list thread
list_sem         -- list semaphore in system
list_mutex       -- list mutex in system
list_event       -- list event in system
list_mb          -- list mail box in system
list_mq          -- list message queue in system
list_memp        -- list memory pool in system
list_timer       -- list timer in system
list_dev         -- list device in system
mkfs             -- make a file system
df               -- get disk free
mkdir            -- create a directory
cd               -- change current working directory
ls               -- list directory contents
rm               -- remove files or directories
cat              -- print file content
copy             -- copy file or dir
list_sd          -- show SD information
--Variable List:
dummy            -- dummy variable for finsh
       0, 0x00000000

ExFAT, LFN and Non-English Characters Support

ExFAT, LFN (long filename) and non-English characters support are not enabled by default (to make the example smaller). Turning on the following config in "rtconfig.h" (located in RT-Thread library directory) to enable those features.

#define RT_DFS_ELM_USE_EXFAT
#define RT_DFS_ELM_USE_LFN              (2)
#define RT_DFS_ELM_MAX_LFN              (255)
#define RT_DFS_ELM_CODE_PAGE            936

RT_DFS_ELM_MAX_LFN represents the maximum length of file name and can be in range of 12 to 255.

RT_DFS_ELM_CODE_PAGE is set to 437 for US by default, changed to 936 will enable simplified Chinese support as shown below.

finsh />ls()
Directory /:
DATALOG.TXT         240
hi_utf8.txt         35
a_really_long_file_name.txt22
      0, 0x00000000
finsh />cat("hi_utf8.txt")
Hello, world!
世界，你好！	0, 0x00000000

Next Steps

• Multitasking on Arduino

• RT-Thread Primer (coming soon)