【嵌入式SD NAND】基于FATFS/Littlefs文件系統(tǒng)的日志框架實(shí)現(xiàn)
【嵌入式】基于FATFS/Littlefs文件系統(tǒng)的日志框架實(shí)現(xiàn)
1. 概述
2. 設(shè)計(jì)概要
3. 設(shè)計(jì)實(shí)現(xiàn)
3.1 初始化 `init`
3.2 日志寫(xiě)入 `write`
3.3 日志讀取 `read`
3.4 注銷 `deinit`
3.5 全部代碼匯總
4. 測(cè)試
5. 總結(jié)
那么在移植好了文件系統(tǒng)之后,我們又應(yīng)該如何應(yīng)用文件系統(tǒng)呢?
很多人會(huì)說(shuō),這個(gè)簡(jiǎn)單,就操作文件嘛!open、read、write、close不就行了嗎!當(dāng)然對(duì)于簡(jiǎn)單的使用,掌握open、read、write、close,去存儲(chǔ)一兩個(gè)文件或者從一兩個(gè)文件中簡(jiǎn)單的讀取下數(shù)據(jù)這確實(shí)沒(méi)有什么難度。但在實(shí)際應(yīng)用中,特別是產(chǎn)品開(kāi)發(fā)過(guò)程中,往往不只是簡(jiǎn)單的操作一兩個(gè)文件就可以的,如果真是這樣,那費(fèi)那么大勁移植文件系統(tǒng)多少有點(diǎn)浪費(fèi)!
在實(shí)際項(xiàng)目開(kāi)發(fā)中,往往需要依托文件系統(tǒng)操作諸多的文件,操作諸多的數(shù)據(jù)。如通過(guò)配置文件配置機(jī)器設(shè)備信息、通過(guò)升級(jí)文件進(jìn)行產(chǎn)品升級(jí)、通過(guò)存放字庫(kù)文件實(shí)現(xiàn)多語(yǔ)言支持等等,這些都是比較簡(jiǎn)單的操作,讀寫(xiě)不是很頻繁,相對(duì)來(lái)說(shuō)實(shí)現(xiàn)比較簡(jiǎn)單,還有一類需求讀寫(xiě)會(huì)相當(dāng)頻繁,且大多數(shù)產(chǎn)品內(nèi)都希望存在的,那便是日志文件,通過(guò)日志文件來(lái)記錄設(shè)備的運(yùn)行數(shù)據(jù)。日志文件不同于其他功能,其往往需要具備幾個(gè)基本特性需求:
單個(gè)文件大小限制
日志總大小空間占用限制
自動(dòng)循環(huán)覆蓋
網(wǎng)上也有一些開(kāi)源的日志框架,如 Log4j,不過(guò)大都是基于 java / c ++ 實(shí)現(xiàn)的,雖然功能比較全面,但比較繁雜,且也難以移植應(yīng)用于嵌入式開(kāi)發(fā)中。而在嵌入式開(kāi)發(fā)中,可能也受限于資源限制,并沒(méi)有發(fā)現(xiàn)不錯(cuò)的基于文件系統(tǒng)的開(kāi)源日志框架(至少博主目前沒(méi)有發(fā)現(xiàn),有的話歡迎大家評(píng)論區(qū)討論 ),對(duì)于如何實(shí)現(xiàn)一個(gè)日志框架很多人一下子可能沒(méi)有頭緒,綜上,本文將分享一個(gè)簡(jiǎn)單的基于文件系統(tǒng)的日志程序以供大家思考。
2. 設(shè)計(jì)概要我們需要實(shí)現(xiàn)的日志模塊的核心需求為:
單個(gè)文件大小限制
日志總大小空間占用限制
自動(dòng)循環(huán)覆蓋
對(duì)于一個(gè)模塊,對(duì)外僅需提供其操作的接口即可,內(nèi)部的算法實(shí)現(xiàn)均無(wú)需對(duì)外開(kāi)放,而對(duì)于此日志模塊,對(duì)外只需提供基本的以下四個(gè)接口即可:
初始化 init
寫(xiě)日志 write
讀日志 read
注銷 deinit
關(guān)于日志存儲(chǔ)的核心思想如下:
寫(xiě)數(shù)據(jù)之前先判斷當(dāng)前操作的文件是否超出單個(gè)文件大小限制,如超出大小限制則進(jìn)行日志輪轉(zhuǎn),創(chuàng)建一個(gè)新的日志文件并判斷日志文件總大小是否超出限制,如果超出則刪除最早的那一份日志文件
關(guān)于日志存儲(chǔ)的詳細(xì)設(shè)計(jì)如下:
日志文件格式采用:<filename>.log ,當(dāng)當(dāng)前文件達(dá)到單個(gè)文件大小之后,進(jìn)行文件輪轉(zhuǎn);
假定當(dāng)前限制日志每個(gè)日志文件大小為2048Byte,最多存儲(chǔ)10個(gè)文件;
當(dāng)當(dāng)前文件達(dá)到單個(gè)文件大小之后,迭代修改日志文件名:
<filename>.log -> <filename>.log.0
<filename>.log.0 -> <filename>.log.1
<filename>.log.1 -> <filename>.log.2
…
<filename>.log.8 -> <filename>.log.9
刪除 <filename>.log.9
ps:注意實(shí)際代碼操作的時(shí)候,文件修改順序是反過(guò)來(lái)的,也就是先 刪除 <filename>.log.9 再將<filename>.log.8 -> <filename>.log.9
3. 設(shè)計(jì)實(shí)現(xiàn)3.1 初始化 init初始化部分代碼主要功能是完成日志數(shù)據(jù)結(jié)構(gòu)體的構(gòu)造,并通過(guò)傳入?yún)?shù) log_file_cfg_t cfg 配置日志文件的配置信息,如單個(gè)日志文件大小、日志文件名、最多存放的日志文件數(shù)等內(nèi)容,日志模塊初始化部分代碼如下:
log_file_t log_storage_init(log_file_cfg_t cfg)
{
log_file_t log = NULL;
log_file_cfg_t log_cfg = NULL;
log_file_read_t log_read = NULL;
log = (log_file_t)malloc(sizeof(struct log_file_config));
if (log == NULL)
goto error;
log_cfg = (log_file_cfg_t)malloc(sizeof(struct log_file_config));
if (log_cfg == NULL) {
free(log);
log = NULL;
goto error;
}
log_read = (log_file_read_t)malloc(sizeof(struct log_file_read));
if (log_read == NULL) {
free(log);
log = NULL;
free(log_cfg);
log_cfg = NULL;
goto error;
}
memcpy(log_cfg, cfg, sizeof(struct log_file_config));
log_read->rotate_index = 0;
log_read->file_offset = 0;
log->cfg = log_cfg;
log->read = log_read;
log->user_data = NULL;
error:
return log;
}
日志寫(xiě)入部分代碼主要分為兩大部分,一部分是正常寫(xiě)入,另一部分是文件輪轉(zhuǎn);當(dāng)寫(xiě)入的文件超過(guò)單個(gè)文件大小限制時(shí),即會(huì)觸發(fā)文件輪轉(zhuǎn)操作。
在文件輪轉(zhuǎn)中,主要做的是:創(chuàng)建一個(gè)新的日志文件并判斷日志文件總大小是否超出限制,如果超出則刪除最早的那一份日志文件,具體設(shè)計(jì)細(xì)節(jié)可參考上文設(shè)計(jì)概要中的詳細(xì)設(shè)計(jì)部分。
實(shí)現(xiàn)代碼如下:
static int log_rotate(log_file_t log)
{
int ret = 0;
FILE *fp;
char old_filename[NAME_MAX + 10] = {0};
char new_filename[NAME_MAX + 10] = {0};
for (int i = log->cfg->rotate_num; i > 0; i --) {
memset(old_filename, 0, sizeof(old_filename));
memset(new_filename, 0, sizeof(new_filename));
snprintf(old_filename, sizeof(old_filename), i ? "%s_%d.log" : "%s.log", log->cfg->filename, i - 1);
snprintf(new_filename, sizeof(new_filename), "%s_%d.log", log->cfg->filename, i);
printf("old:%s new:%s\n", old_filename, new_filename);
if ((fp = fopen(new_filename, "r")) != NULL) {
if (fclose(fp) != 0) {
ret = -1;
goto error;
}
if (remove(new_filename) != 0) {
ret = -2;
goto error;
}
}
if ((fp = fopen(old_filename, "r")) != NULL) {
if (fclose(fp) != 0) {
ret = -1;
goto error;
}
if (rename(old_filename, new_filename) != 0) {
ret = -3;
goto error;
}
}
}
error:
return ret;
}
int log_storage_write(log_file_t log, const unsigned char *buf, unsigned int len)
{
int ret = 0;
int file_size = 0;
char full_filename[NAME_MAX + 5] = {0};
FILE *fp = NULL;
if (log == NULL || log->cfg == NULL || log->read == NULL || buf == NULL || len == 0) {
ret = -1;
goto param_error;
}
snprintf(full_filename, sizeof(full_filename), "%s.log", log->cfg->filename);
printf("fullfilename:%s\n", full_filename);
log_file_lock();
fp = fopen(full_filename, "a+b");
if (fp == NULL) {
ret = -2;
goto error;
}
fseek(fp, 0L, SEEK_END);
file_size = ftell(fp);
printf("file_size:%d\n", file_size);
if ((file_size + len) > log->cfg->max_size) {
if (fclose(fp) != 0) {
ret = -3;
goto error;
}
int j = 0;
j = log_rotate(log);
printf("log rotate:%d\n", j);
fp = fopen(full_filename, "a+b");
if (fp == NULL) {
ret = -2;
goto error;
}
}
if (fwrite(buf, len, 1, fp) != 1) {
fclose(fp);
ret = -4;
goto error;
}
error:
if (fp != NULL) {
if (fclose(fp) != 0) {
ret = -3;
goto error;
}
}
log_file_unlock();
param_error:
return ret;
}
此處日志讀取在本文主題中非重點(diǎn)設(shè)計(jì)內(nèi)容,因此此處做簡(jiǎn)單設(shè)計(jì),通過(guò)傳入?yún)?shù)判斷應(yīng)該讀取哪一份文件之后進(jìn)行直接讀取。設(shè)計(jì)代碼如下:
int log_storage_read(log_file_t log, unsigned int rotate_num, unsigned char *buf, unsigned int *len)
{
int ret = 0;
int file_size = 0;
char full_filename[NAME_MAX + 5] = {0};
FILE *fp = NULL;
if (log == NULL || log->cfg == NULL || log->read == NULL || buf == NULL || len == 0) {
ret = -1;
goto param_error;
}
if (rotate_num == 0)
snprintf(full_filename, sizeof(full_filename), "%s.log", log->cfg->filename);
else
snprintf(full_filename, sizeof(full_filename), "%s.log.%d", log->cfg->filename, rotate_num);
log_file_lock();
fp = fopen(full_filename, "a+b");
if (fp == NULL) {
ret = -2;
goto error;
}
/* check file length. */
fseek(fp, 0L, SEEK_END);
file_size = ftell(fp);
printf("file_size:%d\n", file_size);
if (file_size < *len)
*len = file_size;
fseek(fp, 0L, SEEK_SET);
if (fread(buf, *len, 1, fp) != 1) {
ret = -3;
fclose(fp);
goto error;
}
error:
if (fp != NULL) {
if (fclose(fp) != 0) {
ret = -4;
goto error;
}
}
log_file_unlock();
param_error:
return ret;
}
注銷的主要功能是將我們?cè)?nbsp;init 時(shí)創(chuàng)建的數(shù)據(jù)結(jié)構(gòu)進(jìn)行回收,如果模塊內(nèi)部有功能處于打開(kāi)裝填,也應(yīng)關(guān)閉模塊的功能,此處我們僅需對(duì) init 時(shí)創(chuàng)建的 log_file_t log 數(shù)據(jù)結(jié)構(gòu)體進(jìn)行注銷、內(nèi)存回收即可,具體代碼實(shí)現(xiàn)如下:
int log_storage_deinit(log_file_t log)
{
if (log == NULL)
return -1;
if (log->cfg != NULL)
free(log->cfg);
if (log->read != NULL)
free(log->read);
if (log->user_data != NULL)
free(log->user_data);
free(log);
return 0;
}
日志模塊內(nèi)核頭文件:simple_storage.h
#ifndef __SIMPLE_STORAGE_H__
#define __SIMPLE_STORAGE_H__
#define NAME_MAX 40
struct log_file_config {
const char filename[NAME_MAX]; /* Filename of this type. */
int max_size; /* single file max size. */
int rotate_num; /* The number of files that support rotate. */
};
typedef struct log_file_config* log_file_cfg_t;
struct log_file_read {
int rotate_index; /* The rotate file index. */
int file_offset; /* The offset of the currently read file. */
};
typedef struct log_file_read* log_file_read_t;
struct log_file {
log_file_cfg_t cfg;
log_file_read_t read;
void *user_data;
};
typedef struct log_file* log_file_t;
log_file_t log_storage_init(log_file_cfg_t cfg);
int log_storage_write(log_file_t log, const unsigned char *buf, unsigned int len);
int log_storage_read(log_file_t log, unsigned int rotate_num, unsigned char *buf, unsigned int *len);
int log_storage_deinit(log_file_t log);
#endif /* __SIMPLE_STORAGE_H__ */
日志模塊內(nèi)核文件: simple_storage.c
#include "simple_storage.h"
#include "simple_storage_port.h"
#include <stdio.h>
#include <string.h>
log_file_t log_storage_init(log_file_cfg_t cfg)
{
log_file_t log = NULL;
log_file_cfg_t log_cfg = NULL;
log_file_read_t log_read = NULL;
log = (log_file_t)malloc(sizeof(struct log_file_config));
if (log == NULL)
goto error;
log_cfg = (log_file_cfg_t)malloc(sizeof(struct log_file_config));
if (log_cfg == NULL) {
free(log);
log = NULL;
goto error;
}
log_read = (log_file_read_t)malloc(sizeof(struct log_file_read));
if (log_read == NULL) {
free(log);
log = NULL;
free(log_cfg);
log_cfg = NULL;
goto error;
}
memcpy(log_cfg, cfg, sizeof(struct log_file_config));
log_read->rotate_index = 0;
log_read->file_offset = 0;
log->cfg = log_cfg;
log->read = log_read;
log->user_data = NULL;
error:
return log;
}
static int log_rotate(log_file_t log)
{
int ret = 0;
FILE *fp;
char old_filename[NAME_MAX + 10] = {0};
char new_filename[NAME_MAX + 10] = {0};
for (int i = log->cfg->rotate_num; i > 0; i --) {
memset(old_filename, 0, sizeof(old_filename));
memset(new_filename, 0, sizeof(new_filename));
snprintf(old_filename, sizeof(old_filename), i ? "%s_%d.log" : "%s.log", log->cfg->filename, i - 1);
snprintf(new_filename, sizeof(new_filename), "%s_%d.log", log->cfg->filename, i);
printf("old:%s new:%s\n", old_filename, new_filename);
if ((fp = fopen(new_filename, "r")) != NULL) {
if (fclose(fp) != 0) {
ret = -1;
goto error;
}
if (remove(new_filename) != 0) {
ret = -2;
goto error;
}
}
if ((fp = fopen(old_filename, "r")) != NULL) {
if (fclose(fp) != 0) {
ret = -1;
goto error;
}
if (rename(old_filename, new_filename) != 0) {
ret = -3;
goto error;
}
}
}
error:
return ret;
}
int log_storage_write(log_file_t log, const unsigned char *buf, unsigned int len)
{
int ret = 0;
int file_size = 0;
char full_filename[NAME_MAX + 5] = {0};
FILE *fp = NULL;
if (log == NULL || log->cfg == NULL || log->read == NULL || buf == NULL || len == 0) {
ret = -1;
goto param_error;
}
snprintf(full_filename, sizeof(full_filename), "%s.log", log->cfg->filename);
printf("fullfilename:%s\n", full_filename);
log_file_lock();
fp = fopen(full_filename, "a+b");
if (fp == NULL) {
ret = -2;
goto error;
}
fseek(fp, 0L, SEEK_END);
file_size = ftell(fp);
printf("file_size:%d\n", file_size);
if ((file_size + len) > log->cfg->max_size) {
if (fclose(fp) != 0) {
ret = -3;
goto error;
}
int j = 0;
j = log_rotate(log);
printf("log rotate:%d\n", j);
fp = fopen(full_filename, "a+b");
if (fp == NULL) {
ret = -2;
goto error;
}
}
if (fwrite(buf, len, 1, fp) != 1) {
fclose(fp);
ret = -4;
goto error;
}
error:
if (fp != NULL) {
if (fclose(fp) != 0) {
//TODO: check the amount of disk space, delete if there is not enough space.
ret = -3;
goto error;
}
}
log_file_unlock();
param_error:
return ret;
}
int log_storage_read(log_file_t log, unsigned int rotate_num, unsigned char *buf, unsigned int *len)
{
int ret = 0;
int file_size = 0;
char full_filename[NAME_MAX + 5] = {0};
FILE *fp = NULL;
if (log == NULL || log->cfg == NULL || log->read == NULL || buf == NULL || len == 0) {
ret = -1;
goto param_error;
}
if (rotate_num == 0)
snprintf(full_filename, sizeof(full_filename), "%s.log", log->cfg->filename);
else
snprintf(full_filename, sizeof(full_filename), "%s.log.%d", log->cfg->filename, rotate_num);
log_file_lock();
fp = fopen(full_filename, "a+b");
if (fp == NULL) {
ret = -2;
goto error;
}
/* check file length. */
fseek(fp, 0L, SEEK_END);
file_size = ftell(fp);
printf("file_size:%d\n", file_size);
if (file_size < *len)
*len = file_size;
fseek(fp, 0L, SEEK_SET);
if (fread(buf, *len, 1, fp) != 1) {
ret = -3;
fclose(fp);
goto error;
}
error:
if (fp != NULL) {
if (fclose(fp) != 0) {
ret = -4;
goto error;
}
}
log_file_unlock();
param_error:
return ret;
}
int log_storage_deinit(log_file_t log)
{
if (log == NULL)
return -1;
if (log->cfg != NULL)
free(log->cfg);
if (log->read != NULL)
free(log->read);
if (log->user_data != NULL)
free(log->user_data);
free(log);
return 0;
}
在日志模塊源文件的代碼中,我們可以看到實(shí)際每次操作文件的時(shí)候,都有調(diào)用一個(gè)函數(shù)鎖操作,考慮到不同平臺(tái)的鎖操作實(shí)現(xiàn)不一樣,因此將此部分通過(guò)函數(shù)導(dǎo)出來(lái),放置在模塊的端口文件中。不同的平臺(tái)、系統(tǒng)根據(jù)各自的平臺(tái)和系統(tǒng)的情況進(jìn)行實(shí)現(xiàn),如像裸機(jī)編程這類不需要進(jìn)行鎖操作的不進(jìn)行函數(shù)實(shí)現(xiàn)即可。
日志模塊端口頭文件:simple_storage_port.c
#ifndef __SIMPLE_STORAGE_PORT_H__
#define __SIMPLE_STORAGE_PORT_H__
int log_file_init(void);
int log_file_lock(void);
int log_file_unlock(void);
#endif /* __SIMPLE_STORAGE_PORT_H__ */
日志模塊端口源文件:simple_storage_port.h
#include "simple_storage_port.h"
int log_file_init(void)
{
return 0;
}
int log_file_lock(void)
{
return 0;
}
int log_file_unlock(void)
{
return 0;
}
將以上代碼進(jìn)行運(yùn)行測(cè)試,硬件平臺(tái)如下:
控制器: stm32f103vet6,野火指南者開(kāi)發(fā)板
存儲(chǔ)芯片: CS創(chuàng)世 SD nand,型號(hào):CSNP4GCR01-AMW
文件系統(tǒng): FATFS,注意此日志不受文件系統(tǒng)限制
操作系統(tǒng): RT-Thread,此模塊與操作系統(tǒng)無(wú)關(guān),此處只是方便使用故自行移植了rtthread
應(yīng)用層代碼如下:
int main(void)
{
/* Reset of all peripherals, Initializes the Flash interface and the Systick. */
HAL_Init();
/* USER CODE BEGIN Init */
/* USER CODE END Init */
/* Configure the system clock */
SystemClock_Config();
/* USER CODE BEGIN SysInit */
/* USER CODE END SysInit */
/* Initialize all configured peripherals */
MX_GPIO_Init();
MX_SDIO_SD_Init();
MX_USART1_UART_Init();
MX_FATFS_Init();
/* USER CODE BEGIN 2 */
struct log_file_config log_cfg = {
.filename = "test",
.max_size = 2048,
.rotate_num = 10,
};
log_file_t log = NULL;
log = log_storage_init(&log_cfg);
if (log == NULL)
return;
/* USER CODE END 2 */
/* Infinite loop */
/* USER CODE BEGIN WHILE */
unsigned char buf[2048] = {0};
int len = 0;
while (1) {
// ... 省略用戶代碼
/* 寫(xiě)入測(cè)試 */
for (int i = 0; i < 2048; i++) {
log_storage_write(log, "hello world", sizeof("hello world"));
rt_thread_mdelay(100);
}
/* 讀取測(cè)試 */
len = sizeof(buf);
memset(buf, 0, sizeof(buf));
log_storage_read(log, 1, buf, &len);
for (int i = 0; i < len; i ++)
rt_kprintf("%c", buf[i]);
rt_thread_mdelay(1000);
}
}
測(cè)試結(jié)果如下:
msg> hello worldhello world hello world hello world hello world hello world hello world hello world hello world ...省略
msh > ls
test.log 2046
test.log.0 2046
test.log.1 2046
test.log.2 2046
test.log.3 2046
test.log.4 2046
5. 總結(jié)
綜上便是基于文件系統(tǒng)的簡(jiǎn)易日志模塊設(shè)計(jì)的全部?jī)?nèi)容了,雖然簡(jiǎn)陋了點(diǎn),但相信對(duì)于大部分沒(méi)有接觸過(guò)日志系統(tǒng)設(shè)計(jì)的人來(lái)說(shuō)提供了很好的一條設(shè)計(jì)思路。
也正因?yàn)楹?jiǎn)易,給大家對(duì)于日志系統(tǒng)設(shè)計(jì)的優(yōu)化留足了大量的優(yōu)化空間。比如:
文件輪轉(zhuǎn)的時(shí)候需要對(duì)每個(gè)文件的文件名進(jìn)行修改,是否可以有更好的方式不用每個(gè)文件都修改呢?
文件名的設(shè)計(jì)是不方便閱讀的,是否可以引入時(shí)間參數(shù)?
文件名設(shè)計(jì)如何引入了時(shí)間參數(shù),當(dāng)設(shè)備RTC備用電池掉電的時(shí)候又如何保證文件不會(huì)被錯(cuò)誤覆蓋?
文件的讀取顯然優(yōu)化空間更大,實(shí)際上用戶不應(yīng)該傳入rotate_num 參數(shù),因?yàn)檫@是模塊內(nèi)部的參數(shù),用戶不可感知的
文件讀取如何做到分多次讀取一個(gè)文件的內(nèi)容,且不會(huì)重復(fù),是順序讀???
等等,以上只是我簡(jiǎn)單想到的幾點(diǎn)內(nèi)容,大家不妨思考下如何實(shí)現(xiàn)方案更好呢?當(dāng)然又還有哪些需求是需要引入的呢,也歡迎大家在評(píng)論區(qū)留言,關(guān)注我,后續(xù)抽時(shí)間再分享下改良版日志系統(tǒng)?。?!
*博客內(nèi)容為網(wǎng)友個(gè)人發(fā)布,僅代表博主個(gè)人觀點(diǎn),如有侵權(quán)請(qǐng)聯(lián)系工作人員刪除。