Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LFS Telecommand to list files / directories [Issue #7] #216

Open
wants to merge 27 commits into
base: main
Choose a base branch
from

Conversation

Saksham-P
Copy link
Contributor

Added Telecommand to create directory

  • Arg1: Directory Name

Added Telecommand to list directory entries

  • Arg1: Root Directory
  • Arg2: Offset of how many files to skip at the beginning (including . and ..)
  • Arg3: Number of files to display in the listing

Saksham-P and others added 17 commits October 2, 2024 13:27
Replaced all previous code for FLASH driver for NAND flash module.
Updated littlefs_helper.c configuration to use NAND memory module configuration values
Made changes to flash to handle full page reading and writing, added flash telecommand that unblocks block locks.
Minor changes to flash read hex telecommand for testing, added docstring for FLASH_reset function and telecommand.
Removed all debugging printing lines to prepare for merging. Added new timeout variable for handling HAL_SPI transmits of 2176 bytes.
Removed the telecommand unblock_block_locks and added automatic execution during write functions, minor changes to other flash telecommands.
Fixed issue with improper page addressing where littlefs sent page address in bytes rather than the actual page address.
Removed unnecessary lines of code and added additional helpful comments based on PR comments
Current version of this telecommand doesn't store the directory information anywhere yet, only sends it through UART through debugging.
Added a telecommand for creating a directory and updated warning if trying to create a directory that already exists
@Saksham-P Saksham-P linked an issue Oct 20, 2024 that may be closed by this pull request
@Saksham-P Saksham-P changed the title LFS Telecommand to list files / directories [Issue #7](https://github.com/CalgaryToSpace/CTS-SAT-1-OBC-Firmware/issues/7) LFS Telecommand to list files / directories [Issue #7] Oct 20, 2024
@Saksham-P Saksham-P requested a review from NuclearTea October 20, 2024 00:04
@Saksham-P
Copy link
Contributor Author

The implementation for the listing directory telecommand works as intended, although I just want to make sure if the implementation is as optimized and efficient as it can be.

@Saksham-P Saksham-P self-assigned this Oct 20, 2024
@Saksham-P Saksham-P marked this pull request as ready for review October 22, 2024 03:39
Copy link
Contributor

@NuclearTea NuclearTea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work Overall, just some considerations.

firmware/Core/Src/littlefs/littlefs_helper.c Outdated Show resolved Hide resolved
firmware/Core/Src/littlefs/littlefs_helper.c Show resolved Hide resolved
@NuclearTea
Copy link
Contributor

@Saksham-P @KaleF07 Wondering if the comments made sense?

@KaleF07
Copy link
Contributor

KaleF07 commented Oct 24, 2024

@Saksham-P @KaleF07 Wondering if the comments made sense?

Saksham was mostly the one working on this issue, some of my commits are from previous LFS issues. I will let him respond to the comments as I have not looked at the code yet

Added Break statement within while loop if error reading a directory entry, and return the error after closing the directory
@Saksham-P
Copy link
Contributor Author

I updated the code, and didn't change the loop, but just checked if we read dir gave an error, we break instead of returning the error, and then return the error after closing the directory.

Copy link
Contributor

@NuclearTea NuclearTea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Just need to confirm whether adding the delete file TCMD to this PR makes sense. Otherwise, great job!

DEBUG_uart_print_str("Error reading directory contents.\n");
return read_dir_result;
if (read_dir_result >= 0) {
DEBUG_uart_print_str("Successfully Listed Directory Contents.\n");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small thing here, this if statement seems redundant, as you could just put this debug_uart_print_str in the below else statement.

if (read_dir_result < 0) {
return read_dir_resust;
} else {
// debug statement
return 0;
}

firmware/Core/Src/littlefs/littlefs_helper.c Show resolved Hide resolved
firmware/Core/Src/telecommands/lfs_telecommand_defs.c Outdated Show resolved Hide resolved
firmware/Core/Src/telecommands/lfs_telecommand_defs.c Outdated Show resolved Hide resolved
All functions have some coverage of LOG_message, some don't have it entirely as they are needed to print useful information to the serial
@Saksham-P
Copy link
Contributor Author

I've replaced as many debug statements to LOG_message statements, there are still a few that I intentionally left in the code as they are used to send usefull information to the Serial, for example when listing the directories to the serial, it uses Debug_UART.

I've also left the LFS_delete_file out of this commit, as I saw Kale is already working on that issue for now, I will catch up with him to make sure we are on the same page once going through that.

@DeflateAwning DeflateAwning added the pr-please-review Indicator for someone to review a PR label Nov 16, 2024
@Makiv1
Copy link
Contributor

Makiv1 commented Nov 17, 2024

As per @NuclearTea 's request, I have tested this branch and the code seems to be working correctly.
First I ran fs_list_directory(,0,10) which returned the following:

========================= UART Telecommand Received =========================
CTS1+fs_list_directory(,0,10)!

=========================
Parsed telecommand (len=32): 'CTS1+fs_list_directory(,0,10)!
'
0000000000000+0000233437_N [TELECOMMAND:NORMAL]: Executing telecommand from agenda slot 0, sent at tssent=0, scheduled for tsexec=0.
========================= Executing telecommand 'fs_list_directory'=========================
Name 	 bytes
.
..
lifecycle

Successfully Listed Directory Contents.
0000000000000+0000233608_N [LFS:NORMAL]: Successfully listed contents from directory: 
========================= Response (duration=158ms, err=0) =========================

===========================================================================

This lines up with my expectation as I have created a directory called lifecycle while working on another issue.

Then running fs_make_directory(test_dir) twice gives the following:

Parsed telecommand (len=35): 'CTS1+fs_make_directory(test_dir)!
'
0000000000000+0000524074_N [TELECOMMAND:NORMAL]: Executing telecommand from agenda slot 0, sent at tssent=0, scheduled for tsexec=0.
========================= Executing telecommand 'fs_make_directory'=========================
0000000000000+0000524580_N [LFS:NORMAL]: Successfully created directory: test_dir
========================= Response (duration=494ms, err=0) =========================

===========================================================================
FrontierSat time: 19700101T00:08:44.603, Uptime: 524603 ms
FrontierSat time: 19700101T00:08:45.598, Uptime: 525598 ms
FrontierSat time: 19700101T00:08:46.593, Uptime: 526593 ms
FrontierSat time: 19700101T00:08:47.588, Uptime: 527588 ms
========================= UART Telecommand Received =========================
CTS1+fs_make_directory(test_dir)!

=========================
Parsed telecommand (len=35): 'CTS1+fs_make_directory(test_dir)!
'
0000000000000+0000528223_N [TELECOMMAND:NORMAL]: Executing telecommand from agenda slot 0, sent at tssent=0, scheduled for tsexec=0.
========================= Executing telecommand 'fs_make_directory'=========================
0000000000000+0000528353_N [LFS:WARNING]: Directory test_dir already exists.
========================= Response (duration=117ms, err=1 !!!!!! ERROR !!!!!!) =========================
LittleFS Make Directory Error: -17

===========================================================================

The first time, directory creation succeeds and is confirmed by calling fs_list_directory(,0,10). The second time it fails as the directory already exists. This seems correct.

Finally by calling fs_list_directory(lifecycle,0,10) we list the folders inside of this directory and get:

========================= UART Telecommand Received =========================
CTS1+fs_list_directory(lifecycle,0,10)!

=========================
Parsed telecommand (len=41): 'CTS1+fs_list_directory(lifecycle,0,10)!
'
0000000000000+0000875407_N [TELECOMMAND:NORMAL]: Executing telecommand from agenda slot 0, sent at tssent=0, scheduled for tsexec=0.
========================= Executing telecommand 'fs_list_directory'=========================
Name 	 bytes
.
..
deploy_antenna_on_boot_enabled.bool	1 bytes

Successfully Listed Directory Contents.
0000000000000+0000875775_N [LFS:NORMAL]: Successfully listed contents from directory: lifecycle
========================= Response (duration=357ms, err=0) =========================

===========================================================================

This also seems correct as I have created deploy_antenna_on_boot_enabled.bool while working on another issue.

I have also tried listing a directory which didn't exist: This Failed with error -2.

Some Closing Thoughts

  • code seems to be working correctly when tested on some common scenarios
  • This is super useful for when working with the file system!
  • perhaps the response could be a bit more descriptive (logs could label which entries are files and which are directories. errors could be a bit more verbose, avoiding the need for error code lookups in the lfs.h file). Although the code in its current state resolves the linked issue

Great work this is super sweet! Hope this helps!

Copy link
Contributor

@NuclearTea NuclearTea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! @DeflateAwning wondering what you think. The code has been tested by @Makiv1 and has passed testing.

Copy link
Contributor

@DeflateAwning DeflateAwning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fantastic, thanks! Just a few tiny changes, but awesome overall.

In addition to the review comments, in the display, can we please suffix folders (but not files) with /. So, for example, it would look like this:

========================= Executing telecommand 'fs_list_directory'=========================
Name 	 bytes
./
../
lifecycle/
some_file.txt  12

int8_t read_dir_result = 1;
while (read_dir_result >= 0)
struct lfs_info info;
DEBUG_uart_print_str("Name \t bytes\n");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is gonna be awfully difficult to read when the satellite's in space ;)

Please use the logging function to display this info.

count--;
}

DEBUG_uart_print_str(info.name);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please switch all this to LOG_message

DEBUG_uart_print_str(" bytes");
}
DEBUG_uart_print_str("\n");
}

int8_t close_dir_result = lfs_dir_close(&LFS_filesystem, &dir);
if (close_dir_result < 0)
{
DEBUG_uart_print_str("Error closing directory.\n");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this DEBUG_uart_print when you add LOG_message calls

if (read_dir_result < 0) {
return read_dir_result;
} else {
DEBUG_uart_print_str("Successfully Listed Directory Contents.\n");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove DEBUG_uart_print_str. Please try to find all instances in the new code you're adding here.

int8_t format_result = lfs_format(&LFS_filesystem, &LFS_cfg);
if (format_result < 0) {
LOG_message(LOG_SYSTEM_LFS, LOG_SEVERITY_CRITICAL, LOG_all_sinks_except(LOG_SINK_FILE), "Error formatting FLASH memory!");
return format_result;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please convert this file's indentation to use spaces instead of tabs. There's a few places it's mixed in this repo, but in general, we prefer spaces. Ideally, never mix in the same file.

@NuclearTea NuclearTea added the blocker Other tasks are waiting on this task, but this task is not waiting on other tasks label Dec 1, 2024
@DeflateAwning DeflateAwning added pr-awaiting-changes and removed pr-please-review Indicator for someone to review a PR labels Dec 3, 2024
@Saksham-P
Copy link
Contributor Author

Update:

image
Above image displays how the directory is listed. Got rid of all Debugs in littlefs_helper.c. Removed static_config.h include as well as debug_uart.h include. Could implement sorting in the future based on a criteria.

Concern?

After pushing my changes, I realized that my local build doesn't give an error or warning when I am using %d to print uint32_t, but the build on GitHub does. (littlefs_helper.c, Line 178)

  • LOG_message(LOG_SYSTEM_LFS, LOG_SEVERITY_NORMAL, LOG_all_sinks_except(LOG_SINK_FILE), "%s\t%ld bytes", info.name, info.size);

My current arm compiler is arm-none-eabi-gcc\13.2.1-1.1.1 which may be the issue?

@DeflateAwning
Copy link
Contributor

After merging/working on the version in main, it should warn when you use an invalid string on LOG_message now. Was a recent improvement in #238

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocker Other tasks are waiting on this task, but this task is not waiting on other tasks pr-awaiting-changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Telecommand: list files in flash/LittleFS filesystem
6 participants