zvfs: improve libc FILE to integer fd abstraction

[cfriedt]

Previously, there was an implicit assumption that Zephyr's internal struct fd_entry * was synonymous with FILE * from the C library.

This is generally not the case and aliasing these two distinct types was preventing a fair bit of functionality from Just Working - namely stdio function calls like fgets() and fopen(). The problem count be seen directly when trying to use a function like zvfs_fdopen().

Instead of aliasing the two types, require that all Zephyr C libraries provide

1. FILE *z_libc_file_alloc(int fd, const char *mode), to allocate and populate the required fields of a FILE object

2. int z_libc_file_get_fd(FILE *fp), to convert a FILE* object to an integer file descriptor.

For Picolibc and Newlib-based C libraries, these functions set and get the integer file descriptor from a field of the internal FILE object representation.

For the minimal C library, these functions convert between array index and struct fd_entry *.

[cfriedt]

@tagunil / @evgeniy-paltsev - ARC mwdt might need a similar change. I would be happy to include it here if you have suggestions.

[tagunil]

@tagunil / @evgeniy-paltsev - ARC mwdt might need a similar change. I would be happy to include it here if you have suggestions.

I generally like the idea, but I need time to check if/how it would work for our legacy MW libc.

[keith-packard]

How does z_libc_file_alloc differ from the POSIX fdopen function? How does z_libc_file_get_fd differ from the POSIX fileno function?

[cfriedt]

How does z_libc_file_alloc differ from the POSIX fdopen function? How does z_libc_file_get_fd differ from the POSIX fileno function?

Mostly by name. These functions can't depend on POSIX.

[cfriedt]

@tejlmand - do you know if there are similar changes necessary for the armclang toolchain? If so, I would be happy to include them here.

[keith-packard]

How does z_libc_file_alloc differ from the POSIX fdopen function? How does z_libc_file_get_fd differ from the POSIX fileno function?

Mostly by name. These functions can't depend on POSIX.

So, the picolibc (and newlib) implementations could just be wrappers?

[cfriedt]

Mostly by name. These functions can't depend on POSIX.

So, the picolibc (and newlib) implementations could just be wrappers?

What would the picolibc and newlib implementations be wrappers around? Maybe I've misunderstood the question.

[keith-packard]

What would the picolibc and newlib implementations be wrappers around? Maybe I've misunderstood the question.

#define _POSIX_C_SOURCE 200809L
#include <stdio.h>

FILE *z_libc_file_alloc(int fd, const char *mode)
{
    return fdopen(fd, mode);
}

int z_libc_file_get_fd(FILE *f)
{
    return fileno(f);
}

Alternatively, just add fdopen and fileno to the Zephyr API and use them directly.

[cfriedt]

What would the picolibc and newlib implementations be wrappers around? Maybe I've misunderstood the question.

#define _POSIX_C_SOURCE 200809L
#include <stdio.h>

FILE *z_libc_file_alloc(int fd, const char *mode)
{
    return fdopen(fd, mode);
}

int z_libc_file_get_fd(FILE *f)
{
    return fileno(f);
}

Alternatively, just add fdopen and fileno to the Zephyr API and use them directly.

That would introduce a POSIX dependency (as well as a dependency cycle), which is what we're trying to avoid in this situation.

It could maybe work if there were similar libc-internal functions that were not aliases of fdopen() or fileno().

In other words, here, we are limited to ISO C only and non-POSIX functions.

[keith-packard]

In other words, here, we are limited to ISO C only and non-POSIX functions.

So all we need to do is expose fileno and fdopen as Zephyr functions from the C library, just as we do for other Zephyr APIs which came from POSIX, like strnlen and strtok_r? For newlib, we could use the same hack used for those? Zero code added to Zephyr seems a lot better than copying code from an external library repository.

[cfriedt]

In other words, here, we are limited to ISO C only and non-POSIX functions.

So all we need to do is expose fileno and fdopen as Zephyr functions from the C library, just as we do for other Zephyr APIs which came from POSIX, like strnlen and strtok_r?

Not exactly - strnlen() and strtok_r() have zero side-effects and do not deal with management or validation of kernel resources.

Some of the kernel side of this might eventually hop the fence to syscall neighbourhood, and it seems like there would be some dependency cycles formed by calling userspace-facing APIs. From kernel space.

For newlib, we could use the same hack used for those? Zero code added to Zephyr seems a lot better than copying code from an external library repository.

I realize it's tempting to hack things, blur the lines and take shortcuts, for the sake of size-optimization, but I feel it's the wrong approach. Especially because "Zephyr is not a POSIX OS".

I think it's ok to call functions here that are not directly part of POSIX APIs though, so maybe if there are aliases that exist, those can be used instead (even though it's technically still the same code).

Here is a question: if, for example, only _write_r() and _write() are referenced directly, does the write() symbol get discarded if write() is an alias of _write()?

[cfriedt]

I tried using the aliases as well as the POSIX symbols here, but it unfortunately introduced an infinite recursion. I also tried using the option TC_PROVIDES_POSIX_DEVICE_IO and there were unresolved symbols.

Probably the existing change should suffice, for now.

[cfriedt]

• fixed assignment of reentrant function pointers in the absence of consistent typing.