Allow extern getFd() to get file descriptors from Java for Kiwix Android

[MohitMaliFtechiz]

We have an issue in Android directly accessing the fd via file path on Android 11 and above without MANAGE_EXTERNAL_PERMISSION(Which is special permission in Android and Play Store does not allow us to take this permission).

Archive and SuggestionSearcher need the file path to work properly with the Xapian index as per conversation with @mgautierfr in kiwix/kiwix-android#3636. See kiwix/kiwix-android#3636 (comment) for more details.

So to solve this issue can we open a new fd from the existing fd via fileInputStream API and use it for Xapian without affecting the existing one? Somehow this is possible at the libzim level. Or if you think of any better suggestion that can achieve this with existing fd would be welcome.

[kelson42]

@mgautierfr Any feedback here?

[kelson42]

We have an issue in Android directly accessing the fd via file path on Android 11 and above without MANAGE_EXTERNAL_PERMISSION(Which is special permission in Android and Play Store does not allow us to take this permission).

Phrasing seems incorrect. Correct phrasing is: Issues on Android to fully (suggestions/ft search) use ZIM files when open via fd only (so without a - ZIM - file path). In such a case Xapian indexes can not be open (as they do require a dedicated fd, which can only be properly retrieved in "Java world" from the ZIM file path).

Archive and SuggestionSearcher need the file path to work properly with the Xapian index as per conversation with @mgautierfr in kiwix/kiwix-android#3636. See kiwix/kiwix-android#3636 (comment) for more details.

So to solve this issue can we open a new fd from the existing fd via fileInputStream API and use it for Xapian without affecting the existing one? Somehow this is possible at the libzim level. Or if you think of any better suggestion that can achieve this with existing fd would be welcome.

I answer here under the control of @mgautierfr, but "yes" you can open many fd from the same file. Each fileInputStream will have its own fd, but you can open many fileInputStream for the same file and then you should have many fds.

So. here I see only one solution: the libzim should be modified to allow an "external" getFD() method which would be able to call java code to get one. This is a feature request which implies modifications here, in kiwix/java-libkiwix and kiwix/kiwix-android.

[mgautierfr]

This comment kiwix/libkiwix#1015 (comment) is interesting and must not be forgotten, and here an answer:

Opening an archive with a list of (file-path, offset, size) would be interesting/needed in itself but not enough.
We may have system (android, flatpak,...) where we cannot have full access to filesystem. In those cases, file is opened by an external tool and we only get a fd.

[veloman-yunkan]

This comment kiwix/libkiwix#1015 (comment) is interesting and must not be forgotten, and here an answer:

Opening an archive with a list of (file-path, offset, size) would be interesting/needed in itself but not enough. We may have system (android, flatpak,...) where we cannot have full access to filesystem. In those cases, file is opened by an external tool and we only get a fd.

@mgautierfr As I noted in the said comment, we convert a file descriptor to a file path (of the form /dev/fd/XXX) and actually open the file by path. What I had proposed was to move that trick out of libzim - let the users do that conversion themselves (libzim may simply provide a utility) and the public API will only accept paths.

[mgautierfr]

The problem is that the trick may be not possible at all. Moving it to the user side move the problem outside of libzim but the root issue is still there: We cannot assume we can open by path (even /dev/fd/xxx)