Revisit the Android W^X problem

[iamahuman]

Feature description
Termux should circumvent Play Store policy of restricting execution of arbitrary code from third parties, by imitating what Google Chrome does. Bundling packages into APKs is certainly not the way to go.

1. Consult Firefox and Chrome as to how it executes external code (e.g. JavaScript) while still managing to comply with Play Store policy.
2. Run user code in a sandbox (isolated_app?), emulating forbidden system calls as needed. (Note that we already do this with execve to handle #!/usr/bin/... shebangs).
   • Additional permissions to control system can be granted through some kind of broker, which user code can communicate to via IPC.
   • Don't rely exclusively on ptrace through; rather, approach the problem with a mix of ptrace+seccomp+recompilation.
3. Circumvent W^X restriction by using a custom binary loader (instead of e.g. /system/bin/linker).

Reference implementation

N/A

Related

• Play Store now forbids downloading executable code #655
• No more exec from data folder on targetAPI >= Android Q #1072

[iamahuman]

On the one hand, we need to think about how we can still give the sandboxed code access to storage files. /storage is implemented via FUSE (filesystem in userspace), so perhaps we can just steal some hints from there.

On the other hand, we can use this opportunity to explore support for containers and containerized applications (e.g. Docker, Flatpak, ...), possibly after figuring out how to resolve complications made by the SE Android policies.

[Grimler91]

Termux should circumvent Play Store policy of restricting execution of arbitrary code

We would definitely risk being suddenly removed from play store if we try to circumvent the policy, but as you said I guess it is theoretically possible to comply in the same way as browsers with javascript.

I think what we need is some people to feel strong enough about this "issue" to actually develop and experiment with possible solutions. The apk route has been shown to be possible, but needs much more work.

Alternative ideas are great, but an idea is not really worth anything before it is implemented (as business people like to say).
Leaving the issue open for discussion, but please understand that basically only xeffyr and fornwall has looked into android-11 support so far, and I doubt they would take the lead to implement new ideas now that we have a proof-of-concept of the apk solution.

[RalfWerner]

Bundling packages into APKs is certainly not the way to go.

Yes and No! I studied the use of a boot loops concept and made good experiences. pkg will be preserved but the *.apk will be individualized/customized. Thus, target=29+ and sdcard access are no more problems.

[creepy-pasta101]

I agree with @iamahuman .. @xeffyr If we just use proot to solve this, it may become easier.. but it has its own problems...

1- This is a known fact- performance issues...

2- Usual termux users love the fact that these packages are run directly under the android system.. That's why most users ask for packages natively that adjust without them with proot
The presence of proot would render having no difference between termux's packages and other proot distros

3- Any bugs or issues with proot upstream would also be an issue with Termux....

4- The Build script would have to be modified.. I don't know to what extent....

The best approach, I think would be to modify termux-exec, which currently is a wrapper only for shebangs around execv(). This would mean more work.. but has its benefits...
This would mean that termux-exec, after modification, would be included with termux-app, like the bootstrap packages.. but would be executed by the app , which then executes these packages....

[ghost]

@suhan-paradkar We have only 3 variants of resolving the issue:

1. Leaving everything as-is and continuting with SDK 28.
2. Packaging everything as "native libs" of APK. - Look branch android-10.
3. Using a custom ELF loader: proot or something else.
4. System emulation.

Variant 1 is current one, 3 - against Play Store policy.

This would mean that termux-exec, after modification, would be included with termux-app, like the bootstrap packages.. but would be executed by the app , which then executes these packages....

To do so you need to inject a LD_PRELOAD into Dalvik/ART startup. This is not possible and therefore you can't use shared library approach. It also works on function level and not the system call - will cause troubles with non-C/C++ programs.

You need a program which takes ELF executable, load its sections into memory and passes execution flow. A bare variant of proot. ptrace() there would be required anyway, to be able handle execve calls recursively.

[ghost]

The Build script would have to be modified.. I don't know to what extent....

What is that build script? It won't be hard to make Termux app build proot and bundle into APK.

If you are talking about package scripts, then there no issues at all. It even possible to get rid of most of our packages with proot, they won't be needed in case of use of normal distribution.

---

IMO, impact on user experience is inevitable. You can't keep quality, functionality, performance or application store availability on high level at same time. Even now this isn't possible, not saying about what will be after fixing execve issue. Solution shouldn't bring more problems than fixing. If there is something that can fix more Termux issues than just execve, you should consider it instead.

[RalfWerner]

2. Packaging everything as "native libs" of APK. - Look branch android-10.
3. Using a custom ELF loader: proot or something else. - against Play Store policy.

@xeffyr I've a question: Is the existence of proot in the termux.apk (bootstrap*) already a violation of the Play Store Directive? The insert would only be done/used after the package update/install TERMUX_ANDROID10 or when proot-distro is used and Google should not be interested.
In my bootloop concept, the extension of the elf-splits (not everything) is also outside the control of Google. And the app version update does not change to $PREFIX.

What is that build script? It won't be hard to make Termux app build proot and bundle into APK.

Agree! My current build script support also: "1. Leaving everything as-is and continuting with SDK 28."

[ghost]

Is the existence of proot in the termux.apk (bootstrap*) already a violation of the Play Store Directive?

https://support.google.com/googleplay/android-developer/answer/9888379:

An app distributed via Google Play may not modify, replace, or update itself using any method other than
Google Play's update mechanism. Likewise, an app may not download executable code (e.g. dex, JAR, .so
files) from a source other than Google Play. This restriction does not apply to code that runs in a virtual
machine and has limited access to Android APIs (such as JavaScript in a webview or browser). 

Proot by itself doesn't violate anything. But it lets application to continue violate policy through downloadable packages. Yes, Termux already violates the policy and it doesn't matter whether proot is added or not. It still exists in Play Store, but there are risks.

[RalfWerner]

Yes, Termux already violates the policy

OK - very juristic formulation. Thus, play store as a distributor no longer suitable for a developing app like termux, when Google does not change that. In the bootloop, after the first version are always local dev-*.apk used.
For 29+, only the question of whether this level has advantages over 28, on which one should not do without?

[ghost]

The only "advantage" is being able to submit updates to Play Store. Not so many. Termux works on Android 11 and even on 12 beta. That's expected, because Android currently preserves backwards compatibility. Problems will be only if with specific update Google decides to drop it.

[xloem]

These solutions wouldn't have to overall reduce usability: for example, if binaries could be installed to sdcard storage, that would be a huge plus elsewhere.

[Vixeliz]

Sorry for the dumb question but with the packaging everything into the apk does that mean it is no longer possible for users to compile and run programs? That would be unfortunate.

[ghost]

@Vixeliz PRoot can be used to workaround that but:

a. Any custom executable loader is technically against Play Store app publishing policy. - Matters only if we decide to continue Play Store updates.
b. PRoot does modifications to program execution flow, adds performance penalty and not stable on some devices.
c. It is unknown how exactly this (Android W^X) issue is going to be solved: we may choose a different solution, use other executable loader or not use it at all.

[Vixeliz]

@xeffyr I see, if you were to try and get back on play store is it feasable to have two versions one with proot and one without? My main use case for termux is developing applications and im very worried it won't be possible going forward.

[ghost]

@xeffyr I see, if you were to try and get back on play store is it feasable to have two versions one with proot and one without? My main use case for termux is developing applications and im very worried it won't be possible going forward.

I don't think 2 termux versions with and without proot would be a good idea. because it needs to maintain one for non-proot-ed termux app for target level 29 (assuming in-APK's for example) and for proot-ed termux app it also needs to maintain existing package implementation which is apt and would probably need much maintenance so preferably one choice is a choice

[Vixeliz]

I see well personally i am crossing my fingers for proot then.

[RalfWerner]

In the android-10 branch, proot is part of the bootstrap (W^X violation) and all executables (ELF) outside the Termux-APK (with n=~/../../lib) are usable through the loader (TERMUX_ANDROID10) with the losses described above. Disadvantage of the branch is, that the well-known pkg administration was abolished and everything is in APK. I've studied an alternative that is based on the ELF transfer/split from $PREFIX to $n. This advances an APK build with variable bootstrap (loop) but pkg would stay as it is.

The more ELF in $n are, the more effective Termux works with API>28. I suspect PlayStore would not refuse the installation and many of the discord kids would never notice any difference as they work with proot-distro.
A bigger problem is the withdrawal of @fornwall (currently has other priorities), which was the only one to do/able the PlayStore updates. So it would have to be found a person with the necessary knowledge and time that can replace him or he returns.

[iamahuman]

I suppose withdrawing from Play Store has one big problem: funding.

[ghost]

Play Store is a just one of ways for funding and no one of maintainers except Fornwall shares it (both account & funding) anyway.

While Termux can dance for follow its policies for app publishing, there no guarantees that they would not be more strict in future. Relying on it for publishing in-APK bundled packages potentially creates a point of failure that is beyond our control: every submitted apk goes through moderation, there could be delays and chance that apk will be rejected for (un)known reason. This creates a possibility that something may become broken as publishing of specific dependency version failed.

[Manu99it]

As an user, I think the best approach is the current one: don't use play store for publishing the app. Other ways to overcome this issue seem to have too many disadvantages and in future Google could tighten PS policies even more. There are already too much problems caused by wrong choices made by Google on Android done on the assumption that users are stupid, or for the sake of privacy and security (when instead those should be in a compromise with usability and comfort). You get two advantages by don't publishing the app to PS:

• app is used mainly by who wants really use it and at least is ok with learning HOW to do it, so you potentially get less management problems on GitHub, reviews, already reported issues, etc
• you don't need to refactor the app in a way that may be frustrating for established users, at least until android isn't going to drop backwards compatibility

[agnostic-apollo]

Docs have been added for android app data file execute restrictions at following links.

• https://github.com/agnostic-apollo/Android-Docs/blob/master/site/pages/en/projects/docs/apps/processes/app-data-file-execute-restrictions.md
• https://github.com/agnostic-apollo/Android-Docs/blob/master/site/pages/en/projects/docs/os/selinux/security-context.md
• https://github.com/agnostic-apollo/Android-Docs/blob/master/site/pages/en/projects/docs/os/selinux/context-types.md

---

@DownrightNifty Hey, thanks for the offer. Check https://termux.dev/donate for how to donate to me or preferably other termux devs currently. I already have funding for next couple of months, and it's not as much a money issue, as it's a time issue.

I have a gazillion things to do related to Termux and currently termux app updates and related work takes higher priority since they have been already been delayed for far too long (as LWN points out too) and more people are in greater need of the update with fixes and additions than any future breakage. I have been working on the updates full time for more than a year at this point, there is just so much work to do and everything is broken. Getting the permission added to AOSP is a priority for me too, just not the top priority and I do plan on looking into it after the releases like I have already said.

Go ahead if you want to make a blog post, that would be great. However, I think after all this time, all the important people who may be able to actually get a real fix implemented already know about it, but then again funding could help motivate someone to take charge.

Also a working hack is being worked on at termux/termux-exec#24 as mentioned above.

[twaik]

It is just a random thought, but what will happen in the case if we drop using bionic linker and start using our own linker and executable/library format?
I mean we can use PE file format instead of using ELF, and technically it will not be custom ELF loader and will not violate GP policies.
In this case we will need to implement building our executables and libraries. AFAIK we can mix android ndk's compiler and llvm-mingw's linker for building (I mean build executable/library with llvm-mingw linker using .o files compiled with android NDK's clang).
Also we will need some solution for invoking our executables (for using in something like libtermux-exec.so). I think we can implement something like invoking /system/bin/sh with LD_PRELOAD, overriding __libc_init (__libc_start_main from bionic world). In this case we will be able to start out executable-starting code present in some ELF library inside APK.

Yes, it definitely will break a lot of stuff working with ELF file format, and will cause higher RAM usage (because kernel shares readonly executable pages related to shared libs to save some memory, which will not happen in our case because of W^X restriction), but we will be able to be back to GP.

[tareksander]

technically it will not be custom ELF loader and will not violate GP policies.

The policy isn't specifically about ELF or dex code, but executable code with system API access in general. The only way to really comply would be using Wasm, Wasi, and providing a custom API for things Wasi doesn't provide (like pthreads, somehow shared memory, etc...).

[tareksander]

Alternatively, a gentoo approach would also comply: if only source code is downloaded, technically you're not downloading executables, and if your definition of app only includes the apk, you're also not updating the app from another source. That with a custom loader would probably be the easiest.

[twaik]

executable code with system API access in general

What is considered to be a system API access? We can restrict access to some functions because we will have full control on the linking process.

[twaik]

Alternative solution could be creating archives with all linking data (.o, .a files and linking command) to be linked on device (if building from source is ok).

[SwuduSusuwu]

Alternatively, a gentoo approach would also comply: if only source code is downloaded, technically you're not downloading executables, and if your definition of app only includes the apk, you're also not updating the app from another source

Sounds cool. ArchLinux does this too.

executable code with system API access in general

What is considered to be a system API access? We can restrict access to some functions because we will have full control on the linking process.

It is now about "how much access does this function give you; if it is too much access, block"; Android has app-level permissions for this.
All code which is not scripts (such as Python is) or virtual (such as Java is) has direct use of operating system calls (printf counts).

If it starts with "Standard C library" or "Standard C++ library", it uses operating system calls:
man printf shows the sh version of printf, which as a script is allowed.
man 3 printf shows the system access version of printf, which is an operating system call.

[tareksander]

Alternative solution could be creating archives with all linking data (.o, .a files and linking command) to be linked on device (if building from source is ok).

.o and .a files already contain executable code.

[twaik]

But it is not linked yet.
Also I did not find the exact article about it in Play Policy Centre.

[TomJo2000]

I think we've rules lawyered Google's arbitrary rules that obviously were not written with any application like Termux in mind enough for one day, don't you think?

[tareksander]

I think if .so is not allowed, so is .a and .o.

Also I did not find the exact article about it in Play Policy Centre.

https://support.google.com/googleplay/android-developer/answer/9888379?sjid=9674596579061280027-EU

[SwuduSusuwu]

I think we've rules lawyered Google's arbitrary rules

Truth is that this is solved (or almost):
Google Store now has new versions of Termux (which can apt-get install the most used Unix packages, + execute those); this whole project was produced through Termux (click for view of devops environment on Termux).
Guess is that the recent responses exist as wishes to have one version for F-Droid + Google Store, or perhaps users who did not know that Google Store's Termux is now new.

[patham9]

@SwuduSusuwu Awesome to hear if I read your response correctly. I guess people were also concerned about the long-term feasibility of the solution. How was it solved in the end, I assume in a way that it will not become problematic with deprecation of older SDK's? It would be interesting to learn more about it.

[SwuduSusuwu]

@SwuduSusuwu Awesome to hear if I read your response correctly. I guess people were also concerned about the long-term feasibility of the solution. How was it solved in the end, I assume in a way that it will not become problematic with deprecation of older SDK's? It would be interesting to learn more about it.

If understood (don't remember where this post was), Termux now has a special version which loads all executables into Termux's own process (just for Google Store).
Some results from version on Google Store:

~/SubStack $ date
Wed Nov 20 08:35:11 PST 2024
~/SubStack $ termux-info
Termux Variables:
TERMUX_EXEC__PROC_SELF_EXE=/data/data/com.termux/files/usr/bin/termux-info
TERMUX_VERSION=googleplay.2024.10.24
TERMUX__USER_ID=0
Packages CPU architecture:
aarch64
Subscribed repositories:
# sources.list
deb https://termux.net stable main

[sylirre]

Termux now has a special version which loads all executables into Termux's own process (just for Google Store).

No, it runs every binary via /system/bin/linker64. As dynamic linker since certain Android OS version can load executables, this is a potential workaround for SELinux restriction. Not without drawbacks though.

See termux/termux-exec#24

Original Termux:

Termux from Play Store, see what says cat /proc/$$/comm: